\subsection{Hydrogen wavefunctions}
Hydrogen wavefunctions $\psi$ are solutions to the differential equation
$${\psi\over n^2}=\nabla^2\psi+{2\psi\over r}$$
where $n$ is an integer representing the quantization of total energy and
$r$ is the radial distance of the electron.
The Laplacian operator in spherical coordinates is

$$\nabla^2={1\over r^2}{\partial\over\partial r}
\left(r^2{\partial\over\partial r}\right)
+{1\over r^2\sin\theta}{\partial\over\partial\theta}
\left(\sin\theta{\partial\over\partial\theta}\right)
+{1\over r^2\sin^2\theta}{\partial^2\over\partial\phi^2}$$

The general form of $\psi$ is

$$\psi=r^le^{-r/n}L_{n-l-1}^{2l+1}(2r/n)
P_l^{|m|}(\cos\theta)e^{im\phi}$$

where $L$ is a Laguerre polynomial, $P$ is a Legendre polynomial and
$l$ and $m$ are integers such that

$$1\le l\le n-1,\qquad -l\le m\le l$$

The general form can be expressed as the product of a radial
wavefunction $R$ and a spherical harmonic $Y$.

$$\psi=RY,\qquad R=r^le^{-r/n}L_{n-l-1}^{2l+1}(2r/n),\qquad
Y=P_l^{|m|}(\cos\theta)e^{im\phi}$$

The following script checks $E=K+V$ for $n,l,m=7,3,1$.

\begin{Verbatim}[formatcom=\color{blue},samepage=true]
laplacian(f) = 1/r^2*d(r^2*d(f,r),r)+
  1/(r^2*sin(theta))*d(sin(theta)*d(f,theta),theta)+
  1/(r*sin(theta))^2*d(f,phi,phi)
n = 7
l = 3
m = 1
R = r^l*exp(-r/n)*laguerre(2*r/n,n-l-1,2*l+1)
Y = legendre(cos(theta),l,abs(m))*exp(i*m*phi)
psi = R*Y
E = psi/n^2
K = laplacian(psi)
V = 2*psi/r
simplify(E-K-V)
\end{Verbatim}

This is the result when the script runs.

$\displaystyle 0$


\subsection{Maclaurin series}

A Taylor series around \(c=0\).

\(f(x)=\sum_{i=0}^\infty (x-c)^i \dfrac{f^i(c)}{i!}\)

\(f(x)=\sum_{i=0}^\infty (x)^i \dfrac{f^i(0)}{i!}\)

For example, for:

\(f(x)=(1-x)^{-1}\)

\(f^i(0)=i!\)

So, around \(x=0\):

\(f(x)=\sum_{i=0}^\infty (x)^i\)


\subsection{Cayley's theorem}

Cayley's theorem states that every group \(G\) is isomorphic to a subgroup of the symmetric group acting on \(G\).

Multiplication by a member of \(G\) is a bijective function, as for each \(g\) there is also a \(g^{-1}\).

This means that multiplication of each member of \(G\) is a permutation, and so is a subset of the symmetric group on \(G\).


\subsection{Basis vectors}

\subsubsection{Basis}

We can write vectors as combinations of other vectors.

\(v=\sum_i \alpha_i v_i\)

A subset which spans the vector space, and which is also linearly independent, is a basis of the vector space.

For an arbitrary vector of size \(n\), we cannot use less than \(n\) elementary vectors. We could use more, but these would be redundant.

If we use \(n\) elementary vectors, there is a unique solution of weights of elementary vectors.

If we use more than \(n\) elementary vectors, there will be linear dependence, and so there will not be a unique solution.


\documentclass{article}
\usepackage[utf8]{inputenc}
\usepackage{amsmath,amssymb}
\newcommand\setdef[2]{\left\{\left.{#1}\,\right|\,{#2}\right\}}
\newcommand\card[1]{\left|#1\right|}
\begin{document}

\section[Probability space]{Probability space $(\Omega, \mathcal A, \mathbb P)$}
\begin{description}
  \item[$\Omega$] is the set of outcomes ($n$-tuples, 1 of 365 days for $n$ people)
    \[ \Omega = \left\{(d_1, d_2, \dots, d_n): 1 \leq d_i \leq 365 \: \forall i \in \{1, \dots, n\}\right\} \]
  \item[$\mathcal A$] is the set of events (power set of $\Omega$)
    \[ \mathcal A = \mathcal P(\Omega) \]
    \begin{align*}
      \{\} &\in \mathcal A \\
      \Omega &\in \mathcal A \\
      \{(d_1, d_2, \dots, d_n): d_i = d_j \text{ for any } i \neq j\} &\in \mathcal A \\
      \{(d_1, d_2, \dots, d_n): d_i \neq d_j \forall i \neq j\} &\in \mathcal A
    \end{align*}
  \item[$\mathbb P$] is the probability measure for given $A \in \mathcal A$
    \[ \mathbb P(\{\}) = 0 \]
    \[ \mathbb P(\Omega) = 1 \]
    \[ \mathbb P(\{(d_1, d_2, \dots, d_n): d_i \neq d_j \forall i \neq j\}) = \frac{n! \cdot {365 \choose n}}{365^n} \]
    \[ \mathbb P(\{(d_1, d_2, \dots, d_n): d_i = d_j \text{ for any } i \neq j\}) = 1 - \frac{n! \cdot {365 \choose n}}{365^n} \]
    where ${365 \choose n}$ is the number of possibilities to assign $n$ people to 365 dates without collision. Once you have chosen $n$ slots, there are $n!$ ways to permute the specific assignment for person $1$ to $n$. As usual we divide the number of desired outcomes by the number of possible outcomes. The possible outcomes are left. We have $365^n$ ways to assign one of $365$ days to $n$ people.
\end{description}

\subsection{Whiteboard solution for (a)}
\[ \mathbb P((x_1, \dots, x_n)) = 365^n \]
\[ \mathbb P(X \geq 2) = 1 - \mathbb P(X = 1) \]
Event $A = \setdef{(x_1, \dots, x_n) \in \Omega}{\text{at least two $x_i$ are equal}}$.
\[ A^C = \setdef{(x_1, \dots, x_n) \in \Omega}{x_i \neq x_j, j \neq i} \]
\[ \mathbb P(A^C) = \frac{\card{A^C}}{\card{\Omega}} = \frac{365 \cdot 364 \cdot \dots \cdot (365-n+1)}{365^n} \]

\section{Find $n$ such that $\mathbb P(A) > 0.5$ with $|A| = 1$}

Using R:

\begin{verbatim}
# via https://stackoverflow.com/a/40527881
ramanujan <- function(n){
  n*log(n) - n + log(n*(1 + 4*n*(1+2*n)))/6 + log(pi)/2
}
bignchoosek <- function(n,k){
  exp(ramanujan(n) - ramanujan(k) - ramanujan(n-k))
}
f <- function (n) { 1 - factorial(n) * bignchoosek(365, n)/365^n }
f(5)
# [1] 0.02713187
for (i in 1:365) {
  if (f(i) > 0.5) {
    print(i)
    break
  }
}
# [1] 23
\end{verbatim}

Using Python:

\begin{verbatim}
>>> import math
>>> fac = math.factorial
>>> f = lambda n: 1 - fac(n) * (fac(365) / (fac(n) * fac(365 - n))) / 365.0**n
>>> for i in range(1,365):
...     if f(i) > 0.5:
...         print(i)
...         break
...
23
\end{verbatim}

Answer: 23

\section{Find $n$ such that $\mathbb P(A) > 0.99$ with $|A| = 1$}

Using R:

\begin{verbatim}
for (i in 1:365) {
  if (f(i) > 0.99) {
    print(i)
    break
  }
}
# [1] 57
\end{verbatim}

Using Python:

\begin{verbatim}
>>> for i in range(1,365):
...     if f(i) > 0.99:
...         print(i)
...         break
...
57
\end{verbatim}

\end{document}


\chapter{Functions}

\index{function}

In a pure mathematical sense, a function is defined as taking an input and determining some output based on that input. A function allows us to group a piece of code that we expect to re-use many times.

\section{Inputs and Outputs}

A simple example might be a function that finds the absolute value of a number. That is, it gets rid of the negative sign if the number is negative, or else it leaves it alone.

\codejs{\source{js/absolute.js}}

Let's take a look at all the elements of how we create functions in JavaScript. The \texttt{var absolute} is a variable just like any other variable. It is how we are going to refer to this particular function later on in the program. The assignment \texttt{=} is simply stating that once I create the function, that variable will hold a reference to it.\\

\index{arguments of a function}

Next, I use the \texttt{function} keyword to state that I am defining a new function. What follows is an open and closed set of parenthesis. This defines the argument list to the function: the inputs to the function, and how I will refer to them from within the function. \texttt{function(x)} means that this function will have 1 input, and we will refer to that input as \texttt{x} within the function as if it is a variable. However, \texttt{x} is not accessible outside the function, only inside of it.\\

What follows the argument list is a set of open and closed curly brackets \texttt{\{\}}. All of the code between these brackets is what is run whenever the function is called. This is where I put any of the control structures or calls to other functions I wish to occur to complete the task this function is supposed to perform.\\

\index{return statement}

The \texttt{return} statement causes the function to stop executing the function code, and return immediately back to where the function was called. A single value can also be returned to that point as in this example. The absolute value function checks to see if the number is negative. If it is, it returns the negation of the number causing it to return a positive number. Otherwise it will return the original number, which I know must already be a positive number.\\

I can put a \texttt{return} statement in multiple places within the function code, but only one of them will be called during a particular call to the function since it will immediately terminate the function as soon as it's reached. So, if it returns the negation of a negative number inside the \texttt{if} statement, it will never reach the other \texttt{return} statement.\\

After the closing bracket of the function is a semi-colon, which is placed there because of the assignment operation that assigned this function into the variable \texttt{absolute}, just like every other assignment operation such as \texttt{var z = 4;}.\\

Once the function is defined, I can call the function by referring to it using the \texttt{absolute} variable. Calling the function is performed by placing an open and closed set of parenthesis with the values I wish the arguments to have inside the function, in exactly the same order as they appeared in the argument list when the function was defined. In this example I find the absolute value of \texttt{-3.5} by calling the function with that as the value of the \texttt{x} argument: \texttt{absolute(-3.5)}. This is such a useful function that it is a part of JavaScript already, and can be called by referring to \texttt{Math.abs(x)}.\\

\section{Function Scope}

\index{function scope}

I can put any code inside a function that I wish, including defining new variables. However, when I define a variable in a function, that variable is not directly accessible outside of the function. This is called function scope. This prevents causing confusion when using the same variable name in different functions.\\

\codejs{\source{js/sqrt.js}}

This example shows one particular way of approximately computing the square root of a number using iteration. JavaScript has a square root function already as \texttt{Math.sqrt(x)}, but lets look at how this function works.\\

It begins just like the \texttt{absolute} function, by declaring a new variable \texttt{var sqrt = } to name the \texttt{function}, and an argument list \texttt{(...)} naming a single input \texttt{x}. However, this \texttt{x} is not the same as the previous \texttt{x}. The argument is only referenced inside of the function, and since this is a different function the argument can have the same name.\\

The first thing done when the function is called is to declare a variable \texttt{guess}, which will end up being the approximate value of the square root of the input. The initial value of \texttt{guess} is not super important, but I know that the square root of a number is less than the number itself, so starting with a guess of half the original number is a good start.\\

Another variable \texttt{max\_error} is then declared to hold a value of how accurate I want the answer to be. I might be tempted to just put zero here, but if I did that then the function may not ever finish running! Basically, if I want a more accurate answer, the function will take longer to run since it will have to iterate more times. So, I just pick a value of the error that is small compared to the numbers I am interested in.\\

These variables are not accessible outside of the function. They don't exist until the function is called, and once the function stops they in effect disappear. So, if I want a variable declared in a function to have a particular value, then I have to assign it within the function. If I want to know what the value was once the function returns, then I have to make sure there is another way to access it, such as using the \texttt{return} statement to return the value the variable contained.\\

A while-loop control structure is then used to implement the iteration. The loop should continue to iterate until the error is less-than or equal to the maximum error that I specified. To do that, the square of the \texttt{guess} is computed by multiplying it by itself, and then subtracting from what should be the true value of the square, which was the original number. If the guess was exactly equal to the square root of \texttt{x}, then this difference would be zero. However, it could be more or less than the true value since it is only approximate.\\

The difference \texttt{guess*guess - x} is then fed into the \texttt{absolute()} function from earlier because the difference could be negative, and I only care about how big it is in magnitude so that I can use the comparison operator to see if it's bigger than the maximum error I want.\\

Notice that I did some math inside of the argument list of the \texttt{absolute()} function. That is ok! Before \texttt{absolute} is called, all of the operations inside there are performed to result in a single number that is then assigned as the argument to the function, and only then is \texttt{absolute} actually called. Also, notice that \texttt{absolute} is really a variable declared outside of the \texttt{sqrt} function, but I am referring to it from inside the \texttt{sqrt} function. Function scope works like a one-way mirror: it can see all the variables declared outside, but no-one on the outside can see the variables that are inside.\\

Inside the while loop the \texttt{guess} is made better by using some mathematical magic based on the previous guess and the original number. Every time through the loop the \texttt{guess} gets closer and closer to the real deal, and eventually it's close enough that the loop condition causes the loop to stop.\\

After the loop is done I just need to return the value of \texttt{guess} back to where the function was called. At this point the variables \texttt{guess} and \texttt{max\_error} can no longer be referenced from anywhere, but the returned value is assigned into the variable \texttt{y} in this example, so that is where it is now stored.\\

\section{Recursion}

\index{recursion}

The code inside a function can call other functions, including itself. When a function calls itself, or results in another call to itself, this is called recursion. This is a form of iteration since the same code is executed repeatedly, and so must have some condition when it will stop, just like with a loop.\\

\codejs{\source{js/factorial.js}}

This example shows a recursive function that computes the factorial of an integer: \(n! = n(n-1)(n-2)...(3)(2)1\). The first thing it does is see if the argument to the function is  0, since factorial of 0 is 1. If it's not 0, then it computes the factorial of 1 less than the argument by calling itself on \texttt{n-1}, multiplies the result by \texttt{n} and returns that. It can call itself because the variable \texttt{factorial} is already defined, and the function will be defined before it gets called the first time.\\

Why does this work? The first time the function is called, the argument is equal to the number it's actually computing the factorial of. Say it's the number 5. We know that \(5! = 5*4*3*2*1\), and we noticed that \(4! = 4*3*2*1\). So we can re-write the original problem as \(5! = 5*4!\). Translating that into code means that the factorial of a number can be computed by calling the same function on 1 less than that number, and multiplying the result by the number.\\

When the function is called the second time, the argument n has a value of 4. Technically speaking, the first call to the function hasn't even finished yet. It's waiting for the second call of the function to return before it can multiply by it's version of n to return the total answer. This is ok because every time the function is called it creates an entirely new \texttt{n} to work with, so the computer doesn't get confused about which one is which. That is called a stack frame, and every recursive call creates a new frame on the stack of all the previous calls that are waiting on this particular call to the function to finish.\\

To complete this call, the same logic applies as the first time. \(4! = 4 * 3!\). This keeps working until it gets down to zero. \(1! = 1 * 0!\), but then what is \(0!\)? If we let the function keep doing the same thing it would think that \(0! = 0 * (-1)!\), and then \((-1)! = (-1)*(-2)!\)  and so on. For one thing that's just wrong. For another, it would never stop! That is what the if-statement is for: it's a terminating condition for when it reaches 0.\\

Once the terminating condition is reached, it returns a value without having to call itself anymore. All the previous calls to the function on the call stack that are waiting can then be completed in the reverse order. \texttt{factorial(0)} returns 1. Then \texttt{factorial(1)} returns 1. Then \texttt{factorial(2)} returns 2. \texttt{factorial(3)} returns 6. \texttt{factorial(4)} returns 24. And finally \texttt{factorial(5)} returns 120, which was the first thing to get called and is the final answer.\\

As an exercise, lets see what would happen if we called \texttt{factorial(-1)}? You can code it up and just see. But lets also think about it logically why what happens, happens. If the argument starts out as -1, the first thing it does is check to see if it's equal to 0. It's not, so it does the recursive call sending in \texttt{n-1}, which is -2. As stated before, this will cause an infinite recursion because the terminating condition \texttt{n === 0} will never be reached. However, it won't run forever. What will happen is that it continues to pile recursive calls onto the call stack, potentially thousands of times, but at some point the computer will say that the maximum size of the stack was reached and will terminate the program, called a stack overflow.\\

\section{Exception Handling}

\index{exceptions}
\index{throw statement}

The function call \texttt{factorial(-1)} never returns, but it doesn't run forever either. It simply exhausts the resources of the computer and crashes the program. For one, factorial isn't really even defined for -1. So it shouldn't have even attempted to compute it. This is an undefined scenario. The way I might deal with that is by throwing an exception from THE function to tell whoever tried to compute \((-1)!\) that they're not suppose to do that.\\

\codejs{\source{js/factorial_exception.js}}

The \texttt{throw} keyword works similarly to \texttt{return}: The function will stop executing, and a value will be associate with the termination. In this case I send a message about what happened. When something is thrown, however, it is an error, and causes what is called a program exception (something that isn't supposed to happen). If that exception is not handled by whatever called the function, then the whole program will terminate. This prevents the undefined behavior from happening unknown to anyone. The stack overflow was also an exception, but the exception I add should catch the problem sooner and give more information about what the issue is.\\

The way we can handle possible exceptions from a function is called a try-catch block. I put the code that might cause an exception inside the braces of the \texttt{try}. If no exception occurs, then everything proceeds as if there is no exception handling at all. However, if that code results in an exception, then execution in the try block terminates, and the catch block begins executing and the \texttt{err} argument contains the value that was specified at the \texttt{throw} statement.\\

\index{try-catch}

\codejs{\source{js/factorial_exception_handled.js}}

\section{Side-effects}

\index{side effects of a function}

Since the code inside a function can access the variables declared outside of the function, it can also alter the values stored in those variables. Since those variables are not arguments to the function, nor are they assigned from the \texttt{return} from the function, we call this type of alteration a \textit{side-effect} of calling the function.

\codejs{\source{js/increment.js}}

In this example \texttt{increment} is a function that has no arguments, and does not return anything. However, when it is called it alters the variable \texttt{i} by adding 1 to its current value. Since \texttt{i} is declared outside of the function, it can be accessed directly even after the function returns.\\

This type of behavior is very useful and powerful, but it must be used carefully. Since side-effects can have consequences far away from where the function is actually called, as opposed to the return statement, and persist after the function returns, someone that didn't write the function may not understand what it's doing. Documentation should be used to describe what is happening if a function has side-effects.\\

\section{Closure}

\index{closure}

A closure happens when one function is defined inside the scope of another function, but is still callable after the outer function has already returned. It is also a way of using the power of function side-effects without any of the dangers by containing the side effects to a particular function scope.

\codejs{\source{js/increment_closure.js}}

I've taken the code for the \texttt{increment} function, and placed it inside another function called \texttt{makeIncrement}. When \texttt{makeIncrement} is called, it creates a new instance of \texttt{increment}, which is returned. However, since \texttt{increment} has to have access to the variable \texttt{i} to work, a closure is formed around that function scope so that it still has access to the version of \texttt{i} created at the same time the function was, even after \texttt{makeIncrement} has returned.\\

When \texttt{makeIncrement} returns the inner function, it is stored in a new variable. Whenever that version of \texttt{increment} is called it is working with its own version of \texttt{i}. I can call \texttt{makeIncrement} again, creating a new version of \texttt{increment} with its own version of \texttt{i}, independent of the other one. That is because each call to \texttt{makeIncrement} creates its own scope, and a new closure when it returns.\\

No matter which order the two versions of the function are called, it remembers how many times that particular version has already been called independent from the other one. However, I cannot directly access the variable \texttt{i} because it was inside the function scope of \texttt{makeIncrement}. It is basically a hidden variable that can be used to keep track of some information between function calls, and so as the creator of \texttt{makeIncrement} I don't have to worry about anyone messing with that variable or making any assumptions about it.\\


\subsection{Assumtions}

Generally, k states, markov chain?

Assumptions are:

+ Lasting immunity
+ No births/other deaths

Components:

+ \(S(t)\) - Susceptible
+ \(I(t)\) - Infected
+ \(R(t)\) - Removed (recovered or died)

Proportion of people who recover each period - \(\gamma \).

Each period, infected can transmit to \(\beta \) people. Total of \(I\beta \).

Not everyone susceptible though, so \(I\beta \dfrac{S}{N}\)

Dynamics:

+ \(\dfrac{dR}{dt} = \gamma I\)
+ \(\dfrac{dS}{dt} = -\beta I \dfrac{S}{N}\)
+ \(\dfrac{dI}{dt} = \beta I \dfrac{S}{N} - \gamma I\)

Note that \(dfrac{dR}{dt} + dfrac{dI}{dt} + \dfrac{dS}{dt} = 0\)

We can then work out \(\dfrac{dI}{dS}\)

\(\dfrac{dI}{dS} = \dfrac{\dfrac{dI}{dt}}{\dfrac{dS}{dt}}\)

\(\dfrac{dI}{dS} = \dfrac{\beta I \dfrac{S}{N} - \gamma I}{-\beta I \dfrac{S}{N}}\)

\(\dfrac{dI}{dS} = \dfrac{\beta S - \gamma N}{-\beta S}\)

\(\dfrac{dI}{dS} = - 1 + \dfrac{\gamma }{\beta }\dfrac{N}{S}\)

We can then work out \(\dfrac{dS}{dR}\)

\(\dfrac{dS}{dR} = \dfrac{\dfrac{dS}{dt}}{\dfrac{dR}{dt}}\)

\(\dfrac{dS}{dR} = \dfrac{-\beta I \dfrac{S}{N}}{\gamma I}\)

\(\dfrac{dS}{dR} = -\dfrac{\beta }{\gamma }\dfrac{S}{N}\)

We can rewrite the infection dynamic:

+ \(\dfrac{dI}{dt} = \beta I \dfrac{S}{N} - \gamma I\)
+ \(\dfrac{dI}{dt} = I(\beta \dfrac{S}{N} - \gamma )\)
+ \(\dfrac{dI}{dt} = I\gamma (\dfrac{\beta }{\gamma }\dfrac{S}{N} - 1)\)

This means that outbreak if \(\dfrac{\beta }{\gamma } > \dfrac{S}{N}\)

\(R_0 = \dfrac{\beta }{\gamma }\)

What is steady state?

\(\dfrac{dI}{dt} = I(\beta \dfrac{S}{N} - \gamma )\)

\(\dfrac{dI}{dt} = 0\)

\(R_0 \dfrac{S}{N} = 1 )\)

\(\dfrac{S}{N} = \dfrac{1}{R_0} )\)

\(\dfrac{\beta }{\gamma } > \dfrac{S}{N}\)

\subsection{Vaccinations}

What proportion need vaccination?


\subsection{Fubini's Theorem for Z-simple Regions}
\noindent
There is another case of Fubini’s Theorem that arises in 3D.\\
\begin{theorem}[Fubini's Theorem for Z-simple Regions]
	Let $D \subset \mathbb{R}^3$ and let $\Omega = \left\{(x,y,z) \mid (x,y) \in D, g(x,y) \leq z \leq h(x,y) \right\}$ be a z-simple region.
	\begin{equation*}
		\iiint\limits_{\Omega}{f(x,y,z)\mathrm{d}V} = \iint\limits_{D}{\int_{g(x,y)}^{h(x,y)}{f(x,y,z)\mathrm{d}z}\mathrm{d}A}
	\end{equation*}
\end{theorem}

\noindent
In other words, dependent variables must be integrated before the independent variables on which they rely, but the order of integration of independent variables doesn't matter.\\

\noindent
For example, we an express the unit sphere as a z-simple region where $D = \left\{(x,y) \mid x^2 + y^2 \leq 1\right\}$ and $-\sqrt{1 - x^2 - y^2} \leq z \leq \sqrt{1 - x^2 - y^2}$. Note that $D$ is a Type I Region.
\begin{equation*}
	V_{\text{sphere}} = \int_{-1}^{1}{\int_{-\sqrt{1-x^2}}^{\sqrt{1-x^2}}{\int_{-\sqrt{1-x^2-y^2}}^{\sqrt{1-x^2-y^2}}{\mathrm{d}z}\mathrm{d}x}\mathrm{d}x} = \frac{4\pi}{3}
\end{equation*}


The homogeneous rigid transform is an extremely useful tool used to represent various things in robotics.
First, the rigid transform is defined as a transformation $T$ that when acting on any vector $v$, produces a transformed vector $T(v)$ of the form:
\begin{equation}
	T(v) = R v + t
\end{equation}

where $R^T R^{-1}$, and $t$ is a vector giving the translation of the origin.
This concept can be represented in a form called a {\it homogeneous transformation matrix}.
A homogeneous transformation matrix $T \in \mathbb{R}^{4 \times 4}$ is a member of special euclidean group $SE(3)$, and can be written in the form:
\begin{equation}
	T = \begin{bmatrix}
		R & t \\
		\bf{0} & 1
	\end{bmatrix}
\end{equation}
where $R \in \mathbb{R}^{3 \times 3}$ is a rotation matrix, and $t \in \mathbb{R}^3$ is a translation vector.
The inverse of a homogeneous transformation matrix is:
\begin{align}
	T^{-1} &=
	\begin{bmatrix}
		R^T & -R^Tt \\
		\bf{0} & 1
	\end{bmatrix} \\
	T T^{-1} &= \bf{I}
\end{align}
When using super and subscripts to specify the frames we are referring to, then $T_b^a = (T_a^b)^{-1}$.
We can use a homogeneous transformation to operate on homogeneous points $p, q \in \mathbb{R}^4$,
\begin{align}
	q &= Tp \\
	\begin{bmatrix}
		x_q \\
		y_q \\
		z_q \\
		1
	\end{bmatrix}
	&= \begin{bmatrix}
		R & t \\
		\bf{0} & 1
	\end{bmatrix}
	\begin{bmatrix}
		x_p \\
		y_p \\
		z_p \\
		1
	\end{bmatrix}.
\end{align}

In general, there are three interpretations for homogeneous transformations:
\begin{enumerate}
	\item A description of relative orientation and translation between frames.
	\item A coordinate transform between frames.
	Specifically, $T_j^i$ is a transform from frame ${j}$ to frame ${i}$:
	\begin{equation}
		P^i = T_j^i P^j
	\end{equation}
	\item A motion of a point (or a collection of points) within a single frame. For example, as we saw above point $p$ can be moved to point $q$:
	\begin{align}
		q &= Tp \\
		P_2^i &= T_j^i P^i_1
	\end{align}
	Similarly, a homogeneous transform can represent a motion from one frame to another.
	Specifically, $T_j^i$ moves frame $i$ to frame $j$.
\end{enumerate}
It is important to specify which of these interpretations you are using within your program, as it can get confusing what these transformations represent if people aren't on the same page.

Additionally, one must pay attention to the frame that the operand is in, as this determines the function of the transform to some degree.
$P_2^i = T^i_j P_1^i$ is a motion that operates on point $P_1^i$ within frame $i$ whereas $P^i = T_j^i P^j$ is a coordinate transformation from $j$ to $i$, and therefore if we feed a point in frame $j$ to this transformation, the point will not move, but the coordinates will be transformed into frame $i$, which is often desired.

\subsection{Transforming between frames}
We can chain together homogeneous transforms to represent a traversal through frames.
\begin{figure}[H]
	\centering
	\includegraphics[width=0.8\textwidth]{images/frame_traversal.png}
	\caption{Traversing through different coordinate frames represented by rigid transformation matrices.}
	\label{fig: frames}
\end{figure}

Say we want the transform from $a$ to $d$:
\begin{equation}
	T_d^a = T^a_b T^b_c T^c_d.
\end{equation}
Notice that the bottom subscript and the following top subscript "cancel" out when multiplying frames together.
Additionally when composing transforms like this, the "source" frame (in this case $d$) is included in the right most transform, and the "destination" frame $a$, is included in the left most.

Using this knowledge we can represent frames that are related in this way by a pose graph, or pose tree consisting of parent and child relationships between each frame.

\begin{figure}[H]
	\centering
	\includegraphics[width=0.8\textwidth]{images/pose_graph.png}
	\caption{Frames and their transforms can be represented in graph form.}
	\label{fig: poses}
\end{figure}


\subsection{Mean Squared Error (MSE) and Root Mean Squared Error (RMSE) of an estimator}

\subsubsection{Mean squared error}

Mean squared error

\(MSE = E[(\hat \theta - \theta )^2]=E[((\hat \theta - E[\hat \theta ])+(E[\hat \theta ]-\theta ))^2]\)

\(MSE = E[(\hat \theta - \theta )^2]=E[(\hat \theta - E[\hat \theta ])^2+(E[\hat \theta ]-\theta )^2+2(E[\hat \theta ]-\theta )(\hat \theta- E[\hat \theta ])]\)

\(MSE = E[(\hat \theta - \theta )^2]=E[(\hat \theta - E[\hat \theta ])^2]+E[(E[\hat \theta ]-\theta)^2] +E[2(E[\hat \theta ]-\theta )(\hat \theta- E[\hat \theta ])]\)

\(MSE = E[(\hat \theta - \theta )^2]=Var(\hat \theta )+(E[\hat \theta ]-\theta)^2 +2(E[\hat \theta ]-\theta )E[\hat \theta- E[\hat \theta ]]\)

\(MSE = E[(\hat \theta - \theta )^2]=Var(\hat \theta )+Bias (\hat \theta )^2\)

\subsubsection{Root Mean Square Error (RMSE)}

This is the square root of the MSE.

It is also called the Root Mean Square Deviation (RMSD)


\section{Linear Systems of Equations}
\label{sec:linearequations}

(Real numbers only this time.)

Linear equations are of the form $Ax = b$ where $A$ is a matrix and $x$ and $b$ are vectors. The rows of $A$ and $b$ form a system of equations that must be simultaneously satisfied by the entries of $x$. If $x,b\in\mathbb{R}^n$, then the solutions to the equation of a single row corresponds to an $n-1$-dimensional hyperplane. If the rows of $A$ are linearly independent, then solutions that simultaneously satisfy the equations in $k$-rows correspond to the $n-k$-dimensional intersection of $k$ $n$-dimensional hyperplanes. The solution of $x$ that satisfies all $n$ equations is a $n-n = 0$-dimensional point, and so $x$ is uniquely determined. If any two rows of $A$ are not linearly independent, then the hyperplanes that correspond to values of $x$ that satisfy them overlap exactly, and their intersection is $n$ dimensional, rather than $n-1$ dimensional. In this case, the value of $x$ that satisfies all rows of $A$ is not narrowed down to a single point. The system of equations is said to be *underdetermined*:. This is equivalently the case when $A$ has $m<n$ rows.

\subsection{$A\in\mathbb{R}^{n\times n}$ Square Matrices}
If $A\in\mathbb{R}^{nxn}$, then the solution to the system is formally $x = A^{-1}b$, where $A^{-1}$ is the matrix inverse, satisfying $A^{-1}A=I$, where $I$ is the identity matrix.

Since $Ax=b$ is the same as expressing $b$ in terms of a linear combination of the columns of $A$, the entries of $x$ can be interpreted as the coefficients resulting from the projection of $b$ into the column space of $A$. Therefore, for an orthonormal matrix, the $A^{-1}$ is simply $A^T$

\subsection{$A\in\mathbb{R}^{m\times n}$ Rectangular Matrices, Overdetermined Case}
If $A\in\mathbb{R}^{mxn}$ with $m>n$ rows, then there need not be any point $x\in\mathbb{R}^n$ in which the $m$ hyperplanes all intersect. In that case, the system does not have a solution $x\in\mathbb{R}^n$, and the system is considered *overdetermined*. (The intersection of $m$ distinct hyperplanes in $n$ dimensional space would have negative dimension $(n-m)<0$ if $m>n$, which my feeble brain can't make sense of.)
\\

In the overdetermined case $A^{mxn}$ with $m>n$, the columns of $A$ do not span $\mathbb{R}^m$ and therefore $b\in\mathbb{R}^m$ may have some component $\epsilon$ that lies outside of the column space of $A$. In that case, no linear combination $x$ of the columns of $A$ can express $b$ perfectly, but we might look for approximate solutions $\hat{x}$ so that:

\begin{equation}
A\hat{x} + \epsilon = b
\end{equation}

So that the error $||\epsilon||_{\alpha}$ is minimized. This is the starting point for linear regression from the linear algebra perspective. In practice, the approximation is usually approximated by applying an iterative gradient descent algorithm to minimize the \textit{loss function} $||\epsilon||_{\alpha}$. The choice of metric $||\cdot||_{\alpha}$ is essentially a design choice. For $\alpha=2$, the metric is the $L^2$ norm (cf. section \ref{sec:l2norm}) and an analytic solution exists, named the \textit{normal equations}. The solution minimizes the least squares error and corresponds to the projection of $\mathbf{b}$ into the column space of $a$. The procedure is better known as ordinary least squares regression and therefore I will move a more elaborate discussion to chapter \ref{chap:linearregression}.

\subsection{$A\in\mathbb{R}^{n\times m}$ Rectangular Matrices, Underdetermined Case}
For an underdetermined system with $m<n$, there is either no solution (if the $m$ hyperplanes don't intersect), or there are infinitely many possible solutions that lie on an $n-m$ dimensional hyperplane. One idea is to pick the solution that minimizes $||\hat{x}||_2$ based on the idea that it might generalize better. The least norm solution is $\hat{x} = A^T\left(AAT\right)^{-1}b$, which is the projection of $\vec{0}$ on the solution set.


\section{Richardson's Extrapolation}
Suppose that for each number $h\neq 0$, we have a formula $N_1(h)$ that approximates an unknown constant $M$ with truncation error $O(h)$
\begin{align*}
    M-N_1(h) &= K_1h+K_2h^2+K_3h^3+\ldots \\
    M-N_1\left(\frac{h}{2}\right) &= K_1h+K_2\frac{h^2}{4}+K_3\frac{h^3}{8}+\ldots \\
\end{align*}
If we subtract the first equation from the second equation, then we'll get
\begin{align*}
    & M=N_1\left(\frac{h}{2}\right)+\left[N_1\left(\frac{h}{2}-N_1(h)\right)\right] + K_2\left(\frac{h^2}{2}-h^2\right) + K_3\left(\frac{h^3}{4}-h^3\right) + \ldots \\
    \Rightarrow & N_2(h) = N_1\left(\frac{h}{2}\right) + \left[N_1\left(\frac{h}{2}-N_1(h)\right)\right] \\
    \Rightarrow & M = N_2(h) - \frac{K_2}{2}h^2 - \frac{3K_3}{4}h^3 + \ldots \qquad \text{with truncation error $O(h^2)$}
\end{align*}
\begin{align*}
    & M=N_2\left(\frac{h}{2}\right)+\left[N_2\left(\frac{h}{2}-N_2(h)\right)\right]/3 + \frac{K_3}{8}h^3 + \ldots \\
    \Rightarrow & N_3(h) = N_2\left(\frac{h}{2}\right) + \left[N_2\left(\frac{h}{2}-N_2(h)\right)\right]/3 \\
    \Rightarrow & M = N_3(h) - \frac{K_3}{8}h^3 + \frac{7K_3}{48}h^4 + \ldots \qquad \text{with truncation error $O(h^3)$}
\end{align*}


\section{Eigenvectors of Circulant Matrices}

Define the $n \times n$ matrix $A$ by
\begin{equation}
A = \begin{bmatrix*}[r] -2 & 1 & 0 & 0 & \dots & 0 & 0 & 1 \\ 1 & -2 & 1 & 0 & \dots& 0 & 0 & 0 \\ 0 & 1 & -2 & 1 & \cdots&0 & 0 & 0 \\ 0 & 0 & 1 & -2 & \cdots&0 & 0 & 0 \\ \vdots & \vdots & \vdots & \vdots & \ddots & \vdots & \vdots & \vdots \\ 0 & 0 & 0 & 0 & \cdots & -2 & 1 & 0 \\ 0 & 0 & 0 & 0 & \cdots & 1 & -2 & 1\\ 1 & 0 & 0 & 0 & \cdots & 0 & 1 & -2 \end{bmatrix*}
\end{equation}
In this project, you will explore (and explain) the behavior of the iterative map
    \begin{equation}
      \label{eq:eigen-iterative}
      \bm{x}^{(k+1)} = \left(I + \eta A\right) \bm{x}^{(k)}
    \end{equation}
    for various initial vectors $\bm{x}^{(0)}$, and for $\eta = 0.1$.
\begin{enumerate}[(a)]
  \item Create a function that takes in the argument $n$ and returns the $n \times n$ matrix $A$.  \textit{Hint:} The built-in functions \texttt{diag(...)} (Matlab), or \texttt{np.diag(...)} (Python), or \texttt{diagm(...)} (Julia) may be useful.
  \item Let $n = 32$. Create the $n$ dimensional initial vector $\bm{x}^{(0)}= (1,0,0,\dots,0)$ and plot $\bm{x}^{(0)}$, $\bm{x}^{(1)}$, $\bm{x}^{(2)}, \dots $. Repeat a selection of initial vectors until you can qualitatively describe the behavior of the map $\bm{x} \to (I + \eta A)\bm{x}$?
  \item Compute the eigenvectors of $(I+\eta A)$ by using the function \texttt{[X,V]=eig(...)} (Matlab), \texttt{V,X = np.linalg.eig(...)} (Python), or \texttt{V,X=eigen(...)} (Julia).
  \item \sloppy Plot enough of the eigenvectors so you get a sense for what they look like.\\ \textit{Hint:} If your software returned complex-valued eigenvectors, it may be helpful to plot real and imaginary components separately, for example, in Julia, this would be \texttt{plot(real(X[k,:]))} and \texttt{plot(imag(X[k,:]))}.
  \item[($\ast$)] \textit{Bonus:} Propose a closed form representation for the eigenvectors. Can you prove that every symmetric circulant matrix has eigenvectors in this form?
  \item[($\ast$)] \textit{(Bonus:)} Let $C$ be an (arbitrary) circulant matrix. One of the (many) expressions for the $k$\textsuperscript{th} eigenvector of a circulant matrix is
	  \begin{equation}
		  \label{eq:circulant-eig-vector}
		  \bm{v}_k = \begin{pmatrix} e^{(2 \pi i) 0k/n}\\ e^{(2 \pi i)1k/n}\\e^{(2 \pi i) 2k/n} \\ \vdots \\ e^{( 2 \pi i) (n-1)k/n} \end{pmatrix}
		 \end{equation}
		 with corresponding eigenvalues given by the product of $c$, the first row of $C$, with the eigenvector,
		 \begin{equation}
			 \label{eq:circulant-eig-value}
			 \lambda_k = c v_k
	\end{equation}
	Verify that $\bm{x}_k$ as defined in equation (\ref{eq:circulant-eig-vector}) is an eigenvector of an aribitrary circulant matrix, and that the corresponding eigenvalue is $\lambda_k$, as give by equation (\ref{eq:circulant-eig-value}).
	 \item The $n \times n$ \textit{Fourier Matrix} is the $n \times n$ matrix whose columns are the eigenvectors $\bm{v}_0$, $\bm{v}_1$,\ldots, $\bm{v}_{n-1}$.  Write a function that takes argument $n$ and returns the $n\times n$ Fourier matrix, $F$.
  \item Without any further computation, and by only considering eigenvectors and their corresponding eigenvalues, describe the evolution of the iterative map in equation (\ref{eq:eigen-iterative}) for an initial vector in the form
\begin{equation*}
	\bm{x}^{(0)} = \big(\, \cos(2\pi j / n) + 5 \cos( 4 \cdot 2\pi j / n), \text{ for } j = 1,...,n\,\big).
\end{equation*}
\end{enumerate}
Repeat these steps for the $n \times n$ matrix $B$, defined by
\begin{equation}
B = \begin{bmatrix*}[r] 0 & 1 & 0 & 0 & \dots & 0 & 0 & -1 \\ -1 & 0 & 1 & 0 & \dots& 0 & 0 & 0 \\ 0 & -1 & 0 & \phantom{+}1 & \cdots&0 & 0 & 0 \\ 0 & 0 & -1 & 0 & \cdots&0 & 0 & 0 \\ \vdots & \vdots & \vdots & \vdots & \ddots & \vdots & \vdots & \vdots \\ 0 & 0 & 0 & 0 & \cdots & 0 & 1 & 0 \\ 0 & 0 & 0 & 0 & \cdots & -1 & 0 & 1\\ 1 & 0 & 0 & 0 & \cdots & 0 & -1 & 0 \end{bmatrix*}
\end{equation}


\subsection{Boolean algebra}

\subsubsection{Boolean algebra in propositional logic}

We previously discussed properties of normal form, and the results from these properties.

If another structure shares these properties then they will also share the results.

\subsubsection{Sets satisfy the definitions of a boolean algebra}

If a mathematical structure has the following properties, it shares the results from normal form, and is a boolean algebra.

\begin{itemize}
\item Both binary operators are commutitive - \(A\land B=B\land A\) and \(A\lor B=B\lor A\)
\item Both binary operators are associative - \((A\land B)\land C=A\land (B\land C)\) and \((A\lor B)\lor C=A\lor (B\lor C)\)
\item Completements - \(A\land \neg A=\emptyset \) and \(A\lor \neg A=U\)
\item Absorption - \(A\land (A\lor B)=A\) and \(A\lor (A\land B)=A\)
\item Identity - \(A\land U=A\) and \(A\lor \emptyset =A\)
\item Distributivity - \(A\land (B\lor C)=(A\land B)\lor (A\land C)\) and \(A\lor (B\land C)=(A\lor B)\land(A\lor C)\)
\end{itemize}

These hold for sets, and so boolean algebra holds for sets.


\section{Derivatives of Trig Functions}
Let's find the derivative of $f(x) = \sin{x}$.
Using the limit definition of the derivative,
\begin{align*}
	f^\prime(x) &= \lim_{h \to 0}{\frac{\sin{(x+h)}-\sin{x}}{h}} \\
	&= \lim_{h\to 0}{\frac{\sin{x}\cos{h} + \cos{x}\sin{h} - \sin{x}}{h}} \\
	&= \lim_{h \to 0}{\frac{\sin{x}\left(\cos{h}-1\right) + \cos{x}\sin{h}}{h}} \\
	&= \sin{x}\lim_{h\to 0}{\frac{\cos{h}-1}{h}} + \cos{x}\lim_{h\to 0}{\frac{\sin{h}}{h}} \\
	&= \sin{x}\cdot 0 + \cos{x}\cdot{1} \\
	&= \cos{x}.
\end{align*}

We can now use the chain rule to get the derivative of $\cos{x}$.
\begin{align*}
	f(x) &= \cos{x} \\
	&= \sin{\left(\frac{\pi}{2}-x\right)} \\
	f^\prime(x) &= \cos{\left(\frac{\pi}{2}-x\right)}\cdot -1 \\
	&= -\sin{x}.
\end{align*}

The rest of the derivatives of the common trig functions follow from these results and the quotient rule.
\begin{table}[H]
	\begin{center}
		\begin{tabular}{ l l l }
			$\begin{aligned}\dd{}{x}\sin{x}=\cos{x}\end{aligned}$ & $\begin{aligned}\dd{}{x}\sec{x}=\sec{x}\tan{x}\end{aligned}$ & $\begin{aligned}\dd{}{x}\tan{x}=\sec^2{x}\end{aligned}$ \\
			& & \\
			$\begin{aligned}\dd{}{x}\cos{x}=-\sin{x}\end{aligned}$ & $\begin{aligned}\dd{}{x}\csc{x}=-\csc{x}\cot{x}\end{aligned}$ & $\begin{aligned}\dd{}{x}\cot{x}=-\csc^2{x}\end{aligned}.$
		\end{tabular}
	\end{center}
\end{table}

\begin{example}
	Use the identity $\cos{2x} = \cos^2{x} - \sin^2{x}$ to find the derivative of $\cos{2x}$. Express your answer in terms of $\sin{2x}$. Does this answer the same as what you'd expect by finding the derivative using the chain rule?
\end{example}
\begin{answer}
	Taking the derivative of $\cos^2{x}$,
	\begin{equation*}
		\dd{}{x}\cos^2{x} = \cos{x}\cdot-\sin{x} + \cos{x}\cdot-\sin{x} = -2\sin{x}\cos{x} = -\sin{2x}.
	\end{equation*}

	Taking the derivative of $\sin^2{x}$,
	\begin{equation*}
		\dd{}{x}\sin^2{x} = \sin{x}\cos{x} + \sin{x}\cos{x} = 2\sin{x}\cos{x} = \sin{2x}.
	\end{equation*}

	Combining these results to get the derivative of $\cos{2x}$,
	\begin{equation*}
		\dd{}{x}\cos{2x} = \dd{}{x}\cos^2{x} - \dd{}{x}\sin^2{x} = -\sin{2x} - \sin{2x} = -2\sin{2x}.
	\end{equation*}

	This answer is indeed the same one we'd expect using the chain rule.
\end{answer}


\documentclass{article}
\usepackage{amsmath}
\usepackage[margin=1.0in]{geometry}
\usepackage{xcolor}

\begin{document}

\noindent
Does $\displaystyle \sum_{n=1}^\infty \left(\frac{2\pi+\cos n}{n}\right)^n$
diverge, converge absolutely, or converge conditionally?

\subsection*{Solution}

We consider the Root Test, based on the shape of the terms. (Note that all terms are positive.)
\begin{align*}
L&=\lim_{n \to \infty} \sqrt[n]{|a_n|}\\
&= \lim_{n \to \infty} \sqrt[n]{\left| \left(\frac{2\pi+\cos n}{n}\right)^n \right|}\\
&= \lim_{n \to \infty} \sqrt[n]{\left(\frac{2\pi+\cos n}{n}\right)^n }\\
&= \lim_{n \to \infty}\frac{2\pi+\cos n}{n}
\end{align*}
We cannot evaluate the limit of the sequence above using L'Hopital's rule. (Note the numerator does not go to either $0$ or $\infty$.) Instead, we set up the following for the Squeeze Theorem. Note that
\[ -1 \leq \cos n \leq 1\]
so by adding $2\pi$ to all sides,
\[ 2\pi-1 \leq 2\pi+\cos n \leq 2\pi+1\]
and by dividing,
\[ \frac{2\pi-1}{n} \leq \frac{2\pi+\cos n}{n} \leq \frac{2\pi+1}{n}\]
Since
\[ \lim_{n \to \infty} \frac{2\pi-1}{n}  = 0 \quad\text{and}\quad \lim_{n \to \infty} \frac{2\pi+1}{n}  = 0,\]
by the Squeeze Theorem,
\[\lim_{n \to \infty}\frac{2\pi+\cos n}{n}=0.\]
Since $L < 1$, by the Root Test, the series $\displaystyle \sum_{n=1}^\infty \left(\frac{2\pi+\cos n}{n}\right)^n$ converges absolutely.

\end{document}%%%%%%%%%%%%%%%%%

\begin{align*}
L&=\lim_{n \to \infty} \sqrt[n]{|a_n|}\\
&= \lim_{n \to \infty} \sqrt[n]{\left| \right|}\\
\end{align*}

Since $\sum |a_n| = \sum a_n$, the series $\displaystyle \sum_{n=1}^\infty AAAAAAAAAAAAAA$ converges absolutely.

Since $|r| < 1$, the series ...  converges by the Geometric Series Test.

Since $|r| \geq 1$, the series ...  diverges by the Geometric Series Test.

The function $f(x)=\frac{}{}$ is continuous, positive, and decreasing on $[1,\infty)$.

\subsection*{Solution}


\subsection{Averaging regression predictions}

If we have multiple predictions, we can take an average of these, possibly weighted.

We have \(m\) regressors \(g_j(\mathbf x_i)\).

Our output is:

\(h(\mathbf x_i)=\sum_j w_jg_j(\mathbf x_i)\)


\subsection{Definition and uniqueness}

We now define a matrix operation which in some ways plays the role
of division. We cannot divide by a matrix, but we can multiply by the
inverse of a matrix, which is almost as good.

\begin{definition}{The inverse of a matrix}{invertible-matrix}
  Let $A$ and $B$ be $n\times n$-matrices. We say that $B$ is an
  \textbf{inverse}%
  \index{inverse!of a matrix}%
  \index{matrix!inverse} of $A$ if
  \begin{equation*}
    BA=I\quad\mbox{and}\quad
    AB=I.
  \end{equation*}
  If this is the case, we also write $B=A^{-1}$. When a matrix has an
  inverse, it is called \textbf{invertible}%
  \index{matrix!invertible}%
  \index{invertible matrix}.
\end{definition}

\begin{example}{Verifying the inverse of a matrix}{verifying-inverse}
  Let $A=\begin{mymatrix}{rr}
    1 & 1 \\
    1 & 2
  \end{mymatrix}$. Check that $B=\begin{mymatrix}{rr}
    2 & -1 \\
    -1 & 1
  \end{mymatrix}$ is an inverse of $A$.
\end{example}

\begin{solution}
  To check this, multiply
  \begin{equation*}
    AB = \begin{mymatrix}{rr}
      1 & 1 \\
      1 & 2
    \end{mymatrix} \begin{mymatrix}{rr}
      2 & -1 \\
      -1 & 1
    \end{mymatrix} =\allowbreak \begin{mymatrix}{rr}
      1 & 0 \\
      0 & 1
    \end{mymatrix} = I
  \end{equation*}
  and
  \begin{equation*}
    BA = \begin{mymatrix}{rr}
      2 & -1 \\
      -1 & 1
    \end{mymatrix} \begin{mymatrix}{rr}
      1 & 1 \\
      1 & 2
    \end{mymatrix} =\allowbreak \begin{mymatrix}{rr}
      1 & 0 \\
      0 & 1
    \end{mymatrix} = I.
  \end{equation*}
  This shows that $B$ is indeed an inverse of $A$.
\end{solution}

Unlike multiplication of scalars, it can happen that $A\neq 0$ but $A$
does not have an inverse. This is illustrated in the following
example.

\begin{example}{A non-zero matrix with no inverse}{non-invertible-matrix}
  Let $A=\begin{mymatrix}{rr}
    1 & 1 \\
    1 & 1
  \end{mymatrix}$. Show that $A$ is not invertible.
\end{example}

\begin{solution}
  One might think $A$ has an inverse because it does not equal zero.
  However, note that
  \begin{equation*}
    \begin{mymatrix}{rr}
      1 & 1 \\
      1 & 1
    \end{mymatrix} \begin{mymatrix}{r}
      -1 \\
      1
    \end{mymatrix} =\begin{mymatrix}{r}
      0 \\
      0
    \end{mymatrix}.
  \end{equation*}
  If an inverse $A^{-1}$ existed, we would have the following:
  \begin{eqnarray*}
    \begin{mymatrix}{r}
      -1 \\
      1
    \end{mymatrix}
    &=&
        I\begin{mymatrix}{r}
          -1 \\
          1
        \end{mymatrix} \\
    &=&
        (A^{-1}A) \begin{mymatrix}{r}
          -1 \\
          1
        \end{mymatrix} \\
    &=&
        A^{-1}\paren{A\begin{mymatrix}{r}
            -1 \\
            1
          \end{mymatrix}} \\
    &=&
        A^{-1}\paren{\begin{mymatrix}{r}
            0 \\
            0
          \end{mymatrix}} \\
    &=&
        \begin{mymatrix}{r}
          0 \\
          0
        \end{mymatrix}.
\end{eqnarray*}
  This says that
  \begin{equation*}
    \begin{mymatrix}{r}
      -1 \\
      1
    \end{mymatrix}
    =
    \begin{mymatrix}{r}
      0 \\
      0
    \end{mymatrix},
  \end{equation*}
  which is impossible! Therefore, $A$ does not have an inverse.
\end{solution}

Can a matrix have more than one inverse? It turns out that this is not
the case: the following theorem shows that if $A$ has an inverse, then
the inverse is unique. We can therefore speak of ``the'' inverse,
rather than just ``an'' inverse, of $A$.

\begin{theorem}{Uniqueness of inverse}{unique-inverse}
  Suppose $A$ is an $n\times n$-matrix such that both $B$ and $C$ are
  inverses of $A$. Then $B=C$.
\end{theorem}

\begin{proof}
  By assumption, both $B$ and $C$ are inverses of $A$, so we have
  $AB=I$, $BA=I$, $AC=I$, and $CA=I$. Using the associative and unit
  properties of matrix multiplication, we have:
  \begin{equation*}
    B = BI = B(AC) = (BA)C = IC = C.
  \end{equation*}
  Therefore, $B=C$, as desired.
\end{proof}


\subsection{Divisors and Greatest Common Divisors (GCD)}

\subsubsection{Divisors}

The divisors \(d\) of a natural number \(n\) are the natural numbers such that \(\dfrac{n}{d}\in \mathbb{N}\).

For example, for \(6\) the divisors are \(1, 2, 3, 6\).

Divisors cannot be bigger than the number they are dividing.

\subsubsection{Universal divisors}

For any number \(n \in \mathbb{N}^+\):

\(\dfrac{n}{n}=1\)

\(\dfrac{n}{1}=n\)

Both \(1\) and \(n\) are divisors.

\subsubsection{Common divisors}

A common divisor is a number which is a divisor to two supplied numbers.

\subsubsection{Greatest common divisor}

The greatest common divisor of \(2\) numbers is as the name suggests.

So \(GCD(18,24)=6\)


%
% Chapter 2.5
%

\section*{2.5 Chain Rule}

\subsection*{Definition of the Chain Rule}

If \(g\) is differentiable at \(x\) and \(f\) is differentiable at \(g(x)\), then the composite function \(F=f \circ g\) defined by \(F(x)=f(g(x))\) is differentiable at \(x\) and \(F'\) is given by the product
$$F'(x)=f'(g(x)) \times g'(x)$$
In Leibniz notation, if \(y=f(u)\) and \(u=g(x)\) are both differentiable functions, then
$$ \frac{dy}{dx}=\frac{dy}{du}\frac{du}{dx} $$

\subsection*{Power Rule with the Chain Rule}

If \(n\) is any real number and \(u=g(x)\) is differentiable, the
$$\frac{d}{dx}(u^n)=nu^{n-1}\frac{du}{dx} \Leftrightarrow \frac{d}{dx}{[g(x)]}^n=n{[g(x)]}^{n-1} \times g'(x)$$


\textbf{Find all the roots of $f(z) = z^8 + 15z^4 − 16$ on the complex plane.}

\vspace{0.3in}

In order to use the angle to distinguish the different roots, none of them should share the same angle. To find out if this is the case we compute the roots manually, which is easily done by factorizing the polynomial
\begin{align*}
z^8 + 15z^4 − 16=(z-1)(z+1)(z^2+1)(z^4+16)=0.
\end{align*}
Hence, the roots are
\begin{align*}
x_k=-1,1,\pm i,\sqrt{2}(1\pm i),\sqrt{2}(-1\pm i).
\end{align*}
and are shown in the next figure. Note that the angle can be computed as the arc tangent of quotient of the imaginary and real part of the complex roots, and its different for every root.
\begin{figure}[H]
\centering     %%% not \center
{\includegraphics[scale=0.55]{prob2roots.eps}}
\caption{Newton's method performance.}
\end{figure}

After obtaining the different roots depending on the initial guess we can plot the angle of the Newton's method solution and obtain the following fractals.

\begin{figure}[H]
\centering     %%% not \center
\hspace*{\fill}
\subfigure[Angle fractal]{\includegraphics[scale=0.55]{colormap.eps}}
\hspace*{\fill}
\subfigure[Iterations fractal]{\includegraphics[scale=0.55]{colormap2.eps}}
\hspace*{\fill}
\caption{Fractals obtained with Newton's method.}
\end{figure}
\subsection*{Matlab code for this problem}
\begin{verbatim}
%% Problem 2
n = 200;

[x,y]=meshgrid(linspace(-3,3,n));
z = x+y*1i;
z_roots = [1, -1, -1i, 1i, ...
    sqrt(2)*(1+1i),sqrt(2)*(1-1i),sqrt(2)*(-1+1i),sqrt(2)*(-1-1i)];

% We now compute the roots using Newton's method:
r = nan(size(z)); niter = r; conv = r; % allocate memory
f = @(z) z^8+15*z^4-16;
df = @(z) 8*z^7+60*z^3;
for k = 1:n
    for j = 1:n
        [r(k,j),niter(k,j),conv(k,j)] = newton(f,df,z(k,j),1e-16,200);
    end
end
%%
% Plotting the angle of the Newton's method solution, we obtain a fractal:
close all
figure
pcolor(x, y, round(angle(r),3));
mp = colormap;
colormap jet
shading interp
hold on
plot(z_roots, 'k.','markersize',20); axis equal; axis tight; hold off;
axis([-2 2 -2 2])
xlabel('$\Re(x_k)$','interpreter','latex','fontsize',labelfontsize)
ylabel('$\Im(x_k)$','interpreter','latex','fontsize',labelfontsize)
saveas(gcf,'Latex/FIGURES/colormap','epsc')

% Plot the number of iterations required in a colormap
figure
pcolor(x, y, niter);
shading flat, hold on
plot(z_roots, 'k.','markersize',20); axis equal; axis tight; hold off;
axis([-2 2 -2 2])
colorbar
xlabel('$\Re(x_k)$','interpreter','latex','fontsize',labelfontsize)
ylabel('$\Im(x_k)$','interpreter','latex','fontsize',labelfontsize)
saveas(gcf,'Latex/FIGURES/colormap2','epsc')

% Plot the roots in the complex plane
figure
plot(real(z_roots),imag(z_roots),'b.','markerSize',12)
grid on
axis([-1.5 1.5 -1.5 1.5])
set(gca,'fontsize',axisfontsize)
xlabel('$\Re(x_k)$','interpreter','latex','fontsize',labelfontsize)
ylabel('$\Im(x_k)$','interpreter','latex','fontsize',labelfontsize)
saveas(gcf,'Latex/FIGURES/prob2roots','epsc')
\end{verbatim}


\section{Linear Equations with Constant Equations}

\subsection{Auxiliary Equation}

Linear homogeneous DE with constant coefficients can be expressed as

\begin{equation}
    a_{0} \frac{d^{n} y}{d x^{n}}+a_{1} \frac{d^{n-1} y}{d x^{n-1}}+\cdots+a_{n-1} \frac{d y}{d x}+a_{n} y=0
\end{equation}

Can be written in the form

\begin{equation}
    f(D)y=0
\end{equation}

where $f(D)$ is a linear differential operator. If the algebraic eqn $f(m)=0$
then we know $f(D)e^{mx}=0\implies y=e^{mx}$ is a solution to the form above.
$f(m)=0$ is the auxiliary equation associated with the DE.
Since the DE is of order $n$, the auxiliary equation is of degree $n$ with
roots $m_1,\ldots,m_n$.

Thus we have $n$ solutions $y_1=\exp(m_1x),\ldots,y_n=\exp(m_nx)$ assuming
the roots are \textbf{real} and \textbf{distinct} are then \textbf{linearly independent}.
The general solution is thus

\begin{equation}
    y=c_1\exp(m_1x)+\cdots+c_n\exp(m_nx)
\end{equation}

with arbitrary constants $c_1,\ldots,c_n$.

\subsubsection{Derivation}

We can say that $y^{(k)}$ is the $k$th derivative of $y$, so say the general form is

$$
a_ny^{(n)}+a_{n-1}y^{(n-1)}+\cdots+a_1y^\prime +a_0y=0
$$

If we take $y=e^{rx}$, then observe $y^{(n)}=r^ne^{rx}$. So rewrite the general form as

\begin{align*}
    a_nr^ne^{rx}+a_{n-1}r^{n-1}e^{rx}+\cdots+a_1re^{rx}+a_0e^{rx}&=0\\
    a_nr^n+a_{n-1}r^{n-1}+\cdots+a_1r+a_0&=0
\end{align*}

Solving for the roots $r$ in this characteristic equation helps us obtain the general solution.

\subsection{Auxiliary Equation Repeated Roots}

Need method for obtaining $n$ linearly independent solutions for $n$ equal roots of auxiliary equation.
Suppose auxiliary equation $f(m)=0$ has $n$ roots $m_1=m_2=\cdots=m_n=b$.
Thus, the operator function $f(D)$ has a factor $(D-b)^n$. Want to find $n$ linearly independent $y$ for which $(D-b)^ny=0$.
Use the substitution $y_k=x^ke^{bx}$ such that

\begin{equation}
    (D-b)^n(x^ke^{bx})=0,\;k=0,1,2,\ldots,n-1
\end{equation}

The functions $y_k=x^ke^{bx}$ are linearly independent because the respective powers $x^0,\ldots,x^k$ are linearly independent.
So the general solution takes form

\begin{equation}
    y=c_1e^{bx}+c_2xe^{bx}+\cdots+c_nx^{n-1}e^{bx}
\end{equation}


\chapter{Probability}

\section{Shuffle}
Equal probability shuffle algorithm.

\subsection{Incorrect naive solution}
Swap current card $A_i$ with a random card from the deck.
\begin{java}
for (int i = 0; i < N; i++) {
   int j = (int) Math.random()*N;
   swap(a[i], a[j]);
}
\end{java}
The easiest proof that this algorithm does not produce a uniformly random permutation is that it generates 27 possible outcomes, but there are only 3! = 6 permutations. Since $27\%3 \neq 0$, there must be some permutation is that is picked too much, and some that is picked to little.
\subsection{Knuth Shuffle}
Knuth (aka Fisher-Yates) shuffling algorithm guarantees to rearrange the elements in uniformly random order.
\\
Core clues:
\begin{enumerate}
\item choose index uniformly $\in [i, N)$
\end{enumerate}
\begin{java}
public void shuffle(Object[] a) {
    int N = a.length;
    for (int i = 0; i < N; i++) {
        // choose index uniformly in [i, N)
        int j = i + (int) (Math.random() * (N - i));
        swap(a[i], a[j]);
    }
}
\end{java}

\section{Expected Value}
\subsection{Roll dice until expected value.}


\section{Motivation: from finite element to neural network}\label{FE2NN}
In this section, we will introduce the so-called shallow neural network
(deep neural network with one hidden layer) from the viewpoint of finite element method.

Let us recall the linear finite element functions on the unit interval $\bar{\Omega}=[0,1]$ in Section \ref{linearFE}.
Consider a set of equidistant girds $\mathcal T_\ell$ of level $\ell$ and mesh length $h_\ell = 2^{-\ell}$. The grid points $x_{\ell,i}$ are given by
\begin{equation}
x_{\ell,i}:=ih_\ell,\quad 0\le i\le 2^\ell.
\end{equation}
For $\ell=1$, we denote the special hat function by $\varphi(x)$ and any nodal basis function in \eqref{1dbasis:function} on grid $\mathcal T_\ell$ by $\varphi_{\ell,i} $ as below
\begin{equation}\label{def_g}
\varphi(x) =
\begin{cases}
2x \quad &x\in [0,\frac{1}{2}] \\
2(1-x) \quad &x\in [\frac{1}{2}, 1] \\
0, \quad &\text{others}
\end{cases},\qquad
\varphi_{\ell,i} = \varphi(\frac{x - x_{\ell,i-1}}{2h_\ell}) = \varphi(w_\ell x + b_{\ell,i}).
\end{equation}
That is to say, any $\varphi_{\ell,i}(x)$ can be obtained from $\varphi(x)$ by scaling
(dilation) and translation with
\begin{equation}\label{key}
w_\ell = 2^{\ell-1}, \quad b_{\ell,i} = \frac{-(i-1)}{2},
\end{equation}
in $\varphi_{\ell,i} = \varphi(w_\ell x + b_{\ell,i})$.
\begin{figure}[H]
\centering
\includegraphics[width=4cm]{1dbasis1.pdf}\qquad
\includegraphics[width=5cm]{basisfunction.pdf}
\caption{Diagram of $\varphi(x)$ (left) and $\varphi_{\ell,i}(x)$ (right).}
\end{figure}

Let us recall the finite element interpolation in Section \ref{linearFE} as
\begin{equation}\label{key}
u(x) \approx u_\ell(x) := \sum_{ 0\le i \le 2^\ell} u(x_{\ell,i}) \varphi_{\ell,i}(x),
\end{equation}
for any smooth function $u(x)$ on $(0,1)$. The above interpolation will converge as $\ell \to \infty$, which shows that
\begin{equation}\label{key}
{\rm span} \left\{  \varphi(w_\ell x + b_{\ell,i}) \right\} \quad \text{is dense in} \quad H^1(0,1).
\end{equation}
Thus, we may have the next concise relation:
\begin{equation}\label{key}
\begin{split}
\text{FE space} =  &{\rm span} \left\{  \varphi(w_\ell x + b_{\ell,i}) ~|~ 0\le i \le 2^\ell, \ell = 1, 2, \cdots \right\}
\\
\subset  &{\rm span} \left\{  \varphi(w x + b) ~|~  w, b \in \mathbb{R} \right\}.
\end{split}
\end{equation}
In other words, the finite element space can be understood as the linear combination of $\varphi(w x + b)$ with certain special choice of $w$ and $b$.

Here, we need to point out that this ${\rm span} \left\{  \varphi(w x + b) ~|~  w, b \in \mathbb{R} \right\}$ is exact the deep neural networks with one hidden layer (shallow neural networks) with activation function $\varphi(x)$. More precisely,
\begin{equation}\label{key}
f \in {\rm span} \left\{  \varphi(w x + b) ~|~  w, b \in \mathbb{R} \right\},
\end{equation}
means there exist positive integer $N$ and $w_j, b_j \in \mathbb{R}$ such that
\begin{equation}\label{key}
f = \sum_{j=1}^N a_j \varphi(w_j x + b_j),
\end{equation}
which is also called one hidden neural network function with $N$ neurons.

\begin{remark}
	\begin{enumerate}
		\item By making $w_\ell$ and $b_{\ell,i}$ in \eqref{def_g} arbitrary, we get a much larger class of
		function which is exact a special neural network with activation function $\varphi(x)$.
		\item Generalizations:
		\begin{enumerate}
			\item activation function $\varphi$ can be different, such as ${\rm ReLU}(x) = \max\{0,x\}$.
			\item There is a natural extension for high dimension $d$ as
			\begin{equation}\label{key}
			\left\{  \varphi(w\cdot x + b) \right \},
			\end{equation}
			where $w\in \mathbb{R}^d$, $b\in \mathbb{R}$ and $\displaystyle w\cdot x = \sum_{i=1}^d w_i x_i$.
			This is called ``deep'' neural network with one hidden layer.
		\end{enumerate}
	\end{enumerate}
\end{remark}

%\input{3FEM/2dFEM}

497-6DL/7 Deep Neural network Functions/7.1-FEM2DNN.tex


\section{Mathematical Preliminaries}
\subsection{Directed Graph}
A directed graph $\mathcal{G}$, consists of a pair of finite sets, $(\emph{V},\emph{E})$. The elements of $\emph{V}$ are the vertices and the elements of $\emph{E}$ are the edges. An edge is a set formed by pairs of vertices. Further to this, there is a function, $\tau$, that associates an ordered pair of vertices for each edge in $\emph{E}$. The direction of an edge $\emph{e} = \{v_1, v_2\}$, is said to be $\tau(\emph{e}) = (v_1, v_2)$, where $\tau_{1}(e) = v_1$ is the source vertex and $\tau_{2}(e) = v_2$ is the target vertex.

With our definitions, there comes a constraint for the function $\tau$.
\begin{itemize}
\item There are no self-loops, that is for each $\emph{e} \in \emph{E}$, if $\tau(\emph{e}) = (v_1, v_2)$ then $v_1 \neq v_2$.
\end{itemize}

There may exist neurons that are reciprocally connected, that is, there may be edges whereby we have $\emph{e}, \emph{e}^\prime \in \emph{E}$ such that $\tau(\emph{e})=(v_1,v_2)$ and $\tau(\emph{e}^\prime) = (v_2,v_1)$.

A vertex $\emph{v} \in \mathcal{G}$ is said to be a sink if there exists no $\emph{e} \in \emph{E}$ such that $\emph{v}=\tau_1(\emph{e})$, but there is at least one edge $\emph{e}^\prime \in \emph{E}$ such that $\tau_2(\emph{e}^\prime) = \emph{v}$. Similarly, \emph{v} is said to be a source if there exists no $\emph{e} \in \emph{E}$ such that $\emph{v} = \tau_2(\emph{e})$, but there is at least one $\emph{e}^\prime \in \emph{E}$ such that $\tau_1(\emph{e}^\prime) = \emph{v}$.

A path in a directed graph consists of a sequence of edges $(e_1, e_2, ..., e_n)$ such that for all $1 \leq k \leq n$, the target of $e_k$ is $e_{k+1}$, i.e.~$\tau_2(\emph{e}_k) = \tau_1(\emph{e}_{k+1})$. The length of the path $(e_1, e_2, ..., e_n)$ is n.

If the target of $e_n$ is the source of $e_1$, that is $\tau_2(\emph{e}_n) = \tau_1(\emph{e}_1)$, then $(e_1, ..., e_n)$ is an oriented cycle. A directed graph that contains no oriented cycle is said to be acyclic.

A directed graph is said to be fully connected if for every pair of distinct vertices, there exists a path from one to the other, in at least one direction.
\subsection{Simplices}
The singular of simplices is simplex. A simplex is a generalisation of the notion of a point, a line, a triangle up to some arbitrary dimension. The simplex is named as such since it is the simplest possible polytope in any given space.

We have, for example, the following possible simplices:

\begin{itemize}
    \item A 0 simplex which consists of a vertex.
    \item A 1 simplex which consists of two vertices and an edge with direction.
    \item A 2 simplex which consists of three vertices and three edges all having direction. There is also a clear source and a clear sink. An example is given below.
    \item A 3 simplex consists of 4 vertices and 6 edges all having direction and where we have a clear source and a clear sink.
\end{itemize}
To generalise this, a k-simplex is the convex hull of the k+1 vertices it contains \cite{2018}.
\begin{figure}[H]%
    \centering
    \captionsetup{justification=centering}
    \subfloat[\centering 0 Simplex]{{\includegraphics[width=3cm]{graph/0simplex.png} }}%
    \qquad
    \subfloat[\centering 1 Simplex]{{\includegraphics[width=3cm]{graph/1simplex.png} }}%
    \qquad
    \subfloat[\centering 2 Simplex]{{\includegraphics[width=3cm]{graph/2simplex.png} }}%
    \qquad
    \subfloat[\centering 3 Simplex]{{\includegraphics[width=3cm]{graph/3simplex.png} }}%
    \caption{Simplices from dimension 0 through to 3}%
    \label{fig:example}%
\end{figure}

\subsection{Simplicial Complex}
An abstract simplicial complex $\emph{S}$, is a collection of finite sets. These sets are closed under taking subsets, that is, every subset of a set in the family is also in the family. As an example, in a 2 dimensional simplicial complex, we have triangles, which are sets of size 3, their edges, which are sets of size 2 and their vertices which are sets of size 1 \cite{2008Gregarxiv:0809.4221}.

However, we have a slight variation on this in that we have an abstract $\emph{directed}$ simplicial complex. This is where we again have a collection of sets denoted $\emph{S}$. However, this time, the sets are finite and ordered with the property that if $\sigma \in \emph{S}$, then every subset $\tau$ of $\sigma$, with the natural ordering inherited from $\sigma$, is also a member of $\emph{S}$.

The elements $\sigma$ of a simplicial complex, as mentioned before, are simplices. The set of all simplices of $\emph{S}$ is denoted $\emph{S}_n$. A simplex $\tau$ is said to be a face of $\sigma$ if $\tau$ is a subset of $\sigma$ with a strictly smaller cardinality than $\sigma$. As an example, lets take Figure 3(c). This shows a simplex $\sigma$ of dimension 2, however, we can also consider the edge from vertex 1 to vertex 3 as a face of the 2 dimensional simplex, since this is defined using $\tau$(e) = ($v_1, v_3$) where $\tau$ is a subset of $\sigma$.

These simplices may be joined along these faces. A simplex that is not the face of any other simplex is said to be maximal. Again, with the 2-simplex above, we can see that the maximal dimension of this is 2, and that the edges and vertices that construct this object are faces of the 2-simplex and faces of faces of the 2-simplex respectively. Thus, the set of all maximal simplices of a simplicial complex determines the entire simplicial complex since this is made up entirely of maximal simplicial complexes or a face of a simplicial complex.

\subsection{Simplicial Complex applied to Directed Graphs}
The directed graph $\mathcal{G}$ naturally creates the directed simplicial complex. The directed simplicial complex that is associated to the directed graph $\mathcal{G}$ is called the directed flag complex of $\mathcal{G}$.

The directed flag complex is defined to be the ordered simplicial complex whose n-simplices are all ordered (n+1)-cliques. So, let it be that $\sigma = (v_0, v_1, ...v_n)$, such that $v_i \in \emph{V}$ $\forall i$ and ($v_i, v_j) \in \emph{E}$ for $i < j$. Then the ordered set of these completes the directed flag complex. Further to this, we note that $v_0$ is the initial vertex, or the source of $\sigma$, and that $v_n$ is the final or sink of $\sigma$.

One final thing of note is that because of our assumption on $\tau$, an n-simplex in $\emph{S}$ is characterised by the ordered sequence $(v_o, v_1, ..., v_n)$ and not by the vertices. By example, $(v_0, v_1, v_2)$ and $(v_0, v_2, v_1)$ are two distinct 2-simplices that contain the same set of vertices.

\subsection{Directionality}
The directionality \cite{Reimann_2017} of a directed graph $\mathcal{G}$ is given by:
\begin{equation}
    \Dr(\mathcal{G}) = \sum_{v \in V} \sd(\emph{v})^2 \enspace{,}
\end{equation}
where
\begin{equation}
    \sd(\emph{v}) = \Indeg(\emph{v}) - \outd(\emph{v})\enspace{.}
\end{equation}
In words, the signed degree ($\sd$) is given as the in-degree minus the out-degree for each vertex in $\mathcal{G}$. Following on from this, the directionality of $\mathcal{G}$ is the sum of the squares of all the signed degrees for each vertex in the directed graph $\mathcal{G}$.

Further to this, we can note that the sum of the signed degree for a finite graph is zero. That is,
\begin{equation}
    \sum_{v \in \mathcal{G}}\sd(\emph{v}) = 0 \enspace{.}
\end{equation}
\subsection{Homology}
Simplicial homology arose as a way to study topological spaces. The building blocks of which are our n-simplices. On top of this, we have further numerical quantities in order to describe the abstract directed flag complex. These include Betti numbers and the Euler Characteristic. These are described below.
\subsection{Euler Characteristic}
The Euler Characteristic is simply defined as the alternating sum of the number of simplices in each dimension. This can also be applied to Betti numbers in each dimension, however, for the Betti numbers for our models and even the Erdos-Renyi model in dimensions 1 through 3,  this is not computationally viable since the required memory to compute the Betti numbers in these dimensions is upwards of 16GB.

So, algebraically, the Euler Characteristic is given as follows:
\begin{equation*}
    \chi = k_0 - k_1 + k_2 - k_3 + ...
\end{equation*}
\begin{equation}
    \Rightarrow \chi = \sum_{k \geq 0} (-1)^k \abs{S_k} \enspace{.}
\end{equation}
As a side note, there is a close relationship between the Euler Characteristic and Betti numbers \cite{Reimann_2017}, that are defined in the next section. This relationship is as follows:
\begin{equation}
    \chi(\emph{S}) = \sum_{k\geq 0}(-1)^k \beta_{k}(\emph{S})
\end{equation}

\subsection{Betti Numbers}
Informally, the $k^{th}$ Betti number refers to the number of k-dimensional holes on a topological space, where a k-dimensional hole is a k-dimensional cycle that is not a boundary of a (k+1)-dimensional object.

The definitions of the first few Betti numbers are as follows;
\begin{itemize}
    \item $\beta_0$ is the number of connected components
    \item $\beta_1$ is the number of 1-dimensional or `circular' holes.
    \item $\beta_2$ is the number of 2-dimensional `voids' or `cavities'.
\end{itemize}
By example, lets take our triangular prism in Figure 4. We can see that the vertices are all connected as one object. This, as well as with the Bio-M MC, takes the zeroth Betti number to be 1. That is, $\beta_0 = 1$. Next, we take the informal definition of the 1-dimensional Betti number. We can see here that there are no holes that are of dimension 1, so $\beta_1 = 0$. Finally, we look at the second dimensional definition of a Betti number. We are looking for holes or cavities that are entirely enclosed by 2-dimensional objects. We can see that we have a hole that is enclosed by triangles. Therefore, $\beta_2 = 1$. Applying the Euler Characteristic to the Betti numbers of this example, we have,
\begin{equation}
    \beta_0 - \beta_1 + \beta_2 = 1 - 0 + 1 = 2
\end{equation}
So if we take the classical usage of the Euler characteristic, we can see the relationship, as stated above in the Euler characteristic section, is also 2 as follows:
\begin{equation}
    V - E + F = 6 - 8 + 12 = 2
\end{equation}

\begin{figure}[H]%
    \centering
    \captionsetup{justification=centering}
{{\includegraphics[width=12cm]{graph/simplex_structure.png} }}%
    \caption{Triangular Prism}
    \label{fig:example}%
\end{figure}

Now, informally, let $\mathbb{F}_2$ denote a field of two elements. Let S be a simplicial complex. Define the chain complex C*(S, $\mathbb{F}_2$) to be the sequence ($C_n$ = $C_n$(S,$\mathbb{F}_2$))$_{n\geq0}$, such that $C_n$ is the $\mathbb{F}_2$-Vector space whose basis elements are the n-simplices $\sigma \in  S_n$, for each $n\geq0$. In other words, the elements of $C_n$ are formal sums of n-simplices in S.

For each $n\geq1$, there is a linear transformation called a \textit{differential}.
\begin{equation}
    \partial_n : C_n \rightarrow C_{n-1} \enspace{.}
\end{equation}
specified by $\partial_n(\sigma) = \sigma^0 + \sigma^1 + ... + \sigma^n$ for every n-simplex $\sigma$, where $\sigma^i$ is the $i^{th}$ face of $\sigma$. Having defined $\partial_n$ on the basis, one then extends it linearly to the entire vector space $C_n$. The $n^{th}$ Betti number $\beta_n$(S) of a simplicial complex S is the $\mathbb{F}$2-vector space dimension of its $n^{th}$ mod-2 homology group, which is defined by,
\begin{equation}
    \mathbb{H}_n(S, \mathbb{F}_2) = \frac{\Ker(\partial_n)}{ \ime(\partial_{n+1})} \enspace{,}
\end{equation}
for  $n\geq 1$ and
\begin{equation}
    \mathbb{H}_0(S, \mathbb{F}_2) = C_0/\ime(\partial_{1}) \enspace{.}
\end{equation}
For all $n\geq1$, there is an inclusion of vector subspaces i.e.~$\ime(\partial_{n+1}) \subseteq \Ker(\partial_{n}) \subseteq C_n$, and thus the definition of homology makes sense.

Computing Betti numbers of a simplicial complex is conceptually very easy. Let $|S_n|$ denote the number of n-simplices in the simplicial complex S. If one encodes the differential $\partial_{n}$ as a $(|S_n-1| \times |S_n|)$-matrix $D_n$ with entries in $\mathbb{F}_2$, then one can easily compute its \textit{nullity}, null($\partial_n$), and its rank, rk($\partial_n$), which are the $\mathbb{F}_2$ dimensions of the null-space and the column space of $D_n$ respectively. The Betti numbers of S are then a sequence of natural numbers defined by,
\begin{equation}
    \beta_0(S) = \text{dim}_{\mathbb{F}_2}(C_0) - \text{rk}(\partial_1) \enspace{,}
\end{equation}
\begin{equation}
    \beta_{n}(S) = \text{null}(\partial_{n}) - \text{rk}(\partial_{n+1}) \enspace{.}
\end{equation}

Since Im($\partial_{n+1}$) $\subseteq$ Ker($\partial_n$) for all $n\geq 1$, the Betti numbers are always non negative. The $n^{th}$ Betti number $\beta_n$ gives an indication of the n-dimensional cavities in the geometric realisation of S \cite{Reimann_2017}.

\subsection{Blocks}
Within the MC we have a set of neurons and a set of synapses, represented by vertices and directed edges respectively. Now, when we take into account the direction of information flow, an edge has with it a pair of ordered vertices.

Now, when we subdivide the MC on a layer by layer basis, we are left with 25 subsets of ordered pairs of vertices, each belonging to a ``block''. An example of a block is L1-L4. That is, we have a set of ordered vertices where the pre-synaptic neurons set is from Layer 1 and the post-synaptic neurons are from Layer 4.

This is not to say that there are a specific set of neurons that are pre-synaptic or post-synaptic, but rather, the neuron's state is determined by the order in which it appears in the ordered pair $\emph{e}$.

\subsection{Total Variation distance}
Here, we use the Total Variation (TV) distance to quantify statistical differences between distributions obtained from our models.

Let $f$ and $g$ be two discrete distributions over a common support set $\{1,2,\ldots,n\}$.
Thus, $\sum_{i=1}^{n}f_{i} = 1$ and $\sum_{i=1}^{n}f_{i} = 1$.

Typically, we want to compare distribution $f$ obtained from a model with the distribution $g$ obtained from the Bio-M MC. To compare the distance between these two distributions, we take:
\begin{equation}
\delta(f, g) = \sum_{i=1}^{n}(|f_{i} - g_{i}|)\enspace{.}
\end{equation}

This will output a value in the range $[0, 2]$.
Clearly $\delta(f, g)=0$ if $f=g$ and $\delta(f, g)=2$ if $f$ and $g$ have no common events.


\section{Gaussian Quadrature}
\subsection{Legendre Polynomial}
\begin{theo}
Suppose $x_1,x_2\ldots,x_n$ are the roots of the $n$th Legendre polynomial $P_n(x)$ and that for $i=1,2,\ldots,n$ the number $C_i$ are defined by
\[
C_i = \int_{-1}^1\prod_{j=1\atop j\neq i}^n\left(\frac{x-x_j}{x_i-x_j}\right)\D x.
\]
If $P(x)$ is any polynomial of degree less than $2n$, then
\[
\int_{-1}^1P(x)\D x = \sum_{i=1}^nC_iP(x_i)
\]
\end{theo}

\begin{proof}
\begin{enumerate}[(1)]
    \item $P(x)$ is of degree less than $n$.
        \begin{align*}
        \int_{-1}^1P(x)\D x &= \int_{-1}^1\sum_{i=1}^{n}P(x_i)L_i(x)\D x = \int_{-1}^1\sum_{i=1}^n\prod_{j=1\atop j\neq i}^n\left(\frac{x-x_j}{x_i-x_j}\right)P(x_i)\D x\\
        &= \sum_{i=1}^n\left[\int_{-1}^1\prod_{j=1\atop j\neq i}^n\left(\frac{x-x_j}{x_i-x_j}\right)\D x\right]P(x_i) = \sum_{i=1}^nC_iP(x_i).
        \end{align*}
    \item $P(x)$ is of degree at least $n$ but less than $2n$.
        \begin{align*}
        P(x_i)=Q(x_i)P_n(x_i)+R(x_i)=R(x_i)\quad\text{(degree less than $n$)}.
        \end{align*}
\end{enumerate}
\end{proof}

\subsection{Gaussian Quadrature on Arbitrary Intervals}
An integral $\int_a^bf(x)\D x$ over an arbitrart $[a,b]$ can be transformed into an integral over $[-1,1]$
\[
\int_a^bf(x)\D x = \int_{-1}^1f\left(\frac{(b-a)t+(b+a)}{2}\right)\frac{(b-a)}{2}\D t.
\]


\chapter{Mathematic Core Functions(MathCore)}
\section{logistic() and logit()}
\noindent Formula:

$$logistic(x,x0,L,k)=\frac{L}{1+e^{-k(x-x0)}}$$
$$logistic(x)=\frac{1}{1+e^{-x}}$$
$$logit(x)=\ln(\frac{x}{1-x})$$

\noindent Defination:
\begin{lstlisting}
template<typename T>
inline T logistic(T x, T x0 , T L = 1, T k = 1);

inline double logistic(double x);

inline double logit(double x);
\end{lstlisting}

\section{softmax() and sigmoid()}
\noindent Formula:

$$softmax(\mathbf{x})=\frac{e^{x_i}}{\sum e^{x_k}}$$
$$sigmoid(x_i)=\frac{1}{1+e^{-x_i}}$$

One can see that sigmoid function is simply element-wise logistic function.

\noindent Defination:
\begin{lstlisting}
template<typename EigenV>
inline EigenV softmax(EigenV x);

template<typename EigenV>
inline EigenV sigmoid(EigenV x);
\end{lstlisting}


\lab{Algorithms}{Nested and Lambda Functions}{Nested and Lambda Functions}

\objective{Learn about nested and lambda functions}

\section*{Nested functions}
So far all the functions we have been defining are called \emph{global} functions.  This means that these functions are accessible to any program that imports your module.  In Python, it is possible to define nested functions (functions inside of functions).  Nested functions are called \emph{local} functions.  They have access to all the variables that the parent function has, but they are not accessible outside of the parent function.  Consider an example where you reuse a specific block of code many times inside a single function.  This block of code would be a good candidate for a nested function.  Nested functions, and functions in general, must be defined before they are called. Realize that nested functions do have overhead, so only use them when necessary.  Here's an example of pseudocode for Newton's Method:

\begin{lstlisting}
def NewtonsMethod(f,x0):
    def derivative(f):
        #Our nested function is defined first.
        #It is usally good practice to define
        #your functions as early as possible

        #calculate derivative of f
        return derivative

    #Perform Newton's Method
    #we need the deriviative of f
    df = derivative()

    #more calculations
    return roots
\end{lstlisting}

Our nested function \li{derivative} has access to the variables \li{f} and \li{x0}.  Even though we have access to \li{f}, it would still be a good idea to define our nested to function to take one argument, \li{f}.  Our nested function \li{derivative} however, is not accessible outside of \li{NewtonsMethod}.

\section*{Lambda Functions}
A lambda function allows us to create simple, one expression functions. The syntax is illustrated by this example.  Lambda functions are defined as \emph{lambda} functions.

\begin{lstlisting}
: from math import sin, cos
: sincos = lambda x: sin(x) + cos(x)
\end{lstlisting}

Now we can find the value of $\sin(x) + \cos(x)$ by simply typing \li{sincos(x)}.

Oftentimes in real applications we are attempting to do operations on a function. Examples would include root finding, minimization, integration or solving differential equations. Python allows you to pass entire functions as arguments to another function.  If you recall in the beginning of this book, our timing context is a function that accepts another function as input.  Define the following function in Python.
\begin{lstlisting}
: def myFunc(a,b):
....: return a**b
: myFunc
<function myFunc at 0x8b33f44>
: m = lambda x, y: x**y
: m
<function <lambda> at 0x8b45534>
\end{lstlisting}

Notice how lambda functions are much more compact. We are essentially defining the lambda function on-the-fly.  Note that lambda functions cannot have a return statement and must be a single line (even though it does have a colon).
\begin{lstlisting}
: from timer import timer
: with timer(loops=1000) as t:
....: t.timer(myFunc, 3,5)
....: t.timer(lambda x,y: x**y, 3,5)
\end{lstlisting}

Let's look at another example of a function that accepts a function as an argument.
\begin{lstlisting}
: from scipy import integrate
: integrate.quad(lambda x: sp.exp(-x**2/2.0), 0, 2)
(1.1962880133226084, 1.3281464964738456e-14)
\end{lstlisting}

This code will calculate the integral of $e^{-\frac{x^2}{2}}$ between zero and two. Here we used a lambda function to get the job done.

\begin{problem}
Write a function that accepts a number $n$, creates a random polynomial of degree $n$ and then finds a zero of that polynomial. If there is no zero then have the function return a local minimum instead. You will need to create function handles and use the function \li{scipy.optimize.bisect}.
\end{problem}

\begin{problem}
The Lambert W-function is defined as the inverse of the function $f(x) = xe^x$. The W-function has important applications in atomic physics and differential equations. It has no closed-form solution. Create a function that estimates the Lambert W-function (you'll need to use \li{bisect} again).
\end{problem}

\begin{problem}
Read the help file on the function \li{scipy.integrate.odeint}. Use \li{odeint} to find the solution of the differential equation
\begin{equation*}
        x'' = -3x \quad x(0) = 1 \quad x'(0) = 0
\end{equation*}

To do this, you will want to convert the higher order system into a lower order system of more variables. Try letting $x_1 = x$ and $x_2 = x'$.
Now plot the output. What do you find? Is this what you expect?
\end{problem}

\begin{problem}
Newton's method is an iterative root-finding method. It generally converges very quickly, and is very important in non-linear optimization. Newton's method starts with an initial guess $x_0$ for a root and then calculates the next guess using the following method:
\begin{equation*}
x_{i+1} = x_i - \frac{f(x_i)}{f'(x_i)}
\end{equation*}

This procedure repeats until a tolerance is reached (either $x_i - x_{i+1}$ or $f(x_{i+1})$ is small enough)
One of the important advantages to Newton's method as opposed to functions like \li{bisect} is that they are more robust when functions have asymptotes (try to use \li{bisect} on \li{tan} to see this). Also, Newton's method is generally much faster, and is generalizable to higher dimensions.
Write your own Newton's method. Have it accept as arguments f, f', $x_0$ and tol. Remember that f and f' (which you'll probably have to name something like df or fPrime) will be function handles that are passed in. Hint: Use a while loop that tests for reaching the tolerances.
\end{problem}


%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
% CS624: Analysis of Algorithms
% Copyright 2015 Pejman Ghorbanzade <pejman@ghorbanzade.com>
% Creative Commons Attribution-ShareAlike 4.0 International License
% More info: https://github.com/ghorbanzade/beacon
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%

\section*{Question 6}

Prove that if there are positive constants $a$ and $c$ such that
\begin{equation}
T(n) = \sum_{j=2}^{n}(a + (j-1)c)
\end{equation}
then there are constants $A$, $B$ and $C$ such that
\begin{equation}
T(n) = An^2 + Bn + C
\end{equation}
Of course, $A$, $B$ and $C$ depend on $a$ and $c$, but do not depend on $n$.
You should show also that $A > 0$.

\subsection*{Solution}

\begin{equation}
\begin{aligned}
T(n) &= \sum_{j=2}^{n}(a + (j-1)c)\\
&=(a-c)\sum_{j=2}^{n}1+c\sum_{j=2}^{n}j\\
&=(a-c)(n-1)+c\frac{n(n-1)}{2}-c\\
&=(0.5c)n^2+(a-1.5c)n-a
\end{aligned}
\end{equation}

Where $A = 0.5c$, $B = (a - 1.5c)$ and $C = -a$.
Since $c$ is positive, $A$ is positive as well.


\chapter{Conditional Probability}
In general, the occurrence of an event $B$ changes the probability that another
event $A$ occurs, $\Pr_{\Distribution{D}}(A \cond B)$ denotes the latter
probability. More formally, $\Pr(A \cond B) = \Pr(A \cap B) / \Pr(B)$, where
$(\Omega, \Distribution{D})$ is a finite discrete probability space,
$A, B \subseteq \Omega$, and $\Pr_{\Distribution{D}}(B) \neq 0$.
We say that $\Pr_{\Distribution{D}}(A \cond B)$ is the conditional probability
that $A$ occurs given that $B$ occurs.

For example, let us consider $\Omega = [6]^2$ and the uniform distribution
$\Uniform{\Omega}$ on $\Omega$; i.e., we consider an experiment consisting of
rolling two dice. Let us compute the probability that the sum of numbers on the
dice exceeds $6$ given that the first dice's number is $3$. In other words we
need to compute $\Pr_{\Uniform{\Omega}}(A \cond B)$, where
$A = \set[i + j > 6]{(i, j) \in [6]^2}$ and
$B = \set[{j \in [6]}]{(3, j)}$. It is clear that $\Pr_{\Uniform{\Omega}}(B) =
1 / 6$ and
$\Pr_{\Uniform{\Omega}}(A \cap B) = \set{(3, 4), (3, 5), (3, 6)} = 1 / 12$.
Hence, $\Pr_{\Uniform{\Omega}}(A \cond B) = 1 / 2$.

\begin{exercise}
  A family has two children. What is the probability that both are boys, given
  at least one is a boy? What if it is given that \emph{the first child} is a
  boy. (You assume that the probability distribution of families is uniform.)
\end{exercise}

Let us consider another example known as ``Monty Hall Problem''. On the
television game \emph{Let’s make a deal}, a contestant is presented with a
choice of three closed doors. Behind exactly one door is a prize; the other
doors conceal cheap items. First, the contestant is asked to choose a door. Then
Monty Hall, the host of the show,  shows the contestant one of the worthless
prizes behind one of the other doors. At this point, there are two closed
doors, and the contestant is given the opportunity to switch from his original
choice to the other closed door. The question is, is it better for the
contestant to stick to his original choice or to switch doors?

Let us analyze this question using the conditional probabilities. Without loss
of generality, we may assume that the contestant chooses door $1$.
Note that the sample space is equal to $\set{(1, 2), (1, 3), (2, 3), (3, 2)}$,
where the first number denotes the door with the prize and the second number
denotes the door opened by the host. The probability distribution is equal to
\[
  \Distribution{D}(x) =
  \begin{cases}
    1 / 6 & \text{if } x = (1, 2) \\
    1 / 6 & \text{if } x = (1, 3) \\
    1 / 3 & \text{if } x = (2, 3) \\
    1 / 3 & \text{if } x = (3, 2)
  \end{cases}
\]
since in the first two cases Monty has two possible choices to show a door
without the prize.
Suppose the host revealed the door number $2$ (the probability of this is
$3 / 6$). Then the probability that we win the price if we stick to the original
choice is $(1 / 6) / (3 / 6) = 1 / 3$. However, the probability to win the prize
in case of us swithcing the door is $(1 / 3) / (3 / 6) = 2 / 3$. Which implies,
paradoxically, that it is beneficial to switch the door!

\begin{theorem}[Bayes’ Rules]
  Let $(\Omega, \Distribution{D})$ be a finite discrete probability space.
  \begin{itemize}
    \item Let $A, B \subseteq \Omega$ be two events such that
      $\Pr_{\Distribution{D}}(A) > 0$ and
      $\Pr_{\Distribution{D}}(B) > 0$. Then $\Pr_{\Distribution{D}}(A \cond B) =
      \frac{\Pr_{\Distribution{D}}(B \cond A)
        \Pr_{\Distribution{D}}(B)}{\Pr_{\Distribution{D}}(A)}$.
    \item Let $A, B \subseteq \Omega$ be two events such that
      $\Pr_{\Distribution{D}}(A) < 1$ and
      $\Pr_{\Distribution{D}}(B) < 1$; i.e.,
      $\Pr_{\Distribution{D}}(\bar{A}) > 0$ and
      $\Pr_{\Distribution{D}}(\bar{B}) > 0$, where
      $\bar{A} = \Omega \setminus A$ and $\bar{B} = \Omega \setminus B$.
      Then $\Pr_{\Distribution{D}}(A) =
        \Pr_{\Distribution{D}}(A \cond B)\Pr_{\Distribution{D}}(B) +
        \Pr_{\Distribution{D}}(A \cond \bar{B})\Pr_{\Distribution{D}}(\bar{B})$.
  \end{itemize}
\end{theorem}

Usefulness of this result can be illustrated with the following example. Assume
that there is a rare disease that has the property that if a patient is affected
by the disease, then the test is positive in $99\%$ of the cases. However, it
happens in $2\%$ of the cases that a healthy patient tests positive. Statistical
data shows that one person out of $1000$ has the disease. What is the
probability for a patient with a positive test to be affected by the disease?

Let $(\Omega, \Distribution{D})$ be a finite discrete probability space from
this problem.
Let $S$ be the event that the patient has the disease, and $P$ and $N$ the
events that the test is positive or negative. We know that
$\Pr_{\Distribution{D}}(S) = 0.001$,
$\Pr_{\Distribution{D}}(P \cond S) = 0.99$, and
$\Pr_{\Distribution{D}}(P \cond S) = 0.02$, where $\bar{S}$ is the
event that the patient does not have the desease. Therefore
$\Pr_{\Distribution{D}}(S \cond P) =
\frac{
  \Pr_{\Distribution{D}}(P \cond S) \Pr_{\Distribution{D}}(S)
}{
  \Pr_{\Distribution{D}}(P)
}$
and
$\Pr_{\Distribution{D}}(P) =
  \Pr_{\Distribution{D}}(P \cond S) \Pr_{\Distribution{D}}(S) +
  \Pr_{\Distribution{D}}(P \cond \bar{S}) \Pr_{\Distribution{D}}(\bar{S})$.
As a result $\Pr_{\Distribution{D}}(S \cond P) =
\frac{0.99 \cdot 0.001}{0.99 \cdot 0.001 + 0.02 \cdot 0.999} \approx \frac{1}{20}$.


\subsection{Exponentials \& Logarithms}
\begin{definition}
	e is the base of the natural logarithm. It's definied by the limit
	\begin{equation*}
		e = \lim\limits_{n\rightarrow\infty}{\left(1+\frac{1}{n}\right)^n}
	\end{equation*}
\end{definition}
$\exp{x} = e^x$ and $\ln{x}$ are inverse functions of each other such that
\begin{equation*}
	e^{\ln{x}} = x \text{, } \ln{e^x} = x
\end{equation*}

\noindent
Just like other exponents, the normal rules for adding, subtracting, and multiplying powers apply.
\begin{equation*}
	e^xe^y = e^{x+y}\text{, }\frac{e^x}{e^y}=e^{x-y}\text{, and }\left(e^x\right)^k=e^{xk}
\end{equation*}

\noindent
Similar rules apply for logarithms.
\begin{equation*}
	\ln{x}+\ln{y} = \ln{xy}\text{, }\ln{x}-\ln{y} = \ln{\left(\frac{x}{y}\right)}\text{, and }\ln{\left(a^b\right)}=b\ln{a}
\end{equation*}

\noindent
You can also change a logarithm of any base to a natural logarithm.
\begin{equation*}
	\log_{b}{a} = \frac{\ln{a}}{\ln{b}}
\end{equation*}

\noindent
$e$ is also unique in that it is the only real number $a$ satisfying the equation
\begin{equation*}
	\frac{\mathrm{d}}{\mathrm{d}x}a^x = a^x
\end{equation*}
meaning $e^x$ is its own derivative.