\documentclass[11pt]{amsart}
\usepackage{amsmath,amsfonts,amsthm,amssymb, amsaddr}

\title{Vector Calculus: Grad, Div, and Curl}

\author{Joe Bentley}

\date{\today}

\begin{document}

\maketitle

\newpage

\section{Directional Derivative and Gradient}

Consider a scalar field in three dimensions, given by $f(x, y, z)$. The partial derivatives $\frac{\partial f}{\partial x}$, $\frac{\partial f}{\partial y}$, and $\frac{\partial f}{\partial z}$, are taking the derivatives along the $x$, $y$, and $z$ directions respectively. In other words, they are just like normal derivatives of a curve along a direction; they give the rate of change of $f$ along the given direction. What if we want the derivative in a direction other than $x$, $y$, and $z$? We can define the derivative in an arbitrary direction given by $\hat{n}$ as,

\begin{align*}
  \frac{\partial f}{\partial \hat{n}} = \lim_{\epsilon \to 0} \frac{f(\mathbf{r} + \epsilon \hat{n}) - f(\mathbf{r})}{\epsilon}
\end{align*}

By applying the chain rule to this, it follows,

\begin{align*}
  \frac{\partial f}{\partial \hat{n}} &= \frac{d}{d\epsilon} f(x + \epsilon n_x, y + \epsilon n_y, z + \epsilon n_z) \\
                                      &= \frac{\partial f}{\partial x} \frac{d}{d\epsilon} (x + \epsilon n_x)
                                       + \frac{\partial f}{\partial y} \frac{d}{d\epsilon} (y + \epsilon n_y)
                                       + \frac{\partial f}{\partial z} \frac{d}{d\epsilon} (z + \epsilon n_z) \\
                                      &= n_x \frac{\partial f}{\partial x} + n_y \frac{\partial f}{\partial y} + n_z \frac{\partial f}{\partial z}
\end{align*}

Therefore, the directional derivative in a given direction $\hat{n}$ is given by,

\begin{align*}
  \frac{\partial f}{\partial \hat{n}} = \hat{n} \cdot \nabla f
\end{align*}

Where we define the gradient, $\nabla f$ as,

\begin{align*}
  \nabla f = \frac{\partial f}{\partial x} \hat{\imath} + \frac{\partial f}{\partial y} \hat{\jmath} + \frac{\partial f}{\partial z} \hat{k}
\end{align*}

Note that $\frac{\partial f}{\partial \hat{n}} = \hat{n} \cdot \nabla f$ has its maximum value $|\nabla f|$ when $\hat{n}$ is parallel to $\nabla f$. The maximum value is $|\nabla f|$ as $|\hat{n}| = 1$. Hence $\hat{n} = \nabla f / |\nabla f|$ is the direction in which $f$ is increasing most rapidly.

\section{Physical Interpretation of $\nabla \cdot \mathbf{A}$}

Consider an infinitesimal cuboid situated at the position $(x, y, z)$ with sides of length $dx$, $dy$, $dz$. We denote the surface integral as,

\begin{align*}
  \oint_S \mathbf{A} \cdot d\mathbf{S}
\end{align*}

The vector $\mathbf{A} = A_x \hat{\imath} + A_y \hat{\jmath} + A_z \hat{k}$. First we will consider two opposing faces of the cuboid both in the $\hat{\imath}$, denoted by the surfaces $S_1$ at $(x + dx, y, z)$ and $S_2$ at $(x, y, z)$. For the surface $S_1$, $d\mathbf{S_1}$ is given by $d\mathbf{S_1} = dy dz \hat{\imath}$. The vector $\hat{\imath}$ is the unit normal to the surface $S_1$. The surface integral is then given by,

\begin{align*}
  \oint_{S_1} \mathbf{A} \cdot d\mathbf{S} = A_x(x + dx, y, z) dy dz
\end{align*}

For the surface $S_2$, the surface element $d_\mathbf{S_2}$ is given by $d_\mathbf{S_2} = -dy dz \hat{\imath}$. Note that this time the unit normal is in the opposite direction to the unit normal in $d_\mathbf{S_1}$. The surface integral this time is,

\begin{align*}
  \oint_{S_2} \mathbf{A} \cdot d\mathbf{S} = A_x(x, y, z) dy dz
\end{align*}

By adding together the two surface integrals,

\begin{align*}
  \int_{S_1 + S_2} \mathbf{A} \cdot d\mathbf{S} &= \left[A_x(x + dx, y, z) - A_x(x, y, z)\right] dy dz \\
                                                &= \left[\frac{\partial A_x}{\partial x} dx\right] dy dz \\
                                                &= \frac{\partial A_x}{\partial x} dx dy dz = \frac{\partial A_x}{\partial x} dV
\end{align*}

In ths second line we recognised that it is the definition of the derivative multiplied by $dx$, so we can rewrite it as we have done the line after.

We get similar contributions for each of the other pairs of surfaces which we consider to find the final complete result,

\begin{align*}
  \oint_S \mathbf{A} \cdot d\mathbf{S} &= \left[\frac{\partial A_x}{\partial x} + \frac{\partial A_y}{\partial y} + \frac{\partial A_z}{\partial z}\right] dx dy dz \\
                                       &= \nabla \cdot \mathbf{A} dV
\end{align*}

The divergence of $\mathbf{A}$ is therefore just the flux per unit volume of $\mathbf{A}$.

We can construct a finite volume $V$ from infinitesimal cuboids. The surface integral can then be calculated by adding up the formulae for each cuboid and noting that the integrals over the internal surfaces cancel each other out. We are left with only the outside surface. From this we get the divergence theorem, also known as Gauss' theorem,

\begin{align*}
  \oint_S \mathbf{A} \cdot d\mathbf{S} = \int_V (\nabla \cdot \mathbf{A}) dV
\end{align*}

where $S$ is the boundary of $V$. ($S = \partial V$)

\section{The Continuity Equation}

Consider a volume $V$ bounded by a surface $S = \partial V$. The only way a conserved charge can change within the volume $V$ is if there is a current flow (flow of charge) across $S$. We can see that the rate of decrease of charge within $V$ is equal to the current flow across $S$,

\begin{align*}
  \frac{\partial}{\partial t} \int_V \rho dV = - \oint_S \mathbf{j} \cdot d \mathbf{S}
\end{align*}

where $\rho$ is the charge density and $\mathbf{j}$ is the current density (per unit area). Using the divergence theorem gives that the right hand side can be written such that,

\begin{align*}
  \int_V \frac{\partial \rho}{\partial t} dV = - \int_V \nabla \cdot \mathbf{j} dV
\end{align*}

We have also taken the $\partial/\partial t$ into the integral. For this to be true for all volumes $V$, it is required that,

\begin{align*}
  \frac{\partial \rho}{\partial t} + \nabla \cdot \mathbf{j} = 0
\end{align*}

A familiar case is electric charge, where the density $\rho(\mathbf{r}, t)$ is the charge density, and $\mathbf{j}(\mathbf{r}, t)$ is the current density. Another example would be fluid flow, where the density $\rho(\mathbf{r}, t)$ is just the mass density, and for a velocity field $\mathbf{v}(\mathbf{r}, t)$, the mass current is given by $\mathbf{j}(\mathbf{r}, t) = \rho(\mathbf{r}, t)\mathbf{v}(\mathbf{r}, t)$. Thus in this case, due to mass conservation,

\begin{align*}
  \frac{\partial \rho}{\partial t} + \nabla \cdot (\rho \mathbf{V}) = 0
\end{align*}

In the case of an incompressible fluid flow, the density $\rho$ is constant, and thus

\begin{align*}
  \nabla \cdot \mathbf{v} = 0
\end{align*}

\section{Physical Interpretation of $\nabla \times \mathbf{A}$}

Consider an infinitesimal rectangle in the $xy$-plane situation at $(x, y)$ with sides of length $dx$ and $dy$. In this case we can write the circulation (the closed line integral) around the infinitesimal loop as,

\begin{align*}
  \oint_C \mathbf{A} \cdot d\mathbf{r} = \int_{C_1 + C_2 + C_3 + C_4} \mathbf{A} \cdot d\mathbf{r}
\end{align*}

Where each $C_n$ is each side of the infinitesimal rectangle. For the first side $C_1$ we can write an infinitesimal element $d\mathbf{r_1}$ as $d\mathbf{r_1} = dx \hat{\imath}$. The unit vector $\hat{\imath}$ is the unit normal to the line. We can then write the surface integral as,

\begin{align*}
  \int_{C_1} \mathbf{A} \cdot d\mathbf{r_1} = A_x(x, y, z) dx
\end{align*}

Here we have taken the dot product of $\mathbf{A}$ and $d\mathbf{r_1}$, and since this is only for $A_x$, there is no $\hat{\jmath}$ or $\hat{k}$ component. For the side $C_3$ opposite the side $C_1$ we see that $d\mathbf{r_3} = -dx \hat{\imath}$ as it is in the opposite direction to the unit normal of $C_1$. We can thus write the surface integral,

\begin{align*}
  \int_{C_3} \mathbf{A} \cdot d\mathbf{r_3} = -A_x(x, y + dy, z) dx
\end{align*}

By adding together these two line integrals we then see that,

\begin{align*}
  \int_{C_1 + C_3} &= \left[A_x(x, y, z) - A_x(x, y + dy, z)\right] dx \\
                   &= \left[-\frac{\partial A_x}{\partial y} dy\right] dx = -\frac{\partial A_x}{\partial y} dx dy
\end{align*}

Here we have again noted that the $A_x$ terms are just the derivative of $A_x$ with respect to $y$, multiplied by $dy$.

Similarly we find that the other two sides $C_2$ and $C_4$ give,

\begin{align*}
  \int_{C_2 + C_4} \mathbf{A} \cdot d\mathbf{r} &= \left[A_y(x + dx, y, z) - A_y(x, y, z)\right] dy \\
                                                &= \left[\frac{\partial A_y}{\partial x} dx\right] dy = \frac{\partial A_y}{\partial x} dx dy
\end{align*}

Therefore we can now calculate the integral over the entire rectangle,

\begin{align*}
  \oint_C \mathbf{A} \cdot d\mathbf{r} = \left[\frac{\partial A_y}{\partial x} - \frac{\partial A_x}{\partial y}\right] dx dy
\end{align*}

By noting that in the square brackets is the cross product, we can also write it as the $z$ component of the cross product,

\begin{align*}
  \oint_C \mathbf{A} \cdot d\mathbf{r} = {(\nabla \times \mathbf{A})}_z dx dy
\end{align*}

We can also write the rectangle as a surface element $d\mathbf{S}$ with a width $dx$ and height $dy$. We define the normal to this surface as being in the $\hat{k}$ direction (this is where the cross product comes from) allowing to write the integral as,

\begin{align*}
  \oint_C \mathbf{A} \cdot d\mathbf{r} = (\nabla \times \mathbf{A}) \cdot d\mathbf{S}
\end{align*}

Note that we no longer need just the $z$-component of the cross product as the dot product filters out just the component in the $\hat{k}$ direction for us. From this result we can see that the $z$-component of the curl of $\mathbf{A}$ is the circulation per unit area of $A$ in the $xy$-plane.

Generally, for an infinitesimal surface area $d\mathbf{S} = \hat{n} dS$ where $\hat{n}$ is the unit normal, the circulation over the loop $C = \partial S$ is given by,

\begin{align*}
  \oint_C \mathbf{A} \cdot d\mathbf{r} = \left(\nabla \times \mathbf{A}\right) \cdot d\mathbf{S}
\end{align*}

We can construct a finite surface $S$ from infinitestimal surface elements. We can do this by summing the above equation for each element and noting that the integrals over the internal loops cancel each other out. From this we get Stokes theorem,

\begin{align*}
  \oint_C \mathbf{A} \cdot d\mathbf{r} = \int_S \left(\nabla \times \mathbf{A}\right) \cdot d\mathbf{S}
\end{align*}

\section{Using Divergence and Curl with Maxwell's Equations}

We can use the divergence theorem and Stokes theorem to find the integral versions of Maxwell's equations.

This example shows finding the integral version of Gauss' law. The electric field lines diverge from electric charges such that,

\begin{align*}
  \nabla \cdot \mathbf{E} = \frac{\rho(r)}{\epsilon_0}
\end{align*}

By integrating both sides,

\begin{align*}
  \int_V \nabla \cdot \mathbf{E} dV &= \int_V \frac{\rho(r)}{\epsilon_0} dV \\
  \oint_S \mathbf{E} \cdot d\mathbf{S} &= \frac{Q}{\epsilon_0}
\end{align*}

This is the integral form of Gauss law.

This example shows finding the integral version of Amperes Law. The magnetic field lines curl around the electric currents,

\begin{align*}
  \nabla \times \mathbf{B} &= \mu_0 \mathbf{j}(\mathbf{r}) \\
  \int_S \left(\nabla \times \mathbf{B}\right) \cdot d\mathbf{S} &= \int_S \mu_0 \mathbf{j}(\mathbf{r}) d\mathbf{S} \\
  \oint_C \mathbf{B} \cdot d\mathbf{r} &= \mu_0 I
\end{align*}

\section{Vector Calculus Identities}

In this section we will prove many of the identities that we might need to know in vector calculus. These include vector calculus of sums, products, and multiple derivatives. First we will list which identities we need, and then we will go through and prove them all. In the following text, $v$ will mean vector, and $s$ will mean scalar.

First we will consider the sums. There are two types of summations possible for scalar and vector fields,

\begin{align*}
  \phi + \psi \qquad &\text{(s + s = s)} \\
  \mathbf{A} + \mathbf{B} \qquad &\text{(v + v = v)}
\end{align*}

So we will need the formula for,

\begin{align*}
  &\nabla(\phi + \psi) \\
  &\nabla \cdot (\mathbf{A} + \mathbf{B}) \\
  &\nabla \times (\mathbf{A} + \mathbf{B})
\end{align*}

In fact, we can consider general linear combinations of the scalar and vectors fields,

\begin{align}
  \label{eq:1}
  &\nabla(\lambda \phi + \mu \psi) \\
  \label{eq:2}
  &\nabla \cdot (\lambda \mathbf{A} + \mu \mathbf{B}) \\
  \label{eq:3}
  &\nabla \times (\lambda \mathbf{A} + \mu \mathbf{B})
\end{align}

where $\lambda$ and $\mu$ are constants.

Next we need the identities for products. There are four possible types of products,

\begin{align*}
  \phi\psi \qquad &\text{s $\times$ s = s} \\
  \phi\mathbf{A} \qquad &\text{s $\times$ v = v} \\
  \mathbf{A} \cdot \mathbf{B} \qquad &\text{v $\times$ v = s} \\
  \mathbf{A} \times \mathbf{B} \qquad &\text{v $\times$ v = v}
\end{align*}

So we will need the formula for,

\begin{align}
  \label{eq:4}
  &\nabla(\phi\psi) \\
  \label{eq:5}
  &\nabla\cdot(\phi\mathbf{A}) \\
  \label{eq:6}
  &\nabla\times(\phi\mathbf{A}) \\
  \label{eq:7}
  &\nabla(\mathbf{A}\cdot\mathbf{B}) \\
  \label{eq:8}
  &\nabla\cdot(\mathbf{A}\times\mathbf{B}) \\
  \label{eq:9}
  &\nabla\times(\mathbf{A}\times\mathbf{B})
\end{align}

And finally we need the multiple derivatives,

\begin{align}
  \label{eq:10}
  &\nabla\cdot(\nabla\phi) \\
  \label{eq:11}
  &\nabla\times(\nabla\phi) \\
  &\nabla(\nabla\cdot\mathbf{A}) \notag\\
  \label{eq:12}
  &\nabla\cdot(\nabla\times\mathbf{A}) \\
  \label{eq:13}
  &\nabla\times(\nabla\times\mathbf{A})
\end{align}

So now we will start and find the identity for expression.~\ref{eq:1}. Since this is vector, we only need to prove this for one component,

\begin{align*}
  &{\left[\nabla(\lambda \phi + \mu \psi)\right]}_x = \frac{\partial}{\partial x}\left(\lambda\phi + \mu\psi\right) \\
  =& \lambda \frac{\partial \phi}{\partial x} + \mu \frac{\partial \psi}{\partial x} = \lambda{[\nabla\phi]}_x + \mu{[\nabla \psi]}_x \\
  =& {\left[\lambda \nabla \phi + \mu \nabla \psi\right]}_x \\
  \implies& \nabla[\lambda \phi + \mu \psi] = \lambda\nabla\phi + \mu\nabla\psi
\end{align*}

For the second identity, we have to prove for each component as this isn't a vector as it is the scalar product,

\begin{align*}
  \nabla\cdot[\lambda\mathbf{A} + \mu\mathbf{B}] &= \frac{\partial}{\partial x} {\left[\lambda \mathbf{A} + \mu \mathbf{B}\right]}_x + \frac{\partial}{\partial y} {\left[\lambda \mathbf{A} + \mu \mathbf{B}\right]}_y + \frac{\partial}{\partial z} {\left[\lambda \mathbf{A} + \mu \mathbf{B}\right]}_z \\
                                                 &= \frac{\partial}{\partial x} \left[\lambda A_x + \mu B_x \right] +\frac{\partial}{\partial y} \left[\lambda A_y + \mu B_y \right] +\frac{\partial}{\partial z} \left[\lambda A_z + \mu B_z \right] \\
                                                 &= \lambda \frac{\partial A_x}{\partial x} + \mu \frac{\partial B_x}{\partial x} + \lambda \frac{\partial A_y}{\partial y} + \mu \frac{\partial B_y}{\partial y} + \lambda \frac{\partial A_z}{\partial z} + \mu \frac{\partial B_z}{\partial z} \\
                                                 &= \lambda \left(\frac{\partial A_x}{\partial x} + \frac{\partial A_y}{\partial y} + \frac{\partial A_z}{\partial z}\right) + \mu \left(\frac{\partial B_x}{\partial x} + \frac{\partial B_y}{\partial y} + \frac{\partial B_z}{\partial z}\right) \\
                                                 &= \lambda \nabla \cdot \mathbf{A} + \mu \nabla \cdot \mathbf{B}
\end{align*}

Therefore we have for expression.~\ref{eq:2},

\begin{align*}
  \nabla\cdot[\lambda\mathbf{A} + \mu\mathbf{B}] = \lambda \nabla \cdot \mathbf{A} + \mu \nabla \cdot \mathbf{B}
\end{align*}

For the third identity, we again have another vector as we are taking the cross product, so we only need to prove it for one component,

\begin{align*}
  {\left[\nabla\times(\lambda\mathbf{A}+\mu\mathbf{B})\right]}_x &= \frac{\partial}{\partial y}{[\lambda\mathbf{A}+\mu\mathbf{B}]}_z - \frac{\partial}{\partial z}{[\lambda\mathbf{A}+\mu\mathbf{B}]}_y \\
                                                                 &= \frac{\partial}{\partial y}[\lambda A_z+\mu B_z] - \frac{\partial}{\partial z}[\lambda A_y+\mu B_y] \\
                                                                 &= \lambda\left(\frac{\partial A_z}{\partial y} - \frac{\partial A_y}{\partial z}\right) + \mu\left(\frac{\partial B_z}{\partial y} - \frac{\partial B_y}{\partial z}\right) \\
                                                                 &= \lambda{(\nabla\times\mathbf{A})}_x + \mu{(\nabla\times\mathbf{A})}_x \\
                                                                 &= {[\lambda(\nabla\times\mathbf{A}) + \mu(\nabla\times\mathbf{A})]}_x
\end{align*}

Therefore we have shown that,

\begin{align*}
  \nabla\times(\lambda\mathbf{A} + \mu\mathbf{B}) = \lambda \nabla\times\mathbf{A} + \mu \nabla\times\mathbf{B}
\end{align*}

Next we have our fourth identity and our first product identity, from expression.~\ref{eq:4},

\begin{align*}
  {[\nabla(\phi\psi)]}_x &= \frac{\partial}{\partial x}(\phi\psi) \\
                         &= \frac{\partial \phi}{\partial x}\psi + \phi\frac{\partial\psi}{\partial x} \\
                         &= {[\nabla\phi]}_x\psi + \phi{[\nabla\psi]}_x \\
                         &= {[(\nabla\phi)\psi + \phi(\nabla\psi)]}_x
\end{align*}

Therefore we have our fourth identity,

\begin{align*}
  \nabla(\phi\psi) = (\nabla\phi)\psi + \phi(\nabla\psi)
\end{align*}

Now for our fifth expression, which is a scalar so we have to prove for all components,

\begin{align*}
  \nabla\cdot(\phi\mathbf{A}) &= \frac{\partial}{\partial x}{(\phi\mathbf{A})}_x + \frac{\partial}{\partial y}{(\phi\mathbf{A})}_y + \frac{\partial}{\partial z}{(\phi\mathbf{A})}_z \\
                              &= \frac{\partial}{\partial x}(\phi A_x) + \frac{\partial}{\partial y}(\phi A_y) + \frac{\partial}{\partial z}(\phi A_z) \\
                              &= \frac{\partial\phi}{\partial x} A_x + \frac{\partial\phi}{\partial y} A_y + \frac{\partial\phi}{\partial z} A_z + \frac{\partial A_x}{x} \phi + \frac{\partial A_y}{y} \phi + \frac{\partial A_z}{z} \phi \\
                              &= (\nabla\phi)\cdot\mathbf{A} + \phi \nabla\cdot\mathbf{A}
\end{align*}

We now have the fifth identity,

\begin{align*}
  \nabla\cdot(\phi\mathbf{A}) = (\nabla\phi)\cdot\mathbf{A} + \phi \nabla\cdot\mathbf{A}
\end{align*}

For our sixth expression, we have a vector and thus only have to prove one component,

\begin{align*}
  {[\nabla\times(\phi\mathbf{A})]}_x &= \frac{\partial}{\partial y}{(\phi\mathbf{A})}_z - \frac{\partial}{\partial z}{(\phi\mathbf{A})}_y \\
                                     &= \frac{\partial}{\partial y}(\phi A_z) - \frac{\partial}{\partial z}(\phi A_y) \\
                                     &= \frac{\partial\phi}{\partial y}A_z - \frac{\partial\phi}{\partial z}A_y + \phi\frac{\partial A_z}{y} - \phi\frac{\partial A_y}{z} \\
                                     &= {[\nabla\phi\times\mathbf{A}]}_x + {[\phi\nabla\times\mathbf{A}]}_x \\
                                     &= {[\nabla\phi\times\mathbf{A} + \phi\nabla\times\mathbf{A}]}_x
\end{align*}

Therefore we have our sixth identity,

\begin{align*}
  \nabla\times(\phi\mathbf{A}) = (\nabla\phi)\times\mathbf{A} + \phi(\nabla\times\mathbf{A})
\end{align*}

For the rest of the proofs of the identities see canvas or written notes (the proofs are long and I cannot be bothered to type them up), but I will state all of the identities here,

\begin{align*}
  \nabla(\mathbf{A}\cdot\mathbf{B}) &= (\mathbf{B}\cdot\mathbf{A})\mathbf{A}+\mathbf{B}\times(\nabla\times\mathbf{A})+(\mathbf{A}\cdot\nabla)\mathbf{B}+\mathbf{A}\times(\nabla\times\mathbf{B}) \\
  \nabla\cdot(\mathbf{A}\times\mathbf{B}) &= \mathbf{B}\cdot(\nabla\times\mathbf{A}) - \mathbf{A}\cdot(\nabla\times\mathbf{B}) \\
  \nabla\times(\mathbf{A}\times\mathbf{B}) &= (\mathbf{B}\cdot\nabla)\mathbf{A}-(\mathbf{A}\cdot\nabla)\mathbf{B}+\mathbf{A}(\nabla\cdot\mathbf{B})-\mathbf{B}(\nabla\cdot\mathbf{A}) \\
  \nabla\cdot(\nabla\psi) &= \nabla^2 \psi \\
  \nabla\times(\nabla\psi) &= \mathbf{0} \\
  \nabla\cdot(\nabla\times\mathbf{A}) &= \mathbf{0} \\
  \nabla\times(\nabla\times\mathbf{A}) &= \nabla(\nabla\cdot\mathbf{A}) - \nabla^2 \mathbf{A}
\end{align*}

\section{Using Vector Calculus Identities: Maxwell's Equations and Continuity}

From Coulomb's law, Biot Savart law, and Faraday's laws we can derive the differential form of Maxwell's equations,

\begin{alignat*}{2}
  \nabla \cdot \mathbf{E} &= \frac{\rho}{\epsilon_0} \qquad &&\nabla \times \mathbf{E} = -\frac{\partial\mathbf{B}}{\partial t} \\
  \nabla \cdot \mathbf{B} &= 0 \qquad &&\nabla \times \mathbf{B} = \mu_0 \mathbf{J}
\end{alignat*}

Our equation, $\nabla\times\mathbf{B} = \mu_0 \mathbf{J}$, known as Ampere's circuital law is incomplete as it does not obey the continuity equation. The continuity equation describes that,

\begin{align*}
  \frac{\partial\rho}{\partial t} + \nabla\cdot\mathbf{J} = 0
\end{align*}

but we see that if we take the divergence of $\mathbf{J}$ we have zero,

\begin{align*}
  \mu_0 \nabla\cdot\mathbf{J} = \nabla\cdot(\nabla\times\mathbf{B}) = 0.
\end{align*}

by our identity that the divergence of a curl of a vector field is always zero. This implies that the rate of change of charge density, $\frac{\partial\rho}{\partial t}$ must always be equal to zero for the continuity equation to hold, but this is not necessary to be true for example with a current flowing through a wire. Therefore we must have that,

\begin{align*}
  \nabla\cdot\mathbf{J} = -\frac{\partial\rho}{\partial t}
\end{align*}

Therefore, by applying this,

\begin{align*}
  \mu_0\nabla\cdot\mathbf{J} = -\mu_0\frac{\partial\rho}{\partial t} = -\mu_0\epsilon_0\nabla\cdot\frac{\partial\mathbf{E}}{\partial t}
\end{align*}

Here we have recognised that the rate of change of charge density is given in the time derivative of Gauss' law. It follows that the continuity equation can be satisfied by writing,

\begin{align*}
  \nabla\times\mathbf{B} = \mu_0\left(\mathbf{J} + \epsilon_0 \frac{\partial \mathbf{E}}{\partial t}\right)
\end{align*}

This new term is called the displacement current, and this is known as Maxwell's addition to Ampere's circuit law.

\section{Using Vector Calculus Identities: Electromagnetic Wave Equation}

Maxwell's equations in free space, that is space where there are no charges or currents reduce to,

\begin{alignat*}{2}
  \nabla\cdot\mathbf{E} &= 0 \qquad &&\nabla\times\mathbf{E} = -\frac{\partial \mathbf{B}}{\partial t} \\
  \nabla\cdot\mathbf{B} &= 0 \qquad &&\nabla\times\mathbf{B} = \mu_0\epsilon_0\frac{\partial\mathbf{E}}{\partial t}
\end{alignat*}

Taking the curl of $\nabla\times\mathbf{E}$ gives that,

\begin{align*}
  \nabla\times(\nabla\times\mathbf{E}) = \nabla\times\left(-\frac{\partial\mathbf{B}}{\partial t}\right) = -\frac{\partial}{\partial t} \left(\nabla\times\mathbf{B}\right) = -\mu_0\epsilon_0\frac{\partial^2\mathbf{E}}{\partial t^2}
\end{align*}

Therefore we have using our vector identity for the curl of a curl of a vector field,

\begin{align*}
  \nabla(\nabla\cdot\mathbf{E}) - \nabla^2 \mathbf{E} = -\mu_0\epsilon_0\frac{\partial^2\mathbf{E}}{\partial t^2}
\end{align*}

From our free space version of Gauss' law we can see that the first term goes to zero, so that

\begin{align*}
  \nabla^2\mathbf{E} = \mu_0\epsilon_0\frac{\partial^2\mathbf{E}}{\partial t^2}
\end{align*}

This is just the wave equation, just as in classical wave theory. The only difference between this and how we remember the wave equation from the first year is that instead of taking the second derivative with respect to position in one direction, we now take the second derivative in all directions, which is just the Laplacian. We therefore see that $\mathbf{E}$ satisfies the wave equation with velocity $c = \frac{1}{\sqrt{\mu_0\epsilon_0}}$.

We can do the same thing with the $\mathbf{B}$-field by computing $\nabla\times(\nabla\times\mathbf{B})$,

\begin{align*}
  \nabla\times(\nabla\times\mathbf{B}) &= \mu_0\epsilon_0\frac{\partial}{\partial t}(\nabla\times\mathbf{E}) \\
  \nabla(\nabla\cdot\mathbf{B}) - \nabla^2\mathbf{B} &= \mu_0\epsilon_0\frac{\partial}{\partial t}\left(-\frac{\partial\mathbf{B}}{\partial t}\right)
\end{align*}

Here we have used that $\nabla\times\mathbf{E} = -\frac{\partial\mathbf{B}}{\partial t}$. Again we can say that the first term $\nabla(\nabla\cdot\mathbf{B})$ is zero as Gauss' law for magnetism tells us, so finally we have,

\begin{align*}
  \nabla^2\mathbf{B} = \mu_0\epsilon_0\frac{\partial^2\mathbf{B}}{\partial t^2}
\end{align*}

Which is again the same form of the wave equation satisfied by the electric field. We therefore see that we now have two fields, the magnetic field and the electric field, which both satisfy the classical wave equation of the same form.

\end{document}


\section*{Exercises}

\begin{ex} Show that $1+i,2+i$ are the only two roots to
\begin{equation*}
p(x) =x^{2}-(3+2i) x+(1+3i)
\end{equation*}
Hence complex zeros do not necessarily come in conjugate pairs if the coefficients of the equation
are not real.
\begin{sol}
\[
(x-(1+i)) (x-(2+i))
= x^{2}-(3+2i) x+1+3i
\]
\end{sol}
\end{ex}

\begin{ex} Give the solutions to the following quadratic equations having real
coefficients.

\begin{enumerate}
\item $x^{2}-2x+2=0$

\item $3x^{2}+x+3=0$

\item $x^{2}-6x+13=0$

\item $x^{2}+4x+9=0$

\item $4x^{2}+4x+5=0$
\end{enumerate}
\begin{sol}
\begin{enumerate}
\item Solution is: $1+i,1-i$
\item Solution is: $\frac{1}{6}i\sqrt{35}-\frac{1}{6},-\frac{%
1}{6}i\sqrt{35}-\frac{1}{6}$
\item Solution is: $3+2i,3-2i$
\item Solution is: $i\sqrt{5}-2,-i\sqrt{5}-2$
\item Solution is: $-\frac{1}{2}+i,-\frac{1}{2}-i$
\end{enumerate}
\end{sol}
\end{ex}

\begin{ex} Give the solutions to the following quadratic equations having complex
coefficients.

\begin{enumerate}
\item $x^{2}+2x+1+i=0$

\item $4x^{2}+4ix-5=0$

\item $4x^{2}+(4+4i) x+1+2i=0$

\item $x^{2}-4ix-5=0$

\item $3x^{2}+(1-i) x+3i=0$
\end{enumerate}
\begin{sol}
\begin{enumerate}
\item Solution is : $x=-1+\frac{1}{2}\sqrt{2}-\frac{1}{2}i
\sqrt{2},\;\;x=-1-\frac{1}{2}\sqrt{2}+\frac{1}{2}i\sqrt{2}$
\item Solution is : $x=1-\frac{1}{2}i,\;x=-1-\frac{1}{2}i$
\item Solution is : $x=-\frac{1}{2},\;x=-\frac{1}{2}-i$
\item Solution is : $x=-1+2i,\;\;x=1+2i$
\item Solution is : $x=-\frac{1}{6}+\frac{1}{6}\sqrt{19}+\paren{\frac{1}{6}-\frac{1}{6}\sqrt{19}} i,\;\;x=-\frac{1}{6}-\frac{1}{6}\sqrt{19}+\paren{\frac{1}{6}+\frac{1}{6}\sqrt{19}}i$
\end{enumerate}
\end{sol}
\end{ex}

\begin{ex} \label{exer-complex3}Prove the fundamental theorem of algebra for
quadratic polynomials having coefficients in $\C$. That is, show
that an equation of the form \\ $ax^{2}+bx+c=0$ where $a,b,c$ are complex
numbers, $a\neq 0$ has a complex solution. \textbf{Hint: }Consider the fact,
noted earlier that the expressions given from the quadratic formula do in
fact serve as solutions.
%\begin{sol}
%\end{sol}
\end{ex}


%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
% CS624: Analysis of Algorithms
% Copyright 2015 Pejman Ghorbanzade <pejman@ghorbanzade.com>
% Creative Commons Attribution-ShareAlike 4.0 International License
% More info: https://github.com/ghorbanzade/beacon
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%

\section*{Question 2}

Show that the second smallest of $n$ elements can be found with $n + \lceil \log n \rceil - 2$ comparisons in the worst case.

\subsection*{Solution}

We begin by comparing elements of the array two by two, each time putting the minimum number in a new array of size $\lceil \frac{n}{2} \rceil$.
We will repeat the procedure for the new array until our new array will have only one element which is the element with minimum value.
To calculate the number of comparisons it takes to find the element with minimum value, we use the analogy of a tree with $n$ nodes at height $H$, in which case the number of comparisons would be equal to number of nodes at levels greater than $0$ which is $n - 1$.

The second smallest element is the smallest of all other nodes except the root.
Therefore, there has been a comparison in which this element has lost to the element with minimum value.
As the latter is compared $\lceil \log n \rceil$ times, we can compare all its losers to eachother to obtain the second smallest element, a procedure which takes $\lceil \log n \rceil - 1$ comparisons.

Therefore the total number of comparisons would be $n + \lceil \log n - 2$.


\section*{Ex.2.6}
\subsection*{Multiply-mod-prime hash function where $a=0$ is possible}

\emph{Show it is not universal.}

To be a universal hash function it must hold that for all distinct keys $P(h(x)=h(y))\leq 1/m$.

We will give a concrete example where the above probability is not fulfilled:
\\
Set $p=3$, $m=2$, $x=0$, $y=1$. Then the possible values for $a,b\in\{0,1,2\}$. The table below shows the hash value for $x$ and $y$ with the value set above.

\begin{center}
\begin{tabular}{l|cc|r}
              & $h(x)$ & $h(y)$ & collision\\
\hline
 $(a,b)=(0,0)$& 0 & 0 & true\\
 $(a,b)=(0,1)$& 1 & 1 & true\\
 $(a,b)=(0,2)$& 0 & 0 & true\\
 $(a,b)=(1,0)$& 0 & 1 & false\\
 $(a,b)=(1,1)$& 1 & 0 & false\\
 $(a,b)=(1,2)$& 0 & 0 & true\\
 $(a,b)=(2,0)$& 0 & 0 & true\\
 $(a,b)=(2,1)$& 1 & 0 & false\\
 $(a,b)=(2,2)$& 0 & 1 & false
\end{tabular}
\end{center}

It can be seen that for 5 out of the 9 possibilities, there is a collision. Hence the probability for collision is
$$
P(h(0)=h(1)) = \frac{5}{9} > \frac{1}{2} = \frac{1}{m}
$$
So we have found an instance where this hash function is not universal.

\emph{Prove that it is always 2-universal}

First note that for $a=0$, then $P(h_{0,b}(x)=h_{0,b}(y)) = 1$ for all values of $b, x, y$. The probability $P(a=0)=1/p$. To prove 2-universal we see that
\begin{align*}
P(h_{a,b}(x) = h_{a,b}(y)) &= P([(a=0)\cap(h_{a,b}(x) = h_{a,b}(y))]\cup [(a>0)\cap(h_{a,b}(x) = h_{a,b}(y))])\\
&= P((a=0)\cap(h_{0,b}(x) = h_{0,b}(y))) + P((a>0)\cap(h_{a,b}(x) = h_{a,b}(y)))\\
&= \frac{1}{p} + P((a>0)\cap(h_{a,b}(x) = h_{a,b}(y))) \\
&< \frac{1}{p} + \frac{1}{m} \\
&< \frac{1}{m} + \frac{1}{m} \\
&= \frac{2}{m}
\end{align*}
Hence we have shown that the hash function is 2-universal.


\subsection{Modelling non-linear functions as linear}

\subsubsection{Polynomials}

The function \(y=x^2\) is not linear, however we can model is as linear, by including \(x^2\) as a variable.

We can expand this, and using linear models to estimate parameters for functions such as:

\(y=ax^3+bx^2+xz\)

\subsubsection{Logarithms and exponentials}

We can also transform data using logarithms and exponents.

For example we can model

\(\ln y=\theta \ln x\)


\subsection{Empty set}

We can use this to define the empty set - the set with no members.

\(\varnothing =\{\}\)

Using the above definition this is the same as writing:

\(\forall x \neg (x\in \varnothing )\)


\section{Smoothing and Penalized Least Squares }
In Section 4.4.1 we saw that the smoothing spline solution to a
penalized least squares is a linear smoother.

Using the notation of Section 4.4.1, we can write the penalized
criterion as
\[
(\by - \bB\bg{\theta})'(\by - \bB\bg{\theta}) +
\lambda\bg{\theta}'\bg{\Omega}\bg{\theta}
\]

Setting derivatives with respect to $\bg{\theta}$ equal to 0 gives
the estimating equation:
\[
(\bB'\bB + \lambda\bg{\Omega})\bg{\theta} = \bB'\by
\]
the $\hat{\bg{\theta}}$ that solves this equation will give us the
estimate $\hat{\g} = \bB \hat{\bg{\theta}}$.

Write:
\[
\hat{\g} = \bB \bg{\theta} = \bB(\bB'\bB + \lambda \bg{\Omega})^{-1}
\bB'\by =  ({\mathbf I} + \lambda {\mathbf K})^{-1}\by
\]
where ${\mathbf K} = \bB'^{-} \bg{\Omega} \bB^{-}$.

Notice we can
write the penalized criterion as
\[
(\by - \g)'(\by - \g) + \lambda \g' {\mathbf K} \g
\]

If we plot the rows of this linear smoother we will see that it is
like a kernel smoother.

\begin{figure}[htb]
\caption{Kernels of a smoothing spline.}
\begin{center}
\epsfig{figure=Plots/plot-06-03.ps,angle=270,width=\textwidth}
\end{center}
\end{figure}

Notice that for any linear smoother  with a symmetric and nonnegative
definite $\bS$, i.e. there $\bS^{-}$ exists, then
we can argue in reverse: $\hat{\f}=\bS\by$ is the value that minimizes the
penalized least squares criteria of the form
\[
(\by - \f)'(\by - \f) + \f'(\bS^{-} - I)\f.
\]

Some of the smoothers presented in this class are not symmetrical but
are close. In fact for many of them one can show that asymptotically
they are symmetric.


\section{Non-Parametric Estimation of the CDF and Statistical Functionals}

\subsection{Empirical Distribution Function}
It may be necessary to perform non-parametric estimation of the CDF $F$ of a set of random variables $X_1, X_2, ... , X_n \sim F$.

The empirical distribution function $\hat{F}_n$ is the CDF that puts mass $1/n$ at each point $X_i$.

\begin{equation}
\hat{F}_n(x) = \frac{\sum_{i=1}^n I(X_i \leq x)}{n}
\end{equation}

Where $I(X_i \leq x) = \left\{\begin{array}{c} 1\ \mathrm{if\ } X_i \leq x\\ 0\ \mathrm{if\ } X_i > x \end{array} \right.$.

The empirical CDF is discrete, even when the random variable it is based on may be continuous.

At a given point $x$, $\hat{F}_n(x)$ is an unbiased estimator of $F(x)$.

\begin{itemize}
\item $\mathbb{E}\hat{F}_n(x) = F(x)$
\item $\mathbb{V}\hat{F}_n(x) = 0+\mathrm{MSE} = \frac{F(x)(1-F(x))}{n}$
\item $\hat{F}_n(x) \xrightarrow{P} F(x)$
\end{itemize}

The Glivenko-Cantelli Theorem guarantees that, if $X_1,X_2,...,X_n \sim F$, then:

\begin{equation}
\sup_x |\hat{F}_n(x) - F(x)|\xrightarrow{P} 0
\end{equation}

\subsection{Confidence Measures for the Empirical CDF}
A confidence interval for the empirical CDF is given through the Dvoretzky-Kiefer-Wolfowitz (DKW) Inequality:

\begin{equation}
\mathbb{P}\left(\sup_x|F(x) - \hat{F}_n(x)|>\epsilon \right)\leq 2 e^{-2n\epsilon^2}
\end{equation}

A nonparametric $1-\alpha$ confidence band is then:

\begin{equation}
\begin{array}{c}
L(x) = \max\{\hat{F}_n -\epsilon_n, 0 \}\\
\\
U(x) = \min\{\hat{F}_n + \epsilon_n, 1\}
\end{array}
\end{equation}

with $\epsilon_n = \sqrt{\frac{1}{2n}\log\left(\frac{2}{\alpha}\right)}$.

\begin{equation}
\mathbb{P}\left(L(x) \leq F(x) \leq U(x) \right) \geq 1-\alpha
\end{equation}

\subsection{Statistical Functionals}
A functional is, roughly speaking, a function of a function. The fourier transform of a function is a functional. A \textit{statistical functional} is any function of the CDF, $F$. Examples are the mean, the variance, or the median.

\subsection{Plug-in Estimator}
The plug-in estimator of $\theta = T(F)$ is given by $\hat{\theta} = T(\hat{F}_n)$. In other words, the estimate of the CDF is used instead of the true $F$, resulting in an estimator.

\subsection{Linear Functionals}
Functionals of the form $T(F) = \int r(x) dF(x)$ are linear functionals.

\subsection{Plug-in Estimator for Linear Functionals}
\begin{equation}
T(\hat{F}_n) = \int r(x) d\hat{F}_n(x) = \frac{1}{n}\sum^{n}_{i=1}r(X_i)
\end{equation}

\subsection{Examples: Mean, Variance, Sample Variance, Sample Correlation}
\citeasnoun{wasserman2013all} pp. 100


\section{Limit Definition}
\begin{definition}
	Let $f : D \subseteq \R \to \R$.
	Let $c \in R$ be a limit point (ie $c \in D$ or $c$ is on the boundary of $D$).
	$f$ has a limit $L$ as $x$ approaches $c$ if for any given positive real number $\epsilon$, there is a positive real number $\delta$ such that for all $x \in D$,
	\begin{equation}
		0 < \abs{x-c} < \delta \implies \abs{f(x) - L} < \epsilon.
	\end{equation}
	We write this as
	\begin{equation*}
		\lim_{x \to c}{f(x)} = L.
	\end{equation*}
\end{definition}

\begin{figure}[H]
	\label{epsilon_delta}
	\centering
	\includegraphics[width = 0.5\textwidth]{./limits_continuity/limit_epsilon_delta.png}
	\caption{\hyperref{https://en.wikipedia.org/wiki/(\%CE\%B5,\_\%CE\%B4)-definition\_of\_limit}{}{}{Wikipedia - $(\epsilon, \delta)\text{-definition of limit}$}}
\end{figure}

Visually, what this means is that for any ``error bound'' of $y$ values $\epsilon$, I can give you a corresponding error bound of $x$ values $\delta$ such that all values of $f(z)$ for $z \in (c -\delta, c+ \delta)$ bound are between $L - \epsilon$ and $L + \epsilon$.

We don't use this definition of the limit very often because it's a bit cumbersome.
However, it's important to know that when we use the limit, this is the formal definition making things work.

\begin{example}
	Use the $(\epsilon, \delta)$ definition of the limit to show that
	\begin{equation*}
		\lim_{x\to 0}{x\sin{\frac{1}{x}}} = 0.
	\end{equation*}
\end{example}
\begin{answer}
	Letting $\epsilon > 0$, we need to find corresponding $\delta > 0$ that satisfies the definition for $L = 0$.
	Knowing that $\sin$ is bounded between -1 and 1,
	\begin{equation*}
		\abs{x\sin{\frac{1}{x}} - 0} = \abs{x\sin{\frac{1}{x}}} = \abs{x}\abs{\sin{\frac{1}{x}}} \leq \abs{x}.
	\end{equation*}

	Letting $\delta = \epsilon$, if $0 < \abs{x - 0} < \delta$, then $\abs{x\sin{\frac{1}{x}} - 0} \leq \abs{x} < \epsilon$, as required by the definition.
\end{answer}


\subsection{Limit points and closure}

\subsubsection{Limit points}

A point \(x\) in the topological set \(X\) is a limit point for \(S\subset X\) if every neighbourhood of \(x\) contains another point in \(S\).

For example \(-1\) is a limit point for the real numbers where \(S\) is \([0,1]\) (or \((0,1)\).

\subsubsection{Closure}

The closure of a subset of a topological space is the subset itself along with all limit points.

So the closure of \(|x|<1\) includes \(-1\) and \(1\).


% !TeX root = ../main.tex
% Add the above to each chapter to make compiling the PDF easier in some editors.

\chapter{Groups and Homomorphisms}
\section{Groups}
\begin{defn}[Semigroup, Monoid, and Group]\leavevmode
\begin{defnlist}
    \item A set $G$ with a mapping $\cdot$ on $G$ (that is, $\cdot: G \times G \to G$) is named \begin{itemize}
        \item \emph{semigroup}\index{semigroup} if $\forall a, b, c \in G.\ (a \cdot b) \cdot c = a \cdot (b \cdot c)$; \margintag{\emph{associativity}\index{associativity}}
        \item \mbox{\emph{monoid}\index{monoid} if it is a semigroup and $\exists e \in G.\ \forall a \in G.\ e \cdot a = a \cdot e = a$; \margintag{$e$ is called \emph{neutral element}\index{neutral element}}}
        \item \emph{group}\index{group} if it is a monoid and $\forall a \in G.\ \exists a' \in G.\ a \cdot a' = a' \cdot a = e$. \margintag{$a'$ is the \emph{inverse element}\index{inverse element} of $a$}
    \end{itemize}

    \item A group $(G,\cdot)$ is named \emph{abelian}\index{abelian group} or \emph{commutative}\index{commutativity} if the group operation $\cdot$ is commutative under elements of $G$, \begin{align}
        \forall a, b \in G.\quad a \cdot b = b \cdot a.
    \end{align}

    \begin{marginfigure}
        We denote groups by $(G,\cdot)$. If the group operation $\cdot$ is clear from context, we refer to the group simply as $G$. Subsequently, we also write $a b$ instead of $a \cdot b$.
    \end{marginfigure}
\end{defnlist}
\end{defn}

\begin{rmk}
Let $G$ be a group. Then, \begin{rmklist}
    \item there is exactly one neutral element $e \in G$ and for every $a \in G$ there is exactly one inverse element $a' \in G$, which we call $\inv{a}$;
    \item the mapping on $G$, which we refer to by $\cdot$, may be resembled by any symbol;
    \item if $G$ is abelian, often \begin{itemize}
        \item $+$ is used instead of $\cdot$,
        \item $0$ is used instead of $e$, and
        \item $-a$ is used instead of $\inv{a}$.
    \end{itemize}
\end{rmklist}
\end{rmk}

\begin{ex}{Semigroups, monoids, and groups}{}
\begin{center}
\setlength\tabcolsep{5pt}
\begin{tabular}{lrrrrr}
\toprule
 & $(\NO,+)$ & $(\NZ,+)$ & $(\Z,+)$ & $(\Z,\cdot)$ & $(\woZ{\Q},\cdot)$ \\
 \midrule
 semigroup & yes & yes & yes & yes & yes \\
 \addlinespace
 monoid & no & yes & yes & yes & yes \\
 \addlinespace
 group & yes & no & yes & no & yes \\
 \bottomrule
\end{tabular}
\end{center}
\end{ex}

\begin{ex}{General linear and special linear group}{}
The \emph{general linear group}\index{general linear group} $\GL{n}{K}$ is the group of invertible\marginfootnote{Recall from linear algebra that a matrix $\mA$ is invertible iff $\det{\mA} \neq 0$.} ${n \times n}$ linear maps over a field\marginfootnote[1\baselineskip]{A \emph{field}\index{field} is a set of elements with well-defined operations for addition, subtraction, multiplication, and division. We give a formal definition in \cref{defn:field}. Examples of fields are the rational numbers $\Q$, the real numbers $\R$, and the complex numbers $\C$.} $K$, \begin{align}
    \GL{n}{K} \defeq \{\mA \in K^{n \times n} \mid \det{\mA} \neq 0\}.
\end{align}

The \emph{special linear group}\index{special linear group} $\SL{n}{K}$ is the group of normed linear maps over the field $K$, \begin{align}
    \SL{n}{K} \defeq \{\mA \in K^{n \times n} \mid \det{\mA} = 1\}.
\end{align}

The group operation of $\GL{n}{K}$ and $\SL{n}{K}$ is matrix multiplication. Their neutral element is the identity matrix $\mI$, and the inverse elements are the matrix inverses $\inv{\mA}$.
\end{ex}

\begin{ex}{Abelian groups}{}
\begin{itemize}
    \item $(\Z,+)$
    \item $(\Q,+)$
    \item $(\R,+)$
    \item $(\woZ{\R},\cdot)$
\end{itemize}
\end{ex}

\begin{ex}{Symmetric group}{}
The \emph{symmetric group}\index{symmetric group} $S_n$ is the group of bijections on the set $[n]$, \begin{align}
    S_n \defeq \{\sigma : [n] \to [n] \mid \text{$\sigma$ is bijective}\},
\end{align} with the function composition ``$\circ$'' as mapping.

Elements of the symmetric group ${\sigma \in S_n}$ are called \emph{permutations}\index{permutation}. The neutral element of the symmetric group is the identity $\id$, which maps each input to itself.

There are multiple ways of representing permutations. Perhaps the most natural representation of ${\sigma \in S_n}$ is a mapping in \emph{two-line notation}\index{two-line notation}, \begin{align}
    \sigma = \begin{pmatrix}
        1 & 2 & \cdots & n \\
        \sigma(1) & \sigma(2) & \cdots & \sigma(n) \\
    \end{pmatrix},
\end{align} or in \emph{one-line notation}\index{one-line notation} by simply omitting the first line, \begin{align}
    \sigma = (\sigma(1)\ \sigma(2)\ \cdots\ \sigma(n)).
\end{align}

An alternative characterization of a permutation is as a product of (disjoint) cycles. A \emph{cycle}\index{cycle} ${\rho \in S_n}$ is a permutation that maps a subset of numbers ${\{i_1, i_2, \dots, i_r\} \subseteq [n]}$ in a cyclic fashion. That is, \begin{align}
    \rho(i_1) = i_2,\quad \rho(i_2) = i_3,\quad \cdots\quad \rho(i_{r-1}) = i_r,\quad \rho(i_r) = i_1,
\end{align} leaving all other ${j \in [n]}$ fixed. We denote such a cycle by \begin{align}
    \rho = (i_1\ i_2\ \cdots\ i_r).
\end{align} A cycle of length $r$, is also called \emph{$r$-cycle}. 2-cycles are called \emph{transpositions}\index{transposition}.

Every permutation ${\sigma \in S_n}$ can be written as a ``product'' (i.e., composition), \begin{align}
    \sigma = \rho_1 \cdots \rho_s,
\end{align} where $\rho_i$ are cycles with pairwise disjunct elements. This is also known as the \emph{cycle notation}\index{cycle notation} of $\sigma$. Note that the ordering of $\rho_1, \dots \rho_s$ does not matter, as their elements are disjoint.

The cycle lengths $r_1, \dots, r_s$ (in descending order) of $\rho_1, \dots, \rho_s$ are the \emph{cycle type}\index{cycle type} of $\sigma$.

\begin{rmk}
The symmetric group is not abelian.
\end{rmk}\vspace{-20pt}\begin{proof}
We have $(1\ 2)(2\ 3) = (2\ 3\ 1)$ and $(2\ 3)(1\ 2) = (1\ 3\ 2)$.
\end{proof}
\end{ex}

\begin{lem}[Notation and Rules]
Let $(G,\cdot)$ be a group. \begin{lemlist}
    \item For $a \in G, n \in \NZ$, we write \begin{itemize}
        \item $a^n \defeq \underbrace{a \cdot a \cdots a}_{\text{$n$ many}}$,
        \item $a^0 \defeq e$, and
        \item $a^{-n} \defeq \underbrace{\inv{a} \cdot \inv{a} \cdots \inv{a}}_{\text{$n$ many}}$.
    \end{itemize}

    \item $\forall a, b \in G.\ \forall m, n \in \Z.$ \begin{nestedlemlist}
        \item $\inv{(\inv{a})} = a$
        \item $a^m \cdot a^n = a^{m+n}$, $(a^m)^n = a^{m \cdot n}$
        \item $\inv{(a \cdot b)} = \inv{b} \cdot \inv{a}$
    \end{nestedlemlist}
\end{lemlist}
\end{lem}
\begin{proof}[Proof of (b)(iii)] $(a \cdot b) \cdot \inv{b} \cdot \inv{a} = a \cdot \underbrace{(b \cdot \inv{b})}_{=e} \cdot \inv{a} = a \cdot \inv{a} = e$.
\end{proof}

\begin{defn}[Subgroup]\label{defn:subgroup}
A subset $U \subseteq G$ is called a \emph{subgroup}\index{subgroup} of a group $G$ (denoted $U \subgroup G$) if $U$ itself is a group with mapping $\cdot$. That is, $U$ is a subgroup iff \begin{defnlist}
    \item $e \in U$; \margintag{$U$ contains the neutral element}
    \item $\forall a, b \in U.\quad a \cdot b \in U$; and \margintag{the mapping $\cdot$ is closed wrt. $U$}
    \item $\forall a \in U.\quad \inv{a} \in U.$ \margintag{the mapping $\inv{(\dots)}$ is closed wrt. $U$}
\end{defnlist} Associativity follows from $G$ being a group.
\end{defn}

\begin{ex}{Subgroups}{}
\begin{itemize}
    \item $\{e\}, G \subgroup G$ (\emph{trivial subgroups}\index{trivial subgroups})
    \item $\SL{n}{K} \subgroup \GL{n}{K}$
    \item Let $2\Z \defeq \{2m \mid m \in \Z\}$, then $(2\Z,+) \subgroup (\Z,+)$
\end{itemize}
\end{ex}

\begin{rmk}\label{rmk:cap_subgroup}
If $\{U_i\}_{i \in I}$ are subgroups of $G$, then $\bigcap_{i \in I} U_i \subgroup G$.
\end{rmk}

\begin{defn}[Generated Subgroup]
For any $M \subseteq G$, \begin{align}
    \gen{M} \defeq \bigcap_{\substack{U \subgroup G \\ M \subseteq U}} U \subgroup G, \margintag{by \cref{rmk:cap_subgroup}}
\end{align} is the \emph{subgroup generated by $M$}\index{generated subgroup}. In particular, $\gen{M}$ is the ``smallest'' subgroup that includes $M$.
\end{defn}

\begin{lem}
For $M \neq \emptyset$, we have \begin{align}
    \gen{M} = \{a_1 \cdot a_2 \cdots a_n \mid n \in \N, \text{$a_i \in M$ or $\inv{a_i} \in M$}\}.
\end{align}
\end{lem} \begin{proof}[Proof (sketch)]\leavevmode
Let $N \defeq \{a_1 \cdot a_2 \cdots a_n \mid n \in \N, \text{$a_i \in M$ or $\inv{a_i} \in M$}\}$.
\begin{itemize}
    \item \underline{$\gen{M} \subseteq N$}: $N \subgroup G$ and $M \subseteq N \implies \gen{M} \subseteq N$ \margintag{using that $\gen{M}$ is the smallest subgroup including $M$}
    \item \underline{$N \subseteq \gen{M}$}: if $U \subgroup G$ with $M \subseteq U$, then $U$ includes all of these products $\implies N \subseteq U \implies N \subseteq \gen{M}$ \qedhere
\end{itemize}
\end{proof}

\begin{ex}{Generated subgroup}{}
$S_3 = \{\id, \underbrace{(1\ 2)}_{\tau_1}, \underbrace{(1\ 3)}_{\tau_2}, \underbrace{(2\ 3)}_{\tau_3}, \underbrace{(1\ 2\ 3)}_{\sigma_1}, \underbrace{(1\ 3\ 2)}_{\sigma_2}\} = \gen{\{\underbrace{(1\ 2)}_{\tau_1}, \underbrace{(1\ 2\ 3)}_{\sigma_1}\}}$.
\end{ex}

\begin{defn}[Cyclic Group]
A group $G$ is \emph{cyclic}\index{cyclic group} if $\exists a \in G$ with \begin{align}
    G = \gen{a} \defeq \gen{\{a\}} = \{a^m \mid m \in \Z\}.
\end{align} Such an $a \in G$, is called a \emph{generator}\index{generator} of $G$.
\end{defn}

\begin{marginfigure}
    \includegraphics[width=\textwidth]{c_cyclic_subgroup.png}
    \caption{An illustration of the cyclic subgroup $\gen{i}$ in the complex plane.}
\end{marginfigure}

\begin{ex}{Cyclic groups}{}
\begin{itemize}
    \item $\gen{i} = \{i^m \mid m \in \Z\}$ is a cyclic subgroup of $(\woZ{\C},\cdot)$, \begin{align*}
        \gen{i} &= \{\dots, \underbrace{i^{-2}}_{=-1}, \underbrace{i^{-1}}_{=-i}, 1, i, \underbrace{i^2}_{=-1}, \underbrace{i^3}_{=-i}, \underbrace{i^4}_{=1}, \underbrace{i^5}_{=i}, \dots\} \\
                &= \{1, i, -1, -i\}.
    \end{align*}
    \item $(\Z,+) = \gen{1} = \{\dots, -2, -1, 0, 1, 2, \dots\}$ is cyclic
\end{itemize}
\end{ex}

\begin{defn}[Order]\index{order}
Let $G$ be a group.
\begin{defnlist}
    \item The cardinality $|G| \in \wInfty{\N}$ is called \emph{order} of the group $G$.
    \item For any $a \in G$, $o(a) \defeq |\gen{a}|$ is the \emph{order} of the element $a$.
\end{defnlist} If $|G| < \infty$, $G$ is called \emph{finite}\index{finite group}.
\end{defn}

\begin{lem}
If $k \defeq o(a) < \infty$, we have \begin{lemlist}
    \item $o(a) = \min\{j \in \N \mid a^j = e\}$;
    \item $\gen{a} = \{e, a, a^2, \dots, a^{k-1}\}$; and
    \item\label{lem:order_identity} For any $j \in \Z$, $a^j = e \iff \divides{o(a)}{j}$.\footnote{We use $\divides{a}{b}$ to denote that $a$ divides $b$.}
\end{lemlist}
\end{lem} \begin{proof}
We write $m \defeq \min\{j \in \N \mid a^j = e\}$. \begin{itemize}
    \item \underline{$\{e, a, a^2, \dots, a^{m-1}\} \subseteq \gen{a}$}: Let us fix an arbitrary $j \in \N$. We have, \begin{align*}
        o(a) = |\gen{a}| < \infty &\implies \exists j' > j.\ a^j = a^{j'} \margintag{as the order of $a$ is finite} \\
                                  &\implies a^{j'-j} = e \margintag{by multiplying from the right with $a^{-j}$} \\
                                  &\implies \text{$m$ exists and $\{e, a, a^2, \dots, a^{m-1}\} \subseteq \gen{a}$}. \margintag{as $j'-j \in \N$ is again a natural number}
    \end{align*}
    \item \underline{$\gen{a} \subseteq \{e, a, a^2, \dots, a^{m-1}\}$}: We fix any $n \in \Z$. Then, by \emph{long division}\index{long division}, there exist $q, r \in \Z$ and $0 \leq r < m$ with $n = q \cdot m + r$. This yields, \begin{align*}
        a^n = a^{q \cdot m + r} = \underbrace{(a^m)^q}_{=e} \cdot a^r = a^r \in \{e, a, a^2, \dots, a^{m-1}\}.
    \end{align*}
\end{itemize} This proves that $m = o(a)$ and $\gen{a} = \{e, a, a^2, \dots, a^{m-1}\}$. It also follows that \begin{align*}
    a^n = e \iff r = 0 \iff \divides{m}{n}. &\qedhere
\end{align*}
\end{proof}

\begin{rmk}
For a finite cyclic group $\gen{a} = \{e, a, a^2, \dots, a^{k-1}\}$ of order $k$, we have $a^{k-j} = a^{-j}$.
\end{rmk}

\begin{ex}{Order}{}
\begin{itemize}
    \item Let us consider $\GL{2}{\R}$. Then, \begin{align*}
        \mA = \begin{bmatrix}
            2 & 0 \\
            0 & 2 \\
        \end{bmatrix} &\implies \forall n \in N.\ \mA^n = \begin{bmatrix}
            2^n & 0 \\
            0 & 2^n \\
        \end{bmatrix} \neq \mI \\ &\implies o(\mA) = \infty, \\
        \mB = \begin{bmatrix}
            0 & 1 \\
            -1 & 0 \\
        \end{bmatrix} &\implies \begin{multlined}[t]\mB^2 = \begin{bmatrix}
            -1 & 0 \\
            0 & -1 \\
        \end{bmatrix}, \\ \mB^3 = -\mB = \begin{bmatrix}
            0 & -1 \\
            1 & 0 \\
        \end{bmatrix}, \mB^4 = (\mB^2)^2 = \mI\end{multlined} \\ &\implies o(\mB) = 4.
    \end{align*}

    \item $|S_n| = |\{\sigma : [n] \to [n] \mid \text{$\sigma$ is bijective}\}| = n!$
\end{itemize}
\end{ex}

\begin{ex}{Subgroups of $S_3$}{s3_subgroups}
Let us find the subgroups of \begin{align*}
    S_3 = \{\id, (1\ 2), (1\ 3), (2\ 3), (1\ 2\ 3), (1\ 3\ 2)\}.
\end{align*} We immediately obtain the trivial subgroups $\{\id\}$ and $S_3$ or order $1$ and $6$, respectively. It is a simple exercise to confirm the following cyclic subgroups: \begin{itemize}
    \item $\gen{(1\ 2)} = \{\id, (1\ 2)\}$
    \item $\gen{(1\ 3)} = \{\id, (1\ 3)\}$
    \item $\gen{(2\ 3)} = \{\id, (2\ 3)\}$
    \item $\gen{(1\ 2\ 3)} = \gen{(1\ 3\ 2)} = \{\id, (1\ 2\ 3), (1\ 3\ 2)\}$
\end{itemize} The first three subgroups generated by 2-cycles are of order 2, the last subgroup generated by the 3-cycles is of order 3.

Observe that the subgroup orders are divisors of the group order. This is not coincidental, we will make this precise in the following. In doing so, we will also find that our list of subgroups of $S_3$ was indeed exhaustive.
\end{ex}

\begin{marginfigure}
    \includegraphics[width=\textwidth]{s3_subgroup_graph.png}
    \caption{Subgroup graph of the symmetric group $S_3$. The order of the subgroups is shown in orange.}\label{fig:s3_subgroup_graph}
\end{marginfigure}

The subgroup structure of a group $G$ can be graphically represented in a \emph{subgroup graph}\index{subgroup graph}. Subgroups of $G$ are represented as vertices. Groups $U$ and $V$ are connected if $U \subgroup V$ and there exists no subgroup ``between'' $U$ and $V$. An example is given in \cref{fig:s3_subgroup_graph}.

\begin{defn}[Cosets and Index]
Let $U \subgroup G$ be a subgroup.
\begin{defnlist}
    \item For $a \in G$, \begin{align}
        aU &\defeq \{a \cdot u \mid u \in U\}, \label{eq:left_coset} \\
        Ua &\defeq \{u \cdot a \mid u \in U\},
    \end{align} are the left\index{left coset} and right\index{right coset} \emph{coset}\index{coset} of $U$ in $G$, respectively.
    \item\label{defn:index} The \emph{index}\index{index} $[G : U] \defeq |\{aU \mid a \in G\}|$ of $U$ in $G$ is defined as the number of cosets of $U$ in $G$.\footnote{The number of left cosets is identical to the number of right cosets.}
\end{defnlist}
\end{defn}

\begin{lem}
For all $a, b \in G$, we have
\begin{lemlist}
    \item $aU = U \iff a \in U$
    \item\label{lem:cosets_eq} $aU = bU \iff \inv{a} b \in U$
    \item $aU \cap bU \neq \emptyset \iff aU = bU$
    \item $G = \bigcup_{a \in G} aU$
    \item\label{lem:cosets_size} $|aU| = |U|$
\end{lemlist}
\end{lem} \begin{proof}[Proof of (e)]
$U \to aU, u \mapsto a \cdot u$ is a bijective mapping. Hence, domain and codomain are of the same size.
\end{proof}

\begin{thm}[Lagrange's Theorem]\index{Lagrange's theorem}\label{thm:lagrange}
If $G$ is a finite group and $U \subgroup G$ is some subgroup, then \begin{align}
    |G| = |U| \cdot [G : U].
\end{align} In particular, $|U|$ and $[G : U]$ are divisors of $|G|$.
\end{thm} \begin{proof}
Let $r \defeq \Index{G}{U}$. Then we can write $G$ as a disjoint union of cosets, \begin{align*}
    G = a_1 U \cupdot a_2 U \cupdot \cdots \cupdot a_r U.
\end{align*} We have, \begin{align*}
    |G| = \sum_{i=1}^r |a_i U| = r \cdot |U| = \Index{G}{U} \cdot |U|. \margintag{using that $|a_i U| = |U|$ by \cref{lem:cosets_size}} &\qedhere
\end{align*}
\end{proof}

\begin{cor}
Let $G$ be a finite group. Then we have for any $a \in G$, \begin{corlist}
    \item $\divides{o(a)}{|G|}$
    \item $a^{|G|} = e$ \quad (\emph{Fermat's little theorem}\index{Fermat's little theorem})
\end{corlist}
\end{cor} \begin{proof}
\leavevmode\begin{corlist}
\item By \hyperref[thm:lagrange]{Lagrange's theorem}, $\divides{|\gen{a}|}{|G|}$.
\item By \cref{lem:order_identity}, $a^{|G|} = e \iff \divides{o(a)}{|G|}$. \qedhere
\end{corlist}
\end{proof}

\begin{cor}\label{cor:prime_group_is_cyclic}
Let $G$ be a group such that $|G| = p$ where $p$ is prime. Then, $G$ is cyclic.
\end{cor} \begin{proof}
\begin{align*}
    |G| > 1 &\implies \exists a \in G \setminus \{e\} \\
            &\implies \divides{1 \neq |\gen{a}|}{|G|} \margintag{using that $o(a) \geq 2$ if $a \neq e$ and \hyperref[thm:lagrange]{Lagrange's theorem}} \\[5pt]
            &\implies |\gen{a}| = p = |G|. \margintag{using that $|G|$ only has divisors $1$ and $p$}
\end{align*} Therefore, $\gen{a} = G$.
\end{proof}

\begin{ex}{Subgroups of $S_3$ (continued)}{}
We will now see that $S_3$ has exactly four non-trivial subgroups, proving that we have found all subgroups of $S_3$ in \cref{ex:s3_subgroups}.

Let ${U \subgroup S_3}$ be a non-trivial subgroup. By \hyperref[thm:lagrange]{Lagrange's theorem}, we have ${\divides{|U|}{|S_3| = 3! = 6}}$. As we have excluded the trivial subgroups $\{\id\}$ and $S_3$, we know that ${|U| \neq 1, 6}$. This leaves us with ${|U| \in \{2,3\}}$.

Observe that $2$ and $3$ are prime, hence, by \cref{cor:prime_group_is_cyclic} $U$ must be cyclic. Recall that we have already enumerated all (four) cyclic subgroups of $S_3$ in \cref{ex:s3_subgroups}.
\end{ex}

\section{Homomorphisms}
We will now consider two groups $(G,\cdot)$ and $(H,\cdot)$. To understand the relationship between $G$ and $H$, it is useful to look at mappings between the two groups. A special mapping that (as we will see) preserves the structure of a group, is the group homomorphism.

\begin{defn}[(Group) Homomorphism]
\leavevmode\begin{defnlist}
    \item The mapping $\varphi : G \to H$ is called a \emph{(group) homomorphism}\index{homomorphism}\index{group homomorphism} if \begin{align}
        \forall a, b \in G.\quad \varphi(a \cdot b) = \varphi(a) \cdot \varphi(b). \label{eq:homomorphism}
    \end{align} The homomorphism $\psi : G \to G$ is called \emph{endomorphism}\index{endomorphism} of $G$.
    \item The set of elements that are mapped to the neutral element $e_H$, \begin{align}
        \ker{\varphi} \defeq \{a \in G \mid \varphi(a) = e_H\} \subseteq G, \label{eq:kernel}
    \end{align} is called the \emph{kernel}\index{kernel} of $\varphi$.
    \item The set of elements in the codomain $H$ that $\varphi$ maps to, \begin{align}
        \im{\varphi} \defeq \{\varphi(a) \mid a \in G\} \subseteq H,
    \end{align} is called the \emph{image}\index{image} of $\varphi$.
\end{defnlist}
\end{defn}

\begin{ex}{Homomorphisms}{homomorphisms}
\begin{itemize}
    \item ${\varphi : G \to H, a \mapsto e_H}$ is the \emph{trivial homomorphism}\index{trivial homomorphism}

    \item For any field $K$, ${\det : \GL{n}{K} \to \woZ{K}, \mA \mapsto \det{\mA}}$ is a homomorphism due to the multiplicativity of the determinant.\marginfootnote{Recall that $\det{(\mA \cdot \mB)} = \det{\mA} \cdot \det{\mB}$.} We have for its kernel, \begin{align}
        \ker{\det} = \{\mA \in \GL{n}{K} \mid \det{\mA} = 1\} = \SL{n}{K}. \label{eq:det_kernel}
    \end{align}

    \item Let us consider the \emph{sign}\index{sign} of a permutation, \begin{align}
        \sgn : S_n \to \{-1, 1\}, \sigma \mapsto (-1)^{N(\sigma)},
    \end{align} where $N(\sigma)$ is the number of inversions in $\sigma$. An \emph{inversion}\index{inversion} in $\sigma$ is a pair of elements that is out of order. More formally, \begin{align}
        N(\sigma) = |\{(i,j) \mid \text{$i < j$ and $\sigma(i) > \sigma(j)$}\}|.
    \end{align} For an $r$-cycle the sign reduces to, \begin{align}
        \sgn{\underbrace{(i_1\ \cdots\ i_r)}_{\text{$r$-cycle}}} = (-1)^{r-1}.
    \end{align}

    It can be shown that $\sgn$ is a homomorphism, that is, \begin{align}
        \forall \sigma, \tau \in S_n.\quad \sgn{(\sigma \circ \tau)} = \sgn{\sigma} \cdot \sgn{\tau}.
    \end{align} The kernel of $\sgn$ is the set of permutations with positive sign, \begin{align}
        \ker{\sgn} = \{\sigma \in S_n \mid \sgn{\sigma} = 1\} \eqdef A_n. \label{eq:alternating_group}
    \end{align} This set forms again a group, which is known as the \emph{alternating group}\index{alternating group} $A_n$.

    \item Within the group $(\Z,+)$, $\varphi : \Z \to \Z, m \mapsto 2 m$ is an endomorphism.
\end{itemize}
\end{ex}

\begin{lem}[Properties of Homomorphisms]
Let $\varphi : G \to H$ be a homomorphism. Then,
\begin{lemlist}
    \item $\varphi(e_G) = e_H$
    \item $\forall g \in G.\ \varphi(\inv{g}) = \inv{\varphi(g)}$
    \item\label{lem:homomorphism_kernel_subgroup} $\ker{\varphi} \subgroup G$ and $\im{\varphi} \subgroup H$
    \item $\text{$\varphi$ injective} \iff \ker{\varphi} = \{e_G\}$
    \item if $\psi : H \to K$ is a homomorphism, then $\psi \circ \varphi : G \to K$ is a homomorphism
\end{lemlist}
\end{lem} \begin{marginfigure}
    \includegraphics[width=\textwidth]{injective_surjective.png}
    \caption{An illustration of \emph{injectivity}\index{injective} and \emph{surjectivity}\index{surjective}. When a function $\varphi$ is injective, $\varphi(a) = \varphi(b)$ implies $a = b$. We call a function \emph{bijective}\index{bijective} is it is both injective and surjective, i.e., a one-to-one mapping.}
\end{marginfigure} \begin{proof}
\leavevmode\begin{lemlist}
    \item We have $\varphi(e_G) = \varphi(e_G \cdot e_G) = \varphi(e_G) \cdot \varphi(e_G)$, using the compatibility of a homomorphism with the group structure \eqref{eq:homomorphism}. By multiplying with $\inv{\varphi(e_G)}$ from one side, we obtain that this statement is true if and only if $\varphi(e_G) = e_H$.

    \item Again, using the homomorphism property, we have, \begin{align*}
        e_H = \varphi(e_G) = \varphi(g \cdot \inv{g}) = \varphi(g) \cdot \varphi(\inv{g}).
    \end{align*} By multiplying from the left with $\inv{\varphi(g)}$, we obtain that this statement is true if and only if $\inv{\varphi(g)} = \varphi(\inv{g})$.

    \item Let us confirm the properties of subgroups for $\ker{\varphi} \subgroup G$. The proof is analogous for $\im{\varphi} \subgroup H$. By \cref{defn:subgroup}, we need to show, \begin{nestedlemlist}
        \item $e_G \in \ker{\varphi}$ follows immediately from (a)
        \item $\forall a, b \in \ker{\varphi}.\ \varphi(a \cdot b) = \varphi(a) \cdot \varphi(b) = e_H \cdot e_H = e_H$. Therefore, $\ker{\varphi}$ is closed under the group operation, $a \cdot b \in \ker{\varphi}$.
        \item $\forall a \in \ker{\varphi}.\ \varphi(\inv{a}) = \inv{\varphi(a)} = \inv{e_H} = e_H$. Therefore, $\ker{\varphi}$ is closed under inversion, $\inv{a} \in \ker{\varphi}$.
    \end{nestedlemlist}
    $\implies \ker{\varphi} \subgroup G$.

    \item \begin{itemize}
        \item \underline{``$\Rightarrow$''}: Let $\varphi$ be injective. We want to show that $\ker{\varphi} = \{e_G\}$. Note that (a) already implies $\{e_G\} \subseteq \ker{\varphi}$. To show $\ker{\varphi} \subseteq \{e_G\}$, let $a \in \ker{\varphi}$. Then, \begin{align*}
            \varphi(a) = e_H \overset{(a)}{=} \varphi(e_H).
        \end{align*} As $\varphi$ is injective, it follows that $a = e_H$.
        \item \underline{``$\Leftarrow$''}: Let $\ker{\varphi} = \{e_G\}$. We want to show that $\varphi$ is injective. Let $a, b \in G$ with $\varphi(a) = \varphi(b)$. By multiplying from the right with $\inv{\varphi(b)}$, we obtain, \begin{align*}
            e_H = \varphi(a) \cdot \inv{\varphi(b)} = \varphi(a) \cdot \varphi(\inv{b}) = \varphi(a \cdot \inv{b}).
        \end{align*} As the kernel of $\varphi$ only contains $e_G$, we follow, \begin{align*}
            e_G = a \cdot \inv{b} \overset{\cdot b}{\implies} a = b \implies \text{$\varphi$ injective}.
        \end{align*}
    \end{itemize}

    \item Let $a, b \in G$. Then, \begin{align*}
        (\psi \circ \varphi)(a \cdot b) &= \psi(\varphi(a \cdot b)) = \psi(\varphi(a) \cdot \varphi(b)) \margintag{using that $\varphi$ is a homomorphism} \\
        &= \psi(\varphi(a)) \cdot \psi(\varphi(b)) = (\psi \circ \varphi)(a) \cdot (\psi \circ \varphi)(b). \margintag{using that $\psi$ is a homomorphism} \qedhere
    \end{align*}
\end{lemlist}
\end{proof}

\begin{defn}[Isomorphism]
\leavevmode\begin{defnlist}
    \item The mapping $\varphi : G \to H$ is called an \emph{isomorphism}\index{isomorphism} if $\varphi$ is a homomorphism and bijective. The isomorphism $\psi : G \to G$ is called an \emph{automorphism}\index{automorphism} of $G$.
    \item $G$ and $H$ are called \emph{isomorphic} (denoted $G \isom H$) if there exists an isomorphism $\varphi : G \to H$.
    \item $\Aut{G} \defeq \{\psi : G \to G \mid \text{$\psi$ automorphism}\}$ forms a group under function composition ``$\circ$''. This group is called the \emph{automorphic group}\index{automorphic group} of $G$.
\end{defnlist}
\end{defn}

\begin{rmk}
If $\varphi : G \to H$ is an isomorphism, then $\inv{\varphi} : H \to G$ is an isomorphism.
\end{rmk}

\begin{ex}{Isomorphisms}{}
\begin{itemize}
    \item Given a group $G$ and an arbitrary element $g \in G$, \begin{align}
        i_g : G \to G, x \mapsto g \cdot x \cdot \inv{g},
    \end{align} is the \emph{inner automorphism}\index{inner automorphism} of the so-called \emph{conjugating element}\index{conjugating element} $g$. This isomorphism corresponds to the conjugation group action (also called \emph{(left) conjugation}\index{conjugation} by $g$), which we will encounter again in \cref{sec:groups:actions}.

    \item $\exp : (\R,+) \to (\RgZ,\cdot), x \mapsto e^x$ is an isomorphism.\marginfootnote{$\exp$ is a homomorphism due to $e^{x+y} = e^x e^y$. As $\exp$ is strictly monotonically increasing, it is bijective.}
\end{itemize}
\end{ex}


\subsection{Cumulative distribution functions}

\subsubsection{Definition}

Random variables all valued as real numbers, and so we can write:

\(P(X\le x)=P({\omega |X(\omega)\le x})\)

Or:

\(F_X(x)=\int_{-\infty}^x f_X(u)du\)

\(F_X(x)=\sum_{x_i\le x}P(X=x_i) \)

\subsubsection{Partitions}

\(P(X\le x)+P(X\ge x)-P(X=x)=1\)

\subsubsection{Interval}

\(P(a< X\le b)=F_X(b)-F_X(a)\)


\subsection{Defining linear models}

\subsubsection{Defining}

One option for \(f(X)\) is a linear model.

\(f(X_i)=\hat{Y_i}= \beta_0+\sum_{j=1}^p\beta_iX_{ij}\)

The values for \(\beta \) are the regression coefficients.

So we have:

\(Y_i=\beta_0+\sum_{j=1}^p\beta_iX_{ij}+e(X_i)+e_i\)

We define the error of the estimate as:

\(\epsilon_i=Y_i-\hat{Y_i}\)

\(\epsilon_i=e(X_i)+e_i \)

So:

\(Y_i=\beta_0+\sum_{j=1}^p\beta_iX_{ij}+\epsilon_i\)

The linear model could be wrong for two reasons. No linear model could be appropriate, or the wrong coefficients could be provided for a linear model.

Linear regression if \(f\) is a linear function on \(w\). NB: not linear in \(x\) necessarily. could have \(x^2\) etc, but still linear in \(w\).


\section{Limit Properties}
Limit have many nice properties all allow us to make useful simplifications when evaluating a limit.
Let
\begin{equation*}
	\lim_{x \to c}{f(x)} = L \text{ and } \lim_{x \to c}{g(x)} = M.
\end{equation*}
\begin{align*}
	\textbf{Sum and Difference Rule: }& \lim_{x\to c}{\left(f(x) \pm g(x)\right)} = L \pm M \\
	\textbf{Product Rule: }& \lim_{x\to c}{\left(f(x)g(x) \right)} = LM \\
	\textbf{Constant Multiple Rule: }& \lim_{x \to c}{k\cdot f(x)} = k \lim_{x \to c}{f(x)} = kL \\
	\textbf{Quotient Rule: }& \lim_{x \to c}{\frac{f(x)}{g(x)}} = \frac{\lim_{x \to x}{f(x)}}{\lim_{x \to c}{g(x)}} = \frac{L}{M} \text{, if} M \neq 0 \\
	\textbf{Power Rule: }& \text{If } n \neq  \in \R \text{, } \lim_{x\to c}{\left(f(x)\right)^n} = \left(\lim_{x \to c}{f(x)}\right)^n = L^n
\end{align*}

\subsection{``Substitution Rule''}
Although it may seem obvious from our idea that limits describe behavior at a point that if $f(x)$ is defined at $x=c$, then $\lim_{x\to c}{f(x)} = f(c)$.
However, this is \textit{not} always the case.
Remember that our definition of a limit required these $\epsilon$ and $\delta$ neighborhoods around the limit point.
If $f(x)$ is defined at $x=c$, but $(c, f(c))$ is not a point in these neighborhoods for any $\epsilon > 0$, then the limit will not evaluate to $f(c)$.

\begin{example}
	Find the limit of $f(x)$ as $x$ approaches $2$ for the following function.
	\begin{equation*}
		f(x) = \begin{cases}
			x^2 & x \neq 2 \\
			0 & x = 2
		\end{cases}.
	\end{equation*}
\end{example}
\begin{answer}
	We can clearly see that $f(2) =  0$, but for $\epsilon = 0.1$, for example, there is no $\delta$ that can satisfy our definition, as points like $(2 - \delta, 4 - 2\delta + \delta^2)$ would outside the neighborhood around $(2,0)$.
	In fact, the correct limit value is $4$, the same as if $f(x) = x^2$ for all $x$.
	There are some more nuances we'll need to describe before we can say when it's OK to substitute to evaluate a limit.
\end{answer}


\section{Integer Linear Programming Model}
	In this section will be presented the model implemented using Cplex as solver.
	\subsection{Network Flow Model Representation}
		The problem can be represented on a complete weighted graph $G=(N, A)$ where $N$ is the set of nodes (holes on the board) and $A$ is the set of the arcs $(i, j), ∀ i, j ∈ N$ (trajectory of the drill moving from hole $i$ to hole $j$).To each arc will be associated a weight $c_{ij}$ that represents the time needed to move from starting hole to destination.

		Using this representation the problem is equivalent to determine the minimum weight hamiltonian cycle on G, so it is like the very popular Travelling Salesman Problem (TSP).

		To solve the TSP problem we can formulate it as a network flow model on G. Given a starting node $0 ∈ N$, let $|N|$ be the amount of its output flow, the problem can be solved by finding the path for which:
		\begin{itemize}
			\item Each node receives 1 unit of flow
			\item Each node is visited once
			\item The sum of costs in selected arcs is minimum
		\end{itemize}

	\subsection{Variables and Constraints}
		The problem can then be represented like this:\\
		\newline
		\textbf{SETS:}\\
		\begin{addmargin}[2em]{0em}
			$N$ = the graph nodes (holes)\\
			$A$ = arcs in the form $(i, j), ∀ i, j ∈ N$ (trajectories between holes)\\
		\end{addmargin}
		\textbf{PARAMETERS:}\\
		\begin{addmargin}[2em]{0em}
			$c_{ij}$ = time taken by the drill to move from $i$ to $j$, $∀ (i, j) ∈ A$\\
			$0$ = starting hole, $0 ∈ N$\\
		\end{addmargin}
		\textbf{DECISION VARIABLES:}\\
		\begin{addmargin}[2em]{0em}
			$x_{ij}$ = amount of the flow shipped from $i$ to $j$, $∀ (i, j) ∈ A$\\
			$y_{ij}$ = 1 if arc $(i, j)$ ships some flow, 0 otherwise, $∀ (i, j) ∈ A$\\
		\end{addmargin}
		\textbf{OBJECTIVE FUNCTION:}\\
		\[min \sum_{i, j | (i, j) ∈ A} c_{ij} \cdot y_{ij}\]\\
		\textbf{CONSTRAINTS:}
		\begin{align*}
			&\sum_{j | (0,j) ∈ A} x_{0j} = |N|\\
			\\
			&\sum_{i | (i, k) ∈ A} x_{ik} - \sum_{j | (k, j) ∈ A} x_{kj} = 1 & ∀ k ∈ N \setminus \{0\}\\
			\\
			&\sum_{j | (i, j) ∈ A} y_{ij} = 1 & ∀ i ∈ N\\
			\\
			&\sum_{i | (i, j) ∈ A} y_{ij} = 1 & ∀ j ∈ N\\
			\\
			&x_{ij} \leq |N| \cdot y_{ij} & ∀ (i, j) ∈ A\\
			\\
			&x_{ij} ∈ \mathbb{Z}_{+} & ∀ (i, j) ∈ A\\
			\\
			&y_{ij} ∈ \{0, 1\} & ∀ (i, j) ∈ A
		\end{align*}


\begin{questions}
\question{Consider the predictor-corrector time-stepping scheme:
\begin{align*}
& y_{n+1}^p = y_n + \frac{\Delta t}{12}(23f_n - 16f_{n-1}+5f_{n-2}), \\
&y_{n+1} = y_n + \frac{\Delta t}{12}(5f(t_{n+1},y_{n+1}^p) + 8f)n - f_{n-1})~.
\end{align*}
Here $f_n = f(t_n,y_n)$ and $y'(t) = f(t,y)$, $y(0) = y_0$. Plot the stability region for this scheme.
 }

\begin{solution}

Let $y_{n+1}^p = P$ for simplicity. Then, for stability purposes, $f = \lambda y$. Finally, let $a = \frac{\Delta t \lambda}{12}$. Then,
\begin{align*}
\begin{cases}
P = y^n + a(23y^n - 16y^{n-1}+5y^{n-1} \\
y^{n+1} = y^n + a(5P + 8y^n - y^{n-1})
\end{cases} \Rightarrow y^{n+1} = y^n + a \left[ 5(y^n + a(23y^n- 16y^{n-1}+5y^{n-1})) + 8y^n - y^{n-1} \right]~.
\end{align*}

Now assume $y^{n+1} = gy^n$, and thus $y^n = g^2y^{n-2}$. Plugging this in, we get
\begin{align*}
gy^n = y^n +a\left[ 5y^n + 115ay^n - 80\frac{a}{g}y^n + 25\frac{a}{g^2}y^n\right] + 8ay^n - \frac{a}{g}y^n~.
\end{align*}
Solving for $a$, we get
\begin{align*}
&(115g^2 - 80g +25)a^2 + (13g^2-g)a + (g^2-g^3) = 0 \\
& \Rightarrow a = \frac{-(13g^2 - g) \pm \sqrt{(13g^2-g)^2 - 4(115g^2 - 80g+25)(g^2-g^3)}}{2(115g^2 - 80g +25)}~.
\end{align*}
Lastly we let $z = 12a$ and plot the solution into Matlab to get the following stability region:

\begin{figure}[H]
\center{\includegraphics[scale=.5]{P3F1.eps}}
\end{figure}

\end{solution}

\subsection*{MATLAB}

\begin{verbatim}
close all; clear variables;

theta = linspace(0,2*pi,100);
g = exp(1i*theta);
a = 115*g.^2 - 80*g+25;
b = 13*g.^2 - g;
c = g.^2-g.^3;
sq = sqrt(b.^2 - 4*a.*c);

zp = 12*((-b + sq)./(2*a));
zn = 12*((-b - sq)./(2*a));

figure
plot(real(zp),imag(zp),'.r',real(zn),imag(zn),'.b')
legend('positive square root','negative square root')
grid on
\end{verbatim}

\end{questions}


\subsection{Eigenvalue for rotation transformation}

\textbf{\textit{Claim:}}
If $0< \theta <2\pi$ then transformation $R_\theta : \R^2\rightarrow \R^2$ only has an eigenvector when $\theta=\pi$ (when $\lambda=-1$).
\newline \noindent
\textbf{\textit{Proof:}}
\noindent
The matrix is $A=\begin{bmatrix}\cos\theta & -\sin\theta\\\sin\theta & \cos\theta\end{bmatrix}$. Following by the definition of an eigenvector:

\[
    \begin{aligned}
        A \tb{v} &=\lambda v \Longleftrightarrow \\
        A \tb{v}-\lambda \tb{v} &=\overrightarrow{0} \Longleftrightarrow \\
        A \tb{v}-\lambda(I \tb{v}) &=\overrightarrow{0} \Longleftrightarrow \\
        A \bar{v}-(\lambda I) \tb{v} &=\overrightarrow{0} \Longleftrightarrow \\
        (A-\lambda I) \tb{v} &=\overrightarrow{0} \Longleftrightarrow \\
            \operatorname{det}(A-\lambda I)&=0
        \end{aligned}
\]

Thus,

\[
    \begin{aligned}
        \operatorname{det}(A-\lambda I) &=0 \\
        \operatorname{det}\left(\begin{array}{cc}
        \cos \theta-\lambda & -\sin \theta \\
        \sin \theta & \cos \theta-\lambda
        \end{array}\right) &=0 \\
        \lambda^{2}-2 \lambda \cos \theta+\cos ^{2} \theta+\sin ^{2} \theta &=0 \\
        \lambda^{2}-2 \lambda \cos \theta+1 &=0
        \end{aligned}
\]

The discriminant of this quadratic ($b^2-4ac$) is $4\cos^2\theta-4$, so for a real solution $4\cos^2\theta-4\geq 0$. It then follows
that:

\begin{align*}
    4\cos^2\theta-4&\geq 0\\
    \cos^2\theta&\geq 1\\
    \cos^2\theta&=\pm 1\\
    \theta&=\pi
\end{align*}

Note that because $(A-\lambda I)\tb{v}=\tb{0}$ implies a nontrivial kernel for $A-\lambda I$, $\mathrm{det}(A-\lambda I)=0$.

\subsection{Characteristic Polynomials}

Characteristic polynomial is for $\mathrm{det}(A-\lambda I)$ with variable $\lambda$:

\[\boxed{P_A(\lambda)=\mathrm{det}(A-\lambda I)}\]

General polynomial for $A=\begin{bmatrix}a&b\\c&d\end{bmatrix}$:

\[
    \begin{aligned}
        p_{A}(\lambda) &=\operatorname{det}(A-\lambda I) \\
        &=\operatorname{det}\left(\begin{array}{cc}
        a-\lambda & b \\
        c & d-\lambda
        \end{array}\right) \\
        &=(a-\lambda)(d-\lambda)-b c \\
        &=\lambda^{2}-(a+b) \lambda+(a d-b c)
    \end{aligned}
\]

Ends up that $\mathrm{tr}(A)=a+d$ and $\mathrm{det}(A)=ad-bc$:

\[\boxed{p_{A}(\lambda)=\lambda^{2}-\operatorname{tr} A \lambda+\operatorname{det} A}\]

\subsubsection{General formula}

In general, if $A$ is an $n\times n$ matrix, then

\[\boxed{p_{A}(\lambda)=(-1)^{n} \lambda^{n}+(-1)^{n-1} \operatorname{tr} A \lambda^{n-1}+\cdots+\operatorname{det} A}\]

Conjectures:
\begin{itemize}
    \item By FTLA, degree $n$ polynomial will have $n$ complex roots so at least $n$ real eigenvalues
    \item If all $n$ roots are real, then $\mathrm{tr}(A)$ is sum of eigenvalues and determinant is the product of them
    \item Since roots are either real or in complex conjugate pairs, ($a+bi$ or $a-bi$) then when $n$ is odd $A$ has at least 1 real eigenvalue
\end{itemize}


\chapter{Minimum Spanning Tree}

Given an undirected, connected graph where each edge has positive
weight, the \emph{Minimum Spanning Tree} (MST) is a connected subgraph on the same
vertex set with minimal weight. Intuitively, the MST is the lightest possible
connected subgraph. The MST is not necessarily unique,
for proof of this, consider a graph where all edges are of equal weight.
Any tree is an MST of such a graph.

\section{Kruskal's Algorithm}

%The idea of this algorithm is to begin with the vertex list and an
%empty edge list, and maintain a forest, adding the edge of minimum
%weight that does not make a cycle until the forests are connected.

Kruskal's algorithm for computing the MST relies on the simple observation
that the smallest edge in a graph $G$ is part of \emph{some} MST of $G$.
Kruskal's algorithm starts by constructing a min-heap containing all $m$ edges
in $G$, and a collection of disjoint sets, each containing one of the
$n$ vertices of $G$. We also construct an empty list $T$ that will hold
all the edges of the MST. We then get the minimum edge from the heap, check if
the two nodes the edge joins are from the same set, and if not add
the edge to $T$, and $union$ the two sets that the edge joins.
We repeat this until $T$ contains $n-1$ edges.

Because this algorithm uses
structures we already know and understand, analysis will be fairly easy.
We require $O(m) + O(n)$ time to construct the initial sets and heap. We also
require $O(m \log m)$ to extract the minimums from the heap. Our $n-1$ unions
and $n-1$ finds can be done in $(n \log n + n)$ time. Therefore this algorithm
takes $O(m \log m + n \log n)$.

\section{Prim's Algorithm}
Prim's algorithm for computing the MST is fairly similar to Kruskal's. However,
instead of working with all of the edges and vertices at once, it picks one
vertice and builds from that. We start by constructing an empty list $A$ which
will hold all the vertices that are part of the MST so far, a list $T$ which
will hold all the edges in the MST so far, a min-heap $V$ which will contain all
the vertices not in $A$. At first every node is given a key of infinity.
We then pick a random vertex $v$ from $V$ and move it from $V$
to $A$. Next we look at all the edges of $v$ and set the
keys of the corresponding nodes to the weight of these edges. Now we retrieve
the minimum node $u$ from $V$ and move it to $A$, and its key edge to $T$.
Next we look at all the edges on $u$ and if their weight is smaller than
the current key of their corresponding edge, set the key to that weight. Then
we simply retrieve another node from the heap and repeat. After $n-1$ iterations
we will have a complete MST.

This algorithm requires $O(n)$ time to construct the initial heap and
initialize the first node, and since we must update at most $m$ keys in the
heap of $n$ elements, it requires $O(m \log n)$ time to do this. Therefore the
entire algorithm takes $O(n + m \log n)$ time.