\chapter{Linear Regression}
\label{chap:linreg}

In regression problems, we take a variable (or multiple variables) as input, and try to fit the output to a continuous expected result function.

\section{Univariate Linear Regression}
\label{chaplinreg-sect:univar}

In univariate linear regression, we want to predict a single output value $\hat{y}$ from a single input value $x$. Since this is supervised learning, we already have an idea about what the input/output relationship should look like.

\subsection{The Hypothesis Function}
\label{chaplinreg-sectunivar-subsect:hypfunct}

Imagine we have a problem where the input is $x$ and the output is $y$. In order to do machine learning, there should exist a relationship (a pattern) between the input and output variables. Let's say this function is $y = f\left( x \right)$. In this situation, $f$ is known as the target function. However, this function $f$ is unknown to us, so we need to try and guess what it is. To do that, we form a \textit{hypothesis} function $h\left( x \right)$ that approximates the unknown $f\left( x \right)$.

For single variable linear regression, our hytothesis function takes two parameters: $\theta_0$ and $\theta_1$. As such, we often write it as $h_\theta\left( x \right)$, and it takes the form
\begin{equation}
\hat{y} = h_\theta\left( x \right) = \theta_0 + \theta_1 x
\end{equation}

Note that this is the equation of a straight line ($y = mx + b$). We're trying to find the values for $\theta_0$ and $\theta_1$ to get our estimated output $\hat{y}$. In other words, we're trying to determine the function $h_\theta$ that maps our input data (the $x$'s) to our output data (the $y$'s).

Suppose we have the following set of training data:

\begin{center}
\begin{tabular}{c | c}
\textbf{Input } $x$ & \textbf{Output } $y$ \\
\hline
0 & 4 \\
1 & 7 \\
2 & 7 \\
3 & 8 \\
\end{tabular}
\end{center}

We can plot these points, as shown in Figure \ref{fig:linregeg-justpoints}. Let's make a random guess at our hypothesis function: $\theta_0  = 2$ and $\theta_1 = 2$, making our hypothesis function $h_\theta\left( x \right) = 2 + 2x$, as shown in Figure \ref{fig:linregeg-hypotguess1}.

\begin{figure}[h]
	\centering
	\begin{subfigure}[t]{0.45\textwidth}
		\centering
		\graphicspath{{./Figures/}}
		\includegraphics[scale=0.5]{linreg_eg1_plotpoints_noline.pdf}
		\caption{Plotting our example points.}
		\label{fig:linregeg-justpoints}
	\end{subfigure}
	\begin{subfigure}[t]{0.45\textwidth}
		\centering
		\graphicspath{{./Figures/}}
		\includegraphics[scale=0.5]{linreg_eg1_plotpoints_line1.pdf}
		\caption{Plotting our example points.}
		\label{fig:linregeg-hypotguess1}
	\end{subfigure}
\end{figure}

Using this hypothesis function for $x = 1$, we have $\hat{y} = h_\theta\left( 1 \right) = 2 + 2 \cdot 1 = 4$. In this case, $\hat{y} = 4$, but $y = 7$, so mabe this isn't the best fit hypothesis.

\subsection{The Cost Function}
\label{chaplinreg-sectunivar-subsect:costfxn}

The cost function,\footnote{The cost function can also be called the loss function.} is a function used for parameter estimation, where the input to the cost function is some function of the difference between estimated and the true values for an instance of data. In this case, we can use the cost function to measure the accuracy of our hypothesis function.

The cost function looks at something similar to an average\footnote{It's actually something a bit fancier than a standard average.} of all the results of the hypothesis with inputs from the $x$'s compared to the actual output $y$'s. We define our cost function as follows:

\begin{equation}
J\left( \theta_0, \theta_1 \right) = \frac{1}{2m} \sum_{i=1}^m \left(\hat{y}_i - y_i \right)^2 = \frac{1}{2m} \sum_{i=1}^m \left(h_\theta\left( x_i \right) - y_i \right)^2
\end{equation}

This is known as the \textbf{mean squared error}. If we set $\bar{x}$ equal to the mean of the squares all the $ h_\theta \left( x_i \right) - y_i$, then the cost function is just the mean of $\bar{x}$. The term $\frac{1}{2m}$ is merely a convenience for the computation of gradient descent, which we'll see very shortly.

\subsection{Gradient Descent}
\label{chaplinreg-sectunivar-subsect:graddsc}

We now have our hypothesis function defined, as well as a way of measuring how well it fits the data. Now, we have to estimate the parameters in the hypothesis function, and that's where gradient descent comes in.

Let's graph our cost function as a function of the parameter estimates. This can be somewhat confusing, as we are moving up to a higher level of abstraction. We are not graphing $x$ and $y$ itself, but the parameter range of our hypothesis function and the cost resulting from selecting particular sets of parameters. We put $\theta_0$ on the $x$-axis, and $\theta_1$ on the $y$-axis, with the cost function on the vertical $z$-axis.

\begin{figure}[h] %  figure placement: here, top, bottom, or page
   \centering
    \graphicspath{{./Figures/}} %Use this to import an image from a subfolder.
   \includegraphics[scale=0.4]{linreg_eg2_cost_func_over_thetas.pdf}
   \caption[]{Plot of the cost function $J\left(\theta_0, \theta_1 \right)$ using our hypothesis $h_\theta\left( x \right)$.}
   %If using \listoffigures command, then use \caption[short]{long}, where the short caption will appear in the list of figures, and the long caption will appear next to the figure.
   \label{s}
\end{figure}

Our goal is to take the parameters $\theta_0$ and $\theta_1$ for when the cost function is at its minimum. We can calculate this value by taking the derivative of the cost function, which gives us direction of the steepest gradient to move towards. Take a step in that direction, and repeat.The step size is determined by the parameter $\alpha$, which is called the \textbf{learning rate}. The gradient descent algorithm is:

\textbf{Repeat until convergence:}
\begin{equation}
\theta_j := \theta_j - \alpha \frac{\partial}{\partial \theta_j} J\left( \theta_0, \theta_1 \right)
\end{equation}

where $j = 0, 1$ represents the feature index number.

\subsubsection{Gradient Descent for Linear Regression}
When specifically applied to the case of univariate linear regression, we can derive another form of the gradient descent equation. If we substitute our actual hypothesis function and cost function, we can modify the equation to

\textbf{Repeat until convergence: \{ }
\begin{equation}
\begin{aligned}
\theta_0 &:= \theta_0 - \alpha \frac{1}{m} \sum_{i=1}^m \left( h_\theta\left( x_i \right) - y_i \right) \\
\theta_1 &:= \theta_1 - \alpha \frac{1}{m} \sum_{i=1}^m \left( \left( h_\theta \left( x_i \right) - y_i \right) x_i \right)
\end{aligned}
\end{equation}

\textbf{ \} }

\noindent where $m$ is the size of the training set, $\theta_0$ is a constant that will be changing simultaneously with $\theta_1$, and $x_1$, $y_i$ are values of the given training set. Note that we have separated out the two cases for $\theta_j$ into separate equations for $\theta_0$ and $\theta_1$, and that for $\theta_1$ we are multiplying $x_i$ at the end due to the derivative.

\section{Multivariate Linear Regression}
\label{chaplinreg-sect:multivarreg}
Let's start by looking at some sample housing data with multiple features. \\

\begin{tabular}{c | c | c | c | c }
Size (feet$^2$) & \# of Bedrooms & \# of Floors & Age (years) & Price (in 1000's of \$) \\
$x_1$ & $x_2$ & $x_3$ & $x_4$ & $y$ \\ \hline
2104 & 5 & 1 & 45 & 460 \\
1416 & 3 & 2 & 40 & 232 \\
1534 & 3 & 2 & 30 & 315 \\
852 & 2 & 1 & 36 & 178 \\
560 & 1 & 1 & 12 & 155
\end{tabular}

In this, we can introduce some notation:
\begin{itemize*}
\item The variables $x_1$, $x_2$, etc. are the features.
\item The variable $y$ is the output variable.
\item The number of input features is denoted $n$. In this example, $n = 4$.
\item $m$ specifies the number of training examples (rows). Here, $m = 5$.
\item $x^{\left( i \right)}$ is the input (features) of the $i^{th}$ training example. So $x^{\left( 2 \right)}$ is the column vector $[1416, 3, 2, 40, 232]$.
\item $x_j^{\left( i \right)}$ is feature $j$ in the $i^{th}$ training example. Here, $x_1^{\left( 4 \right)} = 852$.
\end{itemize*}

At this point, we can define the multivariable form of the hypothesis function for linear regression:
\begin{equation}
h_\theta\left( x \right) = \theta_0 + \theta_1 x_1 + \theta_2 x_2 + \theta_3 x_3 + \cdots + \theta_n x_n
\end{equation}

For convenience of notation, we will define $x_0 = 1$ for all feature vectors ($x_0^{\left( i \right)} = 1$). So now, if we include $x_0$, our hypothesis function takes the form:

\begin{equation}
h_\theta\left( x\right) = \sum_{i=0}^n \theta_i x_i
\end{equation}
Now, we can also write the $x$ values and $\theta$ values as vectors:
$$
\vec{x} = \left[ \begin{array}{c}
x_0 \\
x_1 \\
x_2 \\
\vdots \\
x_n
\end{array} \right] \in \mathbb{R}^{n+1}
~~\mbox{\;\;\;\;\;\;\;\;\;\; and \;\;\;\;\;\;\;\;\;\;}~~
\vec{\theta} = \left[\begin{array}{c}
\theta_0 \\
\theta_1 \\
\theta_2 \\
\vdots \\
\theta_n \end{array}\right] \in \mathbb{R}^{n+1}
$$
In vector notation, this is
\begin{equation}
h_\theta\left( x \right) = \vec{\theta}^\intercal\vec{x}
\end{equation}
where we transpose $\vec{\theta}$ into a row vector so we're able to take the inner product.

Now that we have our vector $\vec{\theta} \in \mathbb{R}^{n+1}$, the cost function is
\begin{equation}
J\left(\vec{\theta}\right) = \frac{1}{2m}\sum_{i=1}^m \left( h_\theta\left( x^{\left( i \right)} \right) - y^{\left( i \right)} \right)^2
\end{equation}

\subsection{Gradient Descent for Multiple Variables}
Using our expanded hypothesis and cost functions, the gradient descent algorithm becomes:

\textbf{Repeat until convergence: \{ }
\begin{equation}
\theta_j := \theta_j - \alpha\frac{1}{m} \sum_{i=1}^m \left( h_\theta\left( x^{\left(i\right)} \right) - y^{\left(i\right)} \right) \cdot x_j^{\left( i \right)} ~~\mbox{\;\;\;\;\;\;\;\;\;\; for } j = 0, 1, \cdots, n
\end{equation}
\textbf{ \} }

\subsection{Feature Scaling}
When features are in very different ranges, it can slow down gradient descent dramatically (and also mess up our machine learning algorithms!), because $\theta$ will descend quickly on small ranges and slowly on large ranges, and so will oscillate inefficiently down to the minimum. The way to prevent this is to ensure that all the ranges are roughly the same, ideally:
$$ -1 \leq x \leq 1 $$

Two techniques to accomplish this are \textbf{feature scaling} and \textbf{mean normalization}. Feature scaling involved dividing the input values by the range (max value minus the min value) of the input variable, resulting in a new range of just 1.

\begin{figure}[h] %  figure placement: here, top, bottom, or page
   \centering
    \graphicspath{{./Figures/}} %Use this to import an image from a subfolder.
   \includegraphics[scale=0.6]{linreg_eg3_why_need_feature_scaling.pdf}
   \caption[]{When one feature is on a much larger scale than the other, the plot of the cost function will be stretched out in the direction of the larger feature. Here, imagine that $\theta_1$ is the number of bedrooms a house has, and $\theta_2$ is the size in square feet. }
   %If using \listoffigures command, then use \caption[short]{long}, where the short caption will appear in the list of figures, and the long caption will appear next to the figure.
   \label{s}
\end{figure}

Mean normalization involves subtracting the mean value for an input variable from the value for that input variable, resulting in a new mean of zero. To implement both of these simultaneously, use the following formula:
\begin{equation}
x_i := \frac{x_i - \mu_i}{s_i}
\end{equation}
where $\mu_i$ is the average value of $x_i$, and $s_i$ can either be the range ($x_\text{max} - x_\text{min}$) or the standard deviation.

\subsection{Tips for Gradient Descent}
Here are some of Professor Andrew Ng's tips on implementing gradient descent.

\begin{enumerate}
\item \textbf{Plot \boldmath$J\left(\theta\right)$.} If you plot $J\left(\theta\right)$ as the ordinate and the number of iterations as the abscissa,\footnote{On a Cartesian coordinate plane, the ordinate is the $y$-axis and the abscissa is the $x$-axis.} the graph should be steadily decreasing with increasing number of iterations. If $J\left(\theta\right)$ ever increases, then $\alpha$ is probably too large.
\item If $J\left(\theta\right)$ decreases by less then $E$ in one iteration, where $E$ is some very small number, such as $10^{-3}$, then you can declare convergence.
\item For sufficiently small $\alpha$, $J\left(\theta\right)$ should decrease with every iteration. To choose $\alpha$, try a range of values for $\alpha$ with threefold increases, such as:
$$ \cdots \to 0.001 \to 0.003 \to 0.01 \to 0.03 \to 0.1 \to 0.3 \to 1 \to 3 \to \cdots$$
\item Sometimes, it's better to define new features instead of using the ones given. For example, if we have a house with features frontage\footnote{The width of the land in the front of the house.} and depth,\footnote{The width of the land on the side of the house.} you can combine these into a new feature called area, which is how much land the house sits on.
\end{enumerate}

\subsection{Polynomial Regression}
The form of the hypothesis doesn't necessarily need to be linear of that doesn't fit the data well. We can change the behavior or curve of our hypothesis function by making it quadratic, cubic, square root, or some other form.

For example, if our hypothesis function is $h_\theta \left(x \right) = \theta_0 + \theta_1 x_1$, we can create additional features based on $x_1$, to get the quadratic function $h_\theta\left(x\right) = \theta_0 + \theta_1 x_1 + \theta_2 x_1^2$, or the cubic function $h_\theta\left(x\right) = \theta_0 + \theta_1 x_1 + \theta_2 x_1^2 + \theta_3 x_1^3$.

When thinking about nonlinear features, it is important to keep in mind that features scaling becomes even more essential than it was for linear regression. If $x_1$ has a range of $1$ to $1000$, the $x_1^2$ has a range of $1$ to $1,000,000$.

\section{Vectorized Equations}
Let's revisit our housing example from \S \ref{chaplinreg-sect:multivarreg}. Recall that we looked at the the following example data, and we'll add an extra column for $x_0$ that always takes a value of one: \\

\begin{tabular}{c | c | c | c | c | c }
{} & Size (feet$^2$) & \# Bedrooms & \# Floors & Age (years) & Price (in \$1000's) \\
$x_0$ & $x_1$ & $x_2$ & $x_3$ & $x_4$ & $y$ \\ \hline
1 & 2104 & 5 & 1 & 45 & 460 \\
1 & 1416 & 3 & 2 & 40 & 232 \\
1 & 1534 & 3 & 2 & 30 & 315 \\
1 & 852 & 2 & 1 & 36 & 178 \\
1 & 560 & 1 & 1 & 12 & 155
\end{tabular}

From this, we construct a matrix $X$ that contains all of the features from the training data, and a vector $\vec{y}$ of all the output data.

$$
X = \left[\begin{array}{ccccc}
1 & 2104 & 5 & 1 & 45 \\
1 & 1416 & 3 & 2 & 40 \\
1 & 1534 & 3 & 2 & 30 \\
1 & 852 & 2 & 1 & 36 \\
1 & 560 & 1 & 1 & 12
\end{array}\right]
~~\mbox{\;\;\;\;\;\;\;\;\;\; and \;\;\;\;\;\;\;\;\;\;}~~
\vec{y} = \left[\begin{array}{c}
460 \\ 232 \\ 315 \\ 178 \\ 155
\end{array}\right]
$$

Here, $X$ is a $m \times \left(n + 1\right)$ matrix, and $\vec{y}$ is a $m$-dimensional vector.

Let's go through this again, but this time in full abstraction. Say we have $m$ examples $\left(x^{\left(1\right)}, y^{\left(1\right)}\right),  \left(x^{\left(2\right)}, y^{\left(2\right)}\right), \dots, \left(x^{\left(m\right)}, y^{\left(m\right)}\right)$ and each $x^{\left(i\right)}$ has $n$ features. Then, we have an $\left(n + 1\right)$-dimensional feature vector:

\begin{equation}
x^{\left(i\right)} = \left[\begin{array}{c} x_0^{\left(i\right)} \\ x_1^{\left(i\right)} \\ x_2^{\left(i\right)} \\ \vdots \\ x_n^{\left(i\right)} \end{array}\right] \in \mathbb{R}^{n+1}
\end{equation}

The matrix $X$, which is also called the \textbf{design matrix}, is constructed by taking the transpose of each vector $x^{\left(i\right)}$. Each feature vector $x^{\left(i\right)}$ becomes a row in the design matrix. Just as previously, the output vector $\vec{y}$ is obtained by taking all the labels and stacking them up into an $m$-dimensional vector, and the vector $\vec{\theta}$ is created from stacking all of the parameters for the hypothesis function.

\begin{equation}
X = \left[\begin{array}{ccccc}
-- & -- & \left(x^{\left(1\right)}\right)^\intercal & -- & -- \\
-- & -- & \left(x^{\left(2\right)}\right)^\intercal & -- & -- \\
-- & -- & \left(x^{\left(3\right)}\right)^\intercal & -- & -- \\
-- & -- & \left(x^{\left(4\right)}\right)^\intercal & -- & -- \\
-- & -- & \vdots & -- & -- \\
-- & -- & \left(x^{\left(m\right)}\right)^\intercal & -- & --
\end{array}\right]
~~\mbox{\;\;\;\;\;}~~
\vec{y} = \left[\begin{array}{c} y^{\left(1\right)} \\ y^{\left(2\right)} \\ y^{\left(3\right)} \\ y^{\left(4\right)} \\ \vdots \\ y^{\left(m\right)} \end{array}\right]
~~\mbox{\;\;\;\;\;}~~
\vec{\theta} = \left[\begin{array}{c} \theta_0 \\ \theta_1 \\ \theta_2 \\ \theta_3 \\ \vdots \\ \theta_n \end{array}\right]
\end{equation}

Think back to the start of this section when we separated our table into the design matrix $X$ and the output vector $y$. The design matrix is simply the data as stored in a table put into a matrix.

In our matrix notation for multivariate regression, the hypothesis function takes the form
\begin{equation}
h_\theta\left(X\right) = X\vec{\theta}
\end{equation}
This will always work since $X$ is an $m\times n$ matrix, and $\vec{\theta}$ is an $n \times 1$ vector. In a similar fashion, the cost function in matrix notation is
\begin{equation}
J\left(\vec{\theta}\right) = \frac{1}{2m} \left(X\vec{\theta} - \vec{y}\right)^\intercal \left(X\vec{\theta} - \vec{y}\right)
\end{equation}

The gradient descent rule can be expressed as
\begin{equation}
\vec{\theta} := \vec{\theta} - \alpha \nabla J\left(\theta\right)
\end{equation}

There $\nabla$ is the gradient (vector derivative) operator. If we solve this out using our vectorized hypothesis function, we get
\begin{equation}
\vec{\theta} := \vec{\theta} - \frac{\alpha}{m}X^\intercal \left(X\vec{\theta} - \vec{y} \right)
\end{equation}

\section{The Normal Equation}
The normal equation is a method of solving for the optimal $\theta$ analytically, that is, without iteration. From calculus, if we want to find the minimum of a quadratic equation, we set the derivative equal to zero, and solve. We can apply the same logic to the cost function. If we take the partial derivative $\partial/\partial \theta_j J\left(\theta\right)$ and set this equal to zero for all values of $j$, we'll analytically solve for the minimum.

The derivation of the normal equation is fairly involved from a linear algebra perspective, so at this point just take it as a fact:
\begin{equation}
\vec{\theta} = \left( X^\intercal X \right) ^{-1} X^\intercal \vec{y}
\end{equation}

When deciding whether to use gradient descent of the normal equation, consider the following:

\begin{tabular}{| l | l |} \hline
\textbf{Gradient Descent} & \textbf{Normal Equation} \\
\hline \hline
Need to choose $\alpha$ & No need to choose $\alpha$ \\ \hline
Needs many iterations & No need to iterate \\ \hline
$O\left( kn^2 \right)$ & $O\left(n^3\right)$, need to calculate $X^\intercal X$ \\ \hline
Works well when $n$ is large & Slow if $n$ is very large \\ \hline
\end{tabular}

With the normal equation, computing the inverse has complexity $O\left( n^3 \right)$. If we have a large number of features, this will cause the normal equation to perform slowly. In practice, when $n$ exceeds $10,000$, it would probably be a good idea to use gradient descent.

\subsection{Normal Equation Noninvertibility}
When implementing the normal equation, sometimes the matrix $X^\intercal X$ is noninvertible. The common causes are:
\begin{itemize}
\item Redundant features, where two or more features are linearly dependent
\item Too many features (i.e. $m \leq n$). In this case, delete some features or use regularization (which we'll get to later)
\end{itemize}

We typically avoid this problem by coding a pseudoinverse, instead of taking the actual inverse.

\section{Homework}

\subsection*{Introduction}
In this exercise, you will implement linear regression and get to see it work on data. Before starting on this programming exercise, we strongly recommend watching the video lectures and completing the review questions for the associated topics. To get started with the exercise, you will need to download the starter code and unzip its contents to the directory where you wish to complete the exercise. If needed, use the cd command in Octave/MATLAB to change to this directory before starting this exercise. You can also find instructions for installing Octave/MATLAB in the Environment Setup Instructions" of the course website. \\

To start our code, we'll need to import a few modules that we'll be using.

\begin{minted}{python}
import numpy as np
import pandas as pd

import matplotlib.pyplot as plt
import seaborn as sns

%matplotlib inline	# Use this if you're using Jupyter Notebooks
\end{minted}

\subsubsection*{Simple Octave/MATLAB Function}
The first part of {\texttt ex1.m} gives you practice with Octave/MATLAB syntax and the homework submission process. In the file {\texttt warmUpExercise.m}, you will find the outline of an Octave/MATLAB function. Modify it to return a 5 x 5 identity matrix.

\begin{minted}{python}
A = np.eye(5)
print(A)
\end{minted}

\subsection*{Linear Regression with One Variable}
In this part of this exercise, you will implement linear regression with one variable to predict profits for a food truck. Suppose you are the CEO of a restaurant franchise and are considering different cities for opening a new outlet. The chain already has trucks in various cities and you have data for profits and populations from the cities. You would like to use this data to help you select which city to expand to next.

\subsubsection*{Plotting the Data}
Before starting on any task, it is often useful to understand the data by visualizing it. For this dataset, you can use a scatter plot to visualize the data, since it has only two properties to plot (profit and population). (Many other problems that you will encounter in real life are multi-dimensional and can't be plotted on a 2-d plot.)

We start by loading the data
\begin{minted}{python}
datafile = 'ex1\\ex1data1.txt'
df = pd.read_csv(datafile, header=None, names=['Population', 'Profit'])
\end{minted}

Then we define a function to plot the data and call this function.
\begin{minted}{python}
def plot_data(x, y):
    plt.figure(figsize=(10, 6))
    plt.plot(x, y, '.', label='Training Data')
    plt.xlabel("Population of City in 10,000s")
    plt.ylabel("Profit in $10,000s")

plot_data(df['Population'], df['Profit'])
\end{minted}

\begin{figure}[h] %  figure placement: here, top, bottom, or page
	\centering
	\graphicspath{{./Figures/}} %Use this to import an image from a subfolder.
	\includegraphics[scale=0.5]{linreg_hw_2_1_plot_data.pdf}
%	\caption[]{One-vs-all classification. We're selecting one class to be our positive class, and the rest all become the negative class.}
	\label{linreg_hw_2_1_plot_data.pdf}
\end{figure}

\subsection*{Gradient Descent}
In this part, you will fit the linear regression parameters $\theta$ to our dataset using gradient descent.

\subsection*{Update Equations}
The objective of linear regression is to minimize the cost function
$$
J\left( \theta \right) = \frac{1}{2m} \sum_{i=1}^m \left( h_\theta \left( x^{\left( i\right)} \right) - y^{\left( i \right)} \right)^2
$$
where $h_\theta\left( x \right)$ is the hypothesis given by the linear model
$$
h_\theta\left( x \right) = \theta^\intercal x = \theta_0 + \theta_1 x_1
$$
Recall that the parameters of your model are the $\theta_j$ values. These are the values you will adjust to minimize cost $J(\theta)$. One way to do this is to use the batch gradient descent algorithm. In batch gradient descent, each iteration performs the update
$$
\theta_j := \theta_j - \alpha\frac{1}{m}\sum_{i=1}^m \left( h_\theta\left( x^{\left( i\right)} \right) - y^{\left(i\right)}\right) x_j^{\left(i\right)} \;\;\;\;\;\;\;\;\;\; \text{simultaneously update } \theta_j \text{ for all } j \text{.}
$$
With each step of gradient descent, your parameters $\theta_j$ come closer to the optimal values that will achieve the lowest cost $J(\theta)$.

\subsection*{Implementation}
In the following lines, we add another dimension to our data to accommodate the $\theta_0$ intercept term.

\begin{minted}{python}
# set the number of training examples
m = len(df['Population'])

# create an array from the dataframe (missing column for x_0 values)
X = df['Population'].values

# add in the first column of the array for x_0 values
X = X[:, np.newaxis]
X = np.insert(X, 0, 1, axis=1)

y = df['Profit'].values
y = y[:, np.newaxis]
\end{minted}

\subsection*{Computing the Cost $J\left(\theta\right)$}
Now, we can define our actual hypothesis function for linear regression with a single variable.
\begin{minted}{python}
# define the hypothesis
def h(theta, X):
    """Takes the dot product of the matrix X and the vector theta,
    yielding a predicted result.
    """
    return np.dot(X, theta)

def compute_cost(X, y, theta):
    """Takes the design matrix X and output vector y, and computes the cost of
    the parameters stored in the vector theta.

    The dimensions must be as follows:
    - X must be m x n
    - y must be m x 1
    - theta must be n x 1

    """
    m = len(y)

    J = 1 / (2*m) * np.dot((np.dot(X, theta) - y).T, (np.dot(X, theta) - y))
    return J
\end{minted}

Once you have completed the function, the next step % {\texttt compute_cost}
once using $\theta$  initialized to zeros, and you will see the cost printed to the screen. You should expect to see a cost of 32.07.

\begin{minted}{python}
# define column vector theta = [[0], [0]]
theta = np.zeros((2, 1))

# compute the cost function for our existing X and y, with our new theta vector
# verify that the cost for our theta of zeros is 32.07
compute_cost(X, y, theta)
\end{minted}

This gives us our expected value of $32.07273388$.

\subsection*{Gradient Descent}
Now we'll actually implement the gradient descent algorithm. Keep in mind that the cost $J(\theta)$ is parameterized by the vector $\theta$, not $X$ and $y$. That is, we minimize $J(\theta)$ by changing $\theta$. We initialize the initial parameters to 0 and the learning rate alpha to 0.01.

\begin{minted}{python}
def gradient_descent(X, y, theta, alpha, num_iters):
    m = len(y)
    J_history = []
    theta_history = []

    for i in range(num_iters):
        J_history.append(float(compute_cost(X, y, theta)))
        theta_history.append(theta)
        theta = theta - (alpha / m) * np.dot(X.T, (np.dot(X, theta) - y))

    return theta, J_history, theta_history

# set up some initial parameters for gradient descent
theta_initial = np.zeros((2, 1))
iterations = 1500
alpha = 0.01

# run our gradient descent function
theta_final, J_hist, theta_hist = gradient_descent(X, y,
                                                   theta_initial,
                                                   alpha, iterations)

\end{minted}

\begin{figure}[h] %  figure placement: here, top, bottom, or page
	\centering
	\graphicspath{{./Figures/}} %Use this to import an image from a subfolder.
	\includegraphics[scale=0.5]{linreg_hw_2_3_plot_lin_reg.pdf}
%	\caption[]{One-vs-all classification. We're selecting one class to be our positive class, and the rest all become the negative class.}
	\label{linreg_hw_2_3_plot_lin_reg.pdf}
\end{figure}

\subsection*{Visualizing $J\left(\theta\right)$}
After running the batch gradient descent algorithm, we can plot the convergence of $J\left(\theta\right)$ over the number of iterations.
\begin{minted}{python}
def plot_cost_convergence(J_history):
    abscissa = list(range(len(J_history)))
    ordinate = J_history

    plt.figure(figsize=(10, 6))
    plt.plot(abscissa, ordinate, '.')
    plt.title('Convergence of the Cost Function', fontsize=18)
    plt.xlabel('Iteration Number', fontsize=14)
    plt.ylabel('Cost Function', fontsize=14)
    plt.xlim(min(abscissa) - max(abscissa) * 0.05, 1.05 * max(abscissa))

plot_cost_convergence(J_hist)
plt.ylim(4.3, 6.9)
\end{minted}

\begin{figure}[h] %  figure placement: here, top, bottom, or page
	\centering
	\graphicspath{{./Figures/}} %Use this to import an image from a subfolder.
	\includegraphics[scale=0.5]{linreg_hw_2_4_viz_j_of_theta.pdf}
%	\caption[]{One-vs-all classification. We're selecting one class to be our positive class, and the rest all become the negative class.}
	\label{linreg_hw_2_4_viz_j_of_theta.pdf}
\end{figure}

Now, let's plot the cost minimization on the surface of $J\left(\theta\right)$.

\begin{minted}{python}
from mpl_toolkits.mplot3d import axes3d, Axes3D
from matplotlib import cm

theta_0_hist = [x[0] for x in theta_hist]
theta_1_hist = [x[1] for x in theta_hist]
theta_hist_end = len(theta_0_hist) - 1

fig = plt.figure(figsize=(12, 12))
ax = fig.gca(projection='3d')

theta_0_vals = np.linspace(-10, 10, 100)
theta_1_vals = np.linspace(-1, 4, 100)

theta1, theta2, cost = [], [], []

for t0 in theta_0_vals:
    for t1 in theta_1_vals:
        theta1.append(t0)
        theta2.append(t1)
        theta_array = np.array([[t0], [t1]])
        cost.append(compute_cost(X, y, theta_array))

scat = ax.scatter(theta1, theta2, cost,
                  c=np.abs(cost), cmap=plt.get_cmap('rainbow'))

plt.plot(theta_0_hist, theta_1_hist, J_hist, 'r',
         label='Cost Minimization Path')
plt.plot(theta_0_hist[0], theta_1_hist[0], J_hist[0], 'ro',
         label='Cost Minimization Start')
plt.plot(theta_0_hist[theta_hist_end],
         theta_1_hist[theta_hist_end],
         J_hist[theta_hist_end], 'co', label='Cost Minimization Finish')

plt.xlabel(r'$\theta_0$', fontsize=24)
plt.ylabel(r'$\theta_1$', fontsize=24)
plt.title(r'Cost Function Minimization', fontsize=24)
plt.legend()
\end{minted}

\begin{figure}[h] %  figure placement: here, top, bottom, or page
	\centering
	\graphicspath{{./Figures/}} %Use this to import an image from a subfolder.
	\includegraphics[scale=0.5]{linreg_hw_2_4_plot_surface_plot.pdf}
%	\caption[]{One-vs-all classification. We're selecting one class to be our positive class, and the rest all become the negative class.}
	\label{linreg_hw_2_4_plot_surface_plot.pdf}
\end{figure}

\section{Python Labs: Coding Linear Regression}


%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
% CS624: Analysis of Algorithms
% Copyright 2015 Pejman Ghorbanzade <pejman@ghorbanzade.com>
% Creative Commons Attribution-ShareAlike 4.0 International License
% More info: https://github.com/ghorbanzade/beacon
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%

\section*{Question 8}

Assume you had some kind of super-hardware that, when given two lists of length $n$ that are sorted, merges them into one sorted list, and takes only $n^c$ steps where $c \geq 0$.

\begin{enumerate}[label=(\alph*)]
\item Write down a recursive algorithm that uses this hardware to sort lists of length $n$.
\item Write down a recurrence to describe the run time.
\item For what values of $c$ does this algorithm perform substantially better than $\mathcal{O}(n \log n)$? Why is it highly implausible that this kind of super-hardware could exist for these values of $c$?
\end{enumerate}

\subsection*{Solution}

\begin{enumerate}[label=(\alph*)]
\item Inspired by \textsc{MergeSort}, the proposed algorithm \textsc{Super-Merger-Sort} is given as Algorithm 3.

The call to the super-hardware is performed by calling the \textsc{Super-Merge} method.
Using this algorithm, we can sort an array of length $n$ by a top level call of \textsc{Super-Merger-Sort}(A, $1$, $n$).

\begin{algorithm}[H]
\caption{\textsc{Super-Merger-Sort}($A$, $p$, $r$)}
\begin{algorithmic}[1]
\If {$p < r$}
\State $q \leftarrow \lfloor \frac{p+r}{2} \rfloor$
\State \textsc{Super-Merger-Sort}($A$, $p$, $q$)
\State \textsc{Super-Merger-Sort}($A$, $q+1$, $r$)
\State \textsc{Super-Merge}($A$,$p$,$q$,$r$)
\EndIf
\end{algorithmic}
\end{algorithm}

\item As the original \textsc{MergeSort} algorithm has hardly been modified, the recursion tree will be the same as the recursion tree for \textsc{MergeSort} (Figure 3 of Lecture note 2), the only difference being that the constant runtime of each level of the tree would this time be $n^c$.

\item Based on this fact, the total runtime of the proposed \textsc{Super-Merger-Sort} algorithm with be $\mathcal{O}(n^{c}\log n)$ for sorting an array of length $n$.

The proposed algorithm would therefore perform better when $c < 1$.
This is of course highly implausible because any such super-hardware should at least read once all $n$ elements of the two merged lists to sort them and this itself is $\mathcal{O}(n)$.

\end{enumerate}


%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
% CS624: Analysis of Algorithms
% Copyright 2015 Pejman Ghorbanzade <pejman@ghorbanzade.com>
% Creative Commons Attribution-ShareAlike 4.0 International License
% More info: https://github.com/ghorbanzade/beacon
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%

\section*{Question 1}

Prove that the depth of a binary tree with $L$ leaves is $\Omega(\log L)$.

\subsection*{Solution}

We know that the binary tree is a special kind of tree with at most 2 children for each node.
Since we are interested in the lower bound of the depth of the tree, we will assume the case where every parent has two children which leads to a full binary tree.
We showed before that a full binary tree with depth $H$ has $2^H$ leaves.
Therefore the minimum depth of a binary tree with $L$ leaves will be $\lceil \log L \rceil$ leading to a lower bound of $\Omega (\log L)$ for the depth of the tree.


\lab{Algorithm}{Change of Basis}{Change of Basis}
\label{lab:ChangeBasis}

\objective{Understand how to change the basis of a set of points.}

\section*{Basis}

A basis for a vector space is a set of vectors such that every vector in the space can be expressed uniquely as a linear combination of the basis vectors. In this lab we will take the coordinates in 2-d space and do various affine transformations. For all these exercises we will use the points
\begin{lstlisting}
x=np.array([-1.5,-1.,-.5,0.,.5,1.,1.5,.75,-.75])
y=np.array([0.,-1.,-2.,-2.,-2.,-1.,0.,2.,2.])
\end{lstlisting}
Let be the matrix where each column in a point and M is the matrix that will change the basis. So $M*A$ will be the set of points in the new basis. In our cases the first row of $A$ is $x$ and the second row, $y$.

\section*{Strench}
To strech a set of points M will be a diagonal matrix where the value in each position is the streach in that direction

\begin{problem}
Write a function that will accepts a matrix of points and how much to strench them in each direction. Have the function plot the transformed points.
\end{problem}

\section*{Rotation}
To do a rotation clockwise of angle $\theta$ let
\[
M = \begin{pmatrix}
\cos(\theta) & -\sin(\theta) \\
\sin(\theta) & \cos(\theta)
\end{pmatrix}
\]

\begin{problem}
Write a function that will accepts a matrix of points and how many radians to rotate the points. Have the function plot the transformed points.
\end{problem}

\section*{Shift}
In order to shift a set of points you add to the coordinate how much you would shift it in that direction. You can use array brodcasting to do this.

\begin{problem}
Write a function that will accepts a matrix of points and how much to shift them in each direction. Have the function plot the transformed points.
\end{problem}

\section*{Combination}
Say you have points in a rotated basis and you want to stretch them along that basis. You left mutiply the strech matrix $S$ by the rotation matrix $R$. So $R*S*A$ will strench the points in the rotation in R.

\begin{problem}
Write a function that will accepts a matrix of points and strench, rotate and shift them.
\end{problem}

\section*{Images}

An Image is a 3d array where the first two demensions are the location and the 3rd demension is the RGB content. To apply the above tranformations one would need to moved the position of the RGB arrays to match where the new transformation would have them. Strenching is done by
 interpolation.
\begin{problem}
Write a function that will accepts an image and how many radians to rotate the image. The function will rotate the image around the center of the image and then show the rotated image. HINTS: Make an array of ones that is $1.5$ times greater than the max of the hieght and width of the original image. Change the x,y cordinates in the original image so the the center is $(0,0)$. Rotate the image and then center the image. You might need to use for loops. Allow your test cases to be small images.
\end{problem}


\section{Estimator} \label{sec:estimator}
Consider the commonly used Kullback-Leibler (KL) divergence between two
distributions $p(x)$ and $g(x)$:
\beq
    \kl{p}{q} = \int p(x)~\log\frac{p(x)}{q(x)}~{\rm d}x.
\eeq
The KL divergence between the posterior $p = p(\theta \given D, m)$ and prior
$\pi = p(\theta | m)$ is then
\beq
    \kl{p}{\pi} = \int p(\theta \given D, m)~\log\frac{p(\theta \given D, m)}{p(\theta | m)}~{\rm d}\theta.
\eeq
If we use Bayes' Theorem to substitute for $p(\theta \given D, m)$ in the
numerator, we can rewrite this as
\begin{align}
    \kl{p}{\pi} &= \int p(\theta \given D,
    m)~\log\frac{p(D \given \theta, m)}{p(D | m)}~{\rm d}\theta\\
        &= -\log p(D\given m) + \int p(\theta \given D, m)~\log p(D \given \theta, m)~{\rm d}\theta.
\end{align}
The first term of the right hand side is the log evidence and the second term
is the expectation value of the log likelihood under the posterior, $\big\langle
\log \mathcal{L} \big\rangle$. In standard Bayesian analyses in astronomy where
we use MCMC to sample the posterior, $\big\langle \log \mathcal{L} \big\rangle$
can be easily derived with these samples using Monte Carlo integration:
\beq
\big\langle \log \mathcal{L} \big \rangle \approx \frac{1}{n}
\sum\limits_{i=1}^{n} \log \mathcal{L}(\theta^{(i)})
\eeq
Therefore, if we can estimate the KL divergence between the posterior and prior
we can also estimate the evidence:
\beq \label{eq:estimator}
\log p(D\given m) = \big\langle \log \mathcal{L} \big\rangle - \kl{p(D \given \theta, m)}{\pi}.
\eeq

To estimate the divergence, we can make use of non-parametric divergence
estimators, which have been applied to Support Distribution Machines and
used in the machine learning and astronomical literature~\citep[\emph{e.g.}][]{poczos2011,
poczos2012, poczos2012a, xu2013, ntampaka2015, ntampaka2016, ravanbakhsh2017,
hahn2019}. These estimators allow us to estimate the divergence between
distributions $p$ and $q$, $D(p\,||\,q)$ using samples drawn from them. In
our case, $p$ and $q$ are the posterior and prior distributions. Again,
standard analyses already sample the posterir distribution; the prior
distribution is typcially straightforward to sample.

For the KL divergence, we use the $k$-Nearest Neighbor (NN) estimator presented in
\cite{wang2009}. Let $X_{1:n} = \{ X_1, ... X_n \}$ and $Y_{1:m} = \{ Y_1, ... Y_m \}$
be $n$ and $m$ samples drawn from the $p$ and $q$ $d$-dimensional distributions, respectively.
Then the divergence between $p$ and $q$ can be estimated as:
\beq \label{eq:div_est}
\kl{p}{q} \approx \widehat{D_{\rm KL}}(p\,||\,q) = \frac{d}{n}
\sum\limits_{i=1}^n \Big[\log \frac{\nu_{\ell_i}(i)}{\rho_{k_i}(i)} \Big] +
\frac{1}{n} \sum\limits_{i=1}^n\Big[\psi(\ell_i) - \psi(k_i)\Big] + \log
\frac{m}{n-1}.
\eeq
In the first term, $\rho_k(i)$ denotes the Euclidean distance between $X_i$ and
the $k^\mathrm{th}$-NN of $X_i$ in sample $\{X_j\}_{i\neq j}$. $\nu_\ell(i)$
denotes the Euclidean distance between $X_i$ and the $\ell^\mathrm{th}$ NN of
$X_i$ in the sample $Y_{1:m}$. In the second term, $\psi$ is the Digamma function:
$\psi(k) = \Gamma'(k)/\Gamma(k)$.
This term corrects for the estimation bias that comes from nonuniformity of
the distribution near each sample point and guarantees that the estimator is
asymptotically unbiased. Furthermore, while some $k$-NN estimators require
chome choice in $k$ and $\ell$~\citep[\emph{e.g.}][]{poczos2012}, the
\cite{wang2009} estimator adaptively determines $\ell_i$ and $k_i$ as the number
of samples $X_{1:n}$ and $Y_{1:m}$), respectively, contained in $B(X_i, \epsilon(i))$,
a Euclidean ball centered at $X_i$ with radius $\epsilon(i)$.
\beq
\epsilon(i) = \max(\rho(i), \nu(i))
\eeq
where
\begin{align}
    \rho(i) &= \min_{j \neq i} || X_i - X_j|| \\
    \nu(i) &= \min_{j \neq i} || X_i - Y_j||.
\end{align}
For further details on the estimator and proofs that the estimator is
asymptotically unbiased and mean-square consistent we refer readers to
\cite{wang2009}.


\subsection{Matrix exponentials}

The exponential of a complex number is defined as:

\(e^x=\sum \dfrac{1}{j!}x^j\)

We can extend this definition to matrices.

\(e^X:=\sum \dfrac{1}{j!}X^j\)

The dimension of a matrix and its exponential are the same.


\section{Line search and gradient descent method}
\subsection{Gradient descent method}
For simplicity, let us just consider a general optimization problem
\begin{equation}\label{optmodel}
\min_{x\in \mathbb{R}^n } f(x).
\end{equation}

\begin{figure}[H]
	\centering
	\centering\includegraphics[height = 5cm, width=7cm]{figures/diag_GD.png}
\end{figure}

\paragraph{A general approach:  line search method}
Given any initial guess $x_1$, the line search method uses the following algorithm
$$
\eta_t=\argmin_{\eta\in \mathbb{R}^1} f(x_t - \eta p_t)\qquad \mbox{\scriptsize(1D minimization problem)}
$$
 to produce $\{ x_{t}\}_{t=1}^{\infty}$
\begin{equation}\label{line-search}
x_{t+1} = x_{t} - \eta_t p_t.
\end{equation}
Here $\eta_t$ is called the step size in optimization and also learning
rate in machine learning, $p_t$ is called the descent direction, which
is the critical component of this algorithm. And $x_t$ tends to
$$
x^*=\argmin_{x\in \mathbb{R}^n} f(x) \iff f(x^*)=\min_{x\in \mathbb{R}^n} f(x)
$$
as $t$ tends to infinity. There is a series of optimization
algorithms which follow the above form just using different choices of $p_t$.

Then, the next natural question is what a good choice of $p_t$ is?
We have the following theorem to show why gradient direction is a good choice for $p_t$.
\begin{lemma}
Given  $x \in \mathbb{R}^n$, if $\nabla f(x)\neq 0$, the fast descent direction of $f$ at $x$ is the negative gradient direction, namely
\begin{equation}\label{key}
-\frac{\nabla f(x)}{\|\nabla f(x)\|} = \mathop{\arg\min}_{ p \in \mathbb{R}^n, \|p\|=1} \left. \frac{\partial f(x + \eta p)}{\partial \eta} \right|_{\eta=0}.
\end{equation}
It means that $f(x)$ decreases most rapidly along the negative gradient direction.
\end{lemma}

\begin{proof}
Let $p$ be a direction in $\mathbb{R}^{n},\|p\|=1$. Consider the local decrease of the function $f(\cdot)$ along direction $p$
$$
\Delta(p)=\lim _{\eta \downarrow 0} \frac{1}{\eta}\left(f(x+\eta p)-f(x)\right)=\left. \frac{\partial f(x + \eta p)}{\partial \eta} \right|_{\eta=0}.
$$
Note that
\begin{equation}
\begin{split}
\left. \frac{\partial f(x + \eta p)}{\partial \eta} \right|_{\eta=0}=\sum_{i=1}^n\left. \frac{\partial f}{\partial x_i}(x + \eta p)p_i \right|_{\eta=0} =(\nabla f, p),
\end{split}
\end{equation}
which means that
$$
f(x+\eta p)-f(x)=\eta(\nabla f(x), p)+o(\eta) .
$$
Therefore
$$
\Delta(p)=(\nabla f(x), p).
$$
Using the Cauchy-Schwarz inequality
$
-\|x\| \cdot\|y\| \leq( x, y) \leq\|x\| \cdot\|y\|,
$
we obtain
$$
-\|\nabla f(x)\| \le (\nabla f(x), p)\le \|\nabla f(x)\| .
$$
Let us take
$$
\bar{p}=-\nabla f(x) /\|\nabla f(x)\|.
$$
Then
$$
\Delta(\bar{p})=-(\nabla f(x), \nabla f(x)) /\|\nabla f(x)\|=-\|\nabla f(x)\|.
$$
The direction $-\nabla f(x)$ (the antigradient) is the direction of the fastest local decrease of the function $f(\cdot)$ at point $x.$
\end{proof}

Here is a simple diagram for this property.
	\begin{figure}[H]
		\centering{\includegraphics[width=5cm]{figures/gradientfast.png}}
		\caption{Negative Gradient Direction: $ x_t$ is current point, $ p_t$ is the negative gradient of $ x_t$, i.e., $- \nabla f( x_t)$.}
		\label{functiongradient}
	\end{figure}

Since at each point,  $f(x)$ decreases most rapidly along the negative
gradient direction, it is then natural to choose the search direction
in \eqref{line-search} in the negative gradient direction and the
resulting algorithm is the so-called gradient descent method.
\begin{algorithm}[H]
\caption{Gradient Descent Method}
\label{alg:LR-R}
Given the initial guess $x_0$, learning rate $\eta_t>0$

	{\bf For} t=1,2,$\cdots$, \\
	\begin{equation}\label{key}
	x_{t+1} =  x_{t} - \eta_{t} \nabla f({x}_{t}),
\end{equation}
\end{algorithm}

In practice, we need a ``stopping criterion'' that determines when the above gradient descent
method to stop.  One possibility is
\begin{quote}
	{\bf While} $S(x_t; f) = \|\nabla f(x_t)\|\le \epsilon$ or $t \ge T$
\end{quote}
for some small tolerance $\epsilon>0$ or maximal number of iterations
$T$.   In general, a good stopping criterion is hard to come by and it
is a subject that has called a lot of research in optimization for
machine learning.

In the gradient method, the scalar factors for the gradients, $\eta_{t},$ are called the step sizes. Of course, they must be positive. There are many variants of the gradient method, which differ one from another by the step-size strategy. Let us consider the most important examples.
\begin{enumerate}
\item The sequence $\left\{\eta_t\right\}_{t=0}^{\infty}$ is chosen in advance. For example,
(constant step)
$$
\eta_t=\frac{\eta}{\sqrt{t+1}};
$$
\item Full relaxation:
$$
\eta_t=\arg \min _{\eta \geq 0} f\left(x_t-\eta \nabla f\left(x_t\right)\right);
$$
\item The Armijo rule: Find $x_{t+1}=x_t-\eta \nabla f\left(x_t\right)$ with $\eta>0$ such that
$$
\alpha\left(\nabla f\left(x_t\right), x_t-x_{t+1}\right) \leq f\left(x_t\right)-f\left(x_{t+1}\right),
$$
$$
\beta\left(\nabla f\left(x_t\right), x_t-x_{t+1}\right) \geq f\left(x_t\right)-f\left(x_{t+1}\right),
$$
where $0<\alpha<\beta<1$ are some fixed parameters.
\end{enumerate}
Comparing these strategies, we see that
\begin{enumerate}
\item The first strategy is the simplest one. It is often used in the context of convex optimization. In this framework, the behavior of functions is much more predictable than in the general nonlinear case.
\item The second strategy is completely theoretical. It is never used in practice since even in one-dimensional case we cannot find the exact minimum in finite time.
\item The third strategy is used in the majority of practical algorithms. It has the following geometric interpretation. Let us fix $x \in \mathbb{R}^{n}$ assuming that $\nabla f(x) \neq 0$. Consider the following function of one variable:
$$
\phi (\eta)=f(x-\eta \nabla f(x)),\quad \eta\ge0.
$$
Then the step-size values acceptable for this strategy belong to the part of the graph of $\phi$ which is located between two linear functions:
$$
\phi_{1}(\eta)=f(x)-\alpha \eta\|\nabla f(x)\|^{2}, \quad \phi_{2}(\eta)=f(x)-\beta \eta\|\nabla f(x)\|^{2}
$$
Note that $\phi(0)=\phi_{1}(0)=\phi_{2}(0)$ and $\phi^{\prime}(0)<\phi_{2}^{\prime}(0)<\phi_{1}^{\prime}(0)<0 .$ Therefore, the
acceptable values exist unless $\phi(\cdot)$ is not bounded below. There are several very fast one-dimensional procedures for finding a point satisfying the Armijo conditions. However, their detailed description is not important for us now.
\end{enumerate}


\paragraph{Tutte-Berge formula}
The theorem states that the size of a maximum matching of a graph $G=(V,E)$ equals

$${\frac {1}{2}}\min _{U\subseteq V}\left(|U|-\operatorname {odd} (G-U)+|V|\right)\,,$$
where $\operatorname {odd} (H)$ counts how many of the connected components of the graph $H$ have an odd number of vertices.

\paragraph{Tutte theorem}

A graph, $G = (V, E)$, has a perfect matching if and only if for every subset $U$ of $V$, the subgraph induced by $V-U$ has at most $|U|$ connected components with an odd number of vertices.

\paragraph{Hall's marriage theorem}

A family $S$ of finite sets has a transversal if and only if $S$ satisfies the marriage condition.


% Chapter I: Preliminaries: Set theory and categories
\chapter{Preliminaries: Set theory and categories}

\section{Naive set theory}
\begin{xca}
Let $U = \{ x \mid x \not\in x \}$. Then, $U \not\in U \iff U \in U$, a
contradiction. This is Russell's paradox. Either we assume the \emph{set of all
sets} doesn't exist, or we need to give up the axiom of \emph{unrestricted
comprehension} in set theory.
\end{xca}

\begin{xca}
Suppose $\sim$ is an equivalence relation on a set $S$. For every element $a \in
S$, define the \emph{equivalence class} of $a$ (with respect to $\sim$) by
\begin{center}
$[a]_{\sim} := \{ b \in S \mid b \sim a \}$.
\end{center}
Then, we note that due to \emph{reflexivity}, the equivalence class
$[a]_{\sim}$ of every element $a \in S$ contains $a$, and hence, is nonempty.
Also, $[a]_{\sim} \subset S$, and therefore, $\bigcup_{a \in S} [a]_{\sim} =
S$. Finally, we show the equivalence classes are mutually disjoint. Indeed, for
any two elements $a, b \in S$, if $[a]_{\sim}$ and $[b]_{\sim}$ are disjoint,
then there is nothing to prove. So, suppose $[a]_{\sim} \cap [b]_{\sim}$ is
nonempty. Then, there exists some $c \in S$ that belongs to such an
intersection. Thus, $c \sim a$ and $c \sim b$. By symmetry, $a \sim c$, and
thus, by transitivity, $a \sim b$, which by symmetry again, implies $b \sim a$.
Therefore, for all $x \in [a]_{\sim}$, we have $x \sim a$, and since $a \sim
b$, by transitivity, $x \sim b$, which implies $x \in [b]_{\sim}$, from which
we conclude $[a]_{\sim} \subset [b]_{\sim}$. We can similarly show $[b]_{\sim}
\subset [a]_{\sim}$. Hence, $[a]_{\sim} = [b]_{\sim}$. This establishes
equivalence classes are mutually disjoint. Hence, the set $\mathscr{P}_{\sim}$
of equivalence classes of $S$ is indeed a partition of $S$.
\end{xca}

\begin{xca}
Suppose $\mathscr{P}$ is a partition on a set $S$. Define a relation $\sim$ on
$S$ as follows: For any two elements $a, b \in S$, $a \sim b$ iff $a$ and $b$
belong to the same set in the partition. Then, it is easy to check $\sim$ is
indeed an equivalence relation on $S$. $\mathscr{P}$ is, therefore, the
corresponding partition of the aforesaid equivalence relation, and we are done.
\end{xca}

\begin{xca}
Note the set of equivalence relations on a set $S$ are in a one-to-one
correspondence with the set of partitions of $S$. Thus, the number of different
equivalence relations that may be defined on $S = \{ 1, 2, 3\}$ equals the
number of partitions of $S$, and this number equals $5$, since the partitions
of $S$ are
\begin{center}
    $\{ \{ 1 \}, \{ 2 \}, \{ 3 \} \}, \{ \{ 1, 2 \}, \{ 3 \} \}, \{ \{ 1, 3 \},
    \{ 2 \} \}, \{ \{ 2, 3 \}, \{ 1 \} \}, \{ \{ 1, 2, 3 \} \}$.
\end{center}
The above partitions are also written as $1|2|3, 12|3, 13|2, 23|1, 123$.
\end{xca}

\begin{xca}
An example of a relation $R$ (defined on a set $S$) that is reflexive and
symmetric but not transitive is the following:
\begin{center}
$R = \{ (1,1), (2,2), (3,3), (1,2), (2,1), (2,3), (3,2) \}$, where
$S = \{ 1,2,3 \}$.
\end{center}
\end{xca}

\begin{xca}
Define a relation $\sim$ on the set $\R$ of real numbers by setting
\begin{center}
 $a \sim b \iff b - a \in \mathbb{Z}$.
\end{center}
We claim $\sim$ is an equivalence relation. To that end, note, for all $a \in
\R$, we have $a \sim a$, since $a - a = 0 \in \Z$. Therefore, $\sim$ is
reflexive. Also, if $a \sim b$, then $b - a \in \Z$, which implies $a - b \in
\Z$, and thus, $b \sim a$. Therefore, $\sim$ is symmetric. Finally, suppose $a
\sim b$ and $b \sim c$. Then, $b - a, c - b \in \Z$, and thus, $c - a = (c - b)
+ (b - a) \in \Z$. Thus, $\sim$ is transitive. Therefore, $\sim$ is an
equivalence relation on $\R$.

(Description of $\sim$) Note all reals that have the same decimal expansion
belong to the same equivalence class under $\sim$. Thus, $[0]_{\sim} = \Z$, and
for any $0 < \alpha < 1, [\alpha]_{\sim} = \{ n + \alpha \mid n \in \Z \}$. This
takes care of all the reals, since each real can always be written as $n +
\alpha$, for some $n \in \Z$ and $0 < \alpha < 1$. Therefore, a `compelling'
description for $\R/{\sim}$ is the unit interval $[0,1]$, such that the
endpoints, $0$ and $1$, are `glued' together. In other words, it is a `loop'
or a 1-sphere.\\

Define a relation $\approx$ on the plane $\R \times \R$ as follows:
\begin{center}
$(a_1, a_2) \approx (b_1, b_2) \iff b_1 - a_1 \in \Z \text{ and }
b_2 - a_2 \in \Z$.
\end{center}
Then, just as above, it is easy to show $\approx$ defines an equivalence
relation on $\R \times \R$. We note $[(0,0)]_{\sim} = \{ (m,n) \mid m,n \in \Z
\}$, and for any $0 < \alpha, \beta < 1, [(\alpha, \beta)]_{\sim} = \{ (m +
\alpha, n + \beta) \mid m,n \in \Z \}$. Thus, a `compelling' description of $\R
\times \R/{\approx}$ is the unit square $[0,1] \times [0,1]$ with the four
corners joined together, so that it forms a 2-sphere.
\end{xca}

\section{Functions between sets}
\begin{xca}
We claim the number of bijections from a set $S$ with $n$ elements to itself is
$n!$. To begin with, any element in $S$ can be mapped to any of the $n$ possible
elements in $S$. Then, the next element in $S$ can be mapped to any of the
remaining $n-1$ elements in $S$, and so on, with the last element in $S$ being
mapped to the last remaining element in $S$. Thus, the number of bijections
equals $n \cdot (n-1) \cdot \ldots \cdot 1 = n!$, which proves our claim.
\end{xca}

\begin{xca}
Assume $A \neq \emptyset$, and let $f: A \to B$ be a function. We claim $f$ has
a right inverse iff it is surjective.

($\impliedby$) Suppose $f$ has a right inverse, $g: B \to A$, say. Then,
$f \circ g = 1_B$. Thus, for all $b \in B$, $b = 1_B(b) = (f \circ g)(b) =
f(g(b)) = f(a)$, where $g(b) = a \in A$. This shows $f$ is surjective.

($\implies$) Suppose $f$ is surjective. Then, for any $b \in B$, the fiber of
$f$ over $b$ is nonempty. Thus, $\{ f^{-1}(b) \}_{b \in B}$ is a family of
nonempty sets, and therefore, using the \emph{axiom of choice}, we can construct
a function $g : B \to A$ as follows: For all $b \in B$, $g(b) = a$ for some
$a \in f^{-1}(b)$. Hence, for all $b \in B, (f \circ g) (b) = f(g(b)) = f(a) =
b = 1_{B} (b)$, and so, $f \circ g = 1_B$. This establishes $g$ is the right
inverse of $f$, and we are done.
\end{xca}

\begin{xca}
Suppose $f : A \to B$ is a bijection. Then, $f$ has an inverse $f^{-1}: B \to A$
such that $f^{-1} \circ f = 1_A$ and $f \circ f^{-1} = 1_B$. Clearly, $f$ is an
inverse of $f^{-1}$, showing $f^{-1}$ is also a bijection.

Suppose $f: A \to B$ and $g: B \to C$ are bijections. We claim $g \circ f: A \to
C$ is also a bijection. To that end, we show $f^{-1} \circ g^{-1}: C \to A$ is
the inverse of $g \circ f$. Indeed, $(g \circ f) \circ (f^{-1} \circ g^{-1}) =
g \circ (f \circ f^{-1}) \circ g^{-1} = g \circ 1_B \circ g^{-1} = g \circ
g^{-1} = 1_C$. And, $(f^{-1} \circ g^{-1}) \circ (g \circ f) = f^{-1} \circ
(g^{-1} \circ g) \circ f = f^{-1} \circ 1_B \circ f = f^{-1} \circ f = 1_A$,
and we are done.
\end{xca}

\begin{xca}
We show `isomorphism' is an equivalence relation on any set of sets.\\
(Reflexivity) For all sets $A, 1_A : A \to A$ is a natural bijection, and thus,
$A \cong A$.\\
(Symmetry) Suppose $A \cong B$ for any two sets $A, B$. Then,
there exists a bijection $f : A \to B$, such that its inverse $f^{-1} : B \to
A$ is also a bijection (as shown in the above exercise.) Thus, $B \cong A$.\\
(Transitivity) Finally, suppose for any three sets, $A, B$ and $C$, $A \cong B$
and $B \cong C$, with $f : A \to B$ and $g: B \to C$ as bijections. Then, from
the previous exercise, $g \circ f : A \to C$ is also a bijection, and thus $A
\cong C$.\\
Thus, our original claim is established.
\end{xca}

\begin{xca}
(\emph{\textbf{Epimorphism}}) A function $f : A \to B$ is an \emph{epimorphism}
(or \emph{epi}) if the following holds: For all sets $Z$ and all functions
$\alpha ', \alpha '' : B \to Z$,
\begin{center}
$\alpha ' \circ f = \alpha '' \circ f \Longrightarrow \alpha ' = \alpha ''$.
\end{center}
In other words, an epimorphism $f$ is \emph{right cancellative}.\\

\emph{Proposition}: A function is surjective iff it is an epimorphism.

\emph{Proof}. ($\implies$) Suppose $f: A \to B$ is an epimorphism. Assume, for
the sake of contradiction, $f$ is \emph{not} surjective. Then, there exists an
element $b_0 \in B$, such that, for all $a \in A$, $f(a) \neq b_0$. We now
construct two distinct functions $\alpha', \alpha'' : B \to \{ 0, 1 \}$ as
follows:
\begin{center}
\[
\alpha'(b) = 0
\]

\[
\alpha''(b) =
    \begin{cases}
        0 & \text{if } b \neq b_0 \\
        1 & \text{if } b = b_0
    \end{cases}
\]
\end{center}
Then, it is easy to check that, for all $a \in A$,
$(\alpha'' \circ f)(a) = \alpha''(f(a)) = 0 = \alpha'(f(a)) = (\alpha' \circ
f)(a)$, which implies $\alpha' \circ f = \alpha'' \circ f$. However, $\alpha'
\neq \alpha''$, which contradicts our assumption that $f$ is an epimorphism.
Hence, we conclude $f$ is surjective.

($\impliedby$) Suppose $f: A \to B$ is surjective. Then, it has a right inverse
$g: B \to A$ such that $f \circ g = 1_B$. Now, assume, for any set $Z$ and any
two functions $\alpha', \alpha'': B \to Z$, $\alpha' \circ f = \alpha'' \circ
f$. Then, $\alpha' = \alpha' \circ 1_B = \alpha' \circ (f \circ g) = (\alpha'
\circ f) \circ g = (\alpha'' \circ f) \circ g = \alpha'' \circ (f \circ g) =
\alpha'' \circ 1_B = \alpha''$, thus proving $f$ is an epimorphism.
\end{xca}

\begin{xca}
Any function $f: A \to B$ determines a section $g: A \to A \times B$ of $\pi_A:
A \times B \to A$ by defining $g$ as follows:
\begin{center}
    $a \mapsto (a, f(a))$
\end{center}
Then, for all $a \in A$, $(\pi_A \circ g)(a) = \pi_A(g(a)) = \pi_A(a, f(a)) =
a = 1_A(a)$, which implies $\pi_A \circ g = 1_A$, thereby showing $g$ as
defined above is indeed a section of $\pi_A$.
\end{xca}

\begin{xca}
Let $f: A \to B$ by any function. We show the graph $\Gamma_f$ of $f$ is
isomorphic to $A$. First, recall the definition of $\Gamma_f$:
\begin{center}
    $\Gamma_f := \{ (a, b) \in (A, B) \mid b = f(a) \} \subseteq A \times B$.
\end{center}
We define a function $g: A \to \Gamma_f$ by
\begin{center}
    $a \mapsto (a, f(a))$.
\end{center}
Then, for any $(a, b) \in \Gamma_f$, we have $b = f(a)$, which implies $g(a) =
(a, f(a)) = (a, b)$, proving $g$ is surjective. Next, for any $a', a'' \in A$,
suppose $g(a') = g(a'')$. This implies $(a', f(a')) = (a'', f(a''))$, which
implies $a' = a''$, thus proving $g$ is injective. Hence, $g$ is an isomorphism,
and so, $A \cong \Gamma_f$.
\end{xca}

\begin{xca}
We describe below explicitly all the terms in the canonical decomposition of the
function $f: \R \to \C$ defined by
\begin{center}
    $r \mapsto e^{2\pi ir}$.
\end{center}
Note $f$ determines an equivalence relation $\sim$ on $\R$ as follows: For all
$r', r'' \in \R$,
\begin{center}
    $r' \sim r'' \iff f(r') = f(r'')$.
\end{center}
Now, $f(r') = f(r'')$ whenever $e^{i2\pi r'} = e^{i2\pi r''}$, \emph{i.e.}
$e^{i2\pi(r' - r'')} = e^0$; that is, $2\pi(r' - r'') = 2\pi k$, where $k \in
\Z$. In other words, $f(r') = f(r'')$ iff $r' - r'' = k$, where $k \in \Z$.
This implies
\begin{center}
    $r' \sim r'' \iff r' - r'' \in \Z$.
\end{center}
The above equivalence relation matches the one stated in Exercise 1.6 of
Chapter 1. And, from the solution to the aforesaid exercise, we note that all
equivalence classes are of the form $[\alpha]_{\sim}$, where $\alpha \in [0,
1)$. Therefore, the isomorphism $\tilde{f}: \R/{\sim} \xrightarrow{\sim}
\opn{im} f$ is defined by
\begin{center}
    $\tilde{f}(\alpha) = e^{2\pi i\alpha}, \, \alpha \in [0, 1)$.
\end{center}
Also, note $\opn{im} f$ is the unit circle on the complex plane. So, the entire
decomposition of $f: \R \to \C$ is as shown below:
\[
\begin{tikzcd}
    \R \arrow[r, two heads]
       \arrow[rrr, bend left, "f"]
       & \R/{\sim} \arrow[r, "\sim"]{r}[swap]{\tilde{f}}
                   & \opn{im} f \arrow[r, hook]
                                & \C
\end{tikzcd},
\]
where $\R/{\sim}$ is a `loop' (1-sphere.)
\end{xca}

\begin{xca}
Suppose $A' \cong A''$ and $B' \cong B''$, and further $A' \cap B' = \emptyset$
and $A'' \cap B'' = \emptyset$. Then, there exist isomorphisms $f: A' \to A''$
and $g: B' \to B''$. Define a piecewise function $h: A' \cup B' \to A'' \cup
 B''$ as follows:
\[
    h(x) =
    \begin{cases}
        f(x) & \text{if } x \in A' \\
        g(x) & \text{if } x \in B'
    \end{cases}
\]
We claim $h$ is an isomorphism. Indeed, let $y \in A'' \cup B''$. Since $A''$
and $B''$ are disjoint, there are two cases to consider. Either, $y \in A''$,
in which case, since $f$ is surjective there exists some $x_1 \in A'$ such that
$f(x_1) = y$, which implies $h(x_1) = y$. Or, $y \in B''$, in which case, since
$g$ is surjective there exists some $x_2 \in B'$ such that $g(x_2) = y$, which
implies $h(x_2) = y$. Thus, in either case, $h$ maps some $x \in A' \cup B'$ to
$y \in A'' \cup B''$, thus showing $h$ is surjective.\\
Now, suppose $h(x_1) = h(x_2)$ for any $x_1, x_2 \in A' \cup B'$. There are
three cases to consider. First, if both $x_1, x_2 \in A'$, then $f(x_1) =
f(x_2)$, and since $f$ is injective, $x_1 = x_2$. Second, if both $x_1, x_2 \in
B'$, then $g(x_1) = g(x_2)$, and since $g$ is injective, $x_1 = x_2$. Finally,
without loss of generality, we can assume $x_1 \in A'$ and $x_2 \in B'$, which
implies $f(x_1) = g(x_2)$, but this is impossible, since $f(x_1) \in A''$ and
$f(x_2) \in B''$, but $A'', B''$ are disjoint. Thus, we conclude $h$ is
injective.\\
Hence, $h$ is an isomorphism.

The above demonstrates the \emph{disjoint union} operation $A \amalg B$ for any
two sets $A$ and $B$ is, indeed, well-defined \emph{up to isomorphism}.
\end{xca}

\begin{xca}
Suppose $A$ and $B$ are finite sets. Then, any one of the $|A|$ elements in $A$
can be mapped to any one of the $|B|$ elements in $B$. Thus, there are
exactly $|B|^{|A|}$ ways of constructing a function $A \to B$. Hence, $|B^A| =
|B|^{|A|}$.
\end{xca}

\begin{xca}
Let $A$ be a set, and let $2^A$ denote the set of functions from $A$ to $2 = \{
0, 1\}$. Then, define a mapping $f: 2^A \to \mc{P}A$, where $\mc{P}A$ is the
powerset of $A$, as follows: Any function $A \to 2$ is mapped to the subset
consisting of all those elements of $A$ that are mapped to $1$ under such a
function. Then, it is easy to check $f$ is both surjective and injective, and
hence, a bijection.
\end{xca}

\section{Categories}
\begin{xca}
Let $\mc{C}$ be a category. Consider a structure $\mc{C}^{op}$ with
\begin{itemize}
    \item $\opn{Obj}(\mc{C}^{op}) := \opn{Obj}(\mc{C})$;
    \item for $A, B$ objects of $\mc{C}^{op}$ (hence objects of $\mc{C}$),
    $\Hom_{\mc{C}^{op}}(A, B) := \Hom_{\mc{C}}(B, A)$.
\end{itemize}
We can make $\mc{C}^{op}$ into a category as follows.
\begin{itemize}
    \item For every object $A$ of $\mc{C}^{op}$, its identity morphism in
    $\mc{C}^{op}$ is defined as the identity morphism $1_A$ in $\mc{C}$.
    \item For any two morphisms $f \in \Hom_{\mc{C}^{op}}(A, B)$ and $g \in
    \Hom_{\mc{C}^{op}}(B, C)$, the composite $g \circ f \in
    \Hom_{\mc{C}^{op}}(A, C)$ is defined as the composite $f \circ g$ in
    $\mc{C}$.
    \item The associativity of composition in $\mc{C}^{op}$ is a direct
    consequence of associativity of composition in $\mc{C}$.
    \item The fact that identity morphisms in $\mc{C}^{op}$ are identities with
    respect to composition is a direct consequence of identity morphisms in
    $\mc{C}$ being identities with respect to composition.
\end{itemize}
The above construction, thus, shows $\mc{C}^{op}$ is a category. This
\emph{\textbf{opposite category}} is simply obtained by `reversing all the
arrows' in $\mc{C}$.
\end{xca}

\begin{xca}
If $A$ is a finite set, then $|\End_{\catname{Set}}(A)| = |A^A| = |A|^{|A|}$.
\end{xca}

\begin{xca}
In Example 3.3, to say $1_a = (a, a)$ is the identity of element $a \in S$ with
respect to composition means that any $f = (a, b) \in \Hom(a, b)$ remains
unchanged when composed with $1_a$ or $1_b$ in the appropriate manner. In the
first case, we have $a \sim a$ and $a \sim b$, and thus, by transitivity, $a
\sim b$, from which we conclude $f 1_a = (a, b) = f$. Similarly, in the second
case, we conclude $1_b f = f$.
\end{xca}

\begin{xca}
Suppose we can define a category in the style of Example 3.3 using the relation
$<$ on the set $\Z$. Then, any element (object) $n \in \Z$ will have an identity
morphism $1_n = (n, n)$ that would imply $n < n$, which is impossible. Hence,
we can't define such a category using the relation $<$ on the set $\Z$.
\end{xca}

\begin{xca}
Example 3.4 is an instance of the categories considered in Example 3.3 in the
sense that set inclusion $\subseteq$ (morphisms) between subsets of $S$
(objects) satisfies the reflexivity and transitivity axioms for $\sim$.
\end{xca}

\begin{xca}
(Assuming some familiarity with linear algebra.) Define a category $\mathsf{V}$
by taking $\opn{Obj}(\mathsf{V}) = \N$ and letting $\Hom_{\mathsf{V}}(n, m) =$
the set of $m \times n$ matrices with real entries, for all $n, m \in \N$.
Composition is defined as products of matrices. Note, for all $n \in \N$,
\[
\Hom_{\mathsf{V}}(n, 0) = \emptyset = \Hom_{\mathsf{V}}(0, n).
\]
This category does \emph{not} `feel' familiar, since it is quite unlike any of
the other categories we have seen before. Objects of $\mathsf{V}$ are not like
the usual sets or elements we have encountered so far, and its morphisms are
matrices which are not the typical set-theoretic functions.
\end{xca}

\begin{xca}
Let $\mc{C}$ be a category, and $A$ an object of $\mc{C}$. We define a category
$\mc{C}^A$ as follows:
\begin{itemize}
    \item $\opn{Obj}(\mc{C}^A) :=$ collection of all morphisms from $A$ to any
    object of $\mc{C}$; that is, an object of $\mc{C}^A$ is a morphism $f \in
    \Hom_{\mc{C}}(A, B)$ for some object $B$ of $\mc{C}$.
    \item For any two objects $f_1: A \to B$ and $f_2: A \to C$ of $\mc{C}^A$,
    morphisms $f_1 \to f_2$ are defined to be commutative diagrams
    \[
    \begin{tikzcd}
        & A \arrow[dl, swap, "f_1"]
            \arrow[dr, "f_2"]
            & \\
            B \arrow[rr, swap, "\sigma"]
              & & C
    \end{tikzcd}
    \]
    in the `ambient' category $\mc{C}$. That is, morphisms $f_1 \to f_2$
    correspond precisely to those morphisms $\sigma: B \to C$ in $\mc{C}$ such
    that $\sigma \circ f_1 = f_2$.
\end{itemize}

(Composition) Two morphisms $f_1 \to f_2$ and $f_2 \to f_3$ in $\mc{C}^A$
correspond to putting two commutative diagrams side-by-side:
\[
\begin{tikzcd}
    & A \arrow[dl, swap, "f_1"]
        \arrow[d, swap, "f_2"]
        \arrow[dr, "f_3"]
        & \\
    B \arrow[r, swap, "\sigma"]
      & C \arrow[r, swap, "\tau"]
          & D
\end{tikzcd}
\]
Then, the diagram obtained by removing the central arrow, \emph{i.e.}
\[
\begin{tikzcd}
    & A \arrow[dl, swap, "f_1"]
        \arrow[dr, "f_3"]
        & \\
    B \arrow[rr, swap, "\tau \circ \sigma"]
      & & D
\end{tikzcd}
\]
also commutes, since $\mc{C}$ is a category.
\end{xca}

\begin{xca}
A \emph{subcategory} $\mc{C}'$ of a category $\mc{C}$ consists of a collection
of objects of $\mc{C}$, with morphisms $\Hom_{\mc{C}'}(A, B) \subseteq
\Hom_{\mc{C}}(A, B)$ for all objects $A, B$ in $\opn{Obj}(\mc{C}')$, such that
identities and compositions in $\mc{C}$ make $\mc{C}'$ into a category. A
subcategory $\mc{C}'$ is \emph{full} if $\Hom_{\mc{C}'}(A, B) =
\Hom_{\mc{C}}(A, B)$ for all $A, B$ in $\opn{Obj}(\mc{C}')$.

We can construct a category of \emph{infinite sets} as follows. The objects of
this category are all infinite sets from $\catname{Set}$, and morphisms from an
infinite set $A$ to another infinite set $B$ are just the usual set-functions.
Then, it is clear that this category is indeed a full subcategory of
$\catname{Set}$.
\end{xca}

\begin{xca}
% TODO: Chapter I - Exercise 3.9
\end{xca}

\begin{xca}
Given any set $A$ in $\catname{Set}$, the subsets (subobjects) of $A$ are in a
one-to-one correspondence with functions $A \to \{ 0, 1\}$. Thus, $\Omega =
\{ 0, 1 \}$ is a \emph{subobject classifier} of $\catname{Set}$.
\end{xca}

\begin{xca}
The category $\mc{C}^{A, B}$ consists of the following pieces of data:
\begin{itemize}
    \item $\opn{Obj}(\mc{C}^{A, B})$ = diagrams
    \[
    \begin{tikzcd}
        A \arrow[rd, "f"] & \\
        & Z \\
        B \arrow[ru, swap, "g"]
    \end{tikzcd}
    \]
    \item morphisms $(f_1: A \to Z_1, g_1: B \to Z_1) \to (f_2: A \to Z_2,
    g_2: B \to Z_2)$ are commutative diagrams
    \[
    \begin{tikzcd}
        A \arrow[rd, "f_1"]
          \arrow[rrd, bend left, "f_2"]
          & & \\
        & Z_1 \arrow[r, swap, "\sigma"]
              & Z_2 \\
        B \arrow[ru, swap, "g_1"]
          \arrow[rru, swap, bend right, "g_2"]
          & &
    \end{tikzcd}
    \]
    \item (Composition) Given commutative diagrams
    \[
    \begin{tikzcd}
        A \arrow[rd, "f_1"]
          \arrow[rrd, bend left, "f_2"]
          \arrow[rrrd, bend left, "f_3"]
          & & & \\
        & Z_1 \arrow[r, swap, "\sigma"]
              & Z_2 \arrow[r, swap, "\tau"]
              & Z_3 \\
        B \arrow[ru, swap, "g_1"]
          \arrow[rru, swap, bend right, "g_2"]
          \arrow[rrru, swap, bend right, "g_3"]
          & & &
    \end{tikzcd},
    \]
    the following diagram commutes:
    \[
    \begin{tikzcd}
        A \arrow[rd, "f_1"]
        \arrow[rrd, bend left, "f_3"]
        & & \\
        & Z_1 \arrow[r, swap, "\tau \circ \sigma"]
        & Z_2 \\
        B \arrow[ru, swap, "g_1"]
        \arrow[rru, swap, bend right, "g_3"]
        & &
    \end{tikzcd}
    \]
    \item (Identities) The identity morphism of
    \[
    \begin{tikzcd}
        A \arrow[rd, "f"] & \\
          & Z \\
        B \arrow[ru, swap, "g"]
    \end{tikzcd}
    \]
    is $Z \xrightarrow{1_Z} Z$, thus making the following diagram commute:
    \[
    \begin{tikzcd}
        A \arrow[rd, "f_1"]
          \arrow[rrd, bend left, "f_1"]
          & & \\
        & Z_1 \arrow[r, swap, "1_Z"]
              & Z_1 \\
        B \arrow[ru, swap, "f_2"]
          \arrow[rru, swap, bend right, "f_2"]
          & &
        \end{tikzcd}
    \]
    \end{itemize}~\\

    Given a category $\mc{C}$ and two fixed morphisms $\alpha: C \to A$ and
    $\beta: C \to B$, the category $\mc{C}^{\alpha, \beta}$ consists of the
    following pieces of data:
    \begin{itemize}
        \item $\opn{Obj}(\mc{C}^{\alpha, \beta})$ = commutative diagrams
        \[
        \begin{tikzcd}
            & A \arrow[rd, "f"] & \\
            C \arrow[ru, "\alpha"]
              \arrow[rd, swap, "\beta"]
              & & Z \\
            & B \arrow[ru, swap, "g"] &
        \end{tikzcd}
        \]
        in $\mc{C}$.
        \item Morphisms correspond to commutative diagrams
        \[
        \begin{tikzcd}
            & A \arrow[rd, "f_1"]
                \arrow[rrd, bend left, "f_2"] & \\
            C \arrow[ru, "\alpha"]
              \arrow[rd, swap, "\beta"]
              & & Z_1 \arrow[r, swap, "\sigma"]
                      & Z_2 \\
            & B \arrow[ru, swap, "g1"]
                \arrow[rru, swap, bend right, "g_2"] &
        \end{tikzcd}
        \]
        \item Composition and identities are defined in the obvious way.
        \item The usual composition and identity laws for $\mc{C}^{\alpha,
        \beta}$ follow from the composition and identity laws in $\mc{C}$.
    \end{itemize}
\end{xca}

\section{Morphisms}
\begin{xca}
Suppose $A_0 \xrightarrow{f_1} A_1 \xrightarrow{f_2} A_2 \xrightarrow{f_3}
\ldots \xrightarrow{f_n} A_n$ is a sequence of $n > 0$ morphisms in a
category $\mc{C}$. We claim any choice of placement of parentheses for $f_n
f_{n-1} \ldots f_1$ equals
\[(\ldots ((f_n f_{n-1}) f_{n-2}) \ldots) f_1.\]

We use induction on $n$ to prove our claim.

(Base cases) We note $f_1 = (f_1)$. And, also $f_2 f_1 = (f_2) f_1$. So the
statement holds for $n = 1, 2$.

(Inductive case) Suppose the statement holds for some $n > 1$. Now, consider
the placement of parentheses for the expression $E = f_{n+1} f_n f_{n-1} \ldots
f_1$. Any such choice will correspond to a representation of $E$ in the form
$e e'$, where both $e$ and $e'$ contain at least one term. This also implies
both $e$ and $e'$ contain at most $n$ terms. So, assume $e$ contains the terms
$f_{n+1}, f_n, \ldots, f_i$ and $e'$ contains the terms $f_{i-1}, \ldots, f_1$
for some $1 < i \le n + 1$. Then, by the inductive hypothesis,
\begin{align*}
    e &= (((f_{n+1} f_n) \ldots) f_i) \\
    e' &= (f_{i-1}(\ldots (f_2 f_1)))
\end{align*}
Then, through repeated use of the associativity of morphisms under composition,
we have $E$ \\
\begin{tabular}{l l}
    &= $e e'$ \\
    &= $e (f_{i-1}(\ldots (f_2 f_1)))$ \\
    &= $((((e f_{i-1}) f_{i-2}) f_{i-3}) \ldots) f_1$ \\
    &= $(\ldots ((f_{n+1} f_n) f_{n-1}) \ldots) f_1$
\end{tabular}
~\\

So, the statement holds for $n+1$ as well, and hence, by induction, the
statement holds for all naturals $n > 0$. And, we are done.
\end{xca}

\begin{xca}
We claim if a set $S$ is endowed with an equivalence relation $\sim$, then the
corresponding category $\mc{C}$ is a groupoid. Indeed, if $a \to b$ is a
morphism in $\mc{C}$, then $a \sim b$, which (by symmetry) implies $b \sim a$.
Thus, $b \to a$, and since there is at most one morphism from any object
(element) to another, we must have
\begin{align*}
    a \to b \to a &= 1_a \\
    b \to a \to b &= 1_b
\end{align*}
Hence, $a \to b$ is an isomorphism, for all elements $a, b \in S$, and thus,
$\mc{C}$ is a groupoid.
\end{xca}

\begin{xca}
Let $A, B$ be objects of a category $\mc{C}$, and let $f \in \Hom_{\mc{C}}(A,
B)$ be a morphism.
\begin{itemize}
    \item Suppose $f$ has a right inverse, $g: B \to A$, say. Then, $f \circ g
    = 1_B$. Assume $Z$ is an object of $\mc{C}$ and $\beta, \beta': B \to Z$
    morphisms such that $\beta \circ f = \beta' \circ f$. Then, $\beta = \beta
    \circ 1_B = \beta \circ (f \circ g) = (\beta \circ f) \circ g = (\beta'
    \circ f) \circ g = \beta' \circ (f \circ g) = \beta' \circ 1_B = \beta'$,
    thus showing $f$ is an epimorphism.
    \item In $(\Z, \le)$, considered as a category, any morphism is an
    epimorphism, but the same does not have a right inverse. For example, $1 \le
    2$, \emph{i.e.} $1 \to 2$. But, if the latter were to have a right inverse,
    then we would have $2 \to 1$, which would then imply $2 \le 1$, clearly a
    contradiction.
\end{itemize}
\end{xca}

\begin{xca}
Suppose $A \xrightarrow{f} B$ and $B \xrightarrow{g} C$ are both monomorphisms
in a category $\mc{C}$. Let $Z$ be any object of $\mc{C}$, and let $\alpha',
\alpha'': Z \to A$ be morphisms such that
\begin{center}
    $(g \circ f) \circ \alpha' = (g \circ f) \circ \alpha''$.
\end{center}
Then, $g \circ (f \circ \alpha') = g \circ (f \circ \alpha'')$, which implies
$f \circ \alpha' = f \circ \alpha''$, which implies $\alpha' = \alpha''$. Hence,
$g \circ f$ is a monomorphism. This demonstrates the composition of two
monomorphisms is a monomorphism.

Now, it is easy to check identity morphisms in $\mc{C}$ are monomorphisms. Thus,
one can define a subcategory $\mc{C}_{\text{mono}}$ of a category $\mc{C}$ by
taking the same objects as in $\mc{C}$ and defining
$\Hom_{\mc{C}_{\text{mono}}}$ to be the subset of $\Hom_{\mc{C}}(A, B)$
consisting of monomorphisms, for all objects $A, B$ of $\mc{C}$.\\

(Epimorphisms) Suppose $A \xrightarrow{f} B$ and $B \xrightarrow{g} C$ are both
epimorphisms in a category $\mc{C}$. Let $Z$ be any object of $\mc{C}$, and let
$\beta', \beta'': C \to Z$ be morphisms such that
\begin{center}
    $\beta' \circ (g \circ f) = \beta'' \circ (g \circ f)$.
\end{center}
Then, $(\beta' \circ g) \circ f = (\beta'' \circ g) \circ f$, which implies
$\beta' \circ g = \beta'' \circ g$, which implies $\beta' = \beta''$, thus
showing $g \circ f$ is an epimorphism. In a manner similar to the one shown
above, one can define a subcategory $\mc{C}_{\text{epi}}$ of a category
$\mc{C}$.
\end{xca}

\begin{xca}
% TODO: Chapter I - Exercise 4.5
\end{xca}

\section{Universal properties}
\begin{xca}
Suppose $F$ is a final object in a category $\mc{C}$. Then, for all objects $A$
of $\mc{C}$, there exists a unique morphism $A \to F$ in $C$, which implies for
all objects $A$ of $\mc{C}^{op}$, there exists a unique morphism $F \to A$ in
$\mc{C}^{op}$. We thus conclude $F$ is initial in the opposite category
$\mc{C}^{op}$.
\end{xca}

\begin{xca}
For any set $A$, the function
\begin{center}
    $\emptyset \xrightarrow{(\emptyset, A, \emptyset)} A$
\end{center}
 is the unique
function from $\emptyset$ to $A$. Thus, $\emptyset$ is initial in
$\catname{Set}$. Now, suppose $I$ is a set that's also initial in
$\catname{Set}$. Then, $\emptyset \cong I$. Therefore, $|I| = |\emptyset| = 0$,
which implies $I = \emptyset$. Hence, $\emptyset$ is the unique initial object
in $\catname{Set}$.
\end{xca}

\begin{xca}
Suppose $F_1$ and $F_2$ are final objects in a category $\mc{C}$. Then, there
are unique morphisms $F_1 \to F_2$ and $F_2 \to F_1$. So, we must have
\begin{center}
    $F_1 \to F_2 \to F_1 = 1_{F_1}$ \\
    $F_2 \to F_1 \to F_2 = 1_{F_2}$
\end{center}
Thus, $F_1 \cong F_2$, and hence, we conclude final objects are unique up to
isomorphism.
\end{xca}

\begin{xca}
In the category $\catname{Set}^*$ of pointed sets, any object $(\{*\}, *)$ is
both an initial and a final object.
\end{xca}

\begin{xca}
The final objects considered in the category considered in \textsection 5.3 are
$A \to \{ \bullet \}$.
\end{xca}

\begin{xca}
Consider the category corresponding to endowing the set $\Z^+$ of positive
integers with the \emph{divisibility} relation. Then, for any two positive
integers $m, n$, if their product $m \times n$ exists, then $m \times n$
divides both $m$ and $n$, and further, if some positive integer $d$ divides
both $m$ and $n$, then $d$ divides $m \times n$. $\gcd(m, n)$ satisfies the
aforesaid condition, and hence, it is the product of $m$ and $n$. Similarly, it
is easy to check $\lcm(m, n)$ is the coproduct of any two positive integers $m$
and $n$.
\end{xca}

\begin{xca}
Suppose $A' \cong A''$ and $B' \cong B''$, and further, $A' \cap B' = \emptyset$
and $A'' \cap B'' = \emptyset$. Then, there exist isomorphisms $k: A' \to B'$
and $l: B' \to B''$, say. We claim
\[
A' \xrightarrow{i_{A'}} A' \cup B' \xleftarrow{i_{B'}} B',
\]
where $i_{A'}, i_{B'}$ are inclusion functions, is the coproduct of $A'$ and
$B'$.

Toward that end, for any set $C$ and functions $A' \xrightarrow{f} C
\xleftarrow{g} B'$, we define a function $h: A' \cup B' \to C$ by
\[
h(x) =
    \begin{cases}
        f(x) & \text{if } x \in A' \\
        g(x) & \text{if } x \in B'
    \end{cases}
\]
Then, it is easy to check the following diagram commutes:
\[
\begin{tikzcd}
    & C & \\
    A' \arrow[r, swap, "i_{A'}"]
       \arrow[ur, "f"]
       & A' \cup B' \arrow[u, swap, dashed, "h"]
                    & B' \arrow[l, "i_{B'}"]
                         \arrow[ul, swap, "g"]
\end{tikzcd}
\]
Also, $h$ is necessarily the unique function $A' \cup B' \to C$ that makes the
above diagram commute.

We next verify
\[
A' \xrightarrow{k \circ i_{A'}} A'' \cup B'' \xleftarrow{l \circ i_{B'}} B'
\]
is also a coproduct of $A'$ and $B'$. Indeed, for any set $C$ and functions
$A' \xrightarrow{f} C \xleftarrow{g} B'$, we define a function $h': A'' \cup B''
\to C$ by
\[
h'(x) =
    \begin{cases}
        f \circ k^{-1}(x) & \text{if } x \in A'' \\
        g \circ l^{-1}(x) & \text{if } x \in B''
    \end{cases}
\]
It is, again, easy to check the following diagram commutes:
\[
\begin{tikzcd}
    & C & \\
    A' \arrow[r, swap, "k \circ i_{A'}"]
       \arrow[ur, "f"]
       & A'' \cup B'' \arrow[u, swap, dashed, "h'"]
                    & B' \arrow[l, "l \circ i_{B'}"]
                         \arrow[ul, swap, "g"]
\end{tikzcd}
\]
Also, $h''$ is necessarily the unique function $A'' \cup B'' \to C$ that makes
the above diagram commute.

Thus, we see both $A' \cup B'$ and $A'' \cup B''$ along with their associated
functions are coproducts of $A'$ and $B'$, and hence, we conclude they are
isomorphic to each other.
\end{xca}

\begin{xca}
Suppose $\mc{C}$ is a category and $A, B$ are objects of $\mc{C}$. Assume the
products $A \leftarrow A \times B \to B$ and $B \leftarrow B \times A \to A$
exist. Then, for all sets $C$ and functions $A \xleftarrow{f} C \xrightarrow{g}
B$, there exists a unique function $\langle f, g \rangle: C \to A \times B$
such that the following diagram commutes:
\[
\begin{tikzcd}
    & C \arrow[ld, swap, "f"]
        \arrow[rd, "g"]
        \arrow[d, dashed, "\langle {f, g} \rangle"] & \\
    A & A \times B \arrow[l]
                   \arrow[r]
                   & B
\end{tikzcd}
\]
This implies there is a unique function $\langle g, f \rangle: C \to B \times A$
such that the following diagram commutes:
\[
\begin{tikzcd}
    & C \arrow[ld, swap, "g"]
        \arrow[rd, "f"]
        \arrow[d, dashed, "\langle {g, f} \rangle"] & \\
    B & B \times A \arrow[l]
                   \arrow[r]
                   & A
\end{tikzcd}
\]
We thus see both $A \leftarrow A \times B \to B$ and $B \leftarrow B \times A
\to A$ satisfy the universal property for the product of $A$ and $B$. That is,
they are both final objects in the corresponding category, and hence, by
Proposition 5.4, they are isomorphic.
\end{xca}

\begin{xca}
Let $\mc{C}$ be a category with products. For any three objects $A, B, C$ of
$\mc{C}$, their product $A \times B \times C$ satisfies the following
property:\\
The product is equipped with morphisms $A \times B \times C \to A$, $A \times B
\times C \to B$ and $A \times B \times C \to C$ such that, for all objects $D$
and morphisms $f: D \to A$, $g: D \to B$ and $h: C \to C$, there is a unique
morphism $\langle f, g, h \rangle: D \to A \times B \times C$, making the
following diagram commute:
\[
\begin{tikzcd}
    & D \arrow[ddl, swap, bend right, "f"]
        \arrow[ddd, bend left, crossing over, "g", near end]
        \arrow[ddr, bend left, "h"]
        \arrow[d, swap, dashed, "\langle {f, g, h} \rangle"]
        & \\
    & A \times B \times C \arrow[dl]
                          \arrow[dd]
                          \arrow[dr, crossing over]
                          & \\
    A & & C \\
    & B &
\end{tikzcd}
\]
A routine check can verify $\langle f, \langle g, h \rangle \rangle: D \to
A \times (B \times C)$ satisfies the aforesaid universal property, and so does
$\langle \langle f, g \rangle, h \rangle: D \to (A \times B) \times C$. Hence,
$A \times (B \times C)$ and $(A \times B) \times C$ are isomorphic.
\end{xca}

\begin{xca}
(Product) Let $\mc{C}$ be a category, and let $\{ A_i \}_{i \in I}$ be a family
of objects $A_i$ of $\mc{C}$ indexed by some set $I$. The product of the
family of objects $A_i$ ($i \in I$) is an object
\[
\prod_{i \in I} A_i
\]
of $\mc{C}$ equipped with a family of morphisms
\[
p_j : \prod_{i \in I} A_i \to A_j \q (j \in I)
\]
such that, for all objects $Z$ and morphisms $q_j : Z \to A_j$ ($j \in I$),
there exists a unique morphism $h: Z \to \prod_{i \in I} A_i$ such that
\[
p_j \circ h = q_j \q (j \in I)
\]

(Coproduct) The coproduct of the family of objects $A_i$ ($i \in I$) is an
object
\[
\coprod_{i \in I} A_i
\]
of $\mc{C}$ equipped with a family of morphisms
\[
p_j : A_j \to \coprod_{i \in I} A_i \q (j \in I)
\]
such that, for all objects $Z$ and morphisms $q_j : A_j \to Z$ ($j \in I$),
there exists a unique morphism $h: \coprod_{i \in I} A_i \to Z$ such that
\[
h \circ p_j = q_j \q (j \in I)
\]

These exist in $\catname{Set}$ if we assume the \emph{axiom of choice}.
\end{xca}

\begin{xca}
Let $A$, resp. $B$, be a set, endowed with an equivalence relation $\sim_A$,
resp. $\sim_B$. Define a relation $\sim$ on $A \times B$ by setting
\[
(a_1, b_1) \sim (a_2, b_2) \iff a_1 \sim_A a_2 \text{ and } b_1 \sim_B b_2
\]
We can easily check $\sim$ is an equivalence relation on $A \times B$.
\begin{itemize}
    \item The composite of the first projection function $\pi_A: A \times B \to
    A$ and the natural surjection $\pi_{\sim_A}: A \to A/{\sim_A}$ is a
    function $A \times B \xrightarrow{\pi_{\sim_A} \circ \pi_A} A/{\sim_A}$.
    Note, if $(a_1, b_1) \sim (a_2, b_2)$, then $a_1 \sim_A a_2$, and thus,
    $\pi_{\sim_A} \circ \pi_A (a_1, b_1) = \pi_{\sim_A}(a_1) = [a_1]_{\sim_A}
    = [a_2]_{\sim_A} = \pi_{\sim_A} \circ \pi_A (a_2, b_2)$. That is,
    equivalent elements in $A \times B$ have the same image in $A/{\sim_A}$
    under the map $\pi_{\sim_A} \circ \pi_A$. Therefore, by the universal
    property of quotients, there exists a unique function $\varphi_A: A \times
    B /{\sim} \to A/{\sim_A}$, given by
    \[
    \varphi_A([(a, b)]_{\sim}) = \pi_{\sim_A} \circ \pi_A (a, b)
    \]
    such that the following diagram commutes:
    \[
    \begin{tikzcd}
        A \times B /{\sim} \arrow[rr, "\varphi_A"]
                           & & A/{\sim_A} \\
        & & \\
        & A \times B \arrow[uul, "\pi_{\sim_{A \times B}}"]
                     \arrow[uur, swap, "\pi_{\sim_A} \circ \pi_A"]
                     &
    \end{tikzcd}
    \]
    Similarly, by the universal property of quotients, there exists a unique
    function $\varphi_B: A \times B /{\sim} \to B/{\sim_B}$, given by
    \[
    \varphi_B([(a, b)]_{\sim}) = \pi_{\sim_B} \circ \pi_B (a, b)
    \]
    such that the following diagram commutes:
    \[
    \begin{tikzcd}
        A \times B /{\sim} \arrow[rr, "\varphi_B"]
                           & & B/{\sim_B} \\
    & & \\
    & A \times B \arrow[uul, "\pi_{\sim_{A \times B}}"]
                 \arrow[uur, swap, "\pi_{\sim_B} \circ \pi_B"]
                 &
    \end{tikzcd}
    \]
    The above, thus, demonstrates there are functions $A \times B /{\sim} \to
    A/{\sim_A}$ and $A \times B /{\sim} \to B/{\sim_B}$.

    \item We now claim $A/{\sim_A} \xleftarrow{\varphi_A} A \times B /{\sim}
    \xrightarrow{\varphi_B} B/{\sim_B}$ is the product of $A/{\sim_A}$ and
    $B/{\sim_B}$.

    To that end, suppose $C$ is any set, endowed with an equivalence relation
    $\sim_C$. Let $A/{\sim_A} \xleftarrow{f} C \xrightarrow{g} B/{\sim_B}$ be
    functions, such that, for all $c_1, c_2 \in C$,
    \begin{center}
        $c_1 \sim_C c_2 \implies f(c_1) = f(c_2)$ and $g(c_1) = g(c_2)$. \\
    \end{center}

    Now, define a function $h: C \to A \times B /{\sim}$ by
    \begin{align*}
        h(c) &= [(a, b)]_{\sim}, \text{ where} \\
        f(c) &= [a]_{\sim_A} \text{ for some } a \in A, \text{ and} \\
        g(c) &= [b]_{\sim_B} \text{ for some } b \in B.
    \end{align*}
    Then, $h$ is the unique function $C \to A \times B /{\sim}$ that makes the
    following diagram commute:
    \[
    \begin{tikzcd}
        & C \arrow[ddl, swap, "f"]
            \arrow[ddr, "g"]
            \arrow[dd, dashed, "h"]
            & \\
        & & \\
        A/{\sim_A} & A \times B /{\sim} \arrow[l, "\varphi_A"]
                                        \arrow[r, swap, "\varphi_B"]
                                        & B/{\sim_B}
    \end{tikzcd}
    \]
    We can easily verify, for all $c \in C$,
    \begin{align*}
        \varphi_A \circ h(c) &= \varphi_A([(a, b)]_{\sim}) \\
                             &= \pi_{\sim_A} \circ \pi_A (a, b) \\
                             &= \pi_{\sim_A}(a) \\
                             &= [a]_{\sim_A} \\
                             &= f(c)
    \end{align*}
    That is, $\varphi_A \circ h = f$. Similarly, it is easy to check
    $\varphi_B \circ h = g$. Also, it is necessarily the case that $h$ as
    defined is unique. Hence, our claim is proved.

    \item From the previous proof, it immediately follows
    \[
    A \times B /{\sim} \cong (A/{\sim_A}) \times (B/{\sim_B}).
    \]
\end{itemize}
\end{xca}

\begin{xca}
Let $\mc{C}$ be a category, and let $X \xrightarrow{\alpha} Z \xleftarrow{\beta}
Y$ be two fixed morphisms in $\mc{C}$.

Then, the \emph{fibered product} of $\alpha$ and $\beta$ is a final object
$X \xleftarrow{f} P \xrightarrow{g} Y$ in the category $\mc{C}_{\alpha, \beta}$.
That is, for all objects $X \xleftarrow{f_1} Q \xrightarrow{g_1} Y$ that makes
the following diagram commute
\[
\begin{tikzcd}
    Q \arrow[ddr, swap,  bend right, "f_1"]
      \arrow[drr, bend left, "g_1"]
      & & \\
    & & Y \arrow[d, "\beta"] \\
    & X \arrow[r, swap, "\alpha"]
      & Z
\end{tikzcd}
\]
there exists a unique morphism $h: Q \to P$ that makes the following diagram
commute
\[
\begin{tikzcd}
    Q \arrow[ddr, swap,  bend right, "f_1"]
      \arrow[drr, bend left, "g_1"]
      \arrow[dr, dashed, "h"]
      & & \\
    & P \arrow[r, "g"]
        \arrow[d, swap, "f"]
        & Y \arrow[d, "\beta"] \\
    & X \arrow[r, swap, "\alpha"]
        & Z
\end{tikzcd}
\]

In $\catname{Set}$, the fibered product of two functions $X \xrightarrow{\alpha}
Z \xleftarrow{\beta} Y$ is the object
\[
X \xleftarrow{\pi_X} X \times_Z Y \xrightarrow{\pi_Y} Y,
\]
where $X \times_Z Y := \{ (x, y) \in X \times Y \mid \alpha(x) = \beta(y) \}$,
and $\pi_X$ and $\pi_Y$ are the canonical projection functions. \\

Let $\mc{C}$ be a category, and let $X \xleftarrow{\alpha} Z \xrightarrow{\beta}
Y$ be two fixed morphisms in $\mc{C}$. Then, the \emph{fibered coproduct} of
$\alpha$ and $\beta$ is an initial object $X \xrightarrow{f} P \xleftarrow{g}
Y$ in the category $\mc{C}^{\alpha, \beta}$.
\end{xca}


\subsection{Bilinear forms}

A bilinear form takes two vectors and produces a scalar from the underyling field.

This is in contrast to a linear form, which only has one input.

In addition, the function is linear in both arguments.

\(\phi (au+x, bv+y)=\phi (au,bv)+\phi (au,y)+\phi (x,bv)+\phi (x,y)\)

\(\phi (au+x, bv+y)=ab\phi (u,v)+a\phi (u,y)+b\phi (x,v)+\phi (x,y)\)

\subsubsection{Representing bilinear forms}

They can be represented as:

\(\phi (u,v)=v^TMu\)

\(f(M)=f([v_1,v_2])\)

We introduce \(e_i\), the element vector. This is \(0\) for all entries except for \(i\) where it is \(1\). Any vector can be shown as a sum of these vectors multiplied by a scalar.

\(f(M)=f([\sum^m_{i=1}a_{1i}e_i,\sum^m_{i=1}a_{2i}e_i])\)

\(f(M)=\sum_{k=1}^mf([a_{1k}e_k,\sum^m_{i=1}a_{2i}e_i])\)

\(f(M)=\sum_{k=1}^m\sum^m_{i=1}f([a_{1k}e_k,a_{2i}e_i])\)

Because this in linear in scalars:

\(f(M)=\sum_{k=1}^m\sum^m_{i=1}a_{1k}a_{2i}f([e_k,e_i])\)

\(f(M)=\sum_{k=1}^m\sum^m_{i=1}a_{1k}a_{2i}e_k^TMe_i\)

\subsubsection{Orthonormal basis and \(M=I\)}

\(f(M)=\sum_{k=1}^m\sum^m_{i=1}a_{1k}a_{2i}e_k^Te_i\)

\(f(M)=\sum_{k=1}^m\sum^m_{i=1}a_{1k}a_{2i}\delta_i^k\)

\(f(M)=\sum^m_{i=1}a_{1i}a_{2i}\)