\chapter{Primitive recursive functions}

\section{Introduction}
Primitive recursive functions are a small class of total functions, corresponding to the expressive power of a simple imperative programming language without \textbf{while} loops, in which every program execution terminates.

Primitive recursive functions are total functions from $\mathbb{N}^n$ to
$\mathbb{N}$, for some $n\in\mathbb{N}$. Note that not all
total $n$-ary recursive functions are primitive recursive
(see for instance Sect.~\vref{sect:ack-not-PR}).

The traditional definition of the set of primitive recursive functions is structured as an inductive definition
in five rules: three base cases, and two recursive construction rules.

\begin{description}
  \item[zero] the constant function of value $0$ is primitive recursive.
\item[S] The successor function $S:\mathbb{N}\rightarrow\mathbb{N}$ is primitive recursive.
 \item[projections] For any pair $0< i\leq n$, the projection $\pi_{i,n}: \mathbb{N}^n\rightarrow\mathbb{N}$, defined by $\pi_{i,n}(x_1,x_2,\dots,x_{n})=x_i$, is primitive recursive.
\item[composition] For any $n$ and $m$, if $h: \mathbb{N}^m\rightarrow\mathbb{N}$, and
$g_0,\dots, g_{m-1}: \mathbb{N}^n\rightarrow\mathbb{N}$ are primitive recursive of $n$ arguments, then the function which maps any
tuple $(x_0,\dots,x_{n-1})$ to $h(g_0(x0,\dots,x_{n-1}),\dots, g_{m-1}(x0,\dots,x_{n-1})): \mathbb{N}^n\rightarrow\mathbb{N}$ is primitive recursive.
\item[primitive recursion]
If $g: \mathbb{N}^n\rightarrow\mathbb{N}$ and $h: \mathbb{N}^{n+2}\rightarrow\mathbb{N}$ are primitive recursive, then the function from $\mathbb{N}^{n+1}$ into $\mathbb{N}$ defined by
\begin{align}
f(0,x_1,\dots,x_n)&=g(x_1,\dots,x_n)\\
f(S(p),x_1,\dots,x_n)&=h(p,f(p, x_1,\dots,x_n),  x_1,\dots,x_n)
\end{align}
is primitive recursive.
\end{description}

Please note the use of dots: $\ldots$ in the definition above.
Dots are not part of \gallina's syntax. Thus, the formal definition of the set of primitive recursive function will have to overcome this representation problem.

  We present in this chapter a formalization of  primitive recursive functions, taken from  Russel O'Connor's formalization in \coq{} of
G\"odel's incompleteness theorems~\cite{OConnor05}.

\begin{remark}
 The theory of primitive recursive function is now hosted in
the \texttt{theories/ordinals/Ackermann} directory.
The specific part on G\"odel's theorem,  is also on
coq-community (\url{https://github.com/coq-community/goedel}) and requires the
\href{https://github.com/coq-community/pocklington}{Pocklington library} for lemmas on primality.
\end{remark}

This chapter contains some comments on Russel's library, as well as a few extensions.
Contributions (under the form of comments, new examples or exercises) are welcome!.

\section{First look at the Ackermann library}

O'Connor's library on Gödel's incompleteness theorems contains a little more
than 45K lines of scripts. The part dedicated to primitive recursive functions and Peano arithmetics is 32K lines long and is originally structured in 38 modules.
Thus, we propose a partial exploration of this library, through examples and exercises. Our additions to the original library --- mainly examples and counter-examples ---,
are stored in the directory \texttt{theories/ordinals/MoreAck}.

In particular, the library \href{../theories/html/hydras.MoreAck.AckNotPR.html}{MoreAck.AckNotPR} contains the well-known  proof that the Ackermann function is not primitive recursive (see Section~\vref{sect:ack-not-PR}).
MoreOver, the library \href{../theories/html/hydras.Hydra.Hydra_Theorems.html}{Hydra.Hydra\_Theorems} contains
a proof that the length of an hydra battle (according to the initial replication factor) is not primitive recursive in general.

\section{Basic definitions}
\index{maths}{Primitive recursive functions}

The formal definition of primitive recursive functions lies in the library
\href{../theories/html/hydras.Ackermann.primRec.html}{Ackermann.primRec},
with preliminary definitions in
\href{../theories/html/hydras.Ackermann.extEqualNat.html}{Ackermann.extEqualNat}
and
\href{../theories/html/hydras.Ackermann.misc.html}{Ackermann.misc}.

\subsection{Functions of arbitrary arity}

The  \texttt{primRec} library allows us to consider primitive functions on \texttt{nat}, with any number of arguments, in
curried form. This is made possible in
\href{../theories/html/hydras.Ackermann.extEqualNat.html}{Ackermann.extEqualNat} by the following definition:

\index{primrec}{Types!naryFunc}
\begin{Coqsrc}
Fixpoint naryFunc (n : nat) : Set :=
  match n with
  | O => nat
  | S n => nat -> naryFunc n
  end.
\end{Coqsrc}

For instance (\texttt{naryFunc 1}) is convertible to \texttt{nat -> nat} and (\texttt{naryFunc 3})
to \texttt{nat -> nat -> nat -> nat}.

\vspace{4pt}
\noindent
\emph{From \href{../theories/html/hydras.MoreAck.PrimRecExamples.html}{MoreAck.PrimRecExamples}}.
\begin{Coqsrc}
Require Import primRec.
Import extEqualNat.

Compute naryFunc 3.
\end{Coqsrc}

\begin{Coqanswer}
= nat -> nat -> nat -> nat
  : Set
\end{Coqanswer}

\begin{Coqsrc}
Check plus: naryFunc 2.

Check 42: naryFunc 0.

Check (fun n p q : nat =>  n * p + q): naryFunc 3.
\end{Coqsrc}

Likewise, arbitrary boolean predicates may have an arbitrary number of arguments. The dependent type
(\texttt{naryRel $n$}), defined in the same way as \texttt{naryFunc}, is the type of $n$-ary functions from
\texttt{nat} into \texttt{bool}.

\begin{Coqsrc}
Compute naryRel 2.
\end{Coqsrc}

\begin{Coqanswer}
 = nat -> nat -> bool
     : Set
\end{Coqanswer}

The magic of dependent types makes it possible to define recursively extensional equality between functions of the same arity.

\index{coq}{Dependent types}
\index{coq}{Dependently typed functions}
\vspace{4pt}
\noindent
\emph{From \href{../theories/html/hydras.Ackermann.extEqualNat.html}{Ackermann.extEqualNat}}

\index{primrec}{Predicates!extEqual}

\begin{Coqsrc}
Fixpoint  extEqual (n : nat) : forall  (a b : naryFunc n), Prop :=
  match n with
    0 => fun a b => a = b
  | S p => fun a b => forall c, extEqual p (a c) (b c)
  end.
\end{Coqsrc}

\begin{Coqsrc}
Compute extEqual 2.
\end{Coqsrc}

\begin{Coqanswer}
     = fun a b : naryFunc 2 => forall x x0 : nat, a x x0 = b x x0
     : naryFunc 2 -> naryFunc 2 -> Prop
 \end{Coqanswer}

\begin{Coqsrc}
Example extEqual_ex1 : extEqual 2 mult (fun x y =>  y * x + x - x) .
Proof.
  intros x y.
\end{Coqsrc}

\begin{Coqanswer}
  x, y : nat
  ============================
  extEqual 0 (x * y) (y * x)
\end{Coqanswer}

\begin{Coqsrc}
  cbn.
\end{Coqsrc}

\begin{Coqanswer}
1 subgoal (ID 10)

  x, y : nat
  ============================
  x * y = y * x + x - x
\end{Coqanswer}

\begin{Coqsrc}
  rewrite <- Nat.add_sub_assoc, Nat.sub_diag.
  - ring.
  - apply le_n.
Qed.
\end{Coqsrc}

\subsection{A Data-type for Primitive Recursive Functions}

O'Connor's formalization of primitive recursive functions takes the form of two mutually inductive dependent data types, each constructor of which is associated with one of these  rules.
These two types are (\texttt{PrimRec $n$}) (primitive recursive functions of $n$ arguments), and
(\texttt{PrimRecs $n$ $m$}) ($m$-tuples of primitive recursive functions of $n$ arguments).

\index{coq}{Dependent types}
\index{coq}{Mutually inductive types}

\index{primrec}{Types!PrimRec}
\index{primrec}{Types!PrimRecs}
\label{def:Primrec}
\vspace{4pt}
\noindent
\emph{From \href{../theories/html/hydras.Ackermann.primRec.html}{Ackermann.primRec}.}
\begin{Coqsrc}
Inductive PrimRec : nat -> Set :=
  | succFunc : PrimRec 1
  | zeroFunc : PrimRec 0
  | projFunc : forall n m : nat, m < n -> PrimRec n
  | composeFunc :
      forall (n m : nat) (g : PrimRecs n m) (h : PrimRec m), PrimRec n
  | primRecFunc :
      forall (n : nat) (g : PrimRec n) (h : PrimRec (S (S n))),
      PrimRec (S n)
with PrimRecs : nat -> nat -> Set :=
  | PRnil : forall n : nat, PrimRecs n 0
  | PRcons : forall n m : nat, PrimRec n -> PrimRecs n m ->
                   PrimRecs n (S m).
\end{Coqsrc}

\begin{remark}
\label{projFunc-order-of-args}
Beware of the conventions used in the \texttt{primRec} library!
The constructor (\texttt{projFunc $n$ $m$})  is associated with the projection $\pi_{n-m,n}$ and \emph{not}
$\pi_{n, m}$.
For instance, the projection $\pi_{2,5}$ defined by $\pi_{2,5}(a,b,c,d,e)=b$ corresponds to the term
(\texttt{projFunc 5 3 H}), where \texttt{H} is a proof of $3<5$.
 This fact is reported in the comments of \texttt{primRec.v}. We presume that this convention makes it easier to define the evaluation function (\texttt{evalProjFunc $n$}) (see the next sub-section). Trying the other convention is left as an exercise.
\end{remark}

\subsection{A little bit of semantics}
Please note that inhabitants of type (\texttt{PrimRec $n$}) are not \coq{} functions like \texttt{Nat.mul}, or factorial, etc. The data-type (\texttt{PrimRec $n$}) is indeed an abstract syntax for the language of primitive recursive functions. The bridge between this language and the word of usual functions
is an interpretation function (\texttt{evalprimRec $n$})  of type
$\texttt{PrimRec}\,n \rightarrow  \texttt{naryFunc}\,n$.
This function is defined by mutual recursion,  together with the  function
(\texttt{evalprimRecS $n$ $m$}) of type
$\texttt{PrimRecs}\,n\,m \rightarrow  \texttt{Vector.t}\,(\texttt{naryFunc}\,n)\,m$.

\index{primrec}{Functions!evalPrimRec}
\index{primrec}{Functions!evalPrimRecs}

\index{coq}{Dependent pattern matching}
Both functions are mutually defined through dependent pattern matching. We advise the readers who
would feel uneasy with dependent types to consult Adam Chlipala's \emph{cpdt}  book~\cite{chlipalacpdt2011}. We leave it to the reader  to look also at the helper functions in
\href{../theories/html/hydras.Ackermann.primRec.html}{Ackermann.primRec}.

\begin{Coqsrc}
Fixpoint evalPrimRec (n : nat) (f : PrimRec n) {struct f} :
 naryFunc n :=
  match f in (PrimRec n) return (naryFunc n) with
  | succFunc => S
  | zeroFunc => 0
  | projFunc n m pf => evalProjFunc n m pf
  | composeFunc n m l f =>
      evalComposeFunc n m (evalPrimRecs _ _ l) (evalPrimRec _ f)
  | primRecFunc n g h =>
      evalPrimRecFunc n (evalPrimRec _ g) (evalPrimRec _ h)
  end
with evalPrimRecs (n m : nat) (fs : PrimRecs n m) {struct fs} :
 Vector.t (naryFunc n) m :=
  match fs in (PrimRecs n m) return (Vector.t (naryFunc n) m) with
  | PRnil a => Vector.nil  (naryFunc a)
  | PRcons a b g gs =>
       Vector.cons _ (evalPrimRec _ g) _  (evalPrimRecs _ _ gs)
  end.
\end{Coqsrc}

Looks complicated? The following examples show that, when
the arity is fixed, these definitions behave well w.r.t.
\coq's reduction rules. Moreover, they make the interpretation functions more ``concrete''.

\vspace{4pt}
\noindent
\emph{From \href{../theories/html/hydras.MoreAck.PrimRecExamples.html}{MoreAck.PrimRecExamples}.}
\begin{Coqsrc}
Example Ex1 : evalPrimRec 0 zeroFunc = 0.
Proof. reflexivity. Qed.

Example Ex2 a : evalPrimRec 1 succFunc a = S a.
Proof. reflexivity. Qed.

Example Ex3 a b c d e f: forall (H: 2 < 6),
    evalPrimRec 6
                (projFunc 6 2 H) a b c d e f = d.
Proof. reflexivity. Qed.

Example Ex4 (x y z : PrimRec 2) (t: PrimRec 3):
  let u := composeFunc 2 3
                       (PRcons 2 _ x
                               (PRcons 2 _ y
                                       (PRcons 2 _ z
                                               (PRnil 2))))
                       t in
  let f := evalPrimRec 2 x in
  let g := evalPrimRec 2 y in
  let h := evalPrimRec 2 z in
  let i := evalPrimRec 3 t in
  let j := evalPrimRec 2 u in
  forall a b, j a b = i (f a b) (g a b) (h a b).
Proof. reflexivity. Qed.

Example Ex5 (x : PrimRec 2)(y: PrimRec 4):
  let g := evalPrimRec _ x in
  let h := evalPrimRec _ y in
  let f := evalPrimRec _ (primRecFunc _ x y) in
  forall a b,  f 0 a b = g a b.
Proof. reflexivity.   Qed.

Example Ex6 (x : PrimRec 2)(y: PrimRec 4):
  let g := evalPrimRec _ x in
  let h := evalPrimRec _ y in
  let f := evalPrimRec _ (primRecFunc _ x y) in
  forall n a b,  f (S n) a b = h n (f n a b) a b.
Proof. reflexivity.   Qed.
\end{Coqsrc}

Another example?
Let us consider the following term\footnote{Of course, we never typed this term \emph{verbatim}; we obtained it by an interactive proof the reader will be able to make after
reading Sect.\vref{sect:proofs-of-isPR}.}:

\label{sect:bigfac}
\begin{Coqsrc}
Example bigPR : PrimRec 1 :=
primRecFunc 0
  (composeFunc 0 1 (PRcons 0 0 zeroFunc (PRnil 0)) succFunc)
  (composeFunc 2 2
    (PRcons 2 1
      (composeFunc 2 1
         (PRcons 2 0 (projFunc 2 1 (le_n 2))
                 (PRnil 2))
         succFunc)
      (PRcons 2 0
        (composeFunc 2 1
          (PRcons 2 0
             (projFunc 2 0
                       (le_S 1 1 (le_n 1)))
             (PRnil 2))
          (projFunc 1 0 (le_n 1))) (PRnil 2)))
    (primRecFunc 1 (composeFunc 1 0 (PRnil 1) zeroFunc)
       (composeFunc 3 2
         (PRcons 3 1
            (projFunc 3 1 (le_S 2 2 (le_n 2)))
            (PRcons 3 0 (projFunc 3 0
                          (le_S 1 2
                                (le_S 1 1 (le_n 1))))
                    (PRnil 3)))
         (primRecFunc 1 (projFunc 1 0 (le_n 1))
                      (composeFunc 3 1
                          (PRcons 3 0
                                  (projFunc 3 1 (le_S 2 2 (le_n 2)))
                                  (PRnil 3))
                          succFunc))))).
\end{Coqsrc}

Let us now interpret this term as an arithmetic function.

\begin{Coqsrc}
Example  mystery_fun : nat -> nat := evalPrimRec 1 bigPR.

Compute map mystery_fun[0;1;2;3;4;5;6] : t nat _.
\end{Coqsrc}

\begin{Coqanswer}
 = 1 :: 1 :: 2 :: 6 :: 24 :: 120 :: 720 :: nil
     : t nat 7
\end{Coqanswer}

After this test, the term \texttt{bigPR} looks to be a primitive recursive definition of the factorial function, although we haven't proved this fact yet. Fortunately, we will see in the following sections simple ways to prove that a given function is primitive recursive, whithout building such an unreadable term.

\section{Proving that a given arithmetic function is primitive recursive}
\label{sect:proofs-of-isPR}

The example in the preceding section clearly shows that, in order to prove that a given arithmetic function
(defined in \gallina{} as usual) is primitive recursive, trying to give  by hand a term  of type (\texttt{PrimRec $n$}) is not a good method, since such terms may be huge and complex, even for simple arithmetic functions. The method proposed in Library \texttt{primRec} is the following one:

\begin{enumerate}
\item Define a type corresponding to the statement "the function \texttt{$f$:naryFunc $n$} is primitive recursive ''.
\item Prove handy lemmas which may help to prove that a given function is primitive recursive.
\end{enumerate}

Thus, the proof that a function, like \texttt{factorial}, is primitive recursive may be interactive, whithout having to type complex terms at any step of the development.

\subsection{The predicate \texttt{isPR}}

\index{primrec}{Predicates!isPR}
\index{coq}{Extensionnaly equal functions}

Let $f$ be an arithmetic function of arity $n$. We say that $f$ is primitive recursive if $f$ is \textbf{extensionnaly}
equal to the interpretation of some term of type \texttt{PrimRec $n$}.

\vspace{4pt}
\noindent
\emph{From \href{../theories/html/hydras.Ackermann.primRec.html}{Ackermann.primRec}.}
\begin{Coqsrc}
Definition isPR (n : nat) (f : naryFunc n) : Set :=
  {p : PrimRec n | extEqual n (evalPrimRec _ p) f}.
\end{Coqsrc}

The library \texttt{primRec} contains a large catalogue of lemmas allowing to prove statements
of the form (\texttt{isPR $n$ $f$}). We won't list all these lemmas here, but give a few examples of
how they may be applied.

\begin{remark}
In the library \texttt{primRec}, all these lemmas are opaque (registered with \texttt{Qed}. Thus they do not allow the user to look at the witness of a proof of a \texttt{isPR} statement. Our example of page\pageref{sect:bigfac} was built using a  copy of \texttt{primRec.v} where many \texttt{Qed}s have been replaced with
\texttt{Defined}s.

If it does not cause compatibility problems (with
\href{https://github.com/coq-community/goedel}{goedel library} for instance), we plan to make all theses lemmas transparent.
\end{remark}

\subsubsection{Elementary proofs of \texttt{isPR} statements}

The constructors \texttt{zeroFunc}, \texttt{succFunc},  and \texttt{projFunc} of type
\texttt{PrimRec} allows us to write trivial proofs of primitive recursivity.
Although  the following lemmas are already proven in
\href{../theories/html/hydras.Ackermann.primRec.html}{Ackermann.primRec},
we wrote alternate proofs in
\href{../theories/html/hydras.MoreAck.PrimRecExamples.html}%
{Ackermann.MoreAck.PrimRecExamples.v}, in order to illustrate the main proof patterns.

\begin{Coqsrc}
Module Alt.

Lemma zeroIsPR : isPR 0 0.
Proof.
  exists zeroFunc.
\end{Coqsrc}

\begin{Coqanswer}
1 subgoal (ID 90)

  ============================
  extEqual 0 (evalPrimRec 0 zeroFunc) 0
\end{Coqanswer}

\begin{Coqsrc}
  cbn.
\end{Coqsrc}

\begin{Coqanswer}
1 subgoal (ID 91)

  ============================
  0 = 0
\end{Coqanswer}

\begin{Coqsrc}
  reflexivity.
Qed.
\end{Coqsrc}

Likewise, we prove that the successor function on \texttt{nat} is primitive recursive too.

\begin{Coqsrc}
Lemma SuccIsPR : isPR 1 S.
Proof.
  exists succFunc; cbn; reflexivity.
Qed.
\end{Coqsrc}

Projections are proved primitive recursive, case by case (many examples in
\href{../theories/html/hydras.Ackermann.primRec.html}{Ackermann.primRec}).
\emph{Please notice again that the name of the projection follows the mathematical tradition,
whilst the arguments of  \texttt{projFunc} use another convention (\emph{cf} remark~\vref{projFunc-order-of-args}).}

\begin{Coqsrc}
Lemma pi2_5IsPR : isPR 5 (fun a b c d e => b).
Proof.
 assert (H: 3 < 5) by auto.
 exists (projFunc 5 3 H).
 cbn; reflexivity.
Qed.
\end{Coqsrc}

Please note that the projection $\pi_{1,1}$ is just the identity on \texttt{nat}, and is realized by
(\texttt{projFunc 1 0}).

\vspace{4pt}
\noindent
\emph{From \href{../theories/html/hydras.Ackermann.primRec.html}{Ackermann.primRec}.}

\begin{Coqsrc}
Lemma idIsPR : isPR 1 (fun x : nat => x).
Proof.
  assert (H: 0 < 1) by auto.
  exists (projFunc 1 0 H); cbn; auto.
Qed.
\end{Coqsrc}

\subsubsection{Using function composition}

Let us look at the proof that any constant $n$ of type \texttt{nat} has type (\texttt{PR 0})
(lemma  \texttt{const1\_NIsPR} of \texttt{primRec}). We carry out a proof by induction on $n$, the base case of which is already proven.
Now, let us assume $n$ is \texttt{PR $n$}, with $x:\texttt{PrimRec}\,0$ as a ``realizer''.
Thus we would like to compose this constant function with the unary successor function.

This is exactly the role of the instance \texttt{composeFunc 0 1} of the dependently typed
function \texttt{composeFunc}, as shown by the following lemma.

\begin{Coqsrc}
Fact compose_01 :
    forall (x:PrimRec 0) (t : PrimRec 1),
    let c := evalPrimRec 0 x in
    let f := evalPrimRec 1 t in
    evalPrimRec 0 (composeFunc 0 1
                               (PRcons 0 0 x (PRnil 0))
                               t)  =
     f c.
Proof. reflexivity. Qed.
\end{Coqsrc}

Thus, we get a quite simple proof of \texttt{const1\_NIsPR}.

\vspace{4pt}
\noindent
\emph{From \href{../theories/html/hydras.MoreAck.PrimRecExamples.html}{MoreAck.PrimRecExamples}}.
\begin{Coqsrc}
Lemma  const1_NIsPR n : isPR 0 n.
Proof.
  induction n.
  - apply zeroIsPR.
  - destruct IHn as [x Hx].
   exists (composeFunc 0 1 (PRcons 0 0 x (PRnil 0)) succFunc).
   cbn in *; intros; now rewrite Hx.
Qed.
\end{Coqsrc}

\subsubsection{Proving that \texttt{plus} is primitive recursive}

The lemma \texttt{plusIsPR} is already proven in \href{../theories/html/hydras.Ackermann.primRec.html}{Ackermann.primRec}. We present in
\href{../theories/html/hydras.MoreAck.PrimRecExamples.html}{MoreAck.PrimRecExamples}
a commented version of this proof,

First, we look for lemmas which may help to prove that a given function obtained with the recursor \texttt{nat\_rec} is primitive recursive.

\begin{Coqsrc}
Search (is_PR 2 (fun _ _ => nat_rec _ _ _ _)).
\end{Coqsrc}

\begin{Coqanswer}
ind1ParamIsPR:
  forall f : nat -> nat -> nat -> nat,
  isPR 3 f ->
  forall g : nat -> nat,
  isPR 1 g ->
  isPR 2
    (fun a b : nat =>
     nat_rec (fun _ : nat => nat)
                 (g b) (fun x y : nat => f x y b) a)
\end{Coqanswer}

We prove that the library function \texttt{plus} is extensionally equal to a function defined with
\texttt{nat\_rec}.

\begin{Coqsrc}
Definition plus_alt x y  :=
              nat_rec  (fun n : nat => nat)
                       y
                       (fun z t =>  S t)
                       x.

Lemma plus_alt_ok:
  extEqual 2 plus_alt plus.
Proof.
  intro x; induction x; cbn; auto.
  intros y; cbn; now rewrite <- (IHx y).
Qed.
\end{Coqsrc}

A last lemma before the proof:

\begin{Coqsrc}
Lemma isPR_extEqual_trans n : forall f g, isPR n f ->
                                    extEqual n f g ->
                                    isPR n g.
Proof.
 intros f g [x Hx]; exists x.
 apply extEqualTrans with f; auto.
Qed.
\end{Coqsrc}

Let us start now.

\begin{Coqsrc}
Lemma plusIsPR : isPR 2 plus.
Proof.
  apply isPR_extEqual_trans with plus_alt.
  - unfold plus_alt; apply ind1ParamIsPR.
\end{Coqsrc}

\begin{Coqanswer}
2 subgoals (ID 126)

  ============================
  isPR 3 (fun _ y _ : nat => S y)

subgoal 2 (ID 127) is:
 isPR 1 (fun b : nat => b)
\end{Coqanswer}

We already proved that \texttt{S} is \texttt{PR 1}, but we need to consider a function of three arguments, which ignores its first and third arguments.
Fortunately, the library \texttt{primRec} already contains lemmas adapted to this kind of situation.

\begin{Coqanswer}
filter010IsPR :
forall g : nat -> nat, isPR 1 g -> isPR 3 (fun _ b _ : nat => g b)
\end{Coqanswer}

Thus, our first subgoal is solved easily. The rest of the proof
is just an application of already proven lemmas.

\begin{Coqsrc}
 - unfold plus_alt; apply ind1ParamIsPR.
    + apply filter010IsPR, succIsPR.
    + apply idIsPR.
  - apply plus_alt_ok.
Qed.
\end{Coqsrc}

\begin{todo}
Comment more examples from   \href{../theories/html/hydras.MoreAck.PrimRecExamples.html}{MoreAck.PrimRecExamples}.
\end{todo}

\index{primrec}{Exercises}
\begin{exercise}
There is a lot of lemmas similar to \texttt{filter010IsPR} in the \texttt{primRec} library, useful to control the arity of functions.
Thus, the reader may look at them, and invent simple examples of application for each lemma.
\end{exercise}

\index{primrec}{Exercises}
\begin{exercise}
Multiplication of natural number is already proven in the \texttt{primRec} library. Write a proof of your own, then compare to the library's version.
\end{exercise}

\subsubsection{More examples}

The following proof decomposes the \texttt{double} function as the composition of
multiplication with the identity and the constant function which returns $2$.
\emph{Note that the lemma \texttt{const1\_NIsPR} considers this function as an unary function (unlike \texttt{const0\_NIsPR})}.

\begin{Coqsrc}
Lemma doubleIsPR : isPR 1 double.
Proof.
  unfold double; apply compose1_2IsPR.
  - apply idIsPR.
\end{Coqsrc}

\begin{Coqanswer}
subgoal 1 (ID 110) is:
 isPR 1 (fun _ : nat => 2)
subgoal 2 (ID 111) is:
 isPR 2 Init.Nat.mul
\end{Coqanswer}

 \begin{Coqsrc}
  - apply const1_NIsPR.
  - apply multIsPR.
Qed.
\end{Coqsrc}

\index{primrec}{Exercises}
\begin{exercise}
Prove that the following functions are primitive recursive.

\begin{Coqsrc}
Fixpoint fact n :=
  match n with
          | 0 => 1
          | S p  => n * fact p
  end.

Fixpoint exp n p :=
  match p with
  | 0 => 1
  | S m =>  exp n m * n
  end.

Fixpoint tower2 n :=
  match n with
  | 0 => 1
  | S p => exp 2 (tower2 p)
  end.
\end{Coqsrc}

\textbf{Hint:} You may have to look again at the lemmas of the library
\href{../theories/html/hydras.Ackermann.primRec.html}{Ackermann.primRec} if you meet some difficulty.
You may start this exercise with the file
    \href{https://https://github.com/coq-community/hydra-battles/blob/master/exercises/primrec/MorePRExamples.v}{exercises/primrec/MorePRExamples.v}.
\end{exercise}

\index{primrec}{Exercises}
\begin{exercise}
Show that the function \texttt{min: naryFunc\,2} is primitive
recursive.

\emph{You may start this exercise with
    \href{https://https://github.com/coq-community/hydra-battles/blob/master/exercises/primrec/MinPR.v}{exercises/primrec/MinPR.v}.}

\end{exercise}

\index{primrec}{Exercises}

\begin{exercise}
Write a simple and readable proof that the Fibonacci function is primitive recursive.

\begin{Coqsrc}
Fixpoint fib (n:nat) : nat :=
  match n with
  | 0 => 1
  | 1 => 1
  | S ((S p) as q) => fib q + fib p
  end.
\end{Coqsrc}

\textbf{Hint:}  You may use as a helper the function which computes the pair \linebreak
$(\texttt{fib}(n+1),\texttt{fib}(n))$.
Library \href{../theories/html/hydras.Ackermann.cPair.html}{Ackermann.cPair} contains
the definition of the encoding of $\mathbb{N}^2$ into $\mathbb{N}$, and the proofs that
the associated constructor and projections are primitive recursive.
\emph{You may start this exercise with the file
    \href{https://github.com/coq-community/hydra-battles/blob/master/exercises/primrec/FibonacciPR.v}{exercises/primrec/FibonacciPR.v}.}

\end{exercise}

\index{primrec}{Exercises}
\begin{exercise}
Prove the following lemmas (which may help to solve the next  exercise).

\begin{Coqsrc}
Lemma boundedSearch3 (P: naryRel 2) (b  : nat),
    boundedSearch P b <= b.

Lemma boundedSearch4 (P: naryRel 2) (b  : nat):
  P b b = true -> P b (boundedSearch P b) = true.
\end{Coqsrc}
\end{exercise}

\index{primrec}{Exercises}

\begin{exercise}
Prove that the function which returns the  integer square root of any natural number  is primitive recursive.

\emph{You may start this exercise with the file
    \href{https://github.com/coq-community/hydra-battles/blob/master/exercises/primrec/isqrt.v}{exercises/primrec/isqrt.v}.}

\end{exercise}

\section{Proving that a given function is \emph{not} primitive recursive}
\label{sect:ack-not-PR}

The best known example of a total recursive function which is not primitive recursive is the Ackermann function. We show how to adapt the classic proof (see for instance~\cite{planetmath}) to the constraints of \gallina. We hope this formal proof
 is a nice opportunity to explore
the treatment of primitive recursive functions by R. O'Connor,
and to play with dependent types.

\subsection{Ackermann function}

Ackermann function is traditionally defined as a function from
$\mathbb{N}\times \mathbb{N}$ into $\mathbb{N}$, through
three equations:

\begin{align}
A(0,n)&=n+1\\
A(m+1,0)&=A(m,1)\\
A(m+1,n+1)&=A(m,A(m+1,n))
\end{align}

Let us try to define this function in \coq{} (in curried form).

\begin{Coqsrc}
Fail
  Fixpoint Ack (m n : nat) : nat :=
  match m, n with
  | 0, n => S n
  | S m, 0 => Ack m 1
  | S m0, S p => Ack m0 (Ack m p)
  end.
\end{Coqsrc}

\begin{Coqanswer}
The command has indeed failed with message:
Cannot guess decreasing argument of fix.
\end{Coqanswer}

A possible workaround is to make \texttt{m} be the
decreasing argument, and define --- within \texttt{m}'s scope --- a local helper function which computes (\texttt{Ack m n}) for any \texttt{n}.
This way, both functions \texttt{Ack} and \texttt{Ackm} have a (structurally) strictly decreasing argument.

\begin{Coqsrc}
Module Alt.

   Fixpoint Ack (m n : nat) : nat :=
     match m with
     | O => S n
     | S p => let fix Ackm (n : nat) :=
                  match n with
                  | O => Ack p 1
                  | S q => Ack p (Ackm q)
                  end
              in Ackm n
     end.

    Compute Ack 3 2.
\end{Coqsrc}

\begin{Coqanswer}
  = 29 : nat
\end{Coqanswer}

\begin{Coqsrc}
End Alt.
\end{Coqsrc}

We prefered to define a variant which uses explicitely
 the functional \texttt{iterate},
where (\texttt{iterate\,$f$\,$n$})
is the $n$-th iteration of $f$\,\footnote{Please not confuse with \texttt{primRec.iterate}, which is monomorphic and does not share the same order of arguments.}. It makes it possible to apply a few lemmas proved in
\href{../theories/html/hydras.Prelude.Iterates.html}{Prelude.Iterates}, for instance about the monotony of the $n$-th iterate of a given function.

\vspace{4pt}
\noindent
\emph{From \href{../theories/html/hydras.Prelude.Iterates.html}{Prelude.Iterates}}.
\index{hydras}{Library Prelude!iterate}
\begin{Coqsrc}
Fixpoint iterate {A:Type}(f : A -> A) (n: nat)(x:A) :=
  match n with
  | 0 => x
  | S p => f (iterate  f p x)
  end.
\end{Coqsrc}

\begin{Coqsrc}
Lemma iterate_le_n_Sn:
forall f : nat -> nat,
(forall x : nat, x <= f x) ->
forall n x : nat, iterate f n x <= iterate f (S n) x.
\end{Coqsrc}

Thus, our definition of the Ackermann function is as follows:

\vspace{4pt}
\noindent
\emph{From \href{../theories/html/hydras.MoreAck.Ack.html}{MoreAck.Ack}}.
\index{maths}{Ackermann function}
\index{primrec}{Ackermann function}

\begin{Coqsrc}
Fixpoint Ack (m:nat) : nat -> nat :=
  match m with
  | 0 => S
  | S n => fun k =>  iterate (Ack n) (S k) 1
  end.

Compute Ack 3 2.
\end{Coqsrc}

\begin{Coqanswer}
  = 29 : nat
\end{Coqanswer}

\index{hydras}{Exercises}

\begin{exercise}
The file \href{../theories/html/hydras.MoreAck.Ack.html}{MoreAck.Ack} presents two other definitions\footnote{post on \href{https://stackoverflow.com/questions/10292421/error-in-defining-ackermann-in-coq} by Anton Trunov.} of the Ackermann functions based on the lexicographic ordering on $\mathbb{N}\times\mathbb{N}$.
Prove that the four functions are extensionnally equal.
\end{exercise}

\subsubsection{First properties of the Ackermann function}

The three first lemmas make us sure that our function
\texttt{Ack} satifies the ``usual'' equations.

\begin{Coqsrc}
Lemma Ack_0 : Ack 0 = S.
Proof refl_equal.

Lemma Ack_S_0 m : Ack (S m) 0 = Ack m 1.
Proof. reflexivity. Qed.

Lemma Ack_S_S : forall m p,
    Ack (S m) (S p) = Ack m (Ack (S m) p).
Proof. reflexivity. Qed.
\end{Coqsrc}

The order of growth of the Ackermann function w.r.t. its first argument is illustrated by the following equalities.

\begin{Coqsrc}
Lemma Ack_1_n n: Ack 1 n = S  (S n).

Lemma Ack_2_n n: Ack 2 n = 2 * n + 3.

Lemma Ack_3_n n: Ack 3 n = exp2 (S (S (S n))) - 3.

Lemma Ack_4_n n: Ack 4 n = hyper_exp2 (S (S (S n))) - 3.
\end{Coqsrc}

\begin{remark}
 The statements above can be rewritten in a more uniform way:

 \begin{quote}
   For $m\in 1..4$, $\texttt{Ack}\,m\,n = f_m\,(n+3)-3$, where
   \begin{align*}
   f_1(n)=&\,n+2 \\
   f_2(n)=&\,n\times 2\\
   f_3(n)=&\,2^n\\
   f_4(n)=&\,2^{2^{\dots^2}}\quad(n\;\textit{levels})
   \end{align*}
 \end{quote}
\end{remark}

An important property of the Ackermann function helps us
to overcome the difficulty raised by nested recursion, by climbing up the hierarchy $\texttt{Ack}\,n\,\_\;(n\in\mathbb{N})$.

\noindent
\emph{From \href{../theories/html/hydras.MoreAck.Ack.html}{MoreAck.Ack}}.
\begin{Coqsrc}
Lemma nested_Ack_bound : forall k m n,
    Ack k (Ack m n) <= Ack (2 + max k m) n.
\end{Coqsrc}

Please note also that for any given $n$, the unary function
(\texttt{Ack\,$n$}) is primitive recursive.

\vspace{4pt}

\noindent
\emph{From \href{../theories/html/hydras.MoreAck.AckNotPR.html}{MoreAck.AckNotPR}}.
\begin{Coqsrc}
Theorem Ackn_IsPR (n: nat) : isPR 1 (Ack n).
\end{Coqsrc}

\subsection{A proof by induction on all primitive recursive functions}

Ìn order to prove that \texttt{Ack} (considered as a function of two arguments) is not primitive recursive, the usual method consists in two steps:

\begin{enumerate}
\item Prove that for any primitive recursive function $f:\mathbb{N}\rightarrow\mathbb{N}\rightarrow\mathbb{N}$, there exists some natural number $n$ depending on $f$, such that, for any $x$ and $y$,
$f\,x\,y \leq \texttt{Ack}\,n\,(\textrm{max}\,x\,y)$ (we say that $f$ is \emph{``majorized''}  by \texttt{Ack}).
\item Show that \texttt{Ack} fails to satisfy this property.
\end{enumerate}

First, we prove that any primitive function of two arguments is majorized by \texttt{Ack}.
If we look at the inductive definition of primitive recursive functions, page~\pageref{def:Primrec}, it is obvious that a proof by induction on the construction of primitive recursive functions must consider functions of any arity.

The following scheme allows us to write proofs by induction on the class of primitive recursive functions.

\vspace{4pt}
\noindent
\emph{From \href{../theories/html/hydras.Ackermann.primRec.html}{Ackermann.primRec}.}

\index{coq}{Commands!Scheme}
\begin{Coqsrc}
Scheme PrimRec_PrimRecs_ind := Induction for PrimRec
  Sort Prop
  with PrimRecs_PrimRec_ind := Induction for PrimRecs
  Sort Prop.
\end{Coqsrc}

\begin{Coqanswer}
PrimRec_PrimRecs_ind :
forall (P : forall n : nat, PrimRec n -> Prop)
  (P0 : forall n n0 : nat, PrimRecs n n0 -> Prop),
(* successor *)
P 1 succFunc ->

(* zero *)
P 0 zeroFunc ->

(* projections *)
(forall (n m : nat) (l : m < n), P n (projFunc n m l)) ->

(* composition *)
(forall (n m : nat) (g : PrimRecs n m),
      P0 n m g -> forall h : PrimRec m, P m h ->
      P n (composeFunc n m g h)) ->

(* primitive recursion *)
(forall (n : nat) (g : PrimRec n),
 P n g ->
    forall h : PrimRec (S (S n)), P (S (S n)) h ->
     P (S n) (primRecFunc n g h)) ->

(* empty list of functions *)
(forall n : nat, P0 n 0 (PRnil n)) ->

(* add a function to a list *)
(forall (n m : nat) (p : PrimRec n),
   P n p ->
   forall p0 : PrimRecs n m, P0 n m p0 ->
   P0 n (S m) (PRcons n m p p0)) ->

(* conclusion ! *)
forall (n : nat) (p : PrimRec n), P n p
\end{Coqanswer}

For instance, proving a property shared by any primitive recursive function of arity 2 leads to consider the case where that function is obtained by composition with a function of any
arity $m$. The same problem happens with primitive
recursion, where a function of arity $n$ is built out of a function of arity $n+1$ and a function of arity $n-1$.

Thus the lemma we will have to prove is the following one:

  \begin{quote}
    For any $n$, and any primitive recursive function $f$ of  arity $n$, there exists some natural number $q$ such that the following inequality holds:
 \[
  \forall x_1,\dots,x_n,
      f(x_1,\dots,\,x_n)\leq\textrm{Ack}(q,\textrm{max}(x_1,\dots,x_n))
\]
 \end{quote}

But dots don't belong to \gallina's syntax! So, we may use \coq's vectors for denoting arbitrary tuples.

First, we extend \texttt{max} to vectors of natural numbers (using the notations of module \texttt{VectorNotations} and some more definitions from
\href{../theories/html/hydras.Prelude.MoreVectors.html}{Prelude.MoreVectors}). So, (\texttt{t\,$A$\,$n$}) is the type of vectors of $n$ elements of type $A$, and the constants \texttt{cons}, \texttt{nil}, \texttt{map}, etc., refer to vectors and not to lists. Likewise, the notation \texttt{x::v} is an abbreviation for
\texttt{VectorDef.cons x \_ v}.

\index{coq}{Dependently typed functions}
\begin{Coqsrc}
Fixpoint max_v {n:nat} : forall (v: Vector.t nat n) , nat :=
  match n as n0 return (Vector.t nat n0 -> nat)
  with
    0 => fun v => 0
  | S p => fun (v : Vector.t nat (S p)) =>
             max (Vector.hd v) (max_v  (Vector.tl v))
  end.

Lemma max_v_2 : forall x y,  max_v (x::y::nil) = max x y.

Lemma max_v_lub : forall n (v: t nat n) y,
    (Forall (fun x =>  x <= y) v) -> max_v v <= y.

Lemma max_v_ge : forall n (v: t nat n) y,  In  y  v -> y <= max_v v.
\end{Coqsrc}

We have also to convert any application
$(f\,x_1\,x_2\,\dots\,x_n)$ into an application of a function
to a single argument: the vector of all the $x_i$\,s.
This is already defined in
Library~\href{../theories/html/hydras.Ackermann.primRec.html}{Ackermann.primRec}.

\begin{Coqsrc}
Fixpoint evalList (m : nat) (l : Vector.t nat m) {struct l} :
 naryFunc m -> nat :=
  match l in (Vector.t _ m) return (naryFunc m -> nat) with
  | Vector.nil => fun x : naryFunc 0 => x
  | Vector.cons a n l' => fun x : naryFunc (S n) => evalList n l' (x a)
  end.
\end{Coqsrc}

Indeed, (\texttt{evalList $m$ $v$ $f$}) is the application to the vector $v$ of
an uncurried version of $f$.
In Library\href{../theories/html/hydras.MoreAck.AckNotPR.html}{MoreAck.AckNotPR}, we introduce a lighter notation.

\index{coq}{Dependently typed functions}
\begin{Coqsrc}
(**  uncurried apply:

[v_apply f (x1::x2:: ... ::xn::nil)]  is [f x1 x2 ... xn]
 *)

Notation "'v_apply' f v" := (evalList _ v f)
     (at level 10, f at level 9).

Example Ex2: forall (f: naryFunc 2) x y,
    v_apply f (x::y::nil) = f x y.
Proof.   intros; now cbn. Qed.

Example Ex4: forall (f: naryFunc 4) x y z t,
    v_apply f (x::y::z::t::nil) = f x y z t.
Proof.  intros; now cbn. Qed.
\end{Coqsrc}

We are now able to translate in \gallina{} the notion of ``majorization'':

\index{coq}{Dependently typed functions}
\begin{Coqsrc}
Definition majorized {n} (f: naryFunc n) (A: naryFunc 2) : Prop :=
  exists (q:nat), forall (v: t nat n),
      v_apply f v <= A q  (max_v v).

Definition majorizedPR {n} (x: PrimRec n) A :=
           majorized (evalPrimRec n x) A.

(** For vectors of functions *)

Definition majorizedS {n m} (fs : Vector.t (naryFunc n) m)
           (A : naryFunc 2):=
  exists N, forall (v: t nat n),
      max_v (map (fun f => v_apply f v) fs) <= A N (max_v v).

Definition majorizedSPR {n m} (x : PrimRecs n m) :=
  majorizedS (evalPrimRecs _ _ x).
\end{Coqsrc}

Now, it remains to prove that any primitive function is majorized by \texttt{Ack}.
The three base cases  are as follows:

\begin{Coqsrc}
Lemma majorSucc : majorizedPR  succFunc Ack.

Lemma majorZero : majorizedPR  zeroFunc Ack.

Lemma majorProjection (n m:nat)(H: m < n):
  majorizedPR (projFunc n m H) Ack.
\end{Coqsrc}

The rest of the cases are proved within a mutual  induction.

\index{coq}{Mutual induction}

\begin{Coqsrc}
Lemma majorAnyPR:  forall n (x: PrimRec n),  majorizedPR  x Ack.
Proof.
  intros n x; induction x using PrimRec_PrimRecs_ind with
                  (P0 := fun n m y => majorizedSPR  y Ack).
  - apply majorSucc.
  - apply majorZero.
  - apply majorProjection.
  \end{Coqsrc}

  \begin{Coqsrc}
  - (** function composition *)
\end{Coqsrc}
\begin{Coqanswer}
1 subgoal (ID 265)

  n, m : nat
  g : PrimRecs n m
  x : PrimRec m
  IHx : majorizedSPR g Ack
  IHx0 : majorizedPR x Ack
  ============================
  majorizedPR (composeFunc n m g x) Ack
\end{Coqanswer}

\begin{Coqsrc}
  ...
 - (** primitive recursion *)
\end{Coqsrc}

\begin{Coqanswer}
1 subgoal (ID 265)

  n : nat
  x1 : PrimRec n
  x2 : PrimRec (S (S n))
  IHx1 : majorizedPR x1 Ack
  IHx2 : majorizedPR x2 Ack
  ============================
  majorizedPR (primRecFunc n x1 x2) Ack
\end{Coqanswer}

\begin{Coqsrc}
 assert (L1 : forall i (v: t nat n) ,
               v_apply f (i::v)  <= Ack q (i + max_v v)).
    { induction i.
      ...
    }
    ...
-  (** empty list of functions *)
\end{Coqsrc}

\begin{Coqanswer}
1 subgoal (ID 266)

  n : nat
  ============================
  majorizedSPR (PRnil n) Ack
\end{Coqanswer}

\begin{Coqsrc}
  ...
- (** non-empty list of functions *)
\end{Coqsrc}

\begin{Coqanswer}
1 subgoal (ID 273)

  n, m : nat
  x : PrimRec n
  p : PrimRecs n m
  IHx : majorizedPR x Ack
  IHx0 : majorizedSPR p Ack
  ============================
  majorizedSPR (PRcons n m x p) Ack
\end{Coqanswer}

\begin{Coqsrc}
 ...
Qed.
\end{Coqsrc}

\subsection{Looking for a contradiction}

The following lemma is just a specialization of \texttt{majorAnyPR} to
binary functions (forgetting vectors, coming back to usual notations).

\begin{Coqsrc}
Lemma majorPR2 (f: naryFunc 2)(Hf : isPR 2 f)
  : exists (n:nat), forall x y,  f x y <= Ack n (max x  y).
\end{Coqsrc}

We prove also a strict version of this lemma, thanks to the following property (proved in Library
\href{../theories/html/hydras.MoreAck.Ack.html}{MoreAck.Ack}~).

\begin{Coqsrc}
Lemma Ack_strict_mono_l : forall n m p, n < m ->
                                        Ack n (S p) < Ack m (S p).
\end{Coqsrc}

\vspace{4pt}
\noindent
\emph{From \href{../theories/html/hydras.MoreAck.AckNotPR.html}{MoreAck.AckNotPR}.}

\begin{Coqsrc}
Lemma majorPR2_strict (f: naryFunc 2)(Hf : isPR 2 f):
    exists (n:nat),
    forall x y, 2 <= x -> 2 <= y -> f x y < Ack n (max x  y).
\end{Coqsrc}

If the Ackermann function were primitive recursive, then there would exist some natural number $n$, such that, for all $x$ and $y$, the inequality
$\texttt{Ack}\,x\,y\leq \texttt{Ack}\,n\,(\texttt{max}\,x\,y)$ holds.
Thus, our impossibility proof is just a sequence of easy small steps.

\begin{Coqsrc}
Section Impossibility_Proof.

  Hypothesis HAck : isPR 2 Ack.

  Lemma Ack_not_PR : False.
  Proof.
    destruct (majorPR2_strict Ack HAck) as [m Hm];
    pose (X := max 2 m); specialize (Hm X X).
    rewrite max_idempotent in Hm;
    assert (Ack m X <= Ack X X) by (apply Ack_mono_l; lia).
    lia.
  Qed.

End Impossibility_Proof.
\end{Coqsrc}

\begin{remark}
It is easy to prove that any unary function which dominatates \texttt{fun n => Ack n n} fails to be primitive recursive. We use an instance of \texttt{majorAnyPR} for unary functions.

\vspace{4pt}
\noindent

\emph{From \href{../theories/html/hydras.MoreAck.AckNotPR.html}{MoreAck.AckNotPR}}.
\begin{Coqsrc}
Lemma majorPR1  (f: naryFunc 1)(Hf : isPR 1 f)
  : exists (n:nat), forall x, f x <= Ack n x.
(* ... *)
\end{Coqsrc}

Then, we write  a short proof by contradiction.

\begin{Coqsrc}
Section dom_AckNotPR.

  Variable f : nat -> nat.
  Hypothesis Hf : dominates f (fun n => Ack n n).

 Lemma dom_AckNotPR: isPR 1 f -> False.
  Proof.
    intros H;  destruct Hf as [N HN].
    destruct  (majorPR1 _ H) as [M HM].
    pose (X := Max.max N M).
    specialize (HN X  (Max.le_max_l N M)).
    specialize (HM X);
      assert (Ack M X <= Ack X X) by (apply Ack_mono_l; subst; lia).
    lia.
  Qed.

End dom_AckNotPR.
\end{Coqsrc}
\end{remark}

\begin{remark}
Note that the Ackermann function is a counter-example to the (false) statement:
\begin{quote}
{\color{red}
  ``Let $f$ be a function of type \texttt{naryFunc\,2}. If, for any $n$, the fonction $f(n)$ is primitive recursive, then f is primitive recursive.''}
\end{quote}
\end{remark}

\section{The length of standard hydra battles}
\label{sect:battle-length-notPR}

The module \href{../theories/html/hydras.Hydra.Hydra_Theorems.html}{Hydra\_Theorems} contains a proof that the function which computes the length of standard hydra battles is not primitive recursive. More precisely, we consider, for a given hydra $h=\iota(\alpha)$, the length of a standard battle which starts with the replication factor $k$ (see Sect~\vref{def:L-alpha}).

This proof is  a little more complex than the preceding one.

\subsection{Definitions}

The function we consider is defined and proven correct in
Module~\href{../theories/html/hydras.Hydra.Battle_length.html}{Hydra.Battle\_length}.

\begin{Coqsrc}
Definition l_std alpha k := (L_ alpha (S k) - k)%nat.

Lemma l_std_ok : forall alpha : E0,
    alpha <> Zero ->
    forall k : nat,
      1 <= k -> battle_length standard k (iota (cnf alpha))
                              (l_std alpha k).
(* ... *)
\end{Coqsrc}

\subsection{Proof steps}

Now, let us assume that the function \texttt{l\_std} is primitive recursive.

\emph{From \href{../theories/html/hydras.Hydra.Hydra_Theorems.html}{Hydra.Hydra\_Theorems}}.

\begin{Coqsrc}
Section battle_lenght_notPR.

  (** We assume that the function with computes the length
      of standard battles is primitive recursive *)

  Hypothesis H: forall alpha, isPR 1 (l_std alpha).
\end{Coqsrc}

Let us consider the hydra represented by the ordinal $\omega^\omega$.

\begin{Coqsrc}
Let alpha := Phi0 omega%e0.
Let h := iota (cnf alpha).
\end{Coqsrc}

In order to get rid of the substraction in the definition of \texttt{l\_std}, we work with a helper function.

\begin{Coqsrc}
Let m k := L_ alpha (S k).

Remark m_eqn : forall k, m k = (l_std alpha k + k)%nat.
(* ... *)
\end{Coqsrc}

Under the hypothesis \texttt{H}, $m$ is also primitive recursive.

\begin{Coqsrc}
Remark mIsPR : isPR 1 m.
(* ... *)
\end{Coqsrc}

\subsubsection{Comparison between $F$ and $H'$}

In \href{../theories/html/hydras.Epsilon0.F_alpha.html}{Epsilon0.F\_alpha}, we prove a relation between the $F$ and $H'$ functional. For any $\alpha$ and $k>0$,
$H'_{\omega^\alpha}(k)\geq F_\alpha(k)$.

\begin{Coqsrc}
H'_F : forall (alpha : E0) (n : nat),
     F_ alpha (S n) <= H'_ (Phi0 alpha) (S n)
\end{Coqsrc}

Our proof of this lemma is not trivial at all. One of it sub-goals is the following one:

\begin{Coqsrc}
  alpha : E0
  IHalpha : forall beta : E0, beta o< alpha -> P beta
  Halpha : Limitb alpha
  n : nat
  ============================
  H'_ (Phi0 (CanonS alpha n)) (S n) <=
  H'_ (Phi0 (CanonS alpha (S n))) (S n)
\end{Coqsrc}

In mathematical notation: $H'_{\omega^{\canonseq{\alpha}{n}}}(n+1) \leq
H'_{\omega^{\canonseq{\alpha}{n+1}}}(n+1)$.

\vspace{4pt}

But there exists no lemma saying that, if
$\beta\leq \alpha$, then $H'_\beta(k)\leq H'_\alpha(k)$, for any $\alpha$ and $\beta$. For instance,
$H'_{42}(3)=45> H'_\omega(3)=7$.

Looking for lemmas of the form $H'_\beta(k)\leq H'_\alpha(k)$, we find this one (from our library
\href{../theories/html/hydras.Epsilon0.Hprime.html}{Epsilon0.Hprime}):

\begin{Coqanswer}
H'_restricted_mono_l :
    forall (alpha beta : E0) (n : nat),
      Canon_plus n alpha beta ->
      H'_ beta n <= H'_ alpha n.
\end{Coqanswer}

Thus, it remains to prove that
there exists a path from ${\omega^{\canonseq{\alpha}{n+1}}}$
to ${\omega^{\canonseq{\alpha}{n}}}$ composed of
$n+1$-steps.

Fortunately, the Ketonen-Solovay machinery contains three lemmas which help us to build such a path.

\begin{Coqanswer}
KS_thm_2_4_lemma5 :
  forall [i : nat] [alpha beta : T1],
  const_pathS i alpha beta ->
  nf alpha -> alpha <> zero ->
  const_pathS i (phi0 alpha) (phi0 beta)

KS_thm_2_4 :
  forall [lambda : T1], nf lambda ->limitb lambda ->
  forall i j : nat, i < j ->
   const_pathS 0 (canonS lambda j) (canonS lambda i)

Cor12_1 :
forall [alpha : T1], nf alpha ->
      forall (beta : T1) (i n : nat),
      beta t1< alpha ->
     i <= n -> const_pathS i alpha beta ->
     const_pathS n alpha beta
\end{Coqanswer}

\subsubsection{End of the proof}

We finish the proof by comparing several fast growing functions.

\emph{From \href{../theories/html/hydras.Epsilon0.L_alpha.html}{Epsilon0.L\_alpha}}

\begin{Coqsrc}
H'_L_ : forall (alpha : E0) (i : nat), H'_ alpha i <= L_ alpha (S i)
\end{Coqsrc}

\emph{From \href{../theories/html/hydras.Epsilon0.F_omega.html}{Epsilon0.F\_omega}}

\begin{Coqsrc}
F_vs_Ack : forall n : nat, 2 <= n -> Ack n n <= F_ omega n
\end{Coqsrc}

By transitivity, we get the inequality
$F_\omega(k+1)\leq m(k+1)$, for any $k$.

\begin{Coqsrc}
Remark m_ge_F_omega : forall k,  F_ omega (S k) <= m (S k).
\end{Coqsrc}

We finish the proof by noting that the function $m$ (composed with \texttt{S}) dominates the Ackermann function, which leads to a contradiction.

\begin{Coqsrc}
Remark m_dominates_Ack :
     dominates (fun n =>  S (m n)) (fun n => Ack.Ack n n).
(* ... *)

Remark SmNotPR : isPR 1 (fun n => S (m n)) -> False.
(* ... *)

Theorem LNotPR : False.
  Proof.
    apply SmNotPR,  compose1_1IsPR.
    - apply mIsPR.
    - apply succIsPR.
  Qed.

End battle_lenght_notPR.
\end{Coqsrc}


\section{Random Walk on a Grid}
A `particle' takes a random walk on a $4 \times 8$ grid. At each step of the walk, the particle may move up, down, left, or right with equal probability. The particle continues this process until it exits the grid. If the particle exits the top of the grid, the particle scores 1 point for the walk. If the particle exits the left, bottom or right sides of the grid, the particle scores 0 points for the walk.
In this project, you will compute the expected score of the particle as a function of the starting coordinates.\\

\noindent \textit{Hint:} For each starting point, you may estimate the expected score of the particle by simulating many, many random paths and taking their average score.\\
\begin{center}
\begin{tikzpicture}
    \pgfmathsetmacro{\N}{8}
    \pgfmathsetmacro{\M}{4}
    \draw[thick, densely dotted] (1,1) grid (\N,\M);
    \draw[line width = 3,color = green, opacity = .5,->] (3,2) -- (4,2) --(4,3) --(5,3) -- (6,3) --(6,4) --(5,4) -- (5,4.5);
    \draw[line width = 3,color = purple, opacity = .5,->] (3,2) -- (3,1) --(2,1) --(2,0.5);
    \draw[line width = 3,color = orange, opacity = .5,->] (3,2) -- (2,2) --(2,3) --(1,3) -- (1,2)--(0.5,2);
    % Right
    \foreach \j in {1,...,\M}
    {
    \draw[thick, densely dotted] (\N+1,\j) -- (\N,\j);
    \draw[ultra thick,color=orange!50!black,fill = white] (\N+1,\j) node {0} circle (.3);
    }
    % Top
    \foreach \i in {1,...,\N}
    {
        \draw[thick, densely dotted] (\i,\M) -- (\i,\M+1);
        \draw[ultra thick,color=blue!50!black,fill = white] (\i,\M+1) node {1} circle (.3);
    }
    % Left
    \foreach \j in {1,...,\M}
    {
        \draw[thick, densely dotted] (1,\j) -- (0,\j);
        \draw[ultra thick,color=orange!50!black,fill = white] (0,\j) node {0} circle (.3);
    }
    % Bottom
    \foreach \i in {1,...,\N}
    {
        \draw[thick, densely dotted] (\i,0) -- (\i,1);
        \draw[ultra thick,color=orange!50!black,fill = white] (\i,0) node {0} circle (.3);
    }
    \draw[color = black,fill=black!70!white,opacity = .7] (3,2) circle  (.2);
\end{tikzpicture}
\end{center}
\textit{Illustration:} The figure shows three sample paths of the random walk that all begin at the initial point (3,2). For the red path, the particle (randomly) takes the steps $(\downarrow, \leftarrow, \downarrow)$, and exits the grid at the bottom scoring zero points for the walk. The other paths are the results of the steps:
\begin{align*}
&\text{Steps: }  (\downarrow \ \leftarrow \ \downarrow) &&\text{(See red path)} && \text{Exit: Bottom} && \text{Score: 0}\\
&\text{Steps: }  (\leftarrow \ \uparrow \ \leftarrow \ \downarrow \ \leftarrow) &&\text{(See orange path)} && \text{Exit: Left} && \text{Score: 0}\\
&\text{Steps: }  (\rightarrow \ \uparrow \ \rightarrow \ \rightarrow \ \uparrow \ \leftarrow \ \uparrow) &&\text{(See green path)}&& \text{Exit: Top} && \text{Score: 1}
\end{align*}

\noindent\textit{Bonus:} In this project, we have computed the solution to a problem by simulating many random events. This `indirect' approach is always computationally inefficient, so it is often worth looking for a `direct' approach. For this problem, there \textit{is} a method of computing the solution directly. Can you find it?


\subsection{Degree of Connectedness}
  We would like to have a measure of how well connected is the graph. A network with a lot of disconnected cliques should
  score lower than a loosely connected network. This measure can be expressed as the expected indirect trust, defined
  incrementally as follows:
  \begin{align*}
    Tr(A) = \sum\limits_{B \in \mathcal{V} \setminus \{A\}}Tr_{A \rightarrow B} && \mbox{ (Total indirect trust for player
    }A\mbox{)} \\
    ETr\left(A\right) = \frac{Tr(A)}{|\mathcal{V} \setminus \{A\}|} = \frac{Tr(A)}{|\mathcal{V}| - 1} && \mbox{ (Expected
    indirect trust for player }A\mbox{)} \\
    ETr = \frac{1}{|\mathcal{V}|}\sum\limits_{A \in \mathcal{V}}ETr\left(A\right) && \mbox{ (Expected indirect trust)}
  \end{align*}


\chapter{Selection}

Consider the following problem: given an array $A$ of $n$ elements,
output the $i$-th smallest element of $A$.

As a simple first solution, we can sort $A$ and then return $A[i]$.
Since sorting takes $O(nlogn)$ time and returning $A[i]$ takes $O(1)$
time, this solution takes $O(nlogn) + O(1) = O(nlogn)$ time.

But it should be easy to see that we can do better in specific cases
like $i=1$ or $i=n$.  Simply iterate once over the array and store the
minimum ($i=1$) or maximum ($i=n$) value.  Since looking at a
particular element of the array takes $O(1)$ time, and we look at all
$n$ elements, this takes total $n \cdot O(1) = O(n)$ time.

We can also tell that this is optimal, because we know that to
determine the $i$-th element, we need to look at all $n$ elements in
the array, so we have a lower bound of $\Omega (n)$ time.

But is this possible in general, for any value of $i$?  Yes.

Suppose in linear time we can find element $x$ such that

{
  % Current graphic is hand-drawn by John Howat.
  % Should replace asap with a nicer diagram
  \includegraphics[scale=0.6]{selection}
  %\caption{Desirable properties of element x.}
  \label{fig:selection}
}

$x$ is somewhere around the middle of the array, and is preceded only
by elements smaller than $x$, and followed only by elements larger
than $x$.  We also know that there are $ \geq (1-\alpha)n $ and $ \leq
\alpha n $ elements both before and after $x$.

We can calculate this $x$ as follows:

\begin{enumerate}

\item Split $A$ into groups of 5.  There will be $\frac{n}{5}$ of
  these groups.
\item Compute the median $m_j$ of each group $M_j$ for $ 1 \leq j \leq
  \frac{n}{5} $.
\item Compute the median $x$ of $m_1,m_2,...,m_{n/5}$.

\end{enumerate}

It should be clear that step 1 takes constant time, step 2 takes
constant time for each group of constant size and $O(n)$ time total
for all $\frac{n}{5}$ groups, and step 3 takes $T(n/5)$ time.

\begin{claim}
This $x$ has the properties we needed above.
\end{claim}

\begin{proof}

We know that $\frac{1}{2}$ of $m_j$ are smaller than $x$, and since
there are $ \frac{n}{5} m_j$s, we know $\frac{n}{10}$ of $m_j$ are $
\leq x $.

So for each $m_j$ where $m_j \leq x$

\begin{itemize}

\item there are 3 elements that are $ \leq m_j $
\item so 3 (or more) elements are $ \leq x $

\end{itemize}
%
\end{proof}
%
Now we must put $x$ into its position in the array using partitioning.
As a side note, partitioning is used in quicksort.

\begin{enumerate}

\item Find $x$, put it at the end
\item Partition elements around $x$
\item Put $x$ into its proper position

\end{enumerate}

We now have an $x$ that satisfies the properties we needed, and it is
properly located at position $q$ in $A$.  We are left with 3 cases:

\begin{enumerate}

\item If $i = q$: $x$ is the $i$-th element of $A$.
\item If $i < q$: recurse on the subarray which is $ < x $

\item If $i > q$: recurse on the subarray which is $ > x $, and $ i
  \leftarrow i - q $

\end{enumerate}

This last step gives a recurrence of $ T \left( \frac{7n}{10} \right)
$ in the worst case because at least $ \frac{3n}{10} $ elements in $A$
are smaller than $x$.

\section{Analysis}

We now have the following recurrence:
%
\begin{displaymath}
T(n) = T \left( \frac{n}{5} \right) +T \left( \frac{7n}{10} \right) + O(n)
\end{displaymath}

\begin{claim}
$ T(n) \leq cn $
\end{claim}

\begin{proof}
\begin{align*}
T(n)
&= dn + T \left( \frac{n}{5} \right) +T \left( \frac{7n}{10} \right) \\
&\leq dn + c \frac{n}{5} + c \frac{7n}{10} \\
&= dn + \frac{9}{10}cn \\
&= cn \left( \frac{d}{c} + \frac{9}{10} \right)
&\leq cn
\end{align*}

As long as

\begin{align*}
\frac{d}{c} + \frac{9}{10} &\leq 1 \\
\frac{d}{c} &\leq \frac{1}{10} \\
10 d &\leq c
\end{align*}
\end{proof}

\section{What is special about 5?}

First of all, we need an odd number for there to be a median.
Secondly, notice:
%
\begin{displaymath}
\frac{1}{5} + \frac{7}{10} = \frac{9}{10} < 1
\end{displaymath}

Dividing into groups of 3 doesn't work because:
%
\begin{displaymath}
T(n) = O(n) + T(n/3) + T(2n/3) = \Theta(nlogn)
\end{displaymath}

Dividing into groups of 7 actually does work because:
%
\begin{displaymath}
T(n) = O(n) + T(n/7) + T(5n/7) = \Theta(n)
\end{displaymath}


\subsection{Separable first-order Ordinary Differential Equations}

For some we can write:

\(\dfrac{dy}{dt}=f(t,y)\)

\(\dfrac{dy}{dt}=\dfrac{g(t)}{h(y)}\)

We can then do the following:

\(h(y)\dfrac{dy}{dt}=g(t)\)

\(\int h(y)\dfrac{dy}{dt}dt=\int g(t)dt + C\)

\(\int h(y)dy=\int g(t)dt + C\)

In some cases, these functions can then be integrated and solved.


\section{Application: Matrix powers}

\begin{outcome}
  \begin{enumerate}
  \item Use diagonalization to raise a matrix to a high power.
  \item Use diagonalization to compute a square root of a matrix.
  \end{enumerate}
\end{outcome}

Suppose we have a matrix $A$ and we want to find $A^{50}$. One could
try to multiply $A$ with itself 50 times, but this is a lot of work
(try it!). However, diagonalization allows us to compute high powers
of a matrix relatively easily.  Suppose $A$ is diagonalizable, so that
$P^{-1}AP=D$. We can rearrange this equation to write $A=PDP^{-1}$.
Now, consider $A^2$. Since $A=PDP^{-1}$, it follows that
\begin{equation*}
  A^2 = (PDP^{-1})^2=PDP^{-1}PDP^{-1}=PD^2P^{-1}.
\end{equation*}
Similarly,
\begin{equation*}
  A^3 = (PDP^{-1})^3=PDP^{-1}PDP^{-1}PDP^{-1}=PD^3P^{-1}.
\end{equation*}
In general,
\begin{equation*}
  A^n = (PDP^{-1})^n=PD^nP^{-1}.
\end{equation*}
Therefore, we have reduced the problem to finding $D^n$. But as we
saw in Example~\ref{exa:sums-products-diagonal}, computing a power of
a diagonal matrix is easy. To compute $D^n$, we only need to raise
every entry on the diagonal to the power of $n$. Through this method,
we can compute large powers of matrices.

\begin{example}{Raising a matrix to a high power}{matrix-high-power}
  Let%
  \index{matrix!raising to a power}%
  \index{power!of a matrix} $A=\begin{mymatrix}{rrr}
    2 & 1 & 0 \\
    0 & 1 & 0 \\
    -1 & -1 & 1
  \end{mymatrix}$. Find $A^{50}$.
\end{example}

\begin{solution}
  First, we will diagonalize $A$. Following the usual steps, we find
  that the eigenvalues are $\eigenvar=1$ and $\eigenvar=2$. The basic
  eigenvectors corresponding to $\eigenvar=1$ are
  \begin{equation*}
    \vect{v}_1 = \begin{mymatrix}{r} 0 \\ 0 \\ 1 \end{mymatrix}
    \quad\mbox{and}\quad
    \vect{v}_2 = \begin{mymatrix}{r} -1 \\ 1 \\ 0 \end{mymatrix},
  \end{equation*}
  and the basic eigenvector corresponding to $\eigenvar = 2$ is
  \begin{equation*}
    \vect{v}_3 = \begin{mymatrix}{r} -1 \\ 0 \\ 1 \end{mymatrix}.
  \end{equation*}
  Now we construct $P$ by using the basic eigenvectors of $A$ as the
  columns of $P$.  Thus
  \begin{equation*}
    P =
    \begin{mymatrix}{rrr}
      0 & -1 & -1 \\
      0 & 1 & 0 \\
      1 & 0 & 1
    \end{mymatrix}.
  \end{equation*}
  The inverse of $P$ is
  \begin{equation*}
    P^{-1}=\begin{mymatrix}{rrr}
      1 & 1 & 1 \\
      0 & 1 & 0 \\
      -1 & -1 & 0
    \end{mymatrix}.
  \end{equation*}
  Then
  \begin{equation*}
    P^{-1}AP
    ~=~
    \begin{mymatrix}{rrr}
      1 & 1 & 1 \\
      0 & 1 & 0 \\
      -1 & -1 & 0
    \end{mymatrix} \begin{mymatrix}{rrr}
      2 & 1 & 0 \\
      0 & 1 & 0 \\
      -1 & -1 & 1
    \end{mymatrix} \begin{mymatrix}{rrr}
      0 & -1 & -1 \\
      0 & 1 & 0 \\
      1 & 0 & 1
    \end{mymatrix} \\
    ~=~
    \begin{mymatrix}{rrr}
      1 & 0 & 0 \\
      0 & 1 & 0 \\
      0 & 0 & 2
    \end{mymatrix} \\
    ~=~
    D.
  \end{equation*}
  Now it follows by rearranging the equation that $A=PDP^{-1}$, and
  therefore, as noted above,
  \begin{eqnarray*}
    A^{50}
    \quad=\quad
    PD^{50}P^{-1}
    &=&\begin{mymatrix}{rrr}
      0 & -1 & -1 \\
      0 & 1 & 0 \\
      1 & 0 & 1
    \end{mymatrix} \begin{mymatrix}{rrr}
      1^{50} & 0      & 0 \\
      0      & 1^{50} & 0 \\
      0      & 0      & 2^{50}
    \end{mymatrix} \begin{mymatrix}{rrr}
      1 & 1 & 1 \\
      0 & 1 & 0 \\
      -1 & -1 & 0
    \end{mymatrix} \\
    &=&\begin{mymatrix}{ccc}
      2^{50} & -1+2^{50} & 0 \\
      0 & 1 & 0 \\
      1-2^{50} & 1-2^{50} & 1
    \end{mymatrix}.
  \end{eqnarray*}
  \vspace{-6ex}\par
\end{solution}

Thus, through diagonalization, we have efficiently computed a high
power of $A$. The following example shows that we can also use the
same technique for finding a square root of a matrix.

\begin{example}{Square root of a matrix}{matrix-root}
  Let%
  \index{matrix!square root of}%
  \index{square root!of a matrix} $A=\begin{mymatrix}{rrr}
    1  &  3 & 3 \\
    -1 &  5 & 3 \\
    1  & -1 & 1 \\
  \end{mymatrix}$. Find a square root of $A$, i.e., find a matrix $B$
  such that $A=B^2$.
\end{example}

\begin{solution}
  We first diagonalize $A$. The characteristic polynomial is
  \begin{eqnarray*}
    \begin{absmatrix}{ccc}
      1-\eigenvar  &  3 & 3 \\
      -1 &  5-\eigenvar & 3 \\
      1  & -1 & 1-\eigenvar \\
    \end{absmatrix}
    &=&
    (1-\eigenvar)(5-\eigenvar)(1-\eigenvar) + 9 + 3
    - 3(5-\eigenvar) + 3(1-\eigenvar) + 3(1-\eigenvar)
    \\
    &=& -\eigenvar^3 + 7\eigenvar^2 - 14\eigenvar + 8,
  \end{eqnarray*}
  with roots $\eigenvar=1$, $\eigenvar=2$, and $\eigenvar=4$. The
  corresponding eigenvectors are
  \begin{equation*}
    \vect{v}_1 = \begin{mymatrix}{r} 1 \\ 1 \\ -1 \end{mymatrix},
    \quad
    \vect{v}_2 = \begin{mymatrix}{r} 0 \\ 1 \\ -1 \end{mymatrix},
    \quad\mbox{and}\quad
    \vect{v}_3 = \begin{mymatrix}{r} 1 \\ 1 \\ 0 \end{mymatrix},
  \end{equation*}
  respectively. Therefore we have $P^{-1}AP = D$, where
  \begin{equation*}
    P = \begin{mymatrix}{rrr}
      1  &  0 & 1 \\
      1  &  1 & 1 \\
      -1 & -1 & 0 \\
    \end{mymatrix},
    \quad
    P^{-1} = \begin{mymatrix}{rrr}
      1  & -1 & -1 \\
      -1 &  1 &  0 \\
      0  &  1 &  1 \\
    \end{mymatrix},
    \quad\mbox{and}\quad
    D = \begin{mymatrix}{rrr}
      1 & 0 & 0 \\
      0 & 2 & 0 \\
      0 & 0 & 4 \\
    \end{mymatrix}.
  \end{equation*}
  We can equivalently write $A = PDP^{-1}$. Finding a square root of a
  diagonal matrix is easy:
  \begin{equation*}
    D^{\frac{1}{2}} =
    \begin{mymatrix}{ccc}
      1 & 0        & 0 \\
      0 & \sqrt{2} & 0 \\
      0 & 0        & 2 \\
    \end{mymatrix}.
  \end{equation*}
  If we now define $B=PD^{\frac{1}{2}}P^{-1}$, we clearly have $B^2 =
  PD^{\frac{1}{2}}P^{-1}PD^{\frac{1}{2}}P^{-1} = PDP^{-1} = A$. So the
  desired square root of $A$ is
  \begin{equation*}
    B
    ~=~
    PD^{\frac{1}{2}}P^{-1}
    ~=~
    \begin{mymatrix}{rrr}
      1  &  0 & 1 \\
      1  &  1 & 1 \\
      -1 & -1 & 0 \\
    \end{mymatrix}
    \begin{mymatrix}{ccc}
      1 & 0        & 0 \\
      0 & \sqrt{2} & 0 \\
      0 & 0        & 2 \\
    \end{mymatrix}
    \begin{mymatrix}{rrr}
      1  & -1 & -1 \\
      -1 &  1 &  0 \\
      0  &  1 &  1 \\
    \end{mymatrix}
    ~=~
    \begin{mymatrix}{ccc}
      1           &  1          &  1 \\
      1-\sqrt{2}  &  1+\sqrt{2} &  1 \\
      -1+\sqrt{2} &  1-\sqrt{2} &  1 \\
    \end{mymatrix}.
  \end{equation*}
  Finally, we verify that we have computed $B$ correctly by squaring
  it and double-checking that we really get $A$.
  \begin{equation*}
    B^2 ~=~
    \begin{mymatrix}{ccc}
      1           &  1          &  1 \\
      1-\sqrt{2}  &  1+\sqrt{2} &  1 \\
      -1+\sqrt{2} &  1-\sqrt{2} &  1 \\
    \end{mymatrix}
    \begin{mymatrix}{ccc}
      1           &  1          &  1 \\
      1-\sqrt{2}  &  1+\sqrt{2} &  1 \\
      -1+\sqrt{2} &  1-\sqrt{2} &  1 \\
    \end{mymatrix}
    ~=~
    \begin{mymatrix}{rrr}
      1  &  3 & 3 \\
      -1 &  5 & 3 \\
      1  & -1 & 1 \\
    \end{mymatrix}
    ~=~
    A.
  \end{equation*}
  We note that the square root of a matrix is not unique. In fact,
  $D$ has 8 different square roots, all of the form
  \begin{equation*}
    \begin{mymatrix}{ccc}
      \pm 1 & 0            & 0 \\
      0     & \pm \sqrt{2} & 0 \\
      0     & 0            & \pm 2 \\
    \end{mymatrix}.
  \end{equation*}
  It follows that $A$ has 8 different square roots as well. We leave
  it as an exercise to compute them all.
\end{solution}

The same method can also be used to compute other powers of a matrix,
for example a cube root.


\documentclass[11pt,twoside]{article}

\pagestyle{myheadings}
\markboth{Actor ranking}{Actor ranking}

\input{../../fncextra}

\begin{document}

\begin{center}
  \bf Not Kevin Bacon
\end{center}

Suppose we have a graph or network with $n$ nodes and some connections between them. The matrix $\mA$ whose entries are
\begin{equation}
  A_{ij} =
  \begin{cases}
    1, & \text{if node $i$ connects to $j$},\\
    0, & \text{otherwise,}
  \end{cases}
\end{equation}
is the \textbf{adjacency matrix} of the network. Note that these connections have direction, and $i$ connecting to $j$ does not automatically imply that $j$ connects to $i$ (i.e., this is a directed graph). Let $s_i = \sum_{j=1}^n A_{ij}$ be the number of nodes that node $i$ connects to. If we were to randomly select a link leaving node $i$, each link would have probability $1/s_i$ of being selected.

Let $\bfx$ be a vector of positive values. We will require that $\sum_i x_i = 1$, so that $\bfx$ has the interpretation of a probability distribution over the nodes. If all connections are given equal weighting, the probability of following a connection from any node to node $i$ is
\begin{equation}
  z_i = \sum_{j\in P_i} \frac{x_j}{s_j} ,
\end{equation}
where $P_i$ is the set of nodes that connect to node $i$. These are the rows with ones in column $i$ of $\mA$. By the definition of $\mA$, this is the same as
\begin{equation}
  \label{eq:yvec}
  z_i = \sum_{j=1}^n \frac{A_{ji} x_j}{s_j} =\sum_{j=1}^n B_{ij} x_j,
\end{equation}
where we defined $B_{ij}=A_{ji}/s_j$. Put simply, $\bfz=\mB \bfx$.

It's important to introduce some overall randomness into the jumps between nodes---this is the only way to escape a self-contained clique (disconnected subgraph). The probability of hopping to any node $i$ entirely at random is just $1/n$. We blend link-following with random hopping as follows. Choose some $p\in[0,1]$, and suppose that a hop between nodes follows one of the connections with probability $p$, or is a random hop with probability $1-p$. Using $\bm{1}$ to denote the $n$-vector of all ones, then
\begin{equation}
  \label{eq:totalmap}
  \bfy = p \mB \bfx + \frac{1-p}{n} \bm{1}
\end{equation}
describes how to update probabilities after each hop. Finally, the fact that
\begin{equation}
  1 = \sum_i x_i = \bm{1}^T \bfx,
\end{equation}
allows us to express the map~\eqref{eq:totalmap} as
\begin{equation}
  \label{eq:matvec}
  \bfy =  \left[ p \mB + \frac{1-p}{n} \bm{1}\bm{1}^T \right] \bfx = \mR\bfx,
\end{equation}
for a square matrix $\mR$. If the probabilities are unchanged by hopping (i.e., $\bfz=\bfx$), then $\bfx$ is an eigenvector of $\mR$ with associated eigenvalue $\lambda=1$. We won't prove this, but $\mR$ is guaranteed to have $\lambda=1$ as the leading eigenvalue, making power iteration possible. The resulting eigenvector $\bfx$ can be sorted to find out which nodes are most likely to be visited in the long run.

Note that $\mR$ is \emph{not} sparse and should never be formed. However, $\mR\bfx$ as defined in~\eqref{eq:totalmap} can be computed efficiently if $\mB$ is sparse. That is all we need to do a power iteration with $\mR$. Since we are working with probability, normalization is not required in the power iteration: $\bfx_{k+1}=\mR \bfx_k$, provided $\bfx_1$ has positive entries and $\|\bfx\|_1=1$.

\subsection*{Goals}

You will use an adjacency matrix for movie actors to perform the power iteration and find the leading eigenvector of $\mR$, and using that vector to rank the actors in influence.

\subsection*{Preparation}

Read section 8.2. Answer the following questions based on the above description.

\begin{enumerate}
\item Show using \eqref{eq:yvec} that $\displaystyle \sum_{i=1}^n z_i = 1$. This proves that $\bfz$ is also a probability distribution.
  \item Show using \eqref{eq:totalmap} that $\bfy$ is a probability distribution.
\end{enumerate}

%\vspace{1ex}
%\noindent \textbf{Notable MATLAB functions:} \texttt{whos}, \texttt{nnz}, \texttt{sort}, \texttt{histogram}

\subsection*{Procedure}

Download the script template and the data file \texttt{actornetwork.mat}.

\begin{enumerate}
\item Load \texttt{actornetwork.mat} file from the assignment site. It has a vector \texttt{actor} of unique actor names for all credited roles in films released from 2004 through 2013. It also has a sparse adjacency matrix \texttt{A}, where a (symmetric) link between actors means that they appeared in at least one film together.
\item Use \texttt{nnz} to compute the density (number of nonzeros over total number of elements) of \texttt{A}. Use \texttt{whos} to find the memory usage of \texttt{A} in bytes. Also calculate the memory usage of an equivalent full (non-sparse) matrix.
\item Compute the vector $\mathbf{s}$ whose entries are $s_i$ as defined above. Make a histogram of its entries using 32 bins.
\item  Construct the matrix $\mB$ appearing in~\eqref{eq:yvec} above. It helps to use the fact that $\mA$ is symmetric. (It's reasonable to loop over one dimension of the matrix, but not over all of the elements.)
\item Set $p=0.9$ and let $\bfx_1$ be a random vector of positive numbers. Normalize $\bfx_1$ to be a probability distribution. By repeatedly applying~\eqref{eq:totalmap}, do 100 power iterations to get $\bfx_{101}$. \emph{Important: Do not attempt to define the matrix $\mR$}, and do \emph{not} use the book's power iteration function; instead use~\eqref{totalmap} to compute $\mR\bfx$. Check that $\|\bfx_{101}-\bfx_{100}\|_1$ is less than $10^{-6}$.
\item Sort the entries of $\bfx$ in descending order, using the second output to print out the names of the 10 ``most collaborative'' actors.
\end{enumerate}

\subsection*{Discussion}
\begin{enumerate}
\item The data file also includes an $m\times n$ sparse matrix $\m{M}$. Its $(i,j)$ entry is one if actor $j$ appeared in film $i$, and zero otherwise. Give a simple interpretation of the matrix $\m{M}^T\m{M}$.
%\item[E2.] Repeat the ranking process using $\m{M}^T\m{M}$ in place of $\mA$. How many movies is the top actor credited with appearing in?
\item What is the interpretation of $\m{M}\m{M}^T$? What would a ranking using this matrix reveal?
\end{enumerate}

\end{document}


\subsection{Markov's inequality and Chebyshev's inequality}
\subsubsection{Lemma 1}

\(E[I_{X\ge a}]=P(X\ge a)\)

Consider the indicator function.

\(I_{X\ge a}\)

This is equal to \(0\) if \(X\) is below \(a\) and \(1\) otherwise.

We can take expectations of this.

\(E[I_{X\ge a}]=P(X\ge a).1+P(X<a).0=P(X\ge a)\)

\(E[I_{X\ge a}]=P(X\ge a)\)

\subsubsection{Lemma 2}

\(aI_{X\ge a}\le X\)

While \(X\) is below \(a\) the left side is equal to \(0\), which holds.

While \(X\) is equal to \(a\) the left side is equal to \(X\), which holds.

While \(X\) is above \(a\) the left side is equal to \(a\), which holds.

\subsubsection{Markov’s inequality}

\(P(X\ge a)\le \dfrac{\mu  }{a}\)

From above:

\(aI_{X\ge a}\le X\)

We can take expectations of both sides:

\(E[aI_{X\ge a}]\le E[X]\)

\(aP(X\ge a)\le E[X]\)

\(P(X\ge a)\le \dfrac{\mu  }{a}\)

\subsubsection{Chebyshev’s inequality}

We know from Markov’s inequality that:

\(P(X\ge a)\le \dfrac{\mu }{a}\)

Let’s take the variable \(X\) to be \((X-\mu )^2\)

\(P((X-\mu )^2\ge a)\le \dfrac{E[(X-\mu )^2]}{a}\)

\(P((X-\mu )^2\ge a)\le \dfrac{\sigma^2}{a}\)

\(P(|X-\mu | \ge \sqrt{a})\le \dfrac{\sigma^2}{a}\)

Take \(a\) to be a multiple \(k^2\) of the variance \(\sigma^2\).

\(a=k^2\sigma^2\)

\(P(|X-\mu | \ge k\sigma )\le \dfrac{\sigma^2}{k^2\sigma^2}\)

\(P(|X-\mu | \ge k\sigma )\le \dfrac{1}{k^2}\)