These are mainly notes to myself (i.e., Siddhartha Gadgil) but are shared in case they are useful.

Optimization

• For functions of one variable, application of calculus.
• Have first derivative and second derivative test.
• In higher dimension:
  • to define first derivative: linear map (actually functional in this case).
  • second derivative : also captureb by a linear map - but a square symmetric matrix.
  • we can also have interesting constraints.
  • even more generally, we have calculus of variations.

Linear Algebra

• Study linear maps on vector spaces, because:
  • many maps are linear
  • many functions are quadratic - these also can be expressed in linear algebra
  • smooth functions are locally approximately linear,...

The Optimization problem

Consider a collection of points in the plane, or more generally in $R^n$. In many cases, we want to find the line that is closest to this collection of points - or more generally the plane, or affine space of a fixed dimension. Two common reasons for this are:

• the points represent values of a function, and we want to approximate this by a linear function $y = mx + c”
• the points are data in some high-dimensional space - say many numbers associated to countries - and we want to capture variation in as...

Dual space

• Let $V$ be a vector space over a field $k$.
• A linear functional $\lambda$ on $V$ is a linear transformation $\lambda: V \to k$.
• Linear functionals form a vector space, called the dual space $V^*$ of $V$.

Duality in Euclidean spaces

• Assume now that $V$ is a finite-dimensional vector space over $\mathbb{R}$ equipped with an inner product $\langle\cdot,\cdot\rangle$.
• Let $w\in V$ be a fixed vector. Then we can define a linear functional $\lambda_w: V \to\mathbb{R}$ by $\lambda_w(v) = \langle v, w \rangle$ for an arbitrary vector $v\in V$.
• We thus obtain a linear...

Bilinear and Quadratic forms

• Let $V$ be a vector space over a field $k$ in which $2 \neq 0$ (say real or complex numbers).
• A Bilinear form $B(x, y)$ is a function $B : V \times V \to k$ which is linear in $x$ and $y$.
• We can associate to a Bilinear form $B$ a so called quadratic form $Q : V \to k$ defined by $Q(x) = B(x, x)$. Note that by definition a Quadratic form is associated to a Bilinear form.
• Given the Bilinear form $B$, we can define a symmetric Bilinear form $\bar{B}(x, y) =...