6. Differential calculus of multivariate functions

Definition 1:

Let $X$ and $Y$ be normed spaces. A mapping $f: E \rightarrow Y$ of a set $E \subset X$ into $Y$ is differentiable at an interior point $x \in E$ if there exists a continuous linear transformation $L(x): X \rightarrow Y$ such that

$$f(x+h)-f(x)=L(x) h+\alpha(x ; h)$$

where $\alpha(x ; h)=o(h)$ as $h \rightarrow 0, x+h \in E$.

We often denote that $L(x)$as $\mathrm{d} f(x), D f(x)$, or $f^{\prime}(x)$.

remark: If the condition for differentiability of the mapping $f$ at some point $a$ is written as

$f(x)-f(a)=L(x)(x-a)+\alpha(a ; x)$where $\alpha(a;x) = o(x − a)$ as $x \to a$.

tangent space

We remark that strictly speaking $f^{\prime}(x) \in \mathcal{L}\left(T X_{x} ; T Y_{f(x)}\right)$ here and $h$ is a vector of the tangent space $T X_{x}$. But parallel translation of a vector to any point $x \in X$ is defined in a vector space, and this allows us to identify the tangent space $T X_{x}$ with the vector space $X$ itself.

Consequently, after choosing a basis in $X$, we can extend it to all the tangent spaces $T X_{x}$. This means that if, for example, $X=\mathbb{R}^{m}$, $Y=\mathbb{R}^{n}$,and the mapping $f \in \mathcal{L}\left(\mathbb{R}^{m} ; \mathbb{R}^{n}\right)$ is given by the matrix $\left(a_{ij}\right),$ then at every point $x \in \mathbb{R}^{m}$ the tangent mapping $f^{\prime}(x): T \mathbb{R}_{x}^{m} \rightarrow T \mathbb{R}_{f(x)}^{n}$ will be given by the same matrix.

Chain Rule

If the mapping $f: U \rightarrow V$ is differentiable at a point $x \in U \subset X$, and the mapping $g: V \rightarrow Z$ is differentiable at $f(x)=y \in V \subset Y$, then the composition $g \circ f$ of these mappings is differentiable at $x$, and

$$(g \circ f)^{\prime}(x)=g^{\prime}(f(x)) \circ f^{\prime}(x)$$

Differentiation of the Inverse (#)

Let $f: U \rightarrow Y$ be a mapping that is continuous at $x \in U \subset X$ and has an inverse $f^{-1}: V \rightarrow X$ that is defined in a neighborhood of $y=f(x)$ and continuous at that point. If the mapping $f$ is differentiable at $x$ and its tangent mapping $f^{\prime}(x) \in \mathcal{L}(X ; Y)$ has a continuous inverse $\left[f^{\prime}(x)\right]^{-1} \in \mathcal{L}(Y ; X),$ then the mapping $f^{-1}$ is differentiable at $y=f(x)$ and

$$\left[f^{-1}\right]^{\prime}(f(x))=\left[f^{\prime}(x)\right]^{-1}$$

Partial derivative

Let $U=U(a)$ be a neighborhood of the point $a \in X=X_{1} \times \cdots \times X_{m}$ in the direct product of the normed spaces $X_{1}, \cdots, X_{m}$ , and let $f: U \rightarrow Y$ be a mapping of $U$ into the normed space $V$. In this case

$$y=f(x)=f\left(x_{1}, \ldots, x_{m}\right)$$

and hence, if we fix all the variables but $x_{i}$ in the above equation by setting $x_{k}=a_{k}$ for $k \in \{1, \ldots, m\} \backslash i,$ we obtain a function

$$f\left(a_{1}, \ldots, a_{i-1}, x_{i}, a_{i+1}, \ldots, a_{m}\right)=: \varphi_{i}\left(x_{i}\right)$$

defined in some neighborhood $U_{i}$ of $a_{i}$ in $X$.

We usually denote this partial derivative by one of the symbols

$$\partial_{i} f(a), \quad D_{i} f(a), \quad \frac{\partial f}{\partial x_{i}}(a), \quad f_{x_{i}}^{\prime}(a)$$

In accordance with these definitions $D_{i} f(a) \in \mathcal{L}\left(X_{i} ; Y\right) .$ More precisely, $D_{i} f(a) \in \mathcal{L}\left(T X_{i}\left(a_{i}\right) ; T Y(f(a))\right)$.

Denote that $h=\left(h_{1}, \ldots, h_{m}\right) \in T X_{1}\left(a_{1}\right) \times \cdots \times T X_{m}\left(a_{m}\right)=T X(a)$, we have the equation

$$\mathrm{d} f(a) h=\partial_{1} f(a) h_{1}+\cdots+\partial_{m} f(a) h_{m}$$

partial derivative of higher order

Consider $U=U(x)$ be a neighborhood of the point $x=(x^1,\cdots,x^n)$, Then if $f$has partial derivative with respect to $x^i$, then $\frac{\partial f}{\partial x^{i}}$becomes a new function which is defined in $U$, if:

$$\frac{\partial}{\partial x^{i_{1}}}\left(\frac{\partial}{\partial x^{i_{2}}} \cdots\left(\frac{\partial f}{\partial x^{i_{k}}}\right) \cdots\right)(x)$$

exist, we call it higher-order partial derivative, often denoted as:

$$\frac{\partial^{k} f}{\partial x^{i_{1}} \cdots \partial x^{i_{k}}}(x)$$

Taylor's theorem

If a mapping $f: U \rightarrow Y$ from a neighborhood $U=U(x)$ of a point $x$ in a normed space $X$ into a normed space $Y$ has derivatives up to order $n-1$ inclusive in $U$ and has an nth order derivative $f^{(n)}(x)$ at the point $x$, then

$$f(x+h)=f(x)+f^{\prime}(x) h+\cdots+\frac{1}{n !} f^{(n)}(x) h^{n}+R\left(x;h\right)$$

$R\left(x;h\right)= o(|h|^n)$when $h\to 0$.

We can expand the $f^{(n)}(x)$, such as Jacobian matrix:

$$\mathbf {T}f(x)= \mathrm df(x)=\nabla f(x)=\left({\frac {\partial f}{\partial x_{1}}}(x),{\frac {\partial f}{\partial x_{2}}}(x),\cdots ,{\frac {\partial f}{\partial x_{n}}}(x)\right)$$

or Hessian matrix:

$$\mathbf{H} f(x)=\left(\begin{array}{cccc} \frac{\partial^{2} f}{\partial x_{1}^{2}} & \frac{\partial^{2} f}{\partial x_{1} \partial x_{2}} & \cdots & \frac{\partial^{2} f}{\partial x_{1} \partial x_{n}} \\&&\\ \frac{\partial^{2} f}{\partial x_{2} \partial x_{1}} & \frac{\partial^{2} f}{\partial x_{2}^{2}} & \cdots & \frac{\partial^{2} f}{\partial x_{2} \partial x_{n}} \\&&\\ \vdots & \vdots & \ddots & \vdots \\&&\\ \frac{\partial^{2} f}{\partial x_{n} \partial x_{1}} & \frac{\partial^{2} f}{\partial x_{n} \partial x_{2}} & \cdots & \frac{\partial^{2} f}{\partial x_{n}^{2}} \end{array}\right)$$

$f^n(x)$means all the $n$-th order partial derivatives of $f$

$$f^{(n)}(x)h^n=\sum_{\alpha _{1}+\cdots+\alpha _{n}=n}{\frac {\partial ^{n}f}{\partial x_{1}^{\alpha _{1}}\cdots \partial x_{n}^{\alpha_ {n}}}}(x)\cdot h_1^{\alpha_1}\cdots h_n^{\alpha_n}$$

and

$$R\left(x;h\right)=\frac{1}{n !} \int_{0}^{1}(1-t)^{n} D^{n+1} f(x+t\cdot h)) \mathrm d t$$

Lagrange multiplier

... pass.

$\blacksquare$