> For the complete documentation index, see [llms.txt](https://coldmoon.gitbook.io/mathematical-analysis/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://coldmoon.gitbook.io/mathematical-analysis/mathematical-analysis-2-1/6.-differential-calculus-of-multivariate-functions.md). # 6. Differential calculus of multivariate functions ### Definition 1: Let $$X$$ and $$Y$$ be normed spaces. A mapping $$f: E \rightarrow Y$$ of a set $$E \subset X$$ into $$Y$$ is differentiable at an interior point $$x \in E$$ if there exists a continuous ***linear transformation*** $$L(x): X \rightarrow Y$$ such that $$ f(x+h)-f(x)=L(x) h+\alpha(x ; h) $$ where $$\alpha(x ; h)=o(h)$$ as $$h \rightarrow 0, x+h \in E$$. We often denote that $$L(x)$$as $$\mathrm{d} f(x), D f(x)$$, or $$f^{\prime}(x)$$. > remark: If the condition for differentiability of the mapping $$f$$ at some point $$a$$ is written as > > $$f(x)-f(a)=L(x)(x-a)+\alpha(a ; x)$$where $$\alpha(a;x) = o(x − a)$$ as $$x \to a$$. ### tangent space We remark that strictly speaking $$f^{\prime}(x) \in \mathcal{L}\left(T X\_{x} ; T Y\_{f(x)}\right)$$ here and $$h$$ is a vector of the tangent space $$T X\_{x}$$. But parallel translation of a vector to any point $$x \in X$$ is defined in a vector space, and this allows us to identify the tangent space $$T X\_{x}$$ with the vector space $$X$$ itself. Consequently, after choosing a basis in $$X$$, we can extend it to all the tangent spaces $$T X\_{x}$$. This means that if, for example, $$X=\mathbb{R}^{m}$$, $$Y=\mathbb{R}^{n}$$,and the **mapping** $$f \in \mathcal{L}\left(\mathbb{R}^{m} ; \mathbb{R}^{n}\right)$$ is given by the matrix $$\left(a\_{ij}\right),$$ then at every point $$x \in \mathbb{R}^{m}$$ the **tangent mapping** $$f^{\prime}(x): T \mathbb{R}*{x}^{m} \rightarrow T \mathbb{R}*{f(x)}^{n}$$ will be given by the same matrix. ### Chain Rule If the mapping $$f: U \rightarrow V$$ is differentiable at a point $$x \in U \subset X$$, and the mapping $$g: V \rightarrow Z$$ is differentiable at $$f(x)=y \in V \subset Y$$, then the composition $$g \circ f$$ of these mappings is differentiable at $$x$$, and $$ (g \circ f)^{\prime}(x)=g^{\prime}(f(x)) \circ f^{\prime}(x) $$ ### Differentiation of the Inverse (#) Let $$f: U \rightarrow Y$$ be a mapping that is continuous at $$x \in U \subset X$$ and has an inverse $$f^{-1}: V \rightarrow X$$ that is defined in a neighborhood of $$y=f(x)$$ and continuous at that point. If the mapping $$f$$ is differentiable at $$x$$ and its tangent mapping $$f^{\prime}(x) \in \mathcal{L}(X ; Y)$$ has a continuous inverse $$\left\[f^{\prime}(x)\right]^{-1} \in \mathcal{L}(Y ; X),$$ then the mapping $$f^{-1}$$ is differentiable at $$y=f(x)$$ and $$ \left\[f^{-1}\right]^{\prime}(f(x))=\left\[f^{\prime}(x)\right]^{-1} $$ ### Partial derivative Let $$U=U(a)$$ be a ***neighborhood*** of the point $$a \in X=X\_{1} \times \cdots \times X\_{m}$$ in the direct product of the normed spaces $$X\_{1}, \cdots, X\_{m}$$ , and let $$f: U \rightarrow Y$$ be a mapping of $$U$$ into the normed space $$V$$. In this case $$ y=f(x)=f\left(x\_{1}, \ldots, x\_{m}\right) $$ and hence, if we fix all the variables but $$x\_{i}$$ *in the above equation* by setting $$x\_{k}=a\_{k}$$ for $$k \in {1, \ldots, m} \backslash i,$$ we obtain a function $$ f\left(a\_{1}, \ldots, a\_{i-1}, x\_{i}, a\_{i+1}, \ldots, a\_{m}\right)=: \varphi\_{i}\left(x\_{i}\right) $$ defined in some neighborhood $$U\_{i}$$ *of* $$a\_{i}$$ in $$X$$. We usually denote this partial derivative by one of the symbols $$ \partial\_{i} f(a), \quad D\_{i} f(a), \quad \frac{\partial f}{\partial x\_{i}}(a), \quad f\_{x\_{i}}^{\prime}(a) $$ In accordance with these definitions $$D\_{i} f(a) \in \mathcal{L}\left(X\_{i} ; Y\right) .$$ More precisely, $$D\_{i} f(a) \in \mathcal{L}\left(T X\_{i}\left(a\_{i}\right) ; T Y(f(a))\right)$$. Denote that $$h=\left(h\_{1}, \ldots, h\_{m}\right) \in T X\_{1}\left(a\_{1}\right) \times \cdots \times T X\_{m}\left(a\_{m}\right)=T X(a)$$, we have the equation $$ \mathrm{d} f(a) h=\partial\_{1} f(a) h\_{1}+\cdots+\partial\_{m} f(a) h\_{m} $$ ### partial derivative of higher order Consider $$U=U(x)$$ be a ***neighborhood*** of the point $$x=(x^1,\cdots,x^n)$$, Then if $$f$$has partial derivative with respect to $$x^i$$, then $$\frac{\partial f}{\partial x^{i}}$$becomes a new function which is defined in $$U$$, if: $$ \frac{\partial}{\partial x^{i\_{1}}}\left(\frac{\partial}{\partial x^{i\_{2}}} \cdots\left(\frac{\partial f}{\partial x^{i\_{k}}}\right) \cdots\right)(x) $$ exist, we call it ***higher-order partial derivative***, often denoted as: $$ \frac{\partial^{k} f}{\partial x^{i\_{1}} \cdots \partial x^{i\_{k}}}(x) $$ ## **Taylor's theorem** If a mapping $$f: U \rightarrow Y$$ from a neighborhood $$U=U(x)$$ of a point $$x$$ in a normed space $$X$$ into a normed space $$Y$$ has derivatives up to order $$n-1$$ inclusive in $$U$$ and has an nth order derivative $$f^{(n)}(x)$$ at the point $$x$$, then $$ f(x+h)=f(x)+f^{\prime}(x) h+\cdots+\frac{1}{n !} f^{(n)}(x) h^{n}+R\left(x;h\right) $$ $$R\left(x;h\right)= o(|h|^n)$$when $$h\to 0$$. We can expand the $$f^{(n)}(x)$$, such as **Jacobian matrix**: $$ \mathbf {T}f(x)= \mathrm df(x)=\nabla f(x)=\left({\frac {\partial f}{\partial x\_{1}}}(x),{\frac {\partial f}{\partial x\_{2}}}(x),\cdots ,{\frac {\partial f}{\partial x\_{n}}}(x)\right) $$ or ***Hessian matrix***: $$ \mathbf{H} f(x)=\left(\begin{array}{cccc} \frac{\partial^{2} f}{\partial x\_{1}^{2}} & \frac{\partial^{2} f}{\partial x\_{1} \partial x\_{2}} & \cdots & \frac{\partial^{2} f}{\partial x\_{1} \partial x\_{n}} \\&&\\ \frac{\partial^{2} f}{\partial x\_{2} \partial x\_{1}} & \frac{\partial^{2} f}{\partial x\_{2}^{2}} & \cdots & \frac{\partial^{2} f}{\partial x\_{2} \partial x\_{n}} \\&&\\ \vdots & \vdots & \ddots & \vdots \\&&\\ \frac{\partial^{2} f}{\partial x\_{n} \partial x\_{1}} & \frac{\partial^{2} f}{\partial x\_{n} \partial x\_{2}} & \cdots & \frac{\partial^{2} f}{\partial x\_{n}^{2}} \end{array}\right) $$ $$f^n(x)$$means all the $$n$$-th order partial derivatives of $$f$$ $$ f^{(n)}(x)h^n=\sum\_{\alpha *{1}+\cdots+\alpha *{n}=n}{\frac {\partial ^{n}f}{\partial x*{1}^{\alpha *{1}}\cdots \partial x*{n}^{\alpha* {n}}}}(x)\cdot h\_1^{\alpha\_1}\cdots h\_n^{\alpha\_n} $$ and $$ R\left(x;h\right)=\frac{1}{n !} \int\_{0}^{1}(1-t)^{n} D^{n+1} f(x+t\cdot h)) \mathrm d t $$ ## Lagrange multiplier ... pass. $$\blacksquare$$