Mathwizurd.com is created by David Witten, a mathematics and computer science student at Vanderbilt University. For more information, see the "About" page.

Multivariable Derivative

Geometric Definition of a Derivative

Before we get to the total derivative, what does the single-variable derivative mean? Well it’s a linear approximation at a given point. Now, imagine when we add a second variable, it’s no longer an approximation with a line, it’s a plane. Generalizing to higher dimensions, the derivative is a hyperplane approximation of a function.

Mathematical Definition

MathJax TeX Test Page The derivative $Df(p)$ is a linear transformation from $\mathbb{R}^n \to \mathbb{R}^m$ that satisfies this equation $$\lim_{h \to 0} \dfrac{f(p + h) - f(p) - Df(p)(h)}{||h||} = 0$$ This is also called the total derivative. The equatino above doesn't help us figure out what it is at all. In order to start getting some intuition into what this mysterious function could be, let's look at an example.

Total Derivative of a Linear Transformation

MathJax TeX Test Page Let's start by writing out the definition $$\lim_{h \to 0} \dfrac{||T(p + h) - T(p) - DT(p)(h)||}{||h||} = 0$$ $$\lim_{h \to 0} \dfrac{||T(p) + T(h) - T(p) - DT(p)(h)||}{||h||} = 0$$ $$\lim_{h \to 0} \dfrac{||T(h) - DT(p)(h)||}{||h||} = 0$$ Because we know that both the derivative and T are linear transformations, we can cosnider their matrix definition: $T(x) = [T]x$. $$\lim_{h \to 0} \dfrac{||T*h - DT(p)*h||}{||h||} = \lim_{h \to 0} ||T - DT(p)|| = 0$$ If $DT(p) = T$, then the limit definition is satisfied. Therefore, the derivative of a linear transformation is the linear transformation itself.


Let's look at an example. What's the derivative of $2x + 3y + 4z$? Well let's look at it like a matrix $$\begin{bmatrix}2 & 3 & 4\end{bmatrix}\begin{bmatrix}x\\y\\z\end{bmatrix}$$ The derivative is $\begin{bmatrix}2 & 3 & 4\end{bmatrix}$.

Jacobian

MathJax TeX Test Page The Jacobian matrix is a matrix whose columns consist of partial derivatives. $$\begin{bmatrix}\dfrac{\partial f}{\partial x_1} & \dfrac{\partial f}{\partial x_2} & ... & \dfrac{\partial f}{\partial x_n}\end{bmatrix}$$ So, the Jacobian of $2x + 3y + 4z$ is $\begin{bmatrix}2 & 3 & 4\end{bmatrix}$. Interestingly, that's also the derivative. Is this always the case? Yes! The derivative of a function equals its Jacobian.

Chain Rule

MathJax TeX Test Page We begin with this fact of derivatives. $$D(f \circ g)(p) = J(f \circ g)(p) = Jf(g(p))Jg(p)$$ Now, let's put this into practice! Let $z = xe^{xy} \begin{cases}x = t^2\\ y = \frac{1}{t}\end{cases}$. Find the $\dfrac{dz}{dt}$. Think of this as finding the derivative of $z(x(t), y(t))$. We begin by finding $Jz(x(p), y(p))$. Now, $x(p)$ and $y(p)$ are just x and y. That's it. $$\begin{bmatrix}\left(1 + xy\right)e^{xy} & x^2e^{xy}\end{bmatrix}$$ Now, we look at $x(p)$, $y(p)$. $$\begin{bmatrix}x_t \\ y_t\end{bmatrix} = \begin{bmatrix}2t\\\frac{1}{t^2}\end{bmatrix}$$ Why is this one column? x and y are technically part of the same function $f(t) = < x(t), y(t)>$. We are only taking partials of one variable, so it's one column. Now, let's multiply the values: $$2t(1 + xy)e^{xy} - \frac{x^2e^{xy}}{t^2}$$ Now, in order to use one variable, we can substitute x(t) and y(t) with their values. $$2t(1 + t)e^{t} - t^2e^{t} = 2te^t + t^2e^t$$
David Witten

Gradients

Directional Derivatives