Mathwizurd.com is created by David Witten, a mathematics and computer science student at Vanderbilt University. For more information, see the "About" page.

Gradients

Relationship to Total Derivative

Recall in the article about multivariable derivatives, we talked about how single-variable derivatives approximate a line, and multivariable derivatives approximate a hyperplane.

MathJax TeX Test Page The gradient is the tranpose of the Jacobian. That's it! $$\begin{bmatrix}\frac{\partial f}{\partial x} \\ \frac{\partial f}{\partial y} \\ \frac{\partial f}{\partial x} \end{bmatrix}$$

Orthogonal to the Curve

MathJax TeX Test Page It turns out that the gradient is orthogonal to a level curve. First, what is a level curve? $$\text{Let } w = f(x,y,z) = x^2 + y^2 + z^2$$ Setting $w = 1$ is a level curve. In fact, it is the unit sphere. $$f(x,y,z) = x^2 + y^2 + z^2$$ $$g(t) = f(x,y,z) = 1$$ $$\frac{dg}{dt} = 0 = \begin{bmatrix}\frac{\partial f}{\partial x} & \frac{\partial f}{\partial y} & \frac{\partial f}{\partial z}\end{bmatrix}\begin{bmatrix}x'(t) \\ y'(t) \\ z'(t)\end{bmatrix}$$ $$0 = \begin{bmatrix}\frac{\partial f}{\partial x} \\ \frac{\partial f}{\partial y} \\ \frac{\partial f}{\partial z}\end{bmatrix} \cdot \begin{bmatrix}x'(t) \\ y'(t) \\ z'(t)\end{bmatrix}$$ $$0 = \text{gradient} \cdot \text{tangent line}$$ So, the gradient of that curve, which is $$\begin{bmatrix}2x \\ 2y \\ 2z\end{bmatrix}$$ is orthogonal to the sphere.

Direction of Max Increase

MathJax TeX Test Page Another important property of the gradient is that it points in the direction of greatest increase. For example, consider $f(x) = x^2 + y^2 + z^2$. Each level surface is a sphere.

If you're an ant on the surface of a unit sphere, and you want to move to a larger sphere, you have an infinite number of options. You can move along the sphere, which means your radius, or $f(x)$ value, would the stay the same. If you move toward the center, the radius decreases. If you move perpendicularly away from the sphere, the radius increases most efficiently. The gradient, the perpendicular direction, goes towards the next highest level curve.

Now, let's show this mathematically $$|D_vf(p)| = |\nabla f(p) \cdot v|$$ We can use the C-S Inequality now. $$|D_vf(p)| = |\nabla f(p) \cdot v| \leq ||\nabla f(p)||||v|| = ||\nabla f(p)|| $$ This means that the maximum value of the directional derivative is when the vector points in the direction of the gradient, which is orthogonal to the curve.

Examples

MathJax TeX Test Page What is the tangent plane to $x^2 + y^2 = z$ at $(3, 4, 25)$?

You have one unit sphere at the origin. How far away do you center a sphere of radius 2 so that it's orthogonal to the first sphere?
David Witten

Tangent Spaces

Multivariable Derivative