Chapter 18Scalar Fields, Vector Fields, and the Gradient

18.1 Scalar Fields and Vector Fields

Let us start with a concrete example. Imagine the inside of a room in winter. It is warm near the heater and cold by the window—a single real number, the air temperature, corresponds to every point in the room. We already learned in Chapter 6 that this situation, in which “one number is determined at each point,” can be expressed in formulas as a function $f(x,y)$ or $f(x,y,z)$ of two variables (a plane) or three variables (space). Now imagine the outdoors on a windy day: this time, a vector, that is, the strength and direction of the wind, corresponds to every point. We use the same word “field” in both cases, but we distinguish them according to whether what corresponds to each point is a number (a scalar) or a vector.

From this chapter onward (Chapters 18–20), the multivariable functions $f,g,h$, etc. that appear are all assumed, unless stated otherwise, to be totally differentiable (Chapter 6, Section 6.3). If a function is totally differentiable, all of its partial derivatives exist, and a tangent plane can be properly drawn to the graph of the function—the gradient vector treated in this chapter is a tool for extracting the information carried by this tangent plane.

18.2 Level Curves and Level Surfaces

The tools most often used to grasp the “shape” of a scalar field are the level curve and the level surface. In fact, these are not new concepts. In Chapter 6, Section 6.1, in order to study the shape of the graph (a surface) of a function $z=f(x,y)$ of two variables, we considered “the curve in the $xy$-plane obtained by fixing the height $z=c$”—the contour line. A level curve is precisely this contour line, renamed in the language of scalar fields. When we extend this to three variables (space), the contour line becomes a surface (the level surface).

18.3 Streamlines

For vector fields, too, there is a tool that makes their “shape” visible. Picture the arrows on a weather map showing the flow of the wind—just looking at scattered arrows already tells you how the wind is behaving, but if you draw a single smooth curve along the arrows (the path the wind follows), the overall picture of the flow becomes much easier to grasp. This “curve running along the arrows (the vector field)” is called a streamline.

Let us derive a way to find streamlines by formula. Represent the curve $C$, using a parameter $t$, as $\vec P(t)=[x(t),y(t),z(t)]$ (this is the parametric representation by a vector-valued function that we learned in Chapter 17). Then the tangent vector of $C$ is given by

(this is the differentiation of vector-valued functions that we learned in Chapter 17—all we had to do was differentiate each component with respect to $t$). By Definition 18.5, $\vec f(P)=[f,g,h]$ (each component is a function of $x,y,z$, but on $C$ it is also a function of $t$) is also tangent to $C$ at the same point, so $\vec f(P)$ and $\vec P'(t)$ are parallel vectors. That is, there exists a nonzero real number $k$ for which

holds (we assume $\vec f(P)\neq\vec 0$). That two vectors are parallel, with the same proportionality constant $k$ for every component, means that the ratios of the corresponding components are all equal, so we have

Rewriting this from the ratio form into the fraction form, we obtain

Then, formally canceling the $dt$ that appears in the numerator and denominator of each expression, we obtain the following relation.

This is a differential equation in the variables $x,y,z$ (it does not contain $t$ explicitly), and by solving it (that is, by solving two simultaneous first-order differential equations) we obtain the equations of the streamlines.

18.4 Definition of the Gradient Vector

Now we come to the main character of this chapter. Given a scalar field $f$, we can build a vector that bundles together, at each point, the way it “increases.” Before giving the definition, let us see what this vector looks like with concrete numbers.

Consider the scalar field $f(x,y)=x^2+y^2$ and examine how $f$ “increases” at the point $(1,1)$. Using partial derivatives (Chapter 6, Section 6.2),

so at the point $(1,1)$ both are equal to $2$. Now we form the vector $[2,\,2]$ by lining up the two values of the partial derivatives, and provisionally call it the “slope vector.” Then, starting from the point $(1,1)$, let us move a little in the direction of various unit vectors $\vec e$ and compare the rate at which $f$ increases (we estimate this by the inner product $[2,2]\cdot\vec e$—which is exactly the idea behind the total differential $\Delta f\fallingdotseq\pdiff{f}{x}\Delta x+\pdiff{f}{y}\Delta y$).

(i) $\vec e=[1,0]$ (the $x$-axis direction): rate of increase $= 2\cdot1+2\cdot0 = 2$.
(ii) $\vec e=[0,1]$ (the $y$-axis direction): rate of increase $= 2\cdot0+2\cdot1 = 2$.
(iii) $\vec e=\left[\dfrac{1}{\sqrt2},\dfrac{1}{\sqrt2}\right]$ (the same direction as the slope vector $[2,2]$): rate of increase $=2\cdot\dfrac1{\sqrt2}+2\cdot\dfrac1{\sqrt2}=\dfrac{4}{\sqrt2}=2\sqrt2\approx2.83$.
(iv) $\vec e=\left[\dfrac{1}{\sqrt2},-\dfrac{1}{\sqrt2}\right]$ (the direction perpendicular to the slope vector): rate of increase $=2\cdot\dfrac1{\sqrt2}+2\cdot\left(-\dfrac1{\sqrt2}\right)=0$.

As we can see, moving in the same direction as the slope vector $[2,2]$ (case (iii)) gives a larger rate of increase ($2\sqrt2>2$) than moving in the $x$-axis or $y$-axis direction, and conversely, in the direction perpendicular to the slope vector (case (iv)), $f$ neither increases nor decreases at all. In other words, this “slope vector” points exactly in the direction in which the value of $f$ increases fastest—and this is the true nature of the gradient vector that we are about to define. (A general proof of this fact will be given in Section 18.6.)

In general, we define this “slope vector,” that is, the vector formed by lining up all the partial derivatives, as follows.

18.5 Formulas for the Gradient Vector

Since $\nabla$ is merely a symbol that lines up partial derivatives (that is, derivatives), $\nabla$ also inherits, unchanged, the properties that ordinary differentiation (the one-variable $\dfrac{d}{dx}$) satisfies—linearity, the product rule, the quotient rule, and the chain rule.

To try this out, let us verify that (II) holds in the case of two variables. The first component (the $x$ component) of $\nabla(fg)$ is, by the ordinary product rule (we are only differentiating partially with respect to $x$, regarding everything other than $x$ as constant, so the one-variable product rule can be used as is),

and this coincides exactly with the first component $\pdiff{f}{x}g+f\pdiff{g}{x}$ of $(\nabla f)g+f(\nabla g)$. The second component (the $y$ component) is similar. In other words, (II) simply applies the one-variable product rule to each component separately. (I) and (III) can be verified by exactly the same idea (apply the one-variable formula to each component), so you should try them yourself. Only (IV) requires some care because a composite function is involved, and it is the only one for which a proof is given in the original notes as well. Let us follow that proof below.

Next we derive three formulas for functions of the position vector, which are especially frequently used among the gradient-vector formulas. First let us confirm the notation. The vector from the origin to the point $P(x,y)$ (or $P(x,y,z)$), $\vec P=[x,y]$ (or $[x,y,z]$), is called the position vector, and its magnitude (norm) is written $|\vec P|=\sqrt{x^2+y^2}$ (or $\sqrt{x^2+y^2+z^2}$)—this is the distance from the origin to the point $P$.

18.6 Geometric Meaning of the Gradient Vector

In the numerical example of Section 18.4, we confirmed with concrete numbers that the gradient vector appears to point in “the direction in which the value of $f$ increases fastest.” Here we prove that this holds in general.

18.6.1 The case of a planar scalar field

Let $f(x,y)$ be a totally differentiable scalar-valued function of two variables. Suppose that a curve in the plane can be expressed with a parameter $t$ as $x=x(t),\,y=y(t)$, and let $P(x_P,y_P)$ be a point on this curve. That is, for some $t_P$ we have $x_P=x(t_P),\,y_P=y(t_P)$. What we should note here is that $f(x,y)$ is a function in which $x,y$ can be varied freely, whereas $f(x_P,y_P)$, obtained by fixing the single point $P$, is just a number (a constant).

Now let us consider the level curve $C$ through the point $P$ (Definition 18.3). The points on $C$ satisfy the equation

Since the right-hand side $f(x_P,y_P)$ is a constant, this equation means that “the value of the left-hand side does not change even if we vary $t$.” If we rewrite both sides of this equation using the total differential in $x,y$ (the idea of $\Delta f\fallingdotseq\pdiff{f}{x}\Delta x+\pdiff{f}{y}\Delta y$ that we learned in Chapter 6, Section 6.3, turned into an equality by taking a limit: $df=\pdiff{f}{x}dx+\pdiff{f}{y}dy$), we obtain

(The right-hand side is $0$ because the left-hand side $f(x_P,y_P)$ is a constant and its total differential is always $0$.) Next, to relate both sides to the changes in $x,y$ (as functions of $t$), we differentiate with respect to $t$. Regarding $dx,dy$ as $\dfrac{dx}{dt}dt,\ \dfrac{dy}{dt}dt$, respectively, and dividing by $dt$, we obtain

(Since $f(x_P,y_P)$ is a constant, the right-hand side stays $0$ when differentiated with respect to $t$.) Looking closely at the left-hand side of ①, we notice that it has the form of the inner product of the two vectors $\left[\pdiff{f}{x},\,\pdiff{f}{y}\right]$ and $\left[\dfrac{dx}{dt},\,\dfrac{dy}{dt}\right]$. In fact, applying the definition of the inner product (the sum of the products of corresponding components), ① can be written directly as

The vector on the left is precisely $\nabla f$ of Definition 18.6, and the vector on the right is the derivative $\vec P'(t)$ of $\vec P(t)=[x(t),y(t)]$ (Chapter 17). Therefore we obtain

Now, $\vec P'(t)$ is none other than a tangent vector to the curve $C$ (the level curve). Eq. \eqref{eq:18-grad-perp} says that “the inner product of $\nabla f$ and the tangent vector $\vec P'(t)$ is $0$,” so by the property that the inner product of two vectors is $0$ when they are perpendicular (high school mathematics; High School Mathematics, Section 7.4), we have $\nabla f\perp\vec P'(t)$. That is,

So far we have learned that “$\nabla f$ is perpendicular to the level curve,” but there are two perpendicular directions (inward and outward). Which way does it point? To find out, let us return to the approximation formula for the total differential (Chapter 6, Section 6.3)

Suppose we keep $\Delta\vec x$ at a fixed magnitude (say, of length $1$) and change only its direction—that is, we take one step of fixed length from the point $P$ in every possible direction. Then the inner product of the two vectors, $\nabla f\cdot\Delta\vec x=|\nabla f|\,|\Delta\vec x|\cos\theta$ ($\theta$ is the angle between the two vectors), is largest when $\theta=0$ ($\Delta\vec x$ points in the same direction as $\nabla f$), smallest (most negative) when $\theta=180°$ (opposite direction), and equal to $0$ when $\theta=90°$ (perpendicular)—and this is exactly what we confirmed in the numerical example of Section 18.4. We therefore learn the following.

18.6.2 The case of a spatial scalar field

The very same argument holds for a scalar-valued function $f(x,y,z)$ of three variables as well. The “level curve $C$” of the planar case is replaced in space by the “level surface $S$,” and all the tangent vectors of curves lying on a level surface are contained in the tangent plane of that level surface. Therefore Theorem 18.1 of the planar case ($\nabla f$ is perpendicular to the tangent vectors of the level curve) is upgraded in space to the following.

(Outline of the proof: Take, on the level surface $S$, an arbitrary curve through $P$, $\vec P(t)=[x(t),y(t),z(t)]$. By exactly the same computation as in Section 18.6.1 (starting from the fact that $f(x(t),y(t),z(t))$ is constant, take the total differential and then differentiate with respect to $t$), we can show that $\nabla f\cdot\vec P'(t)=0$. There are infinitely many curves on $S$ through the point $P$, and the set of all their tangent vectors $\vec P'(t)$ spans the tangent plane, so $\nabla f$ is perpendicular to every direction in that tangent plane—that is, to the entire tangent plane.)

18.7 Equation of the Tangent Plane via the Gradient Vector

Theorem 18.2 ($\nabla f$ is perpendicular to the level surface) becomes a very powerful tool in practice. This is because the equation of the tangent plane to a surface can be written down immediately as long as we know “a point on the plane” and “a normal vector perpendicular to that plane” (High School Mathematics, Section 7.6).

Now suppose we want the tangent plane at a point $(x_1,y_1,z_1)$ on the surface $z=f(x,y)$. If we rewrite the equation of the surface in the form $F(x,y,z)=f(x,y)-z=0$ (one of the level surfaces, $F=0$), then by Theorem 18.2, $\nabla F$ is a normal vector of the tangent plane. In other words, the effort of finding the normal vector $\vec n$ ourselves is replaced by the computation of $\nabla F$. Let us check this with concrete examples.

The strength of this method is that it can be used even when the surface is not given in the form $z=f(x,y)$ (an explicit function). In the next example, let us apply the same method to the sphere, a surface that is hard to write in the form “$z=$ (an expression in $x,y$).”

18.8 Summary and Exercises

18.8.1 Summary

• Scalar fields and vector fields (Definitions 18.1 and 18.2): an assignment of a number (a scalar) or a vector to each point of a region.
• Level curves and level surfaces (Definitions 18.3 and 18.4): the set of points satisfying $f(x,y)=$ (constant) or $f(x,y,z)=$ (constant). A generalization of the “contour lines” of Chapter 6.
• Streamline (Definition 18.5): a curve that is tangent to the vector field $\vec f$ at every point. It is found by solving the differential equation $\dfrac{dx}{f}=\dfrac{dy}{g}=\dfrac{dz}{h}$.
• Gradient vector (Definition 18.6): $\nabla f=\left[\pdiff{f}{x},\pdiff{f}{y}\right]$ (in the plane) or $\left[\pdiff{f}{x},\pdiff{f}{y},\pdiff{f}{z}\right]$ (in space). $\nabla$ is a differential vector operator called nabla (del, or the Hamilton operator).
• Gradient formulas (Formulas 18.1–18.4): the formulas for linearity, products, quotients, and composite functions, together with $\nabla|\vec P|=\dfrac{\vec P}{|\vec P|}$, $\nabla|\vec P|^n=n|\vec P|^{n-2}\vec P$, and $\nabla\ln|\vec P|=\dfrac{\vec P}{|\vec P|^2}$.
• Geometric meaning of the gradient (Theorems 18.1 and 18.2): $\nabla f$ is perpendicular to the level curves and level surfaces, and points in the direction in which $f$ increases fastest (the direction of steepest ascent).
• Equation of the tangent plane: put the surface in the form $F(x,y,z)=0$, and obtain the plane from the normal vector $\nabla F$ and the point that the plane passes through, in the form $a(x-x_1)+b(y-y_1)+c(z-z_1)=0$ (where $[a,b,c]=\nabla F$).

18.8.2 Exercises

18.8.3 References