Chapter 1The Rigorous Definition of Limits and an Introduction to Differentiation

1.1 Writing the Convergence of a Sequence as a Logical Formula: The $\varepsilon$-$N$ Argument

When we say in high school mathematics that “the sequence $\{a_n\}$ converges to $\alpha$,” what we have in mind is probably an image like the following. “As we increase $n$ through $1,2,3,\dots$, $a_n$ gradually approaches $\alpha$, and once $n$ is sufficiently large, the difference between $a_n$ and $\alpha$ becomes as small as we like.” This image itself is correct. The problem is that the phrases “as small as we like” and “sufficiently large” rely on feel rather than on mathematical formulas. As it stands, we cannot prove, in a way that leads everyone to the same conclusion, whether a given sequence really converges.

So we change our way of thinking. We fix the “standard of closeness” first. No matter how strict a standard (that is, no matter how small a positive number) is brought to us, we can prepare in response an index such that “from that index on, the standard is satisfied”—we regard this as the true nature of the convergence of a sequence. The positive number expressing this “standard” is conventionally written $\varepsilon$ (epsilon, the Greek letter corresponding to the English $e$), and the natural number that guarantees “from that index on” is written $N$.

Before getting into abstract symbols, let us first check with a concrete sequence. The sequence $a_n=\dfrac1n$ ($n=1,2,3,\dots$) seems, intuitively, to converge to $0$. Let us fix the “standard of closeness” $\varepsilon$ at several concrete values and actually look for the corresponding $N$.

As this table shows, each time we tighten the “standard of closeness” $\varepsilon$ to $1$/$10$ of its previous value, the required $N$ grows by a factor of roughly $10$. However small we make $\varepsilon$, though, a corresponding (finite) $N$ can always be found—the following Definition 1.1 formulates this fact, “for every $\varepsilon$ there exists a corresponding $N$,” not as a statement about a particular sequence or particular numerical values but as a property that should hold for any sequence and any $\varepsilon$.

For readers seeing them for the first time, let us explain the three symbols that appear in this definition.

• $\forall$ (for all): read as “for every” or “for any.” “$\forall\varepsilon\gt0$” means “for every $\varepsilon$ (satisfying the condition of being a positive number),” and it expresses that the side of $\varepsilon$ can raise no objection, whichever value is chosen.
• $\exists$ (there exists): read as “there exists ….” “$\exists N\in\N$” means “there exists some $N$ (among the natural numbers) such that,” and it expresses that the side of $N$ may choose conveniently after seeing the value of $\varepsilon$.
• $\text{s.t.}$ (such that): read as “such that it satisfies ….” It serves to specify, by the condition that follows, the property of the existence symbol $\exists N$ placed immediately before it.

What matters is the order in which the symbols are lined up. In Eq. \eqref{eq:1-epsilon-N}, “$\forall\varepsilon$” is written first and “$\exists N$” afterward. This means the following order: first the opponent (or you yourself) chooses $\varepsilon$ arbitrarily, and only after that may you choose $N$ according to the $\varepsilon$ that was chosen. In general the value of $N$ may depend on the value of $\varepsilon$ (in fact, in most cases it does). For example, if we tighten $\varepsilon$ from $1/10$ to $1/1000$, we will usually need to re-choose $N$ as well, making it larger.

1.2 Writing the Limit of a Function as a Logical Formula: The $\varepsilon$-$\delta$ Argument

In the previous section we gave a rigorous definition of the limit of a sequence (which may be regarded as a function whose variable is a natural number $n$). The same idea applies to the limit of a general function $f(x)$ whose variable is a real number $x$. For sequences, the condition “once $n$ has advanced far enough” was expressed by a natural number $N$. For functions, the way the condition is written changes subtly depending on the target that $x$ “approaches arbitrarily closely.” The original notes divide this into three cases, so let us go through them in order. In every case the idea is exactly the same as in the $\varepsilon$-$N$ argument, with a positive number $\delta$ (delta) taking over the role of $N$.

Here, too, before moving on to abstract symbols, let us check with concrete numbers. Let $f(x)=3x-1$ and consider the limit as $x\to2$. Since $f(2)=5$, it seems that $\displaystyle\lim_{x\to2}f(x)=5$. Because $|f(x)-5|=|(3x-1)-5|=|3x-6|=3|x-2|$ is a simple proportional relation, once we decide the “precision at the destination” $\varepsilon$, we can determine the “closeness at the starting point” $\delta$ concretely in terms of $\varepsilon$.

Here, too, each time $\varepsilon$ is tightened to $1$/$10$ of its previous value, the required $\delta$ also shrinks to about $1$/$10$ of its previous value (in this example there is the simple proportional relation $\delta=\varepsilon/3$). The following Definition 1.2 formulates this fact, “for every $\varepsilon$ there exists a corresponding $\delta$,” not for a particular function and particular numbers, but as a property that should hold for a general function $f(x)$ and a real number $a$.

Let us break down the meaning of the condition $0\lt|x-a|\lt\delta$ in (III). $|x-a|\lt\delta$ means “$x$ lies at a distance less than $\delta$ from $a$” (the same as $a-\delta\lt x\lt a+\delta$). On top of that, there is the further condition $0\lt|x-a|$, that is, $x\neq a$. This expresses that the limit of a function does not concern the value $f(a)$ at $x=a$ itself (a limit can be considered even if $f$ is not defined at $a$). What the limit looks at is strictly the behavior of $f(x)$ near $a$ (but with $a$ itself excluded).

1.3 Mastering the $\varepsilon$-$N$ Argument

Memorizing a definition does not by itself make it a tool you can use. In this section we put the $\varepsilon$-$N$ argument to actual use: we give examples of proving the convergence of concrete sequences, and we prove head-on, with the $\varepsilon$-$N$ argument, two important theorems that you may know only by name from high school (or may be hearing of for the first time)—d'Alembert's ratio test (named after the French mathematician Jean le Rond d'Alembert) and the squeeze theorem. In high school we merely used these as “that's just how it is,” but with the help of the argument we can explain completely why they are correct.

Next we prove a theorem about general sequences. Here, for the first time, we deal not with a concrete sequence $a_n$ but with a property that holds for “any sequence satisfying $\lim a_n=0$.”

Next we prove d'Alembert's ratio test, which decides the convergence or divergence of a positive-term series (a series all of whose terms are positive). A “series” is the sum $\sum_{n=1}^\infty a_n$ of a sequence, and its “convergence” means that the sequence $\{S_n\}$ formed by the partial sums $S_n=\sum_{k=1}^na_k$ converges to some real number (in the sense of Definition 1.1).

Finally, we prove the squeeze theorem. This is also a theorem that we used in high school as something that “seems intuitively correct,” but with the $\varepsilon$-$N$ argument it can be proved completely.

1.4 Introducing Differentiation: From the Average Rate of Change to the Instantaneous Rate of Change

From here we change the topic and take another look at “differentiation,” which you also learned in high school, using the same language of “limits” as in the $\varepsilon$-$N$ and $\varepsilon$-$\delta$ arguments. Before explaining differentiation, let us first confirm what a “slope” is. The slope is

For example, on the graph of $y=x^2$, let us find the slope when moving from $x=1$ to $x=4$ (the average rate of change). Since $y=1^2=1$ when $x=1$ and $y=4^2=16$ when $x=4$, we obtain

as the slope.

Now let $f(x)=y=x^2$. When a value $x$ increases by $\Delta x$ (“delta x,” the increment of $x$), the average rate of change can be expressed as

(Here $\Delta y=f(x+\Delta x)-f(x)$ is the increment of $y$.)

Now, what happens as $\Delta x$ comes arbitrarily close to $0$?

If we attach $\displaystyle\lim_{\Delta x\to0}$ to the average rate of change found above, we get $\displaystyle\lim_{\Delta x\to0}\frac{f(x+\Delta x)-f(x)}{\Delta x}$. To see what this means, let us actually compute it using the example $f(x)=x^2$.

Let us check the manipulation in each line. The first line simply substitutes $f(x)=x^2$. In the second line, we expand $(x+\Delta x)^2$ by the expansion formula $(a+b)^2=a^2+2ab+b^2$. In the third line, the two $x^2$ in the numerator cancel each other and vanish, so we factor $\Delta x$ out of the remaining $2x\cdot\Delta x+(\Delta x)^2$. In the fourth line, since $\Delta x\neq0$ before the limit $\Delta x\to0$ is taken, we can divide the numerator and denominator by $\Delta x$ and cancel (being able to cancel is the benefit of the fact that “$\Delta x$ is not $0$”). In the fifth line, in the expression $2x+\Delta x$ obtained after cancellation, we now at last take the limit $\Delta x\to0$; the term in $\Delta x$ vanishes and only $2x$ remains.

In this way, attaching $\lim$ to the average rate of change gives the instantaneous slope. And attaching $\lim$ to the average rate of change is called differentiation. Also, the function that comes out when we differentiate a function is called the derivative.

1.5 The Definition of Differentiation and the Introduction of Napier's Constant $e$

Generalizing the computation of the previous section, we now state the definition of differentiation.

The $dx$ and $dy$ that appear in the notation $\dfrac{dy}{dx}$ (Leibniz notation, named after the German mathematician Gottfried Wilhelm Leibniz) mean the infinitesimal increments of $x,y$, respectively. That is, in the limit $\Delta x\to0$, the “infinitely small increment” that $\Delta x$ finally becomes is called $dx$, the corresponding increment of $y$ is called $dy$, and their ratio $dy/dx$ is the derivative at a point (= the instantaneous slope). In terms of a figure: if we magnify the immediate neighborhood of the point where the tangent line touches the graph by an enormous factor, the tangent line itself appears as a small right triangle—it helps to picture the base of that triangle as $dx$, its height as $dy$, and the slope of its hypotenuse as $dy/dx$.

Now that we have the definition of differentiation in hand, let us take on one mystery—why is the base of the natural logarithm (Napier's constant) $e$ defined by the seemingly strange expression $\displaystyle\lim_{x\to0}(1+x)^{1/x}$? The key to solving this mystery lies in the question, “What happens when we differentiate a logarithmic function?”

Take a logarithmic function $y=\log_c x$ ($c$ is some positive real number, $c\neq1$), and differentiate it according to the definition of differentiation.

Let us check the transformation on each line. The first line is the definition of differentiation itself (substitute $f(x)=\log_c x$). In the second line, we used the property of logarithms $\log_c A-\log_c B=\log_c(A/B)$ to combine $\log_c(x+h)-\log_c x=\log_c\dfrac{x+h}{x}=\log_c\left(1+\dfrac hx\right)$. The third line merely rewrites $\dfrac1h=\dfrac1x\cdot\dfrac xh$ (assuming $x\neq0$). In the fourth line, we put $\dfrac hx=u$. Since $u=h/x\to0$ as $h\to0$, we can replace the limit variable $h$ by $u$. Also, since $\dfrac xh=\dfrac1u$, we can combine $\dfrac1u\cdot\log_c(1+u)=\log_c(1+u)^{1/u}$ (using the property of logarithms $k\log_c A=\log_c A^k$).

At the stage of Eq. ①, we cannot yet say that “the differentiation is complete.” This is because the limit $\displaystyle\lim_{u\to0}\log_c(1+u)^{1/u}$ still remains. If this limit (the value that $(1+u)^{1/u}$ approaches as $u\to0$) happens to coincide with the base $c$ itself, then $\log_c c=1$, and the whole of Eq. ① settles into the clean expression $\dfrac1x\cdot1=\dfrac1x$. So we reverse our line of thinking—we choose the base $c$ cleverly so that this limit becomes $1$. To do so, it suffices to define a real number $C$ by

(we have changed the variable back from $u$ to $x$; the choice of letter is free) and to adopt $c=C$ as the base. To this special real number $C$ we conventionally assign the symbol $e$.

The logarithm $\log_e x$ to the base $e$ (Napier's constant, named after the Scottish mathematician John Napier) is called the natural logarithm and is also written $\ln x$. If we choose $c=e$ according to Definition 1.4, the discussion of Eq. ① yields the clean formula $\dfrac{d}{dx}\log_e x=\dfrac1x$ (a complete proof of this formula itself will be given again in Chapter 3).

Since $(\log_e x)'=\dfrac1x$, at the point where the graph of $y=\ln x$ has $y=0$, that is, at $x=1$, the slope of the tangent line at that point is $\dfrac11=1$.

Let us derive this fact properly as a limit formula. Take the natural logarithm of both sides of the formula $e=\displaystyle\lim_{x\to0}(1+x)^{1/x}$ of Definition 1.4 (Eq. ①’).

(The first equality uses the fact that, after applying $\ln$ to both sides, the order of the limit and the continuous function $\ln$ can be exchanged. The second equality uses the property of logarithms $\ln A^{k}=k\ln A$.) Now recall that $e$ itself was chosen as the “base of $\log_e$.” We have $\ln e=\log_e e=1$ (whatever base we take, the logarithm of the base itself is $1$: $\log_c c=1$). Hence the equation above becomes

Thus we have confirmed that “indeed $\ln e=1$,” a fact that looks obvious at first sight but is in fact derived from the very definition of $e$.

Take the reciprocals of both sides of Eq. ②. Since the limit value on the right-hand side of Eq. ② is $1$ ($\neq0$), we may exchange the order of taking the reciprocal and taking the limit ($\lim(1/b_n)=1/\lim b_n$ holds only when the limit is not $0$).

Let us transform Eq. ③ further to derive a limit involving $e^x$. Put $x=e^{u}-1$ (this may seem forced, but as we shall see, this substitution makes $\ln(1+x)$ come out neatly as just $u$). Then $1+x=e^u$, so $\ln(1+x)=\ln e^u=u$. Also, when $x\to0$ we have $e^u-1\to0$, that is, $e^u\to1$, and therefore $u\to0$ (conversely, if $u\to0$ then $x=e^u-1\to0$ as well, so $x\to0$ and $u\to0$ occur simultaneously). Substituting these into Eq. ③, we obtain

Replacing the variable $u$ by $x$ (any letter will do), we get

Let us consider the meaning of Eq. ④. With $f(x)=e^x$, the left-hand side of Eq. ④ is

that is, it is nothing other than the derivative $f'(0)$ of $f(x)=e^x$ at $x=0$ (we used $e^0=1$). Hence Eq. ④ means that $f'(0)=1$, that is, the slope of the tangent line to the graph of $y=e^x$ at $x=0$ is $1$.

1.6 The Defining Formula of the Derivative, Summary, and Exercises

Finally, let us rewrite the definition of differentiation (Definition 1.3) in the form most commonly used. It is customary to use the letter $h$ in place of $\Delta x$ and to write $\Delta x\to0$ as $h\to0$.

The middle expression uses the point $x+h$ reached by moving forward from $x$ by $h$ (the forward difference; either $h\to0^+$ or $h\to0^-$ is allowed). The right-hand expression uses the point $x-h$ lying $h$ behind $x$ (the backward difference). When both converge to the same function $f'(x)$ as $h\to0$, we define it to be the derivative of $f(x)$. (This condition, “the limit of the forward difference and the limit of the backward difference coincide,” is in fact not trivial—there are examples in which they do not coincide. After studying “continuity of functions” in Chapter 2, we shall treat this point in more depth.)

1.6.1 Summary

• The rigorous definition of the convergence of a sequence (the $\varepsilon$-$N$ argument, Definition 1.1): $\forall\varepsilon\gt0,\ \exists N\in\N\ \text{s.t.}\ n\ge N\Rightarrow|a_n-\alpha|\lt\varepsilon$. What is essential is the order: $\forall$ (“for all”) comes first and $\exists$ (“there exists”) comes afterward.
• The rigorous definition of the limit of a function (the $\varepsilon$-$\delta$ argument, Definition 1.2): there are three cases, $x\to\infty,\,x\to-\infty,\,x\to a$. In particular, the case $x\to a$ reads $0\lt|x-a|\lt\delta\Rightarrow|f(x)-P|\lt\varepsilon$ (the point $x=a$ itself is excluded).
• Applications of the $\varepsilon$-$N$ argument: we proved the Cesàro-mean-type theorem (Theorem 1.1), d'Alembert's ratio test (Theorem 1.2: a positive-term series $\sum a_n$ converges if $r=\lim a_{n+1}/a_n$ satisfies $r\lt1$, and diverges if $r\gt1$), and the squeeze theorem (Theorem 1.3), each completely by the $\varepsilon$-$N$ argument alone.
• The definition of differentiation (Definition 1.3): $dy/dx$ is the limit of the average rate of change $\Delta y/\Delta x$ as $\Delta x\to0$. For $f(x)=x^2$, $dy/dx=2x$.
• The definition of Napier's constant (Definition 1.4): $e=\displaystyle\lim_{x\to0}(1+x)^{1/x}$. This is what results from choosing the base $c$ so that “the derivative of $\log_c x$ takes the simplest form $1/x$.” From it follow $\ln e=1$, $\displaystyle\lim_{x\to0}\frac{\ln(1+x)}{x}=1$, and $\displaystyle\lim_{x\to0}\frac{e^{x}-1}{x}=1$ (Eq. ④), and these mean that the slopes of the tangent lines to $y=\ln x$ (at $x=1$) and to $y=e^x$ (at $x=0$) are both $1$.
• The defining formula of the derivative (Definition 1.5): $f'(x)=\displaystyle\lim_{h\to0}\frac{f(x+h)-f(x)}{h}=\lim_{h\to0}\frac{f(x)-f(x-h)}{h}$.

1.6.2 Exercises

1.6.3 References