Chapter 14Expected Value — From Discrete to Continuous

Example 14.1: Expected value of the number rolled on a die

When one die is rolled once, find the expected value $E(X)$ of the number $X$ rolled.

Solution. The faces 1 through 6 of a die are equally likely to come up, so $x_k=k$ and $P_k=\dfrac16$ ($k=1,2,\dots,6$). Substituting into Eq. \eqref{eq:14-discrete} gives the following:

(SymPy check: the value of sum(k*Rational(1,6) for k in range(1,7)) is exactly $7/2$; they agree.) Although the number rolled takes only the integer values from $1$ to $6$, the expected value is the non-integer $3.5$—this is because the expected value represents not “one of the values that can actually occur” but “the average value over many repetitions.”

Example 14.2: Expected value of a prize drawing and the “fair entry fee”

In a certain prize drawing, the prizes and the probabilities of winning are fixed as in the following table.

Find the expected value of the prize $X$ in this drawing, and determine what entry fee for one play would be a “fair” amount at which neither the organizer nor the participant loses.

Solution. First we find the probability of “no prize” from the fact that the probabilities add up to $1$. Since $\displaystyle\sum_k P_k=1$, the probability of no prize is

(indeed it is at least $0$, and the whole is consistently normalized to $1$). By Eq. \eqref{eq:14-discrete}, the expected value is

(Verified with SymPy: exact fraction arithmetic also gives $E(X)=300$.) The reason the expected value serves as a guide to a “fair entry fee” is as follows. If the entry fee is $c$ yen, the expected net profit per play (prize $-$ entry fee) is $E(X)-c$. If this is positive, the participants gain on average and the organizer loses. Conversely, if it is negative, the organizer gains. Therefore, the participants' average gain or loss when this drawing is repeated many times is exactly $0$ only when $c=E(X)=300$ yen, and only in that case can we call it “fair.” (The entry fees of actual lotteries and prize drawings are usually set higher than the expected value in order to cover operating costs.)

Example 14.3: A uniform density $P(x)=1$ (on the interval $[0,1]$)

Suppose that on the interval $[0,1]$ ($L=1$) the probability density is the constant $P(x)=1$. When this interval is divided into $n$ equal parts, compute the left-hand side $\sum_{k=1}^n P_k$ of Eq. \eqref{eq:14-norm-density}, and verify that it is always $1$ regardless of the value of $n$.

Solution. Since $P(x)=1$ (a constant), we have $P\!\left(\frac{k}{n}\right)=1$. Hence

which shows that it is indeed always $1$ regardless of $n$ (without even taking the limit $n\to\infty$). (SymPy check: simplifying summation(1/n,(k,1,n)) gives exactly $1$.) This example is special: precisely because the density is constant, independent of position, no approximation error arises at all even for finite $n$. When the density varies with $x$, as in the next example, $\sum P_k$ in general deviates slightly from $1$ for finite $n$, and becomes exactly $1$ only in the limit $n\to\infty$.

Example 14.4: A linearly increasing density $P(x)=\dfrac{2}{L^2}x$ (on the interval $[0,L]$, with $L=2$)

Suppose that on the interval $[0,2]$ the probability density is $P(x)=\dfrac{2}{L^2}x=\dfrac12 x$ (for the case $L=2$). (a) For each of $n=5,10,100,1000$, compute $\sum_{k=1}^n P_k$ and observe how it approaches $1$ as $n\to\infty$. (b) Separately from the computation with $n\to\infty$, compute the integral $\int_0^L P(x)\,dx$ directly, and verify that for general $L$ its value is always $1$, independent of $L$.

Solution. (a) Since $x_k=\dfrac{2k}{n}$ and the width is $\dfrac{2}{n}$,

(In the last step we used the formula $\displaystyle\sum_{k=1}^n k=\frac{n(n+1)}{2}$ from high school mathematics.) Substituting $n=5,10,100,1000$ gives $\frac65=1.2$, $\frac{11}{10}=1.1$, $\frac{101}{100}=1.01$, and $\frac{1001}{1000}=1.001$, respectively (verified with SymPy as exact fractions), so the sums indeed approach $1$. For finite $n$, however, the sum is always slightly larger than $1$—this is because the density is an increasing function, so using the value $P(x_k)$ at the right endpoint of each small interval as the height slightly overestimates the true area.

(b) For general $L$,

(Verified with SymPy for general $L$ (a symbol declared positive, $L>0$).) The value is always $1$ regardless of $L$—this is because the density function is constructed in the first place so that its integral over the whole interval is $1$ (the proportionality constant $\frac{2}{L^2}$ is chosen precisely for that purpose). The numerical computation in (a) is nothing other than an approximation of this integral by the Riemann-sum method (next section), and it is expected to agree with the exact value $1$ of (b) as $n\to\infty$.

Example 14.5: Converting the limit of a sum into an integral with the Riemann-sum method ($f(x)=x$)

Using the Riemann-sum method, rewrite the limit $\displaystyle\lim_{n\to\infty}\sum_{k=1}^{n}\frac{k}{n}\cdot\frac1n$ as a definite integral and find its value.

Solution. We regard the sum $\displaystyle\sum_{k=1}^n\frac{k}{n}\cdot\frac1n=\sum_{k=1}^n\left(\frac{k}{n}\right)\frac1n$ as the sum $\displaystyle\sum_{k=1}^n f\!\left(\frac{k}{n}\right)\frac1n$ in Formula 14.1 with $L=1$ and $f(x)=x$. Hence

holds. (Check: using the formula $\sum_{k=1}^n k=\frac{n(n+1)}{2}$ from high school mathematics, the left-hand side becomes $\dfrac{1}{n^2}\cdot\dfrac{n(n+1)}{2}=\dfrac{n+1}{2n}$, and we have verified with SymPy that its limit as $n\to\infty$ agrees exactly with $\frac12$.) This result also serves as an alternative proof, by the Riemann-sum method, of the fact that the area of the right triangle (with vertices $(0,0),(1,0),(1,1)$) is $\frac12$.

Example 14.6: Expected value of the uniform distribution

Consider a random variable that takes values with the same density everywhere in the interval $[0,L]$ (the uniform distribution). From the normalization condition, the density is $P(x)=\dfrac1L$ ($0\le x\le L$). Find the expected value of this random variable.

Solution. Substituting $P(x)=\frac1L$ into Eq. \eqref{eq:14-key} gives

(Verified with SymPy for general $L$.) If every point of the interval $[0,L]$ is equally likely, then the expected value being exactly the midpoint $\frac L2$ of the interval agrees perfectly with our intuition (Figure 14.4).

Example 14.7: Expected value of a linearly increasing density (comparison with the uniform distribution)

Find the expected value of the random variable with the density $P(x)=\dfrac{2}{L^2}x$ ($0\le x\le L$) treated in Example 14.4, and compare it with the expected value of the uniform distribution of Example 14.6 (on the same interval $[0,L]$).

Solution. Substituting into Eq. \eqref{eq:14-key} gives

(Verified with SymPy.) Comparing this with the expected value $\frac L2$ of the uniform distribution (Example 14.6), we have $\frac{2L}{3}\gt\frac L2$. This agrees with the intuition that, because this density $P(x)=\frac{2}{L^2}x$ increases monotonically with $x$—that is, the density is higher toward the right of the interval (the side where $x$ is closer to $L$)—the expected value (the center of mass) is likewise pulled to the right of the midpoint of the interval. In general, the expected value is pulled toward the side where the density is higher—we will confirm this property once more in the exercises of Section 14.6.

Example 14.8: Expected value of the exponential distribution (integration by parts)

Phenomena in which the probability that an event occurs in the next instant, given the condition of having “survived up to a certain moment,” is constant in time—such as the time until a radioactive atom decays or the time until a light bulb fails—are known to follow the exponential distribution, whose density function is $P(x)=\lambda e^{-\lambda x}$ ($x\ge0$, where $\lambda>0$ is a constant). Find the expected value $E=\displaystyle\int_0^\infty x\,P(x)\,dx$ of this distribution.

Solution. First, let us confirm that $P(x)$ is indeed normalized as a density: $\displaystyle\int_0^\infty \lambda e^{-\lambda x}\,dx=\Big[-e^{-\lambda x}\Big]_0^\infty=0-(-1)=1$ (since $e^{-\lambda x}\to0$ as $x\to\infty$, because $\lambda>0$). Next, we compute the expected value following Eq. \eqref{eq:14-key}.

Since this integral has the form of a product of $x$ and $e^{-\lambda x}$, we use integration by parts (a technique that uses $\int u'v=uv-\int uv'$; it is one of the first integration techniques one learns at university). Let $u=x$ and $v'=\lambda e^{-\lambda x}$. Then $u'=1$ and $v=-e^{-\lambda x}$ (we only need to integrate $v'$). Hence

The first term is $0$ at $x=0$, and as $x\to\infty$ we have $xe^{-\lambda x}\to0$ (because the exponential function $e^{-\lambda x}$ approaches $0$ faster than any power of $x$), so the first term is $0$ as a whole. The second term is

so, in the end, we obtain

(It has also been verified that SymPy's integrate(x*lam*exp(-lam*x),(x,0,oo)) agrees exactly with $1/\lambda$ under the assumption $\lambda>0$.) Since $\lambda$ is a constant representing “the rate at which an event occurs per unit time” (the decay rate or failure rate), it is natural that its reciprocal $1/\lambda$ is “how long it takes on average” (the mean lifetime). Figure 14.5 shows the graph of the density $P(x)$ and the position of the expected value. The density is largest at $x=0$, but the expected value (the mean) is located not at $x=0$ but away from it, at $x=1/\lambda$. Note that the “most likely value” (the mode) and the “mean” (the expected value) do not in general coincide unless the distribution is symmetric.

Exercise 14.1: Expected value of the sum of the numbers on two dice

When two dice are thrown at the same time, find the expected value $E(X)$ of the sum $X$ of the numbers shown.

Hint: There are $6\times6=36$ possible cases in all. You may count the number of cases in which the sum is $2$, the number in which it is $3$, and so on, and substitute them into Definition 14.1; or you may use the fact that “the expected value of the number on one die is $\frac72$ by Example 14.1” together with the linearity of the expected value ($E(X_1+X_2)=E(X_1)+E(X_2)$; this is not proved in this chapter, but it is a property that holds naturally if we regard the number on each die as a random variable).

→ Solution

Exercise 14.2: Expected value and spread of a random variable taking three values (Advanced)

Suppose that a random variable $X$ takes the $3$ values $2,5,9$, each with equal probability $\frac13$. (a) Find the expected value $E(X)$. (b) Find $E(X^2)$ (the expected value of $X^2$; it can be computed from the values $x_k^2$ that $X^2$ takes and their probabilities $P_k$, by a formula of the same form as Definition 14.1). (c) The variance is a quantity that expresses how widely the data are scattered around the expected value, and it is properly defined as $V(X)=E\big[(X-E(X))^2\big]$ (the expected value of the square of how far each value is from the expected value; we do not go deeply into it in this chapter, but it is one of the most important quantities in statistics). Show that expanding this definition gives the formula $V(X)=E(X^2)-\{E(X)\}^2$, which is convenient for computation, and use the results of (a) and (b) to find the value of $V(X)$.

Hint: The values taken by $X^2$ are $4,25,81$, and each occurs with the same probability $\frac13$ as the original value. For the expansion in (c), expand $E\big[(X-E(X))^2\big]=E\big(X^2-2XE(X)+\{E(X)\}^2\big)$ term by term using the linearity of the expected value (since $E(X)$ is a constant, $E\big(2X\cdot E(X)\big)=2E(X)\cdot E(X)$).

→ Solution

Exercise 14.3: Normalization constant and expected value of a density function

Suppose that on the interval $[0,2]$ the probability density is $P(x)=cx$ ($c$ is a constant). (a) Find the constant $c$ from the normalization condition $\int_0^2 P(x)\,dx=1$. (b) Find the expected value of a random variable with this density.

Hint: For (a) use Eq. \eqref{eq:14-norm-integral}, and for (b) use Eq. \eqref{eq:14-key}. Both reduce to computing the definite integral $\int_0^2 x^m\,dx$ ($m=1,2$).

→ Solution

Exercise 14.4: Practice with the Riemann-sum method ($f(x)=x^2$)

Using the Riemann-sum method (Formula 14.1), convert the limit $\displaystyle\lim_{n\to\infty}\sum_{k=1}^{n}\left(\frac{k}{n}\right)^2\cdot\frac1n$ into a definite integral and find its value.

Hint: Just repeat exactly the same procedure as in Example 14.5 (the case $f(x)=x$) for $f(x)=x^2$. You can also check your answer by computing the sum directly with the high-school formula $\sum_{k=1}^n k^2=\frac{n(n+1)(2n+1)}{6}$ and then taking the limit $n\to\infty$.

→ Solution

Exercise 14.5: Mean lifetime and half-life of radioactive decay

Suppose that the time $T$ until a single atom of a certain radioactive isotope decays follows the exponential distribution with density $P(t)=\lambda e^{-\lambda t}$ ($t\ge0$). (a) Using the result of Example 14.8, express the mean lifetime $\tau=E(T)$ in terms of $\lambda$. (b) Find the time for the number of atoms to decrease to exactly half (the half-life $T_{1/2}$) by solving the equation $e^{-\lambda T_{1/2}}=\frac12$, and express its relation to $\tau$ by a formula. (c) When $\tau=10$ (the unit of time is arbitrary), find an approximate value of $T_{1/2}$.

Hint: In (b), take the natural logarithm of both sides. Note that $\ln\frac12=-\ln2$.

→ Solution

Exercise 14.6: Comparing the expected values of the uniform and triangular distributions

On the interval $[0,L]$, compare the following two densities: (i) the uniform distribution $P(x)=\frac1L$, and (ii) the “triangular” density $P(x)=\dfrac2L\left(1-\dfrac xL\right)$, which takes its maximum value at $x=0$ and decreases linearly until $x=L$. (a) Verify that the density (ii) satisfies the normalization condition $\int_0^L P(x)\,dx=1$. (b) Find the expected value of (ii) and compare it with the expected value $\frac L2$ of (i) (Example 14.6). (c) How is the result of (b) consistent with the property stated in Example 14.7 that “the expected value is pulled toward the side where the density is higher”? Explain, referring to Figure 14.4.

Hint: Both (a) and (b) reduce to computing $\int_0^L x^m\,dx$ ($m=0,1,2$). For (c), note that the density (ii) is largest near $x=0$.

→ Solution