Chapter 48Microstates and the Principle of Equal A Priori Probability — The Microcanonical Distribution

48.1 Microstates and Macrostates

Consider an experiment in which $4$ coins are tossed at the same time. So that we can tell the coins apart, we label them with serial numbers from $1$ to $4$. A result that specifies completely “which coins are heads and which are tails”—for example, “coins $1$ and $3$ are heads, and coins $2$ and $4$ are tails”—we will call in this chapter a microstate. On the other hand, coarse information that ignores the distinction between individual coins, namely “how many heads came up”—for example, “$2$ heads”—we call a macrostate. The arrangements of heads and tails, coin by coin (microstates), number $2^4=16$, but only $6$ of them correspond to the macrostate “$2$ heads”—we will verify this later in Example 48.1.

The very same distinction is meaningful for a collection of gas molecules as well. The state in which the position and velocity of every single gas molecule are specified is a microstate, and the set of a few quantities that can be measured in the laboratory with a thermometer or a pressure gauge, such as pressure, volume, and temperature, is a macrostate. Since there are as many as $N_{\mathrm A}\approx6.02\times10^{23}$ gas molecules per $1\ \mathrm{mol}$, we cannot write down the microstates one by one concretely. However, just as in the coin example, it is possible to count “how many microstates correspond to a given macrostate”—or, for continuous quantities, to measure it as a “volume”—and this is precisely the central task of statistical mechanics.

48.2 Phase Space in Classical Mechanics — Counting Microstates

The microstate of a coin was a choice among a finite number of options, “heads or tails,” but the microstate of a mechanical system (particles, gas molecules, and so on) is specified by continuously varying quantities—positions and momenta. So let us first recall the phase space that we have already introduced in mechanics (Section 45.5). For a system with $1$ degree of freedom (for example, the one-dimensional motion of a single particle connected to a wall by a spring), the state at time $t$ is represented by the pair $(x,p)$ of the position $x(t)$ and the momentum $p(t)=m\dot x(t)$—that is, by one point in the $x$-$p$ plane (the phase space). As the system evolves in time according to the equation of motion, this point moves in the plane, tracing out a single curve (a phase trajectory).

For example, the Hamiltonian of a one-dimensional harmonic oscillator of mass $m$ and spring constant $k$ (Chapter 7) is $H=p^2/(2m)+\frac12kx^2$ (Section 45.3). When the energy is kept at a constant value $E$ (the system is isolated), the set of points of constant energy in the phase space is the curve satisfying the equation $p^2/(2m)+\frac12kx^2=E$, that is, an ellipse like the one in Figure 48.1. The larger the energy $E$, the larger this ellipse becomes—this is the simplest example of an idea we will use repeatedly in this chapter: “a system with larger energy occupies a wider region of the phase space.”

In a system made of an enormous number of molecules, such as a gas, the dimension of the phase space becomes tremendously large. Suppose that $n\ \mathrm{mol}$ of a monatomic ideal gas is contained in a box of volume $V$. There are $N=nN_{\mathrm A}$ molecules ($N_{\mathrm A}$ is the Avogadro constant), and each molecule $i=1,\dots,N$ has a position $\bm{r}_i=(x_i,y_i,z_i)$ and a momentum $\bm{p}_i=(p_{ix},p_{iy},p_{iz})$ in three-dimensional space. We write the positions and momenta of all the molecules together as

(each is a set of $3N$ real numbers). One microstate of this gas corresponds to one point in the phase space that collects the positions and momenta of all $N$ molecules together—a space of dimension $2\times3N=6N$. For $1\ \mathrm{mol}$ we have $N\approx6.02\times10^{23}$, so $6N$ is a staggeringly huge number, of the order of $10^{24}$ (we check this concretely in Example 48.6). We cannot draw this many dimensions in a figure, but the rule itself—“six numbers, $x,y,z,p_x,p_y,p_z$, are needed for each molecule”—is the same whether there is one molecule or $N$ molecules: even when the dimension increases, the idea remains the same.

Let us summarize in a table how the dimension grows as we vary the number of particles and the dimension of space.

Each additional particle adds six dimensions—three for position and three for momentum. That is the whole rule.

If we write the infinitesimal volume element of this phase space as $\dd\{x\}=\dd x_1\dd y_1\dd z_1\cdots\dd z_N$ and $\dd\{p\}=\dd p_{1x}\dd p_{1y}\dd p_{1z}\cdots\dd p_{Nz}$ (each is the integrand part of a $3N$-fold integral, obtained by extending the idea of the multiple integral in University Mathematics, Section 7.1 directly to $3N$ variables), then we can naively regard the number of microstates contained in this volume element as

This is a naive way of counting, in which the “volume” of the phase space is directly proportional to the “number of microstates.” (However, $\dd\{x\}\,\dd\{p\}$ has the dimension of “length × momentum” raised to the $3N$th power, so it is not yet a true count (a dimensionless number). Dividing by $h^{3N}$ makes it a dimensionless number, as we see below.)

Let us apply this fact to the whole $6N$-dimensional phase space. “One state per area $h$” is a relation for one pair of a position and a momentum (one of the directions $x,y,z$ and the corresponding momentum component), and $h$ has the dimension of “length $\times$ momentum.” For $1$ molecule there is one such pair of a position component and a momentum component for each of the three directions $x,y,z$, so the volume of the cell occupied by one microstate in the phase space of one molecule (six-dimensional) is $h\times h\times h=h^3$. With $N$ molecules, the same argument is repeated $N$ times, so the volume of the cell occupied by one microstate in the whole $6N$-dimensional phase space is

(that is, $h^3$ multiplied $N$ times). In other words, the $6N$-dimensional phase space is divided into small cells (squares of a grid) of volume $h^{3N}$, and the innumerable points inside one cell (distinct points in classical mechanics) are all regarded, quantum mechanically, as one and the same microstate.

Therefore, the number of microstates contained in an infinitesimal volume element of the quantized phase space is obtained by dividing the naive volume of Eq. \eqref{eq:48-domega-classical} by the volume $h^{3N}$ of one cell in Eq. \eqref{eq:48-cell-volume}:

Let us check the dimensions: $[\dd x\,\dd p]=\mathrm{m}\cdot\mathrm{kg\,m/s}=\mathrm{kg\,m^2/s}=\mathrm{J\,s}$ (we used $\mathrm{J}=\mathrm{kg\,m^2/s^2}$), which agrees with the unit $\mathrm{J\,s}$ of $h$. Hence $\dd x\,\dd p/h$ is a dimensionless number and has meaning as a true count.

Now, this is still not enough. The gas molecules are all particles of the same kind (for example, the same helium atoms) and are quantum mechanically “indistinguishable”—if we exchange molecule number 1 and molecule number 2, the state is physically exactly the same. However, the notation $\{x\},\{p\}$ uses the molecule numbers explicitly, so two points that differ only by the exchange of molecules 1 and 2 are counted twice, as different microstates.

Finally, let us confirm once more, with an intuitive figure, the property that we have already seen in the ellipses of Figure 48.1—that a system with larger energy has more microstates—this time for a collection of gas molecules. We keep the number $N$ of molecules the same and compare cases that differ only in energy.

48.3 The Ergodic Hypothesis — Time Averages and Phase Averages

So far, we have prepared a way of counting the microstates of a many-particle system such as a gas as “a point in a $6N$-dimensional quantized phase space.” However, what we can measure in the laboratory with a thermometer or a pressure gauge is not an individual microstate but a macroscopic quantity such as pressure or temperature. A measurement always takes a finite time, whereas the microstate of the system is constantly changing, through molecular collisions and the like, on a time scale far shorter than the time the measurement takes—a system in thermal equilibrium appears, macroscopically, not to change as time passes, but microscopically the individual microstates must be replacing one another in succession at a speed too fast for the eye to follow. Therefore, it is natural to regard the value obtained in a measurement as really “an average over the enormous number of microstates that the system passed through during the time the measurement took.”

In the theory of statistical mechanics, however, instead of actually following the time evolution, we take the approach of considering “a hypothetical collection of all the conceivable microstates corresponding to the same macrostate (the same $E,V,N$, etc.)—an ensemble, which means a ‘group’” and calculating the average within that collection (the phase average, discussed below). The assertion that these two averages—the time average, which follows the actual time evolution, and the phase average, which is calculated over the collection of microstates that can exist at a given instant—are equal is the ergodic hypothesis, one of the most fundamental assumptions of statistical mechanics.

When a physical quantity $A$ can be written as $A(l(t))$, a function of the microstate $l(t)$ of the system (a point in phase space at time $t$), the time average is defined as the following limit, in which the measurement time $T$ is taken (in theory) to be infinitely long:

$A(l(t))$ is a value determined by where the system is in phase space at time $t$ (namely $l(t)$), and, as in the left panel of Figure 48.4, taking the time average means adding up the values of $A$ at the points along the trajectory (integrating over time). The factor $1/T$ on the right-hand side is needed because, whereas the integral $\int_0^T A(l(t))\dd t$ itself grows larger the longer $T$ is taken (in general, even when $A$ is bounded), an “average” must be a value per unit time—for example, if $A(t)$ is a constant $A_0$, then without $1/T$ the right-hand side would become $A_0T\to\infty$, which does not hold up as a definition of an “average.” Multiplying by $1/T$ gives the correct answer $\bar A_{\text{time}}=A_0$.

The phase average, on the other hand, is defined for the collection (ensemble) of all the microstates corresponding to the same macrostate. Let $P_i$ be the probability that each microstate $i$ is realized (the definition of probability in University Mathematics, Section 13.4; $\sum_iP_i=1$), and let $A_i$ be the value that the physical quantity $A$ takes in that microstate. The phase average is then defined as the expectation value (University Mathematics, Section 14.1):

(When the microstates are distributed over a continuous phase space, we replace the sum $\sum_i$ by an integral over phase space and the probability $P_i$ by the probability density $\rho(\{x\},\{p\})\dd\Omega$.) Here nothing has yet been assumed about how the probability $P_i$ (or the density $\rho$) is determined—the assumption that fixes it concretely for an isolated system is the principle of equal a priori probability, which we introduce in the next section, Section 48.4.

import numpy as np
rng = np.random.default_rng(1)          # fix the random seed (for reproducibility)
T = 6000
time_avg  = rng.integers(1, 7, T).mean()   # average of the faces from rolling one die 6000 times
phase_avg = rng.integers(1, 7, T).mean()   # average of the faces from rolling 6000 dice once
print(time_avg, phase_avg)

The ergodic hypothesis is an important assumption that underlies the whole theory of statistical mechanics (in particular, the idea of the microcanonical distribution introduced in the next section), but it is not a proven theorem—it is not necessarily the case that every isolated system is ergodic (that is, has its time average equal to its phase average). For example, consider two harmonic oscillators that exert no force on each other (angular frequencies $\omega_1\neq\omega_2$). Phase space is four-dimensional, $(x_1,p_1,x_2,p_2)$, and the surface on which the total energy $E=E_1+E_2$ is constant is a three-dimensional hypersurface (a generalization to higher dimensions of a two-dimensional surface in three-dimensional space; in general, for a system of $N$ particles the constant-energy surface is a ($6N-1$)-dimensional hypersurface). In this system, however, the energies $E_1$ and $E_2$ of the two oscillators are conserved separately, so the trajectory does not fill the entire three-dimensional constant-energy surface; it keeps moving only on a two-dimensional surface on which $E_1,E_2$ are both held constant—a torus (a surface like that of a doughnut)—and it never visits the surface as a whole. A system that, like this one, has many conserved quantities besides the energy (quantities that stay constant during the motion, as we saw in Sections 46.1–46.2) and whose motion is regular is called an integrable system. For an integrable system, the time average is an average over only the region that the trajectory actually traces out, and in general it does not coincide with the phase average taken over the whole constant-energy surface (the unperturbed two-body Kepler motion is a similar example). It is also known that in certain systems such as glass, only a tiny fraction of the microstates that the system could in principle take can be reached within the time scales observable in the laboratory (this is called “breaking of ergodicity”), and that it takes an astronomically long time to reach true thermal equilibrium. The ideal gas treated in this book and the models of solids treated in later chapters are all systems that may be regarded as sufficiently ergodic.

48.4 The Principle of Equal A Priori Probability and the Microcanonical Distribution

In the previous section we stated that computing the phase average requires the probability $P_i$ that each microstate $i$ is realized, but we had not yet assumed anything about how to determine its value. The assumption that determines how this probability is fixed for an isolated system (a system that does not exchange heat or matter with the outside at all, so that the energy $E$, volume $V$, and number of particles $N$ are all kept constant) is the principle of equal a priori probability (also called the principle of equal probability).

This principle is closely related to the ergodic hypothesis of the previous section. According to the ergodic hypothesis, an isolated system (if observed for a sufficiently long time) is thought to pass through all the allowed microstates one after another with equal “frequency of visits”—in that case, when we are asked which microstate the system is in at a given instant, there is no reason for any particular microstate to be favored, and it is natural to think that all microstates are found with equal probability. In other words, the ergodic hypothesis serves as an inductive basis that makes the principle of equal a priori probability plausible (rather than proving it). The converse view is also possible: if we assume the principle of equal a priori probability, the phase average can be computed explicitly, and we can check that its predictions (the equation of state, the specific heat, and so on) agree with experimental results measured over a long time, so the ergodic hypothesis too is gradually verified through such agreement with experiments.

Let us apply the principle of equal a priori probability to the case in which the energy of an isolated system lies exactly within a certain narrow range $[E,E+\Delta E]$ ($\Delta E$ is a small width determined by, for example, the measurement precision). First we organize the notation. We denote by $\Omega(E)$ the total number of microstates with energy at most $E$ (we carry out the concrete calculation for a gas in Section 48.6). Then the total number $W$ of microstates whose energy lies in the range $[E,E+\Delta E]$ (as in Figure 48.5, the number of quantized cells contained in the “energy shell” sandwiched between the constant-energy surface $E$ and the constant-energy surface $E+\Delta E$ in the $6N$-dimensional phase space) is given by

(Assuming that $\Delta E$ is small, we approximated the difference by “derivative $\times$ width”; for the idea of the derivative, see University Mathematics, Chapter 4.) The quantity $\dd\Omega/\dd E$ is the “number of states per unit width of energy,” and in Chapter 49 we write $W(E)\equiv\dd\Omega/\dd E$ and call it the thermodynamic weight. That is, $W$ in this chapter (the dimensionless number of states) corresponds to $W(E)\,\Delta E$ in Chapter 49. The principle of equal a priori probability means assigning the same probability $1/W$ to each of the $W$ microstates in this energy shell.

Eq. \eqref{eq:48-microcanonical} automatically satisfies the normalization condition (the condition, required by the definition of probability (University Mathematics, Section 13.4), that the sum of all the probabilities equals $1$):

This is almost obvious from the very definition of assigning the same value $1/W$ to all $W$ states.

For a classical gas, whose phase space is continuous, we use the probability density $\rho(\{x\},\{p\})$ in place of the probabilities $P_i$. The microcanonical distribution is the one in which $\rho$ is constant inside the energy shell and $0$ outside it, which can be written as

Here $\dd\Omega=\dd\{x\}\dd\{p\}/(N!\,h^{3N})$, and since the number of states in the shell is $W=\int_{\text{shell}}\dd\Omega$, the normalization $\int\rho\,\dd\Omega=\frac1W\cdot W=1$ holds.

For a system with discrete energies (a two-level system or an Einstein solid) we count the number of states $W$ of the shell directly, whereas for a gas with continuous energies we calculate $\Omega(E)$ and obtain $W$ from Eq. \eqref{eq:48-shell-count}. We carry this out in practice starting from the next section. The discussion of how to derive the entropy from $W$ is given in Chapter 49.

48.5 Examples of Counting States — The Two-Level Spin System and the Einstein Solid

From here on, we actually count, for concrete models, the number of states corresponding to the energy $E$. First, let us extend the two-level system treated in Example 48.9 of Section 48.4 to the general case of $N$ particles.

48.5.1 The two-level spin system

Suppose there are $N$ independent spins (or two-level atoms), each of which is in either the ground state (energy $0$) or the excited state (energy $\varepsilon$). Consider the macrostate in which $n$ of the $N$ spins ($0\le n\le N$) are in the excited state; the total energy is $E_n=n\varepsilon$. The number of ways to choose which $n$ spins are excited is exactly the same combinatorial problem as the counting of coins in Example 48.1 (University Mathematics, Section 13.5), and the following formula holds.

This $W(N,n)$ is the number of microstates whose energy is exactly $E_n=n\varepsilon$ (it is a different quantity from $\Omega(E)$, the “number of states with energy at most $E$,” so we write the symbol as $W$). For a system with discrete energies, this is precisely the number of states $W$ of the shell in Section 48.4 (because, if the width $\Delta E$ of the energy shell is taken smaller than the spacing between adjacent energy levels, only the states with energy exactly $E_n$ fall in the shell).

48.5.2 The Einstein solid

Next we consider the Einstein solid, a simple model of the thermal vibrations of atoms in a solid. We regard the atoms that make up the solid as quantum-mechanical harmonic oscillators that vibrate independently, with the same angular frequency $\omega$, in the three directions $x,y,z$. Here, for simplicity, we count the system as a collection of $N$ independent one-dimensional harmonic oscillators. For a real solid with $N_{\text{at}}$ atoms, counting the three directions, we simply read $N=3N_{\text{at}}$. According to quantum mechanics, the energy levels of a one-dimensional harmonic oscillator are known to take only the discrete values given, in terms of integers $n_i=0,1,2,\dots$, by

(here $\hbar=h/(2\pi)$ is the reduced Planck constant, read “h-bar.” The quantum-mechanical derivation of this fact is outside the scope of this textbook, and we use it as it stands, as a result supplied by quantum mechanics). The energy of the $N$ oscillators as a whole, apart from the total zero-point energy $\frac12N\hbar\omega$ (the fixed energy that remains even when every oscillator is in the ground state), is determined only by the sum of the extra quantum numbers $n_i$ that the individual oscillators have:

We call $q$ the “number of energy quanta (phonons, the units of the energy of lattice vibrations, which will be treated in detail in a later chapter)” distributed among the $N$ oscillators. The number of microstates corresponding to the macrostate whose total energy is exactly $E=\left(q+\frac N2\right)\hbar\omega$ becomes a counting problem: “in how many ways can $q$ identical quanta be distributed among $N$ distinguishable oscillators?”

This $W_{\text{Einstein}}(N,q)$ is also the number of states whose energy is exactly $E=(q+N/2)\hbar\omega$, and since the system is discrete it corresponds to the number of states $W$ of the shell. Also, when the formula is applied to a real solid, read $N\to3N_{\text{at}}$ if there are $N_{\text{at}}$ atoms.

48.6 Phase-Space Volume of an Ideal Gas — Calculation with an $n$-Dimensional Ball

Finally, let us return to the monatomic ideal gas itself, the protagonist of this chapter, and actually integrate the volume element of the quantized phase space of Definition 48.3, $\dd\Omega=\dd\{x\}\,\dd\{p\}/(N!\,h^{3N})$, to calculate the total number $\Omega(E)$ of microstates with energy at most $E$. What we need here is the formula for the “volume of an $n$-dimensional ball” that we prepared in Chapter 47.

Simply gazing at Eq. \eqref{eq:48-idealgas-omega} as it stands gives us little feel for it, because $N$ is an enormous number. So let us first confirm, in the simplest case ($N=1$, a “gas” of just one molecule), that the formula is correct and reduces to a familiar result.

Here, for the ideal gas, let us look concretely at the relation, previewed in Section 48.4, between the number of states $W$ of the energy shell and $\Omega(E)$. In Eq. \eqref{eq:48-idealgas-omega} only $E^{3N/2}$ depends on $E$, so by the derivative of the power function, $\dd(E^a)/\dd E=aE^{a-1}$, we have

Therefore the number of states in the shell, Eq. \eqref{eq:48-shell-count}, becomes

(where $\Delta E/E$ is a small number determined by the measurement precision). Taking the logarithm, we obtain

in which the second term is negligible compared with the first. For example, even for $N=10^{23}$ and $\Delta E/E=10^{-20}$, the second term is only $\ln(1.5\times10^{23}\times10^{-20})=\ln(1.5\times10^{3})\approx7.3$, whereas the first term $\ln\Omega(E)$ has a magnitude proportional to $N$ ($\sim10^{23}$). That is, however we choose $\Delta E$, $\ln W$ hardly differs from $\ln\Omega(E)$. This fact is used in Chapter 49.

For general $N$ (in particular, when $N\sim10^{23}$, of the order of Avogadro's number), it is not realistic to handle $N!$ and $\Gamma(3N/2+1)$ in Eq. \eqref{eq:48-idealgas-omega} directly as numbers; we need to simplify the logarithm using Stirling's formula learned in Section 47.3 (in logarithmic form, $\ln n!\approx n\ln n-n$, where $e$ is Napier's number). That calculation, and the argument that proceeds from it, via the thermodynamic weight $W(E)=\dd\Omega/\dd E$ (as stated in Section 48.4, the number of states $W$ of the shell in this chapter corresponds to $W(E)\,\Delta E$), to the derivation of Boltzmann's principle $S=k_{\mathrm B}\ln W$ (the definition of entropy in statistical mechanics), are carried out in the next chapter, Chapter 49. What we have confirmed in this chapter is that the task of “actually counting the microstates with energy at most $E$” can be carried out concretely with the same idea as for the two-level system and the Einstein solid in the previous sections—except that the positions give the volume of the box, $V^N$, and the momenta give the $n$-dimensional ball of Chapter 47.

48.7 Summary and Exercises

48.7.1 Summary

• A microstate is a state in which every element of the system is completely specified, whereas a macrostate is a coarse description specified only by a small number of measurable quantities; a single macrostate generally corresponds to many microstates (Definition 48.1).
• The state of a mechanical system is represented as a single point in phase space (the space spanned by the generalized coordinates and generalized momenta) (Definition 48.2, Chapter 45). The phase space of a gas consisting of $N$ identical particles has as many as $6N$ dimensions.
• According to quantum mechanics, there is one state per area $h$ of phase space (the uncertainty principle $\Delta x\,\Delta p\sim h$ serves as a guide), so the $6N$-dimensional phase space is divided into cells of volume $h^{3N}$. In order not to count exchanges of identical particles as distinct, we divide the number of states by $N!$ (Definition 48.3, Eq. \eqref{eq:48-domega-quantum}).
• The ergodic hypothesis (Law 48.1): In thermal equilibrium, the time average (Eq. \eqref{eq:48-time-average}) and the phase average (Eq. \eqref{eq:48-phase-average}) of a physical quantity are equal. There are, however, exceptions in which this hypothesis does not hold, such as integrable systems and systems in which ergodicity is broken.
• The principle of equal a priori probability (Law 48.2): In an isolated system ($E,V,N$ fixed), all allowed microstates are realized with equal probability. This is not a theorem that can be proved but a fundamental assumption that serves as the starting point of statistical mechanics. The ergodic hypothesis provides an inductive justification for it, and by Liouville's theorem a uniform distribution is preserved as time passes.
• The microcanonical distribution (Definition 48.4) is the distribution that assigns to each of the $W$ microstates in the energy shell $[E,E+\Delta E]$ ($W=\Omega(E+\Delta E)-\Omega(E)\approx(\dd\Omega/\dd E)\Delta E$, Eq. \eqref{eq:48-shell-count}) the probability $P_i=1/W$ (Eq. \eqref{eq:48-microcanonical}).
• For $N$ two-level systems, the number of states with energy exactly $n\varepsilon$ is the binomial coefficient $W(N,n)=\binom Nn$ (Formula 48.1), and the number of states of the Einstein solid ($q$ quanta distributed among $N$ oscillators) is given by $W_{\text{Einstein}}(N,q)=\dfrac{(q+N-1)!}{q!\,(N-1)!}$ (Formula 48.2) (in both cases the elements are distinguishable and fixed at lattice points, so we do not divide by $N!$). As $N$ becomes larger, an overwhelming majority of the microstates concentrates in particular macrostates.
• The phase-space volume $\Omega(E)$ of a monatomic ideal gas can be calculated as the product of the integral over positions ($V^N$) and the integral over momenta (the $n$-dimensional ball of Chapter 47, with $n=3N$ and radius $\sqrt{2mE}$), and it has the strong energy dependence $\Omega(E)\propto E^{3N/2}$ (Formula 48.3). The number of states in the shell is $W=\dfrac{3N}{2}\dfrac{\Delta E}{E}\Omega(E)$ (Eq. \eqref{eq:48-W-ideal}), and $\ln W\approx\ln\Omega(E)$. The thermodynamic weight $W(E)=\dd\Omega/\dd E$ and Boltzmann's principle $S=k_{\mathrm B}\ln W$, which come next, are treated in Chapter 49.

48.7.2 Exercises

48.7.3 References