3.3 The Cumulative Distribution Function

Overview¶

Although a discrete distribution is characterized by its PMF and a continuous distribution is characterized by its PDF, every distribution has a common characterization through its (cumulative) distribution function (CDF). The inverse of the CDF is called the quantile function, and it is useful for indicating where the probability is located in a distribution.

3.3.1 Definition and Basic Properties¶

It should be emphasized that the cumulative distribution function is defined as above for every random variable $X$, regardless of whether the distribution of $X$ is discrete, continuous, or mixed. For the continuous random variable in Example 3.3.1, the CDF was calculated in eq-3-3-1. Here is a discrete example:

We shall soon see (Theorem 3.3.2) that the CDF allows calculation of all interval probabilities; hence, it characterizes the distribution of a random variable. It follows from eq-3-3-2 that the CDF of each random variable $X$ is a function $F$ defined on the real line. The value of $F$ at every point $x$ must be a number $F(x)$ in the interval $[0, 1]$ because $F(x)$ is the probability of the event $\{X \leq x\}$. Furthermore, it follows from eq-3-3-2 that the CDF of every random variable $X$ must have the following three properties.

An example of a CDF is sketched in Figure 3.6. It is shown in that figure that $0 \leq F(x) \leq 1$ over the entire real line. Also, $F(x)$ is always nondecreasing as $x$ increases, although $F(x)$ is constant over the interval $x_1 \leq x \leq x_2$ and for $x \geq x_4$.

The limiting values specified in Property 3.3.2 are indicated in Figure 3.6. In this figure, the value of $F(x)$ actually becomes 1 at $x = x_4$ and then remains 1 for $x > x_4$. Hence, it may be concluded that $\Pr(X \leq x_4) = 1$ and $\Pr(X > x_4) = 0$. On the other hand, according to the sketch in Figure 3.6, the value of $F(x)$ approaches 0 as $x \rightarrow −\infty$, but does not actually become 0 at any finite point $x$. Therefore, for every finite value of $x$, no matter how small, $\Pr(X \leq x) > 0$.

A CDF need not be continuous. In fact, the value of $F(x)$ may jump at any finite or countable number of points. In Figure 3.6, for instance, such jumps or points of discontinuity occur where $x = x_1$ and $x = x_3$. For each fixed value $x$, we shall let $F(x^-)$ denote the limit of the values of $F(y)$ as $y$ approaches $x$ from the left, that is, as $y$ approaches $x$ through values smaller than $x$. In symbols,

Similarly, we shall define $F(x^+)$ as the limit of the values of $F(y)$ as $y$ approaches $x$ from the right. Thus,

If the CDF is continuous at a given point $x$, then $F(x^{-}) = F(x^+) = F(x)$ at that point.

It follows from Property 3.3.3 that at every point $x$ at which a jump occurs,

In Figure 3.6 this property is illustrated by the fact that, at the points of discontinuity $x = x_1$ and $x = x_3$, the value of $F(x_1)$ is taken as $z_1$ and the value of $F(x_3)$ is taken as $z_3$.

Determining Probabilities from the Distribution Function¶

The type of reasoning used in Example 3.3.3 can be extended to find the probability that an arbitrary random variable $X$ will lie in any specified interval of the real line from the CDF. We shall derive this probability for four different types of intervals.

For example, if the CDF of $X$ is as sketched in Figure 3.6, then it follows from Theorems Theorem 3.3.1 and Theorem 3.3.2 that $\Pr(X > x_2) = 1 − z_1$ and $\Pr(x_2 < X \leq x_3) = z_3 − z_1$. Also, since $F(x)$ is constant over the interval $x_1 \leq x \leq x_2$, then $\Pr(x_1 < X \leq x_2) = 0$.

It is important to distinguish carefully between the strict inequalities and the weak inequalities that appear in all of the preceding relations and also in the next theorem. If there is a jump in $F(x)$ at a given value $x$, then the values of $\Pr(X \leq x)$ and $\Pr(X < x)$ will be different.

For example, for the CDF sketched in Figure 3.6, $\Pr(X < x_3) = z_2$ and $\Pr(X < x_4) = 1$.

Finally, we shall show that for every value $x$, $\Pr(X = x)$ is equal to the amount of the jump that occurs in $F$ at the point $x$. If $F$ is continuous at the point $x$, that is, if there is no jump in $F$ at $x$, then $\Pr(X = x) = 0$.

In Figure 3.6, for example, $\Pr(X = x_1) = z_1 − z_0$, $\Pr(X = x_3) = z_3 − z_2$, and the probability of every other individual value of $X$ is 0.

The CDF of a Discrete Distribution¶

From the definition and properties of a CDF $F(x)$, it follows that if $a < b$ and if $\Pr(a < X < b) = 0$, then $F(x)$ will be constant and horizontal over the interval $a < x < b$. Furthermore, as we have just seen, at every point $x$ such that $\Pr(X = x) > 0$, the CDF will jump by the amount $\Pr(X = x)$.

Suppose that $X$ has a discrete distribution with the pmf $f(x)$. Together, the properties of a CDF imply that $F(x)$ must have the following form: $F(x)$ will have a jump of magnitude $f(x_i)$ at each possible value $x_i$ of $X$, and $F(x)$ will be constant between every pair of successive jumps. The distribution of a discrete random variable $X$ can be represented equally well by either the pmf or the CDF of $X$.

The CDF of a Continuous Distribution¶

Thus, the CDF of a continuous random variable $X$ can be obtained from the pdf and vice versa. eq-3-3-7 is how we found the CDF in Example 3.3.1. Notice that the derivative of the $F$ in Example 3.3.1 is

and $F'$ does not exist at $x = 0$. This verifies eq-3-3-8 for Example 3.3.1. Here, we have used the popular shorthand notation $F'(x)$ for the derivative of $F$ at the point $x$.

The Quantile Function¶

The value $x_0$ that we seek in Example 3.3.5 is called the 0.5 quantile of $X$ or the 50th percentile of $X$ because 50% of the distribution of $X$ is at or below $x_0$.

The notation $F^{-1}(p)$ in Definition 3.3.2 deserves some justification. Suppose first that the CDF $F$ of $X$ is continuous and one-to-one over the whole set of possible values of $X$. Then the inverse $F^{-1}$ of $F$ exists, and for each $0 < p < 1$, there is one and only one $x$ such that $F(x) = p$. That $x$ is $F^{−1}(p)$. Definition 3.3.2 extends the concept of inverse function to nondecreasing functions (such as CDFs) that may be neither one-to-one nor continuous.

Quantiles of Continuous Distributions: When the CDF of a random variable $X$ is continuous and one-to-one over the whole set of possible values of $X$, the inverse $F^{-1}$ of $F$ exists and equals the quantile function of $X$.

Note: Quantiles, Like CDFs, Depend on the Distribution Only: Any two random variables with the same distribution have the same quantile function. When we refer to a quantile of $X$, we mean a quantile of the distribution of $X$.

Quantiles of Discrete Distributions: It is convenient to be able to calculate quantiles for discrete distributions as well. The quantile function of Definition 3.3.2 exists for all distributions whether discrete, continuous, or otherwise. For example, in Figure 3.6, let $z_0 \leq p \leq z_1$. Then the smallest $x$ such that $F(x) \geq p$ is $x_1$. For every value of $x < x_1$, we have $F(x) < z_0 \leq p$ and $F(x_1) = z_1$. Notice that $F(x) = z_1$ for all $x$ between $x_1$ and $x_2$, but since $x_1$ is the smallest of all those numbers, $x_1$ is the $p$ quantile. Because distribution functions are continuous from the right, the smallest $x$ such that $F(x) \geq p$ exists for all $0 < p < 1$. For $p = 1$, there is no guarantee that such an $x$ will exist. For example, in Figure 3.6, $F(x_4) = 1$, but in Example 3.3.1, $F(x) < 1$ for all $x$. For $p = 0$, there is never a smallest $x$ such that $F(x) = 0$ because $\lim_{x \rightarrow −\infty}F(x) = 0$. That is, if $F(x_0) = 0$, then $F(x) = 0$ for all $x < x_0$. For these reasons, we never talk about the 0 or 1 quantiles.

: Table 3.1: Quantile function for Example 3.3.9. {#tbl-3-1}

Certain quantiles have special names.

Note: The Median Is Special¶

The median of a distribution is one of several special features that people like to use when sumarizing the distribution of a random variable. We shall discuss summaries of distributions in more detail in Chapter 4: Expectation. Because the median is such a popular summary, we need to note that there are several different but similar “definitions” of median. Recall that the $1/2$ quantile is the smallest number $x$ such that $F(x) \geq 1/2$. For some distributions, usually discrete distributions, there will be an interval of numbers $[x_1, x_2)$ such that for all $x \in [x_1, x_2)$, $F(x) = 1/2$. In such cases, it is common to refer to all such $x$ (including $x_2$) as medians of the distribution. (See Definition 4.5.1.) Another popular convention is to call $(x_1 + x_2)/2$ the median. This last is probably the most common convention. The readers should be aware that, whenever they encounter a median, it might be any one of the things that we just discussed. Fortunately, they all mean nearly the same thing, namely that the number divides the distribution in half as closely as is possible.

One advantage to describing a distribution by the quantile function rather than by the CDF is that quantile functions are easier to display in tabular form for multiple distributions. The reason is that the domain of the quantile function is always the interval $(0, 1)$ no matter what the possible values of $X$ are. Quantiles are also useful for summarizing distributions in terms of where the probability is. For example, if one wishes to say where the middle half of a distribution is, one can say that it lies between the 0.25 quantile and the 0.75 quantile. In sec-8-5, we shall see how to use quantiles to help provide estimates of unknown quantities after observing data.

In Div, you can show how to recover the CDF from the quantile function. Hence, the quantile function is an alternative way to characterize a distribution.

Summary¶

The CDF $F$ of a random variable $X$ is $F(x) = \Pr(X \leq x)$ for all real $x$. This function is continuous from the right. If we let $F(x^-)$ equal the limit of $F(y)$ as $y$ approaches $x$ from below, then $F(x) − F(x^-) = Pr(X = x)$. A continuous distribution has a continuous CDF and $F'(x) = f(x)$, the pdf of the distribution, for all $x$ at which $F$ is differentiable. A discrete distribution has a CDF that is constant between the possible values and jumps by $f(x)$ at each possible value $x$. The quantile function $F^{-1}(p)$ is equal to the smallest $x$ such that $F(x) \geq p$ for $0 < p < 1$.

Exercises¶