Cumulative Distribution Function (CDF)

In the world of statistics and probability, the Cumulative Distribution Function (CDF) plays a pivotal role in understanding how random variables behave. Whether you're dealing with continuous or discrete variables, the CDF offers a comprehensive view of the probability distribution, helping us understand the likelihood of outcomes in a given scenario. This blog explores the CDF's definition, its relationship with probability density functions (PDF), and its application across various types of distributions, such as normal, uniform, and exponential.

1.0What is the Cumulative Distribution Function (CDF)?

The Cumulative Distribution Function (CDF) is a function that provides the probability that a random variable X will take a value less than or equal to a given number x. Simply put, it accumulates the probability of all outcomes up to a certain point. The CDF is defined for both discrete and continuous random variables.

In mathematical terms, for a continuous random variable X, the CDF is represented as:

$F(x)=P(X\le x)={\int }_{-\infty }^{x}f(t)dt$

Where:

• F(x) is the CDF of the random variable X,
• $P(X\le x)$ is the probability that the random variable X is less than or equal to x,
• f(t) is the probability density function (PDF) of the random variable X,
• The integral computes the total area under the PDF curve from −∞ to x.

The CDF increases monotonically from 0 to 1, meaning that as x increases, the probability of X ≤ x either stays the same or increases.

2.0The Relationship Between the CDF and Probability Density Function (PDF)

For continuous random variables, the Cumulative Distribution Function is directly related to the probability density function (PDF). The PDF describes the likelihood of a random variable taking a specific value, while the CDF accumulates this probability over a range of values.

The key relationship is:

• The CDF is the integral of the PDF.
• Conversely, the PDF is the derivative of the CDF.

Thus, the formula to transition from a PDF to a CDF is: $F(x)={\int }_{\infty }^{x}f(t)dt$

And the reverse, from CDF to PDF, is: $f(x)=\frac{d}{dx}F(x)$

3.0Cumulative Distribution Function for Different Types of Distributions

1. Cumulative Distribution Function of Normal Distribution

The Normal Distribution (or Gaussian Distribution) is one of the most commonly encountered distributions in probability and statistics. Its CDF is typically expressed in terms of the error function (erf), and its formula is complex, but it provides crucial insights for calculating probabilities associated with data that follows a bell-shaped curve.

For a normal distribution with mean μ and standard deviation σ, the CDF is:

$F(x)=\frac{1}{2}[1+erf(\frac{x-\mu }{\sigma \sqrt{2}})]$

This formula allows you to calculate the probability that a value from a normal distribution is less than or equal to x.

2. Cumulative Distribution Function of Exponential Distribution

For an Exponential Distribution, which is often used to model the time between events in a Poisson process, the CDF is simpler. For a random variable X that follows an exponential distribution with rate parameter λ, the CDF is:

$F(x)=1-e^{-kx}$

This CDF gives the probability that the random variable X is less than or equal to x, where λ is the rate parameter.

3. Cumulative Distribution Function of Uniform Distribution

The Uniform Distribution is a simple and intuitive distribution where all outcomes are equally likely within a certain interval. For a random variable X that follows a uniform distribution between a and b, the CDF is:

$F(x)=\frac{x-a}{b-a},fora\le x\le b$

The CDF increases linearly from 0 to 1 as x moves from a to b.

4.0Special Cases of CDF

1. Standard Normal Cumulative Distribution Function: The Standard Normal Distribution is a special case of the normal distribution where the mean μ = 0 and the standard deviation σ = 1. The CDF of the standard normal distribution is often denoted as Φ(x), and it is widely used in statistical analysis and hypothesis testing.
2. Inverse Cumulative Distribution Function (Quantile Function): The inverse CDF, also known as the quantile function, is the function that returns the value of xx given a probability. In other words, it helps find the threshold value corresponding to a particular cumulative probability. If F(x) is the CDF, the inverse function $F^{-1}(p)$ gives the value xx such that: $F^{-1}(p)whereF(x)=p$. This is particularly useful in simulations, for example, when generating random variables with a given distribution.
3. Joint Cumulative Distribution Function: In the case of multiple random variables, the joint CDF refers to the probability that two or more random variables simultaneously fall within certain ranges. For 2 random variables X and Y, the joint CDF is defined as: $F(x,y)=P(X\le x,Y\le y)$. It provides the joint probability that X and Y both lie below certain thresholds.

5.0Solved Example of Cumulative Distribution Function (CDF)

Example 1: Suppose we have a normal distribution with a mean μ = 50 and a standard deviation σ = 10. We want to calculate the probability that a randomly selected value X from this distribution is less than or equal to 60. In other words, we want to find: P(X ≤ 60)  

Solution: 

Step 1: Write the Formula for the CDF of a Normal Distribution

For a normal distribution, the CDF is given by the following formula:

$F(x)=\frac{1}{2}[1+erf(\frac{x-\mu }{\sigma \sqrt{2}})]$

Where:

• x = 60 
• μ = 50 
• σ = 10 
• erf is the error function, a special mathematical function used in the calculation of the normal CDF.

Step 2: Standardize the Value

To apply this formula, we first need to standardize the value of x=60x = 60 by converting it into a z-score:

$z=\frac{x-\mu }{\sigma }=\frac{60-50}{10}=1$

Now we have the standardized value z = 1.

Step 3: Use the Standard Normal CDF Formula

Now that we have the z-score, we can plug it into the standard normal distribution CDF formula: 

$F(60)=\frac{1}{2}[1+erf(\frac{1}{\sqrt{2}})]$

Using a calculator or a table for the error function, we find: $erf(1/\sqrt{2})\approx 0.6827$

Now substitute this value into the formula:

$$\begin{gathered} F(60)=\frac{1}{2}[1+0.6827] \\ F(60)\frac{1}{2}\times 1.6827=0.84135 \end{gathered}$$ 

Step 4: Interpret the Result

The value 0.841350.84135 represents the probability that a randomly selected value from the normal distribution will be less than or equal to 60. Therefore: $P(X\le 60)\approx 0.84137$

This means that there is an 84.14% chance that a randomly selected value from this normal distribution will not exceed than or equal to 60.

Example 2: Now, let’s consider a different example using a uniform distribution. Suppose we have a uniform distribution between 0 and 10, and we want to calculate the probability that a random variable X falls between 3 and 7. The probability we are interested in is:

Solution: 

Step 1: Write the Formula for the CDF of a Uniform Distribution

For a uniform distribution between a and b, the CDF is given by:

$F(x)=\frac{x-a}{b-a},fora\le x\le b$

Where:

• a = 0
• b = 10
• x = 3 and x = 7

Step 2: Calculate F(7) and F(3)

First, we calculate the CDF at x = 7: $F(7)=\frac{7-0}{10-0}=\frac{7}{10}=0.7$

Next, we calculate the CDF at x = 3: $F(3)=\frac{3-0}{10-0}=\frac{3}{10}=0.3$

Step 3: Calculate the Probability

Now, we can find the probability that X falls between 3 and 7 by subtracting the CDF values:

$P(3\le X\le 7)=F(7)-F(3)=0.7-0.3=0.4$

Step 4: Interpret the Result

The value 0.40.4 means that there is a 40% chance that a randomly selected value from this uniform distribution will lie between 3 and 7.

Example 3: Let X be the outcome of a fair six-sided die. Write the CDF $F_X(x)$ and compute

Solution

For a fair die, $P(X=k)=\frac{1}{6}$for k=1,2,....,6. The CDF:

$F_X(x)=P(X\le x)=\{\begin{array}{ll}0,& x<1,\\ \frac{x}{6},& 1\le x<6,\\ 1,& x\ge 6.\end{array}$

Compute $P(2<X\le 5)=P(X\in \{3,4,5\})=\frac{3}{6}=\frac{1}{2}$

Example 4: Suppose CDF of an integer-valued XX is $F_X(x)=\{\begin{array}{ll}0,& x<0,\\ 0.2,& 0\le x<1,\\ 0.5,& 1\le x<3,\\ 1,& x\ge 3.\end{array}$. Find P(X = 0), P(X = 1), P(X = 2).

Solution

For discrete distributions, mass at integer kk equals jump of CDF at k: 

$P(X=k)=F(k)-{\lim }_{x\to k^{-}}F(x)$

• $P(X=0)=F(0)-F(0^{-})=0.2-0=\mathrm{0.2.}$
• $P(X=1)=F(1)-F(1^{-})=0.5-0.2=\mathrm{0.3.}$
• $P(X=2)=F(2)-F(2^{-})=F(2)-0.5=0.5-0.5=0$

Finally $P(X\ge 3)=1-0.5=0.5$ corresponds to mass at 3 (or mass spread on ≥3 depending on support); since jump at 3 is $1-0.5=0.5,P(X=3)=\mathrm{0.5.}$

Example 5: Let X be uniform on [2,5]. Write its CDF and PDF, then compute

Solution

Uniform [2,5]: PDF $f_x(x)=\frac{1}{5-2}=\frac{1}{3}$ for $2\le x\le 5$ CDF:

$F_X(x)=P(X\le x)=\{\begin{array}{ll}0,& x<2,\\ \frac{x-2}{3},& 2\le x\le 5,\\ 1,& x>5.\end{array}$

For continuous variable, $P(3<X\le 4.5)=F(4.5)-F(3)=\frac{4.5-2}{3}-\frac{3-2}{3}=\frac{2.5}{3}-\frac{1}{3}=\frac{1.5}{3}=\frac{1}{2}.$

(Alternate: integrate PDF:  ${\int }_3^{4.5}\frac{1}{3}dx=\frac{1.5}{3}=\frac{1}{2}.$)

Example 6: CDF is given by $F_X(x)=\{\begin{array}{ll}0,& x<0,\\ 0.3,& 0\le x<1,\\ 0.3+\frac{1}{10}(x-1),& 1\le x<6,\\ 1,& x\ge 6.\end{array}$. Find

(a) P(X = 0), 

(b) density for 1 < x < 6, 

(c) P(0 < X < 2).

Solution

(a) Mass at 0 is jump at 0: P(X = 0) = 0.3 – 0 = 0.3.

(b) On 1 < x < 6, $F_x(x)=0.3+\frac{1}{10}(x-1)$ Differentiate to get PDF: $f_X(x)=\frac{d}{dx}{F}_X(x)=\frac{1}{10}$ for 1 < x < 6.

$P(0<X<2)=F(2^{-})-F(0)=(0.3+\frac{1}{10}(2-1))-0.3=0.1$

(c) Check total mass: $0.3+{\int }_1^6\frac{1}{10}dx=0.3+\frac{5}{10}=0.3+0.5=\mathrm{0.8.}$

The remaining 0.2 sits as jump at 6: indeed 

$F(6)-F(6^{-})=1-(0.3+\frac{5}{10})=0.2,$

$P(X=6)=0.2$

Example 7: Let X have CDF $F_X(x)=1-e^{-x}$ for (exponential, $\lambda =1$). Find the median mm (value with $P(X\le m)=0.5$).

Solution

Set F(m)=0.5: $1-e^{-m}=0.5⟹e^{-m}=0.5⟹m=\ln 2.$

So median $m=\ln 2\approx 0.6931$

Example 8: Let $X\sim \mathrm{Uniform}[0,1]$. Find CDF of $Y=X^2$ and then PDF.

Solution

For $F_Y(y)=P(Y\le y)=P(X^2\le y)=P(X\le \sqrt{y})=\sqrt{y}$. Since $x\ge 0$. Thus

$F_Y(y)=\{\begin{array}{ll}0,& y<0,\\ \sqrt{y},& 0\le y\le 1,\\ 1,& y>1.\end{array}$

Differentiate for PDF on (0,1): $f_Y(y)=\frac{d}{dy}{F}_Y(y)=\frac{1}{2\sqrt{y}},0<y<1.$ (Check integral: ${\int }_0^1\frac{1}{2\sqrt{y}}dy=[\sqrt{y}{]}_0^1=1.$)

Example 9: Let $X_1,X_2,...,X_n$ be i.i.d. with CDF $F_X(x)$. Let $M=max\{X_1,....,X_n\}$. Find CDF of M.

Solution

$F_M(x)=P(M\le x)=P(X_1\le x,\dots ,X_n\le x)={\prod }_{i=1}^{n}P(X_i\le x)=[F_X(x){]}^n.$

Example numeric: if $X_i$ uniform [0,1], for $F_M(x)=x^n,for0\le x\le 1$

Example 10: CDF: $F_X(x)=\{\begin{array}{ll}0,& x<0,\\ 0.4,& 0\le x<2,\\ 0.7,& 2\le x<3,\\ 1,& x\ge 3.\end{array}$

Compute $P(0<X\le 2),P(X=2),P(2<X\le 3)$

Solution

• $P(0<X\le 2)=F(2)-F(0)=0.7-0.4=0.3$ (Note F(2) denotes right-continuous value at 2.)
• P(X = 2) = jump at $=F(2)-{\lim }_{x\to 2^{-}}F(x)=0.7-0.4=\mathrm{0.3.}$ So all mass in (0,2] comes from the atom at 2 here.
• $P(2<X\le 3)=F(3)-F(2)=1-0.7=\mathrm{0.3.}$

Check totals: P(X = 0) = 0.4 (mass at 0), P(X = 2) = 0.3, P(2)=0.3 at 3? Actually mass at 3 is 1-0.7=0.3. Total 0.4 + 0.3 + 0.3 = 1.

Example 11: Let discrete nonnegative integer X have CDF $F(k)=1-(1/2{)}^k$ for integers $k\ge 1$. Find PMF and E[X].

Solution

First obtain PMF: for integer $k\ge 1$

$P(X=k)=F(k)-F(k-1).$

$$\begin{gathered} P(X=k)=\left(1-{\left(\frac{1}{2}\right)}^k\right)-\left(1-{\left(\frac{1}{2}\right)}^{k-1}\right) \\ ={\left(\frac{1}{2}\right)}^{k-1}-{\left(\frac{1}{2}\right)}^k \\ ={\left(\frac{1}{2}\right)}^k(2-1) \\ ={\left(\frac{1}{2}\right)}^k \end{gathered}$$

So $P(X=k)=Z^{-k}$ for . (Check sum: ${\sum }_{k=1}^{\infty }{2}^{-k}=1.$)

Compute expectation:

$E[X]={\sum }_{k=1}^{\infty }k{2}^{-k}$

Use known sum: ${\sum }_{k=1}^{\infty }k{r}^k=\frac{r}{(1-r{)}^2}$ for $|r|<1$. Here $r=\frac{1}{2}$ but note the exponent is k with coefficient $2^{-k}$ so  $E[X]={\sum }_{k=1}^{\infty }k{\left(\frac{1}{2}\right)}^k=\frac{\frac{1}{2}}{(1-\frac{1}{2}{)}^2}=\frac{\frac{1}{2}}{(1/2{)}^2}=\frac{1/2}{1/4}=2.$

Example 12: Suppose two random variables have CDFs $F_x(X)=1-e^{-x}$ and $F_Y(x)=1-e^{-2x}$ for $x\ge 0$. Which variable is stochastically larger (i.e., which has higher values on average)?

Solution

A variable U is stochastically larger than V if $F_U(x)\le F_V(x)$ for all x (so mass is shifted to the right). For  $x\ge 0,F_X(x)=1-e^{-x},F_Y(x)=1-e^{-2x}$

Compare: $e^{-x}\ge e^{-2x}$ for $x\ge 0$ (since $e^{-x}$ decreases slower), so $1-e^{-x}\le 1-e^{-2x}$. Thus $F_X(x)\le F_Y(x)$ ⇒ X is stochastically larger than Y. Intuitively, X has mean 1, Y has mean 1/2.

6.0Sample Questions on Cumulative Distribution Function

1. What is the difference between a Cumulative Distribution Function and a Probability Density Function?

Ans: The Probability Density Function (PDF) describes the likelihood of the random variable taking specific values. The Cumulative Distribution Function (CDF), however, accumulates the probabilities and gives the cumulative probability up to a point. The CDF is the integral (or sum for discrete variables) of the PDF: So, while the PDF shows density, the CDF shows cumulative probability.

2. How do you calculate the CDF for discrete random variables?

Ans: For discrete random variables, the CDF is found by summing the probabilities of all values less than or equal to a given value x. If X takes values with probabilities P(X = x_i), the CDF is: This gives the cumulative probability at or below each value.

3. How do you calculate the CDF for continuous random variables?

Ans: For continuous random variables, the CDF is the integral of the probability density function (PDF) from negative infinity to a given value xx. The CDF is computed as: where f(t) is the PDF of the random variable. This gives the total probability up to x.