Gamma Distribution

In the world of probability and statistics, the Gamma Distribution is crucial in various fields, such as engineering, finance, and biology. Whether you're dealing with waiting times, lifetimes of products, or modelling certain processes, the Gamma Distribution can be a powerful tool. This blog will delve into the Gamma Distribution—its properties, its formula, and how to use it effectively. 

1.0What is the Gamma Distribution?

The Gamma Distribution is a continuous probability distribution that is characterised by two parameters. It's often used to model waiting times or lifetimes of events that occur at a constant rate. The distribution is commonly used in queuing theory, reliability analysis, and Bayesian statistics.

The Gamma Distribution has two key parameters:

• Shape Parameter (k or α): Determines the shape of the distribution.
• Rate Parameter (λ or β): Determines the rate of events.

When both parameters are integers, the Gamma Distribution becomes a Gamma Process, often used in the analysis of the lifetimes of systems or events.

2.0Gamma Distribution Probability Density Function (PDF)

The Probability Density Function (PDF) for the Gamma Distribution is the mathematical expression that defines the distribution’s behavior. It is given by the formula:

$f(x;k,\lambda )=\frac{{\lambda }^k{x}^{k-1}{e}^{-\lambda x}}{\Gamma (k)}$

Where:

• x ≥ 0 
• k is the shape parameter
• λ is the rate parameter
• $\Gamma (k)$ is the Gamma function evaluated at k, which generalizes the factorial function to non-integer values.

3.0CDF of Gamma Distribution

The Cumulative Distribution Function (CDF) of the Gamma Distribution is the integral of the PDF. It represents the probability that the random variable X takes a value less than or equal to a particular value x. The CDF of the Gamma Distribution is expressed as:

$F(x;k,\lambda )=\frac{\gamma (k,\lambda x)}{\Gamma (k)}$

Where:

• $\gamma (k,\lambda x)$ is the lower incomplete Gamma function.

This function is crucial when calculating the probability of a given range of values for the Gamma random variable.

4.0Expectation and Mean of Gamma Distribution

The Expectation (or Mean) of a Gamma Distribution provides the central tendency or the average value of the random variable. The mean of the Gamma Distribution is given by:

$E[X]=\frac{k}{\lambda }$

Where k is the shape parameter, and λ is the rate parameter. This formula shows that the mean depends directly on the shape and rate of the distribution.

5.0Gamma Distribution Formula

The Gamma Distribution formula combines both the PDF and CDF, and it serves as the backbone of the distribution's application.

As mentioned earlier, the Gamma Distribution PDF is:

$f(x;k,\lambda )=\frac{{\lambda }^k{x}^{k-1}{e}^{-\lambda x}}{\Gamma (k)}$

And the CDF is:

$F(x;k,\lambda )=\frac{\gamma (k,\lambda x)}{\Gamma (k)}$

Example of Gamma Distribution

Let’s look at an example of Gamma Distribution to better understand its application.

Suppose a server in a data center receives requests at an average rate of 2 requests per minute. If the waiting time between requests follows a Gamma Distribution with shape parameter k = 3 and rate parameter λ = 2, then we can calculate the probability of waiting less than 5 minutes for the next request. 

Using the Gamma Distribution CDF, we can determine this probability:

$F(5;3,2)=\frac{\gamma (3,2\times 5)}{\Gamma (3)}$

By evaluating this, we find the cumulative probability.

6.0Inverse Gamma Distribution

An interesting extension of the Gamma Distribution is the Inverse Gamma Distribution. While the Gamma Distribution models waiting times or lifetimes, the Inverse Gamma Distribution models the reciprocal of these quantities, and it’s used in fields like Bayesian statistics.

The Inverse Gamma Distribution has the following PDF:

$f(x;\alpha ,\beta )=\frac{{\beta }^{\alpha }}{\Gamma (\alpha )}{x}^{-(\alpha +1)}{e}^{-\frac{\beta }{x}}$

Where α is the shape parameter and β is the rate parameter. The inverse distribution is often used to model variance, and it can be helpful in situations where the distribution of a quantity is expected to be highly skewed.

7.0Gamma Distribution Parameters

The two primary parameters that define the Gamma Distribution are the shape parameter k (or α) and the rate parameter λ (or β). The Gamma Distribution Parameters determine the overall shape and spread of the distribution, which in turn influences its applications:

• The Shape Parameter controls how "peaked" or "spread out" the distribution is.
• The Rate Parameter controls the scale and how fast the events occur.

Key Properties of the Gamma Distribution

• Skewness: The Gamma Distribution is skewed, and its skewness decreases as the shape parameter k increases.
• Variance: The Variance of the Gamma Distribution is given by $\frac{k}{{\lambda }^2}$, which also depends on the shape and rate parameters.

8.0Solved Examples on Gamma Distribution

Example 1: Suppose the waiting time for the next customer at a service center follows a Gamma Distribution with a shape parameter k = 2 and a rate parameter λ = 3. Find the following:

1. The probability that the waiting time is less than 1 unit of time (i.e., P(X < 1)).
2. The expected waiting time (i.e., the mean of the distribution).
3. The variance of the waiting time.

Solution: 

Step 1: Define the Problem

The given information is:

• Shape parameter (k) = 2
• Rate parameter (λ) = 3

We need to calculate:

1. Probability P(X < 1), i.e., the CDF at x = 1.
2. The mean of the Gamma Distribution.
3. The variance of the Gamma Distribution.

Step 2: Calculate the Probability P(X < 1)

We use the Cumulative Distribution Function (CDF) of the Gamma Distribution to find the probability that the waiting time is less than 1. The CDF is given by:

$F(x;k,\lambda )=\frac{\gamma (k,\lambda x)}{\Gamma (k)}$

Where:

• $\gamma (k,\lambda x)$ is the lower incomplete Gamma function.
• Γ(k) is the Gamma function, which is the generalization of the factorial.

For our case:

• k = 2
• λ = 3
• x = 1

First, we calculate the CDF at x = 1:

$F(1;2,3)=\frac{\gamma (2,3\times 1)}{\Gamma (2)}=\frac{\gamma (2,3)}{1!}$

Since Γ(2) = 1, we just need to evaluate the incomplete Gamma function γ(2, 3).

The lower incomplete Gamma function γ(2,3) is equal to:

$\gamma (2,3)={\int }_0^3{t}^{2-1}{e}^{-t}dt=1-e^{-3}(1+3)$

Using this, we find that:

$\gamma (2,3)=1-e^{-3}(4)\approx 1-0.0498(4)\approx 1-0.1992\approx 0.8008$

Therefore, the CDF at x=1x = 1 is approximately:

F(1; 2, 3) = 0.8008 

Thus, the probability that the waiting time is less than 1 unit of time is:

P(X < 1) ≈ 0.8008  

Step 3: Calculate the Expected Waiting Time (Mean)

The mean (expectation) of the Gamma Distribution is given by the formula:

$E[X]=\frac{k}{\lambda }$

Substituting the values:

$E[X]=\frac{2}{3}\approx 0.6667$

Thus, the expected waiting time is approximately 0.6667 units of time.

Step 4: Calculate the Variance of the Waiting Time

The variance of the Gamma Distribution is given by:

$\mathrm{Var}(X)=\frac{k}{{\lambda }^2}$

Substituting the values:

$\text{Var}(X)=\frac{2}{3^2}=\frac{2}{9}\approx 0.2222$

Thus, the variance of the waiting time is approximately 0.2222 units of time squared.

Final Answers:

1. The probability that the waiting time is less than 1 unit of time is 0.8008.
2. The expected waiting time (mean) is 0.6667 units of time.
3. The variance of the waiting time is 0.2222 units of time squared.

Example 2: Show

${\int }_0^{\infty }\frac{1}{\Gamma (\alpha ){\beta }^{\alpha }}{x}^{\alpha -1}{e}^{-x/\beta }dx=1$

Solution. 

Let $t=\frac{x}{\beta }\Rightarrow x=\beta t,dx=\beta dt$ Then integrand becomes

$\frac{1}{\Gamma (\alpha ){\beta }^{\alpha }}(\beta t{)}^{\alpha -1}{e}^{-t}\beta$

$=\frac{1}{\Gamma (\alpha )}{t}^{\alpha -1}{e}^{-t}.$

Integrate t from 0 to $\infty$ gives $\frac{\Gamma (\alpha )}{\Gamma (\alpha )}=1$. QED.

Example 3: For $X\sim \Gamma (\alpha ,\beta )$ compute E[X] and $Var(X)$_.

Solution. Use known results or compute via moments:

$E[X]={\int }_0^{\infty }x{f}_X(x)dx=\frac{1}{\Gamma (\alpha ){\beta }^{\alpha }}{\int }_0^{\infty }{x}^{\alpha }{e}^{-x/\beta }dx.$

With substitution $t=x/\beta :\int x^{\alpha }{e}^{-x/\beta }dx={\beta }^{\alpha +1}\Gamma (\alpha +1)$. So

$E[X]=\frac{{\beta }^{\alpha +1}\Gamma (\alpha +1)}{\Gamma (\alpha ){\beta }^{\alpha }}=\beta \frac{\Gamma (\alpha +1)}{\Gamma (\alpha )}=\beta \alpha .$

Similarly $E[X^2]={\beta }^2\alpha (\alpha +1)$_. Thus

$Var(X)=E[X^2]-E[X{]}^2={\beta }^2\alpha (\alpha +1)-(\alpha \beta {)}^2=\alpha {\beta }^2.$

Example 4: $X\sim \Gamma (n,\beta )withintegern.ComputeP(X\le t)$ (closed form using sums).

Solution. For n integer, CDF is $F_X(t)=1-e^{-t/\beta }{\sum }_{k=0}^{n-1}\frac{(t/\beta {)}^k}{k!}.$

Derivation: use relation to sum of nn independent Exponential (1/\beta) waiting times (Poisson process); complementary probability equals probability of fewer than n arrivals by time t. So formula above is standard (Erlang CDF).

Example numeric: for $n=3,\beta =2,t=4,$

$F=1-e^{-2}\left(1+\frac{2}{1!}+\frac{2^2}{2!}\right)=1-e^{-2}(1+2+2)=1-5e^{-2}.$

Example 5: Let $Y_1,\dots ,Y_n$ be i.i.d. $Exp(\lambda )$ (rate $\lambda$ ). Show $S={\sum }_{i=1}^n{Y}_i\sim \Gamma (n,1/\lambda )$_.

Solution. PDF of $Exp(\lambda )$ is $\lambda e^{-\lambda y}$_. The convolution of n identical exponentials gives Erlang with shape nn and scale $1/\lambda$_. Equivalently, Laplace transform: ${\mathbf{L}}_{Y_i}(s)=\lambda /(\lambda +s)$. Product for sum is $(\lambda /(\lambda +s){)}^n$ which is Laplace of $\Gamma (n,1/\lambda )$. Hence S has PDF

$f_S(x)=\frac{{\lambda }^n{x}^{n-1}{e}^{-\lambda x}}{(n-1)!},x>0.$

Example 6: Find the mode (value of x maximizing the PDF) of $\Gamma (\alpha ,\beta )$ for $\alpha >1$_.

Solution 

Differentiate log-PDF:

$f(x)=-\alpha \ln \beta -\ln \Gamma (\alpha )+(\alpha -1)\ln x-x/\beta .$

Set derivative = 0:

$\frac{\alpha -1}{x}-\frac{1}{\beta }=0\Rightarrow x=(\alpha -1)\beta .$

For $\alpha >10<\alpha \le 1$ this gives the unique interior maximum. If $0<\alpha \le 1$ the density is decreasing so mode at x = 0.

Example 7: Show that if $X\sim {\chi }_{\nu }^2$ (chi-square with v degrees), then $X\sim \Gamma (\alpha =\nu /2,\beta =2).$

Solution 

Chi-square PDF:

$f(x)=\frac{1}{2^{\nu /2}\Gamma (\nu /2)}{x}^{\nu /2-1}{e}^{-x/2},x>0,$

which matches Gamma with $\alpha =\nu /2,\beta =2.$ So all chi-square properties follow from Gamma.

Example 8: Find MGF $M_X(t)=E[e^{tX}]$, for Gamma $(\alpha ,\beta )$ and use it to compute E[X].

Solution. For $t<1/\beta .$

$M_X(t)={\int }_0^{\infty }{e}^{tx}\frac{1}{\Gamma (\alpha ){\beta }^{\alpha }}{x}^{\alpha -1}{e}^{-x/\beta }dx=\frac{1}{\Gamma (\alpha ){\beta }^{\alpha }}{\int }_0^{\infty }{x}^{\alpha -1}{e}^{-x(1/\beta -t)}dx.$

Let $c=1/\beta -t>0$_. Then integral $=\Gamma (\alpha )c^{-\alpha }$ So

$M_X(t)={\left(\frac{1}{1-\beta t}\right)}^{\alpha },t<1/\beta .$

Differentiate at t = 0:

$E[X]=M_X^{\prime }(0)=\alpha \beta$, agreeing with previous result.

Example 9: Suppose $X_1,\dots ,X_{n}i.i.d.\Gamma (\alpha ,\beta )$ with known α and unknown β (scale). Find the MLE $\hat{\beta }.$

Solution. 

Likelihood:

$L(\beta )\propto {\beta }^{-n\alpha }\exp \left(-\frac{1}{\beta }{\sum }_i{x}_i\right).$

Log-likelihood:

$\ell (\beta )=-n\alpha \ln \beta -\frac{1}{\beta }{\sum }_i{X}_i+\text{const}.$

Differentiate w.r.t. $\beta$ and set 0:

$-\frac{n\alpha }{\beta }+\frac{1}{{\beta }^2}{\sum }_i{X}_i=0\Rightarrow \hat{\beta }=\frac{1}{n\alpha }{\sum }_{i=1}^n{X}_i=\frac{\bar{X}}{\alpha }.$

So MLE $\hat{\beta }=\bar{X}/\alpha .$ (Check the second derivative negative ⇒ maximum.)

Example 10: Let $X\sim \Gamma (4,2)$. Compute P(X > 6).

Solution. 

For integer shape n = 4 use Erlang complementary form:

$P(X>6)=e^{-6/2}{\sum }_{k=0}^3\frac{(6/2{)}^k}{k!}=e^{-3}{\sum }_{k=0}^3\frac{3^k}{k!}.$

Compute terms: $3^0/0!=1,3^1/1!=3,3^2/2!=9/2=4.5,3^3/6=27/6=\mathrm{4.5.}$ Sum = 1 + 3 + 4.5 +  4.5 = 13. So

$P(X>6)=13e^{-3}\approx 13\times 0.0497871\approx \mathrm{0.6472.}$

(You can leave answer as $13e^{-3}$ or approximate.)

Also Read: