Covariance

In statistics, covariance is a measure that tells us how two random variables vary together. If both variables tend to rise or fall at the same time, their covariance is positive. If one increases while the other decreases, it's negative.

Understanding covariance meaning helps us explore the strength and direction of a linear relationship between two variables.

1.0Covariance Meaning

Covariance describes the extent to which two random variables vary or move together. It’s not standardized, meaning its magnitude depends on the scale of the variables. The covariance of X and Y is given by:

$\mathrm{Cov}(X,Y)=\frac{1}{n}{\sum }_{i=1}^n\left(X_i-\bar{X}\right)\left(Y_i-\bar{Y}\right)$

• If Cov(X, Y) > 0: X and Y tend to move in the same direction.
• If Cov(X, Y) < 0: X and Y tend to move in opposite directions.
• If Cov(X, Y) = 0: No linear relationship.

2.0Correlation and Covariance: The Difference

Covariance and correlation both describe how two variables are related. However, they differ:

So, while correlation and covariance both indicate the direction of a relationship, only correlation indicates its strength.

3.0How to Find Covariance

Let’s break down the steps to find covariance between two variables:

1. Find the mean of both variables.
2. Subtract the mean from each value.
3. Multiply the deviations of corresponding values.
4. Take the average of these products.

4.0Solved Examples on Covariance 

Example 1: Let’s compute the covariance of X and Y with the following data:

Solution: 

Step 1: Means

$\bar{X}=4,\bar{Y}=8$

 Step 2: Deviations and Products

Step 3: Average

$\mathrm{Cov}(X,Y)=\frac{4+0+4}{3}=\frac{8}{3}\approx 2.67$

The positive covariance indicates that X and Y increase together.

Example 2: Given: X = [1, 2, 3] Y = [4, 5, 6]

Solution:

• Mean of X: $\bar{X}=\frac{1+2+3}{3}=2$
• Mean of Y: $\bar{Y}=\frac{4+5+6}{3}=5$

$\mathrm{Cov}(X,Y)=\frac{1}{3}\sum \left(X_i-\bar{X}\right)\left(Y_i-\bar{Y}\right)$

$\mathrm{Cov}(X,Y)=\frac{1+0+1}{3}=\frac{2}{3}\approx 0.67$

Example 3: Given: X = [1, 2, 3] Y = [6, 4, 2]

Solution:

• $\bar{X}=2,\bar{Y}=4$

 $\mathrm{Cov}(X,Y)=\frac{-2+0-2}{3}=\frac{-4}{3}\approx -1.33$

Negative covariance indicates inverse relationship.

Example 4: Given: X = [2, 4, 6, 8] Y = [5, 10, 15, 20]

Solution:

Step 1: $\bar{X}=\frac{20}{4}=5,\bar{Y}=\frac{50}{4}=12.5$

$\mathrm{Cov}(X,Y)=\frac{22.5+2.5+2.5+22.5}{4}=\frac{50}{4}=12.5$

Strong positive linear relationship.

Example 5: Given: X = [1, 2, 3, 4], Y = [7, 7, 7, 7]

Solution:

Since Y is constant, $\bar{Y}=7$ and all deviations from the mean will be zero.

$\mathrm{Cov}(X,Y)=\frac{1}{n}\sum \left(X_i-\bar{X}\right)\left(Y_i-\bar{Y}\right)=0$

Covariance = 0 when one variable is constant ⇒ No linear relationship.

Example 6 : Given: Math Scores (X) = [80, 85, 90], Science Scores (Y) = [78, 82, 88]

Solution:

Step 1: $\bar{X}=85,\bar{Y}=82.67$

$\mathrm{Cov}(X,Y)=\frac{23.35+0+26.65}{3}=\frac{50}{3}\approx 16.67$

Strong positive covariance: students doing well in one subject also do well in the other.

5.0Covariance in Statistics: Applications

Understanding covariance in statistics is essential in:

• Portfolio theory (to measure asset relationships)
• Regression analysis
• Principal component analysis (PCA)
• Machine learning algorithms

Covariance helps us grasp how two variables relate, even if it doesn’t tell us how strong that relation is.