Total Probability Theorem
The Total Probability Theorem is a fundamental principle in probability theory that allows us to compute the probability of an event based on conditional probabilities associated with a partition of the sample space. Often, direct computation of an event's probability is challenging, especially when the event depends on several different underlying scenarios. In such cases, the total probability theorem provides a structured approach by breaking down the event into simpler, mutually exclusive cases and summing the weighted probabilities.
This theorem is particularly useful in real-life situations involving uncertainty and multiple contributing factors—such as medical diagnosis, risk analysis, or decision-making processes. By understanding and applying this theorem, one can tackle complex probability problems more systematically and accurately.
1.0Total Probability Theorem Statement
The Total Probability Theorem provides a way to compute the probability of an event based on a partition of the sample space.
Statement:
Let B1,B2,...,Bn be a partition of the sample space S, such that all Bi are mutually exclusive and exhaustive events, and P(Bi)>0∀i. Then, for any event A,
P(A)=∑i=1nP(Bi).P(A∣Bi)
This is particularly useful when direct computation of P(A) is complex, but conditional probabilities P(A∣Bi) are known.
2.0Total Probability Theorem Formula
The general Total Probability Theorem formula can be rewritten as:
P(A)=P(B1)P(A∣B1)+P(B2)P(A∣B2)+...+P(Bn)P(A∣Bn)
This formula is widely applied in fields like data science, finance, medicine, and engineering, especially for modeling risk and decision-making under uncertainty.
3.0Total Probability Theorem Proof
Let’s walk through the Total Probability Theorem proof step-by-step:
- Since B1,B2,...,Bn is a partition of the sample space S, we know that A∩S=A∩(B1∪B2∪...∪Bn)
- Using distributive law:A=(A∩B1)∪(A∩B2)∪...∪(A∩Bn). Since the Bi’s are mutually exclusive, so are the A∩Bi′s
- Then, P(A)=∑i=1nP(A∩Bi)=∑i=1nP(Bi)⋅P(A∣Bi)
4.0Solved Examples on Total Probability Theorem
Example 1: A factory has 3 machines: M1,M2,andM3 producing 30%, 45%, and 25% of the total output respectively. The probability that these machines produce a defective item are 0.01, 0.03, and 0.02 respectively. What is the probability that a randomly selected item is defective?
Solution:
Let D be the event that an item is defective.
Using Total Probability Theorem:
P(D)=P(M1)P(D∣M1)+P(M2)P(D∣M2)+P(M3)P(D∣M3)
=(0.3)(0.01)+(0.45)(0.03)+(0.25)(0.02)= (0.3)(0.01) + (0.45)(0.03) + (0.25)(0.02)
= 0.003 + 0.0135 + 0.005 = 0.0215
So, the probability of selecting a defective item is 0.0215 or 2.15%.
Example 2: A company has three branches: B1,B2,andB3. The probability of a security breach occurring in each branch is given as follows:
- Probability of a breach in B1 is P(B1) = 0.5
- Probability of a breach in B2 is P(B2) = 0.3
- Probability of a breach in B3 is P(B3) = 0.2
If a breach happens in a branch, the probability that the breach is classified as severe is as follows:
- P(S∣B1)=0.4
- P(S∣B1)=0.6
- P(S∣B1)=0.3
Example 3: What is the overall probability that a breach, if it occurs, will be classified as severe?
Solution:
We need to find P(S), the total probability of a severe breach. Using the Total Probability Theorem, we have:
P(S)=P(B1)P(S∣B1)+P(B2)P(S∣B2)+P(B3)P(S∣B3)
Substitute the values:
P(S) = (0.5)(0.4) + (0.3)(0.6) + (0.2) (0.3)
P(S) = 0.2 + 0.18 + 0.06 = 0.44
Thus, the overall probability that a breach will be classified as severe is 0.44 or 44%.
Example 4: A data scientist is analyzing customer churn for a telecom company. The company has two types of customers:
- Type A Customers: They represent 60% of the customer base, and the probability of a type A customer churning is 0.1.
- Type B Customers: They represent 40% of the customer base, and the probability of a type B customer churning is 0.3.
What is the probability that a randomly selected customer has churned?
Solution:
We are given:
P(Churn|A) = 0.1 and P(Churn|B) = 0.3
Let’s find the probability that a randomly selected customer has churned, i.e., P(Churn).
Using the Total Probability Theorem:
P(Churn)=P(A)P(Churn∣A)+P(B)P(Churn∣B)
Substitute the values:
P(Churn) = (0.6)(0.1) + (0.4)(0.3)
P(Churn) = 0.06 + 0.12 = 0.18
Thus, the probability that a randomly selected customer has churned is 0.18 or 18%.
Example 5: In a manufacturing plant, there are three machines producing different types of components:
- Machine 1 produces 25% of the components, with a defect rate of 0.04.
- Machine 2 produces 50% of the components, with a defect rate of 0.02.
- Machine 3 produces 25% of the components, with a defect rate of 0.01.
What is the probability that a randomly chosen component is defective?
Solution:
Let D be the event that a component is defective.
We are given:
- P(M1)=0.25,P(M2)=0.50,P(M3)=0.25
- P(D∣M1)=0.04,P(D∣M2)=0.02,P(D∣M3)=0.01
Using the Total Probability Theorem, we can compute P(D)P(D):
P(D)=P(M1)P(D∣M1)+P(M2)P(D∣M2)+P(M3)P(D∣M3)
Substitute the values:
P(D) = (0.25)(0.04) + (0.50)(0.02) + (0.25)(0.01)
P(D) = 0.01 + 0.01 + 0.0025 = 0.0225
Thus, the probability that a randomly selected component is defective is 0.0225 or 2.25%.
5.0Practice Questions on Total Probability Theorem
- A company sources products from 2 suppliers. 60% come from Supplier A, 40% from B. The defect rate is 2% and 5% respectively. Find the overall defect rate.
- Given three weather forecasts, each with a different prediction and probability, calculate the probability of rain.
- If an email is classified by three filters with known false positive and detection rates, what's the probability that a spam email passes through?
6.0Total Probability Theorem and Bayes Theorem
The Total Probability Theorem and Bayes Theorem are deeply connected.
- The Total Probability Theorem helps compute P(A).
- Bayes Theorem uses that to compute P(Bi∣A), which is:
P(Bi∣A)=P(A)P(Bi).P(A∣Bi)
Thus, Bayes' Theorem relies on the total probability to "reverse" the condition.
7.0Applications of Total Probability Theorem
The applications of Total Probability Theorem are vast, including:
- Medical Diagnosis: Calculating the overall probability of a disease based on various risk factors.
- Spam Filtering: Determining the chance a message is spam based on conditional probabilities.
- Finance: Risk assessment and modeling in uncertain environments.
- Machine Learning: Probabilistic models like Naive Bayes classifiers.
- Quality Control: Estimating defect rates from multiple production lines