# ECSE 305 Lecture Notes - Mutual Exclusivity, Symmetric Relation, Socalled

50 views34 pages
School
McGill University
Department
Electrical Engineering
Course
ECSE 305
Professor
Page:
of 34
Chapter 4
Conditional Probability and
Independence
In the context of a random experiment, knowing that a certain event
Bhas occured may completely change the likelihood we associate to
another event A.
For example, suppose we roll two fair dice:
- The sample space is S={(x, y) : x, y ∈ {1,2, ..., 6}}.
- Let Adenote the event that the sum x+y= 11, i.e., A={(5,6),(6,5)},
and let Bdenote the event that x= 1, i.e. B={(1,1),(1,2), ..., (1,6)}.
- Assuming that the dice are fair, the probability of Ais P(A)=2/36.
- Now, suppose we know that Boccurred, i.e. the ﬁrst die shows 1.
- Under this “condition”, event Ais impossible, and its likelihood or
probability becomes 0.
83
84
Conditional probabilities provide quantitative measures of likelihood
(probability) under the assumption that certain events have occurred,
or equivalently, that certain a priori knowledge is available.
In certain situations, knowing that Bhas occurred does not change the
likelihood of A; this idea is formalized via the mathematical concept of
independence.
The concepts of conditional probability and independence play a ma-
jor role in the design and analysis of modern information processing
compression algorithms, etc.
c
2003 Benoˆıt Champagne Compiled February 2, 2012
4.1 Conditional probability 85
4.1 Conditional probability
Relative frequency interpretation:
Consider a random experiment. Let Aand Bdenote two events of
interest with P(B)>0.
Suppose this experiment is repeated a large number of times, say n.
According to the relative frequency interpretation of probability, we have
P(A)η(A)
n, P (B)η(B)
n, P (AB)η(AB)
n(4.1)
where η(A), η(B) and η(AB) denote the number of occurrences of
events A,Band ABwithin the nrepetitions.
Provided η(B) is large, the probability of A, knowing or given that B
has occurred, might be evaluated as the ratio
P(Agiven B) = η(AB)
η(B),(4.2)
also known as a conditional relative frequency.
Using this approach, we have
P(Agiven B) = η(AB)
η(B)=η(AB)/n
η(B)/n P(AB)
P(B)(4.3)
This and other considerations lead to the following deﬁnition.
c
2003 Benoˆıt Champagne Compiled February 2, 2012