# ECSE 305 Lecture Notes - Mutual Exclusivity, Symmetric Relation, Socalled

50 views34 pages

Published on 18 Apr 2013

Department

Electrical Engineering

Course

ECSE 305

Professor

Chapter 4

Conditional Probability and

Independence

•In the context of a random experiment, knowing that a certain event

Bhas occured may completely change the likelihood we associate to

another event A.

•For example, suppose we roll two fair dice:

- The sample space is S={(x, y) : x, y ∈ {1,2, ..., 6}}.

- Let Adenote the event that the sum x+y= 11, i.e., A={(5,6),(6,5)},

and let Bdenote the event that x= 1, i.e. B={(1,1),(1,2), ..., (1,6)}.

- Assuming that the dice are fair, the probability of Ais P(A)=2/36.

- Now, suppose we know that Boccurred, i.e. the ﬁrst die shows 1.

- Under this “condition”, event Ais impossible, and its likelihood or

probability becomes 0.

83

84

•Conditional probabilities provide quantitative measures of likelihood

(probability) under the assumption that certain events have occurred,

or equivalently, that certain a priori knowledge is available.

•In certain situations, knowing that Bhas occurred does not change the

likelihood of A; this idea is formalized via the mathematical concept of

independence.

•The concepts of conditional probability and independence play a ma-

jor role in the design and analysis of modern information processing

systems, such as digital radio receivers, speech recognition systems, ﬁle

compression algorithms, etc.

c

2003 Benoˆıt Champagne Compiled February 2, 2012

4.1 Conditional probability 85

4.1 Conditional probability

Relative frequency interpretation:

•Consider a random experiment. Let Aand Bdenote two events of

interest with P(B)>0.

•Suppose this experiment is repeated a large number of times, say n.

According to the relative frequency interpretation of probability, we have

P(A)≈η(A)

n, P (B)≈η(B)

n, P (A∩B)≈η(A∩B)

n(4.1)

where η(A), η(B) and η(A∩B) denote the number of occurrences of

events A,Band A∩Bwithin the nrepetitions.

•Provided η(B) is large, the probability of A, knowing or given that B

has occurred, might be evaluated as the ratio

P(Agiven B) = η(A∩B)

η(B),(4.2)

also known as a conditional relative frequency.

•Using this approach, we have

P(Agiven B) = η(A∩B)

η(B)=η(A∩B)/n

η(B)/n ≈P(A∩B)

P(B)(4.3)

•This and other considerations lead to the following deﬁnition.

c

2003 Benoˆıt Champagne Compiled February 2, 2012