An Overview of Basic Probability Theory: Part I
Propositions and Events
What sort of thing has a probability? We often talk about the probability of a particular
event occurring: “what are the chances that it will snow tomorrow?” But we can also
talk about whether or not a proposition has a probability: what is the probability that “it
will snow tomorrow” is true?
Don’t worry too much for now about distinguishing between propositions and events
for the purposes of our class – this is not going to play a huge role in what we do in here.
But this difference matters quite a bit as you get into more serious theoretical work in
probability theory. Different positions will take different notions as fundamental: some
will speak in terms of events (for good reaso n) and some will speak in terms of
propositions (also for good reason).
We will get into some of this material in the second half of the book – so be prepared for
that. But don’t worry too much about it now.
We are going to represent pr opositions or events with capital letters: A, B, C…etc. So, for
example, we will represent the proposition “it will snow tomorrow” by simply using S.
Frequently, we will want to talk about compound events: what is the probability that it
is both cold and snowy tomorrow? And so on. Here are the basic ways that we’re going
to do that.
A v B: A or B will happen (or both)
A & B: A and B will happen
~A: A will not happen
We will use the notation Pr ( ) to represent the proposition the probability of ___
Any instance of the logical notation that we’ve just seen can be put between the
parentheses. Let’s try an example. Take the proposition “I will receive an A or a B in this
class.” We will symbolize that proposition: A v B. Now, if we’re trying to represent “the
probability that I will receive an A or a B in this class” we would write: Pr (A v B).
Take another example. What is the probability that it will not be cold tomorrow? The
proposition is: “it will be cold tomorrow. ” We can symbolize that is C. We’re wondering
what the probability of that proposition being false is. So we need to negate the
proposition: ~C. And now we need to attach our probability operator to it: Pr (~C).
Two Conventions of Probability Theory .
1) All probabilities lie between 0 and 1. The probability of something impossible
occurring is 0 and the probability of something certain occurring is 1. It is quite
rare indeed to find something that has a probability of 1.
2) The probability of something certain occurring is 1.
Mutually Exclusive and Independent
Two events are mutually exclusive when they cannot both occur (at the same time). So,
for example, the proposition “It is overcast” and the proposition “It is sunny” are
mutually exclusive propositions.
Here are some more examples:
1) “The wheel will stop on red” and “The wheel will stop on black.” Pr (R & B) = 0
2) “I will receive an A” and “I will receive a B.” Pr(A & B) = 0
Two events are independent when the occurrence of one does not influence the
probability of the other occurring. So, for example, the propositions “I will get a phone
call today” and “I will eat breakfast today” are independent of one another: it is not
plausible to suppose that one occurrence exerts any influence on the other. To put this in
terms of some terminology we have already looked at in this course, we can say that
event A is independent of event B if A is neither positively relevant nor negatively
relevant to the occurrence of B.
Adding and Multiplying Probabilities
We’re now going to look at the two most basic rules of probability theory – rules that we
will formalize in Part III . So these two rules are expressed here in informal terms. For the
purposes of assignments and exams, you should appeal to the more formal
characterizations of them below.
Rule 1: The probabilities of mutually exclusive events add up
Take the propositions “I will get an A in this class” and “I will get a B in this class.” Let’s
symbolize them as A and B. Let’s suppose that we know that Pr(A) = .1 and Pr(B) = .2.
So now you know the probability that you will get an A in this class and you know the
probability that you will get a B in this class.
But now suppose you want to know what the probability that you will get at least a B in
this class is. What is the right way to symbolize this? Here are two options:
Pr (A v B)
Pretty clearly, the first possibility is impossible: you can’t get an A and a B in this course.
You get only one grade. So if you’re wondering what the probability is that y ou will get
at least a B, then you’re wondering what the probability is that you will get an A or a B:
Pr(A v B). And we said above that we can add them up.
Pr (A v B) = Pr(A) + Pr(B) .
Pr(A) = .1
Pr(B) = .2
Pr(A v B) = .3
This rule can be confusing: you are adding probabilities when you are calculating the
probability of one or the other of two mutually exclusive events occurring . You will be
tempted to apply the addition rule when figuring out Pr(A & B). But just because you’re
adding events (in a sense) doesn’t mean you are adding probabilities. So this rule does not
apply to Pr(A&B).
Rule 2: Multiply probabilities to find the probability of two independent events both
Pr (A & B) = Pr(A) x Pr(B)
Suppose I know that the probability that I w ill receive a phone call today is .8. And
suppose I know that the probability I will eat dinner today is .9. Let’s calculate the
probability of both happening.
If we add probabilities, we get: Pr(P) + Pr(D) = .8 + .9 = 1.7 . But that means the event is
more than certain! This is an impossible probability. Let this be a warning: even though
the notation uses “&,” you do not want to add probabilities in this case.
The correct way to calculate this is: Pr(P & D) = Pr(P) x Pr(D) = .8 x .9 = .72 . So there is a
roughly 3 in 4 chance that I will both eat dinner and receive a phone call today.
Beware of dependent events!
What is the probability that you will roll both an even and a prime when you roll one
Pr(E) = 3/6 = 1/2
Pr(P) = 3/6 = 1/2
Pr(E & P) = 1/2 x 1/2 = 1/4
So you have a .25 probability of rolling both an even and a prime with one die?
No, this is a mistake: E and P are not independent . They are dependent events. In fact,
there is exactly one even prime: 2. So Pr(E&P) = Pr(2) = 1/6.
When events are dependent, you may not multiply probabilities – not exactly, anyway.
We’ll consider this in more detail below.
With two dice, how can you roll a 6 or a 7?
To roll a 6:
(1, 5) (5, 1)
(2, 4) (4, 2)
To roll a 7:
(1, 6) (6, 1)
(2, 5) (5, 2)
(3, 4) (4, 3)
Since there are 36 possible outcomes on a roll of two dice, the probability of rolling a 6 is
5/36 and the probability of rolling a 7 is 6/36 or 1/6. So, with two dice, the probability
of rolling a 7 is higher than the p robability of rolling a 6. But most people don’t realize
Suppose you have two urns full of marbles. In the first urn, there are 3 red marbles and 1
green marble. In the second urn, you have 1 red marble and 3 green marbl es. So this can
be represented as follows:
Urn 1: 3 R, 1 G. Urn 2: 1 R, 3 G.
Flip a coin to select which urn to draw from and then draw two balls from that urn with
replacement. (This means that after you draw the first marble, you note its color and the n
put it back into the urn before you draw again.) What is the probability of getting two
reds on this setup?
We know that the probabilit y of drawing exactly one red ball is 1/2.
Pr (R 1 = 1/2 x 3/4 = 3/8. (The probability that you draw from Urn 1 is 1/ 2 – since it
depends upon a coin flip – and the probability of then drawing a red marble is 3/4, since
there are three red marbles and one green marble.)
Pr (R 2 = 1/2 x 1/4 = 1/8 (The probability that you draw from Urn 2 is ½ - since it depends
upon a coin flip – and the probability of then drawing a red marble is ¼, since there is
one red marble and three green marbles.)
Now, applying Rule 1 – which says that we must add the probabilities of mutually
exclusive events (and these are mutually exclusive, s ince you cannot draw from both
urns) – we get:
Pr (R 1 R ) 2 1/8 + 3/8 = 4/8 = 1/2
This is the probability of drawing exactly one red marble. So now we’re wondering
about the probability of drawing two reds. Since we know the probability of drawing a
single red is 1/2, shouldn’t we just say that the probability of drawing two reds is 1/2 x
1/2 = 1/4?
The answer is no. This is a tough question, but the probability is not 1/4. The reason is
that there are two different ways of drawing two reds in a row on this setup.
First way: Coin flip indicates urn 1; you draw two reds in a row from urn 1
Second way: Coin flip indicates urn 2; you draw two reds in a row from urn 2
Pr (First way): 1/2 x 3/4 x 3/4 = 9/32. Here, the probability of drawing from Urn 1 is 1 /2
and then the probability of drawing a red on the first draw is 3/4 . Since we are drawing
with replacement, the probability of drawing a red on the second draw is the same.
Pr (Second way): 1/2 x 1/4 x 1/4 = 1/32. The same reasoning applies.
Now, to figure out the probability of drawing two reds, we must account for both ways
of doing so. Since these are mutually exclusive (you cannot draw from both urns), we
must add the probabilities together:
Pr (F v S) = 9/32 + 1/32 = 10/32 = 5/16
An Overview of Basic Probability Theory: Part II
Categorical and Conditional Probability
Categorical probabilities state probabilities that contain, we might say, no “ifs, ands or
buts.” Here are some examples:
The probability it will rain tomorrow is .1
I have a 1/13 chance of drawing a spade
Conditional probabilities give the probability of something happening conditional upon
something else happening. Here are some examples:
The probability that I will be hungover given that I had 12 beers last night
The probability that you will walk to school given that it is snowing
Categorical Probability: Pr ( )
Conditional Probability: Pr ( / )
The probability that it will rain tomorrow: Pr (R)
The probability that I will be hungover given that I had twel ve beers last night: Pr (H/T)
So far, we have looked only at categorical probabilities. Now, however, we will look at
some additional rules that concern conditional probabilities.
Bingo: bingo players calculate conditional probabilities all the time. As they fill up a line