INTRO TO PSYCHOLOGY
Set #2 (Lectures 4, 5, 6)
1 Lecture 4
September 10 2009
Design and sampling:
We were talking about experimental design and sampling. The reason is that the size of your
sample, that is the number of people or animals you have to test, is related to how much
variability you expect to find in your experiment. This is a key concept. Whether were doing
any of the 3 kinds of studies; an observational study, a correlational study, or a true
experiment a controlled study, the number of people we need to study is going to relate to how
much variability there is.
The idea is that the thing youre interested in varies very little or not at all, you only need to
sample one or two individuals. But if the thing varies a lot youre going to need to sample a lot.
And this should be intuitive, in order for that variability to show itself.
Take a study, suppose youre interested in knowing what kinds of TV shows people watch.
Back in the old days, when they were only 3 or 4 channels, it was never the case that everybody
was watching the same channel. People would be spread around the 3 or 4 channels. And
maybe you could give a survey and among those whore watching TV, you could ask what
station are you watching? And maybe after talking to 20 or 50 or a hundred people, you have a
pretty good idea. Now suppose in that particular case, 3 channels get most of the viewing, and
only one out of a hundred people watches that 4th channel. You can see intuitively that if you
ask only 10 people, its unlikely that youll find somebody watching that 4th channel. Now,
with the 50 or 100 or 500 channels that people have, you can see you have to ask more than 5
people what they are watching in order to get an accurate representation across the different
channels. If I tell you there are 300 channels, it should be your intuition that youre not going to
get a satisfactory survey if you speak to less than 300 people.
There are well-developed methods in statistics to determine how many people youd have to
ask. The point is the conceptual idea that the size of your sample has to do with the variability.
Suppose I want to figure out: do all humans have lungs or a beating heart? I dont need to look
at every single being on the planet in order to falsify the hypothesis because we have an
understanding of the way things work. We have a scientific theory, we have some principles
about the way the world works and one of them is that human beings have to have lungs and
they have to have a beating heart.
Sometimes we think we have a theory about how things work, and then an observation turns
out and it suggests that we were wrong, and we follow it and it turns out that we were wrong.
This is in fact the principal way which science develops. Billions of years the people assumed
that the earth is flat, and then some observation came in suggesting this might not be the case,
and we followed them out and now we know that the world is not flat.
2 Falsifiable hypothesis:
The way that we talk about these things is important. A real scientist rarely says that he or she
has proven something. What we talk about is hypothesis and evidence; we dont talk about
proof. Usually we try to formulate a hypothesis that is falsifiable. An example of a good
hypothesis is: No swans are white. We go out and look for swans. We find a white one, and
we falsify the hypothesis. Now you have the hypothesis No swans are black. You go out and
look for swans, you dont find any. You cant now say that weve proven that there are no
black swans. The failure to falsify your hypothesis does not prove your hypothesis. All you can
say is we looked at all the reasonable places, we didnt find a black swan, there might be one
out there, we dont know for sure.
Once youve falsified the hypothesis, you can reject a hypothesis based on your observations.
You cannot prove a negative in science you can gather evidence. For example, you might
have two groups of students and you might ask them to study in different ways. Maybe you
have one group to study with coffee and another group to study without coffee and you might
want to carefully control the material theyre studying and the time of the day. If youre
looking for a difference between coffee drinkers and no coffee drinkers, and you dont find it,
you really cant say anything for certain. You cant say coffee doesnt help. All you can say is
that you didnt find evidence that coffee helps in this particular case. The coffee versus no
coffee study, if you hypothesize that there is no difference between the groups and you find a
difference, then youre on to something because you falsified the hypothesis.
The problem is: if I formulate the question the other way, how much of a difference is a big
enough difference? If I say were looking for a difference and then your experiment fails to find
one, there are a hundred reasons why your experiment failed to find it. So the failure to find
something is not as strong as actually finding something. Thats why usually when scientists
formulate a hypothesis, it is formulated in a way that makes it easier to reject. We usually say
that were hypothesizing no difference, so when we find a difference, we falsified it.
Who is the sample?
One of the important things here is the kind of people who are going to be your samples, or in
the case of black swans, where do you look? Most of what we know about psychology, we
know from North American college students. So we dont know a whole lot about how people
with different backgrounds and ages and cultures respond to number of basic things.
There are variables that we can manipulate in experiments. If you were doing a study on
peoples perception of light, you want to establish the smallest amount that you can change the
luminance of the room before somebody noticed. Although there is going to be variability, this
is something physiological and we basically are given the same physiological vision. We
3 believe that the eye function is in similar ways in most people. We probably dont need to take
into account where you were born or what youre majoring.
When it comes to choosing how youre going to conduct your study, and who will be the
participants, considering: first, is it something we believe has a low variability in the
population? What are the things that we have to take into account?
Then, if you want to be able to capture rare events, you have to ask enough people. So if you
want to know whether students at McGill are Christian or not, you might ask a few people. But
if you want to figure out all the possible religions, what percentage of people subscribes or
identify with them? And you want to include religions that statistically dont show up very
often, because it is just statistically less likely, that should be intuitive.
Lets suppose you want to know how much the college students drink on average, or what days
of the week they drink. You might have a reason to believe that McGill is a typical college and
youll only survey students here. What would be wrong with that? Drinking age varies from
place to place. Its colder here in the winter so people are drinking to warm up. The level at
McGill is more different; they work so hard they need to relax. McGill has a reputation for
being a party school. Now you need to find representative samples from different populations.
If McGill is a party school and Concordia is not, then you want to sample both. If McGill has
twice as many students as Concordia, you probably sample in proportion to their populations. If
you want to take this notion of proportions, there might be a small college, 300 hundred
students. If you sample 1000 students here at McGill, youre only going to sample 1 at that
small college, somehow that doesnt seem right either.
There are fancy words like random sampling and representative sampling for this and I
dont think you should worry about memorizing those terms, but I do want you to be aware of
the different new ones. So that as youre reading studies, you can ask yourself whether they
used a representative population for the thing theyre looking for and whether they are over-
generalizing. Are they taking the data from one population and trying to generalize it to a
larger population where it may not be suitable?
When youre dealing with small samples, you could expect greater variability. In other words,
if you flip a coin only four times that doesnt give you a lot of information to go on because
there is 25% of chance that youll get 2 heads in a row. You have to flip the coin dozens or
hundreds of times in order to see how it is behaving and you may never get to 50-50 exactly.
When we take national polls, we can get pretty stable predictions with a very small number of
people. In US for example, where there are 300 million people and typically on the order of 50
or 60 million people voting in a national election, they can survey as few as 2 to 5 thousand