Chapter 8 – Experimental Design
• In the experimental method, all extraneous variables are controlled
Confounding and Internal Validity
• The experimental method provides an unambiguous interpretation of results
because the independent variable is manipulated by the researcher to create groups
that differ in the levels of the variable, which are then compared in terms of their
scores of the dependant variable
• All other variables are kept constant, either through experimental control or
• If the scores of the groups are different, can conclude that it was caused by the
independent variable (because that was the only difference between the groups).
• A confounding variable is a variable that varies along with the independent
variable – confounding occurs when the effects of the independent variable and the
uncontrolled variable are intertwined so you cannot determine which variable
caused the observed effect.
• Good experimental design eliminates possible confounding that results in possible
alternative explanations, because only by eliminating competing, alternative
explanations can we draw a causal relationship from the independent variable.
• Internal Validity – when the results of an experiment can confidently be attributed
to the independent variable (and not any alternate explanations).
• The simplest experimental design has two variables – the independent and
• The independent has two levels – the control group and the experimental group
• Researchers make every effort to ensure the only difference between groups is the
manipulated variable – remember experiments involve control over extraneous
variable through keeping such variables constant (control group) or by
• There are two types of basic experiments – Posttest-only Design and Pretest-
Posttest-Only Design (diagram pg 151)
• Researcher must:
1. obtain two equivalent groups of participants – to eliminate and potential
selection differences: the people selected to be in the conditions cannot differ in
any systematic way. The groups can be made equivalent by randomly assigning
2. introduce the independent variable – the researcher must choose 2 levels of the
independent variable, such as the experimental which receives a treatment and the
control which does not. The researcher could also choose to use two different amounts of the independent variable (eg. Effect of the amount of relaxation
training on quitting smoking). Both methods provide a basis comparing the two
3. measure the effect on the dependant variable – the measurement procedure is
kept the same for both groups so that comparison is possible. While a statistics test
would be conducted, for our purposes, know that this produces an internally valid
• Differs from the “Posttest-Only Design” because it gives a pretest before the
experimental manipulation to ensure that the groups were actually equivalent before
• This is usually not necessary if the participants were randomly assigned. Larger
sample sizes produce groups that are virtually identical in every aspect.
• the larger the sample, the less likelihood that the groups will be systematically
different and the more likely that the effect viewed in the dependant variable is due
to the independent variable.
• Rule of thumb: at least 20-30 participants per group.
Advantages and Disadvantages of the Two Designs
• While randomization is likely to produce equivalent groups, when there are smaller
samples it is possible that they are not equal, so a pretest allows the researcher to
assess whether the groups are in fact equivalent.
• Sometimes a pretest is necessary to select the participants of the experiment. For
example, it may be used to locate the highest and lowest scores on a smoking
measure. Once identified, the participants will be randomly assigned to the
experimental and control groups.
• The pretest can also be used to the extent of change in each individual (eg compare
the smoking measure before and after the treatment).
• A pretest is necessary whenever there is a possibility that the participants will drop
out of the experiment (eg studies over a long period of time). The dropout factor in
experiments is called mortality.
• Even if the groups are equivalent to begin with, different mortality rates will effect
the results greatly. For example, if the heaviest smoker from one group drops out,
and only the lighter smokers are left, the treatment will seem more effective than it
actually is. In this way, mortality can become an alternate explanation for the
• A pretest allows you to asses the effects of mortality – you can look at the pretest
scores of the dropouts and know whether mortality affected the final results.
• One disadvantage of a pretest is they may be time consuming and awkward to
• The most important disadvantage of a pretest is that it may sensitize participants
and allow them to figure out your hypothesis, therefore changing the way they react
to the manipulation – therefore, the independent variable may not have an effect in
the real world, where pretests are not given. • To overcome awareness of the pretest, it can be disguised by administering it in a
completely different setting by a different experimenter.
• A second measure of disguise is to embed the pretest in a set of irrelevant measures
so it is not obvious to the participant what the research topic is
• Solomon Four-Group Design - It is possible to test the impact of the pretest with a
combination of the posttest-only and the pretest-posttest designs. Half the
participants receive only the posttest and half the participants receive both the
pretest and the posttest. If there is no impact of the pretest, the posttest scores will
be the same in both control groups and in both experimental groups.
• Look at table 8.1 on page 154 and the graphs on page 155
Assigning Participants to Experimental Controls
• There are two basic ways of assigning participants to experimental conditions.
• Independent groups design – participants are randomly assigned to the various
conditions so that each participates in one group only
• Repeated measures design – participants are in all conditions. The participants are
measured after receiving each level of the independent variable.
Independent Groups Design
• Participants are assigned to each of the conditions using random assignment.
• In practice, researchers usually use a sequence of random numbers to determine
assignment (Appendix C). the table is made up of a list of numbers from 0 – 99 that
were randomly assigned by a computer
• Randomization eliminates any systematic biased and groups will be equivalent (eg
on age, race, income, education etc) therefore participant differences can not be an
explanation for the results of the experiment.
Repeated Measure Design
• The same individuals participate in all of the groups.
• Eg. In an experiment investigating the relationship between the meaningfulness of
information and learning of that information, the same individuals might first read
low-meaningful material and then take a memory recall test to measure their
learning, then those same individuals may read high-meaningful material and take a
memory recall test to measure their learning. The two results of the groups can then
• You can see why it is called a repeated measure design – because the participants
are repeatedly measured on the dependant variable after being in each condition of
Advantages and Disadvantages of Repeated Measure Design
• Advantages • Fewer research participants are needed and therefore also less costly
• It is very useful in research on perception because extensive training of participants
is required before the experiment can actually begin
• Extremely sensitive to finding statistically significant differences between groups
because the data is from the same group of people in both conditions. The
individual differences can be seen and explained. With the Repeated measure
design, it is much easier to separate the systematic individual differences from the
effect of the independent variable – therefore we are much more likely to detect an
effect of the independent variable on the dependant variable.
• The major problem is that different conditions must be presented in a particular
• Order effect – the order of presenting the treatments affects the dependant
variable. Eg. Greater recall on the high-meaningful material might be d