Chapter X: Non-Experimental & Quasi-Experimental Strategies Non-Equivalent
Group, Pre-Post, and Developmental Designs
10.1 Non-Experimental & Quasi-Experimental Research Strategies
• Five basic research strategies: experimental, non-experimental, quasi-experimental,
correlational, and descriptive.
• A researcher can often devise a research strategy (a method of collecting data) that is
similar to an experiment but fails to satisfy at least one of the requirements of a true
• Such studies are generally called non-experimental or quasi-experimental research
• Although these studies resemble experiments, they always contain a confounding
variable or other threat to internal validity that is an integral part of the design and simply
cannot be removed.
• The existence of a confounding variable means that these studies cannot establish
unambiguous cause-and-effect relationships and, therefore, are not true experiments.
• The distinction between the non-experimental research strategy and the quasi-
experimental research strategy is the degree to which the research strategy limits
confounding and controls threats to internal validity.
• If a research design makes little or no attempt to minimize threats, it is classified as non-
experimental. A quasi-experimental design, on the other hand, makes some attempt to
minimize threats to internal validity and approaches the rigor of a true experiment.
• A quasi-experimental design, on the other hand, makes some attempt to minimize threats
to internal validity and approaches the rigor of a true experiment.
• The fact that quasi-experimental and non-experimental studies are not true experiments
does not mean that they are useless or even second-class research studies.
• Non-experimental and quasi-experimental studies often look like experiments in terms of
the general structure of the research study.
• A non-experimental or quasi-experimental study typically involves comparing groups of
• One variable is used to create groups or conditions, then a second variable is measured
to obtain a set of scores within each condition.
• The different groups or conditions are not created by manipulating an independent
• Groups are usually defined in terms of a pre-existing participant variable or in terms of
• Two methods of defining groups produce two general categories of non-experimental and
1. Between-subjects designs, also known as non-equivalent group designs.
2. Within-subjects designs, also known as pre-post designs.
The non-experimental research strategy makes little or non attempt to control threats to
internal validity whereas the quasi-experimental research strategy attempts to limit threats to
internal validity. Both strategies, like true experiments, typically involve a comparison of
groups or conditions. However, these two strategies use a non-manipulated variable to define
the groups or conditions being compared. The non-manipulated variable is usually a
participant variable (such as male versus female) or a time variable (such as before versus
10.2 Between-Subjects Non-Experimental & Quasi-Experimental Designs: Non-
Equivalent Group Designs
• Between-subjects experimental design refers to a method of comparing two or more
treatment conditions using a different group of participants in each condition. • A common element to between-subjects experiments is the control of participant
variables by assigning participants to specific treatment conditions.
• The goal is to balance or equalize participant variables across treatment conditions by
using a random process or by deliberately matching participants.
• Note that the researcher attempts to create equivalent groups of participants by actively
deciding which individuals go into which groups.
• When the researcher cannot use random assignment or matching to balance participant
variables across groups, there is no assurance that the two groups are equivalent. In this
situation, the research study is called a non-equivalent group design.
A non-equivalent group design is a research study in which the different groups of
participants are formed under circumstances that do not permit the researcher to control the
assignment of individuals to groups, and the groups of participants are, therefore, considered
non-equivalent. Specifically, the researcher cannot use random assignment to create groups
• A non-equivalent group design has a built-in threat to internal validity that precludes an
unambiguous cause-and-effect explanation: assignment bias.
• Assignment bias occurs whenever the assignment procedure produces groups that have
different participant characteristics
• In a non-equivalent group design, there is no random assignment and there is no
assurance of equivalent groups.
• Three common examples of non-equivalent group designs: (1) the differential research
design, (2) the post-test-only non-equivalent control group design, and (3) the pretest-
posttest non-equivalent control group design.
• Non-equivalent group research involves no manipulation but simply attempts to compare
pre-existing groups that are defined by a particular participant variable
• A research study that simply compares pre-existing groups is called a differential
research design because its goal is to establish differences between the pre-existing
• This type of study is often called ex post facto research because it looks at differences
“after the fact;” that is, at differences that already exist between groups.
• Differential research design makes no attempt to control the threat of assignment bias; it
is classified as a non-experimental research design.
A research study that simply compares pre-existing groups is called a differential research
design. A differential study uses a participant characteristic such as gender, race, or
personality to automatically assign participants to groups. The researcher does not randomly
assign individuals to groups. A dependent variable is then measured for each participant to
obtain a set of scores within each group. The goal of the study is to determine whether the
scores for one group are consistently different from the scores of another group. Differential
research is classified as a non-experimental research design.
• Many researchers place differential research in the same category as correlational
research based on their similarities.
• In differential and correlational studies, a researcher simply observes two naturally
occurring variables without any interference or manipulation.
• The subtle distinction between differential research and correlational research is whether
or not one of the variables is used to establish separate groups to be compared.
• In differential research, participant differences in one variable are used to create separate
groups, and measurements of the second variable are made within each group.
• The researcher then compares the measurements for one group with the measurements
of another group, typically looking at mean differences between groups.
• A correlational study treats all participants as a single group and simply measures the
two variables for each individual. • Both designs allow researchers to establish the existence of relationships and to describe
relationships between variables, but neither design permits a cause-and-effect
explanation of the relationship.
Posttest-Only Non-Equivalent Control Group Design
• Non-equivalent groups are commonly used in applied research situations in which the
goal is to evaluate the effectiveness of a treatment administered to a pre-existing group
• A second group of similar but non-equivalent participants is used for the control condition.
• The researcher uses pre-existing groups and does not control the assignment of
participants to groups.
• The researcher does not randomly assign individuals to groups.
A non-equivalent control group design uses pre-existing groups, one of which serves in
the treatment condition and the other in the control condition. The researcher does not
randomly assign individuals to groups.
• One common example of a non-equivalent control group design is called a posttest-only
non-equivalent control group design.
• This type of study is also called a static group comparison.
• In this type of design, one group of participants is given a treatment and then is
measured after the treatment,
• The scores for the treated group are then compared with the scores from a non-
equivalent group that has not received the treatment.
A posttest-only non-equivalent control group design, also known as a static group
comparison, compares two non-equivalent groups of participants. One group is observed
(measured) after receiving a treatment, and the other group is measured at the same time but
receives no treatment. This is an example of a non-experimental research design.
• The posttest-only non-equivalent control group design does not address the threat of
assignment bias; it is considered a non-experimental design.
Pretest-Only Non-Equivalent Control Group Design
• A much stronger version of the non-equivalent control group design is often called a
pretest-posttest non-equivalent control group design.
• Both groups are observed (measured) prior to the treatment.
• The treatment is then administered to one group, and following the treatment, both
groups are observed again.
• The addition of the pretest measurement allows researchers to address the problem of
assignment bias that exists with all non-equivalent groups’ research.
• The researcher then compares the observations before treatment to establish whether or
not the two groups really are similar.
• This type of design allows a researcher to compare the pretest scores and posttest
scores for both groups to determine whether the treatment or some other, time related
factor is responsible for changes.
• In the pre-test-posttest non-equivalent groups design, time-related threats are minimized
because both groups are observed over the same time period and, therefore, should
experience the same time-related factors.
• The pre-test-posttest non-equivalent control group design reduces the threat of
assignment bias, limits threats from time-related factors, and can provide some evidence
to support a cause-and-effect relationship.
• As a result, this type of research is considered quasi-experimental.
A pre-test-posttest non-equivalent design compares two non-equivalent groups. One group is
measured twice, once before a treatment is administered and once after. The other group is
measured at the same two times but does not receive any treatment. Because this design to
limit threats to internal validity, it is classified as quasi-experimental. • The addition of a pre-test to the non-equivalent control group design reduces some
threats to internal validity; it does not eliminate them completely.
• The fact that the groups are non-equivalent and often separated creates the potential for
• It is possible for a time-related threat to affect the groups differently.
• The history for one group could be different from that for the other group.
• History effects that differ form one group to another are called differential history
• In a non-equivalent group design, there is always the possibility that differential history
(and not the treatment) is causing the groups to be different.
• The different groups are not selected from the same source; they may be exposed to
different environmental factors that can influence their scores.
• The internal validity of a pre-test-posttest non-equivalent control group design may be
threatened by differential instrumentation, differential testing effects, differential
maturation, or differential regression.
10.3 Within-Subjects Non-experimental & Quasi-Experimental Designs: Pre-Post
• In a typical pre-post study, one group of participants is observed (measured) before and
after a treatment or event.
• The goal of the pre-post design is to evaluate the influence of the intervening treatment or
event by comparing the observations made before treatment with the observations made
• Whenever a research study involves repeated observations over time, time-related
factors can threaten internal validity.
• History, instrumentation, testing effects, maturation, and statistical regression are five
kinds of time-related threats.
• Any differences found between the pre-treatment observations and the post-treatment
observations could be explained by history, instrumentation, testing effects, maturation,
• In a pre-post design, it is impossible to counterbalance the order of treatments.
• Specifically, the before-treatment observations (Pretest) must always precede the after-
treatment observations (posttest).
• The internal validity of a pre-post study is threatened by a variety of factors related to the
passage of time.
• During the time between the first observation and the last observation, any one of these
factors could influence the participants and cause a change in their scores.
• Unless these factors are controlled or minimized by the structure of the research design,
a pre-post study cannot approach the internal validity of a true experiment.
One Group Pretest-Posttest Design
• The simplest version of the pre-post design consists of only one observation for each
participant made before the treatment or event, and only one observation made after it.
In the one-group pretest-posttest design, each individual in a single group of participants is
measured once before treatment and once after treatment. This type of research is classified
as a non-experimental design.
Time-Series & Interrupted Time-Series Designs
• A time-series design requires a series of observations for each participant before and
after the treatment or event.
• A time-series design requires a series of observations for each participant before and
after the treatment or event.
• The intervening treatment may or may not be manipulated by the researcher. • When the event that occurs in the middle of the series of observations is actually a
treatment manipulated or administered by the researcher, the research study is called a
• A study in which the intervening event is not manipulated by the researcher is called an
interrupted time-series design.
A time-series design has a series of observations for each participant before a treatment
and a series of observations after the treatment. The treatment is administered by the
An interrupted time-series design consists of a series of observations for each participant
before an event occurs and after the event occurs. The event is not a treatment or an
experience that is created or manipulated by the researcher.
• For both time-series design, the pretest and posttest series of observations serve several
• Pretest observations allow a researcher to see any trends that may already exist in the
data before the treatment is even introduced.
• Trends in the data are an indication that the scores are influenced by some factor
unrelated to the treatment.
• If the data show no trends or major fluctuations before the treatment, the researcher can
be reasonably sure that potential threats to internal validity are not influencing the
• The series of observations allows a researcher to minimize most threats to internal
validity. As a result, the time-series and interrupted time-series designs are classified as
• Time-series and interrupted time-series designs minimize most threats to internal validity.
• However, it is possible for some outside event – history – to influence the data.
• If the outside event occurs at any time other than the introduction of the treatment, it
should be easy to separate the history effects from the treatment effects.
• History effects (outside events) are a threat to validity only when there is a perfect
correspondence between the occurrence of the event and the introduction of the
• The series of observations after the treatment or event also allows a researcher to
observe any post-treatment trends.
Equivalent Time-Samples Design
• One way of minimizing the threat of coincidence in a time series design is to expand the
simple time series into an equivalent time-samples design.
• In an equivalent time-samples study, the treatment is repeatedly administered and
removed during the series of observations.
• The study begins with a series of observations, which is followed by a treatment and
another series of observations.
• Then the treatment is removed for a series of no-treatment observations.
• This sequence is repeated with at least one more set of treatment observations and no-