Chapter 11: Single-case, Quasi-experimental, Developmental research
Single-case designs and reasons to use it.
5 types of evaluations in program evaluation research: needs assessment, program assessment,
process evaluation, outcome evaluation, efficiency assessment.
One-group posttest only.
One-group pretest-posttest design, threats to internal validity: history, maturation, testing,
instrument decay, regression toward mean.
Nonequivalent control group design and nonequivalent control group pretest-protest design,
advantages of having control.
Interrupted time series design vs. control series design.
Cross-sectional, longitudinal, sequential research designs, advantages + disadvantages.
Internal validity: independent variable truly is the cause of observed responses on dependent variable.
-Often true in the classical design used in behavioral sciences, however, can use other designs for special
-3 types of special research situations:
a) effect of independent variable is inferred from an experiment with only one participant: single-case.
b) Pre-experimental and quasi-experimental designs.
c) research designs for studying changes occurring with age.
Single-case experimental designs (single-subject designs)
Early interest came from research on operant conditioning by Skinner.
Today, often seen in clinical, counseling, educational, applied settings.
Used to determine if an experimental manipulation had an effect on a single participant.
Measure behavior over time during baseline control periodthen introduce manipulation during
treatment period. Change in behavior from baseline treatment period is evidence for
effectiveness of the manipulation.
Issue: many possible alternative explanations, ex: other events coincided with intro of treatment.
Reversal designs: demonstrate reversibility of the manipulation. (ABA design—ABAB, ABABAB..)
A (baseline)B (treatment) A (baseline). Observe behavior again after treatment back in
baseline, withdraw the treatment, aka withdrawal design.
Ex: child’s learning behavior. Baseline treatment (reinforcing program) withdraw
treatment (2 baseline). Found, behavior changed when treatment was introduced and
reversed when treatment was withdrawn is evidence for its effectiveness.
But, some variables may require longer time to show an impact.
2 issues with ABA design: a) a single reversal is not extremely powerful evidence for
effectiveness of treatment, may merely be random fluctuation.
b) Ethic. Unethical to end the design with the withdrawal of treatment
that may be beneficial. ABAB ends with treatment rather than withdrawal seems better. Multiple baseline designs
May be unethical or not possible to reverse the manipulation. Ex: dangerous or illegal
behaviors, treatment produce long-lasting change in behavior. Take multiple measures
over time before and after manipulation.
If the manipulation is effective, a change in behavior will be observed, and continue to
reflected in further measures of behavior.
Effectiveness of treatment is shown when behavior changes only after manipulation is
introduced & change must be observed under multiple circumstances to rule out chance.
Variations of the multiple baseline design:
a) Across subject: behavior of several subjects is measured over time. For each subject,
introduce manipulation at different point in time. Ex: smoking reduction. Shown a change
in behavior after introduction of manipulation for each subject, and because the change
occurred across all individuals and manipulation introduced at a different time for them,
can rule out chance, historical events etc.. alternative explanation.
b) Across behaviors: different behaviors of single subject are measured over time. At
different time, apply the same manipulation to each behaviors. Ex: reward system to
increase socializing, grooming, reading behavior.
c) Across situations: same behavior measured in different settings. Ex: home and at
work. Again, manipulation introduced at different time in each setting.
Replications in single-case designs
Replicated with other subjects, enhancing generalizability (often in several settings too).
Better to present results from each subject individually rather than as group mean.
Group mean can give misleading picture of individual responses, manipulation may be effective
in changing behavior of some subjects but not other.
Single-case designs are useful to study many problems, valuable for people applying some
change technique in natural environment: teacher trying new technique in classroom.
Also, complex statistical analyses are not required.
Program evaluation: research on programs that are implemented to achieve some positive effect.
Can be implemented in schools, work, communities. Ex: DARE (Drug Abuse Resistance
Education) to reduce drug use.
At first, initial focus of evaluation research was outcome evaluation—did the program result in
positive outcome that it was designed for? But now it is much more.
Five types of evaluations: each attempts to answer a different question about the program.
Five phases of program evaluation research:
Program theory assessment
Efficiency Assessment Needs assessment: whether there are in fact, problems that need to be addressed. Data gathered
from surveys interviews, stat data maintained by public health, criminal justice, other agencies.
Program theory assessment: collaboration of researchers, service providers, prospective clients
of programs to determine that the proposed program does in fact address the needs of target
population in appropriate ways.
Ex: homeless program. Men’s need: living, job, job skills, drinking + drug problems. Women:
health. So the program should take these specific needs in to consideration.
Process evaluation/program monitoring: assuring the implementation of the program is correct.
Ex: reaching target, enough clients, planned services, staffs trained. Done with questionnaires and
interviews, observational studies, analysis of records kept by staff.
Outcome evaluation/impact assessment: are the intended outcomes realized? Goals achieved.
Want to know what the participants are like, what they would be if not completed the program.
Efficiency assessment: worth the resources it consumes. Cost of program weighed against its
benefits. Determine if the resources used to implement the program might be put to better use.
Study effect of independent variable in settings which the control features of true experimental
designs cannot be achieved. Causal inference is difficult because lack features like random
assignment. (a plan B for a true experiment).
Out-group posttest-only design: one-shot case study
Lacks control or comparison group. Deficit in internal validity.
Ex above: how many seconds does it take for a stranger to move away when you sit beside them.
Suppose the mean is 9.6s, this finding is not interpretable, perhaps they wanted to leave, perhaps
they would’ve left faster if it hadn’t been you there.
This design is sometimes used as evidence for program effectiveness. Ex: after 4 hour session of
seminar, do a test, average score = 90%. Concluded the program is successful. But really, lack
internal validity. Don’t know if the score have been equal, lower, or higher without the program.
The reason why they are sometimes accepted is because we may have an implicit idea of how a
control group would perform. But we need that comparison data.
One-group pretest-posttest design
Measure participants before the manipulation