false

Unlock Document

Psychology

PSYB01H3

David Nussbaum

Fall

Description

True Experiments II: Multifactorial Designs Chapter 7
Multifactorial Designs
Also called factorial designs
Two or more independent variables that are qualitatively different
Each has two or more levels
Can be within- or between-subjects
Efficient design
Good for understanding complex phenomena
Multifactorial Design Example-in book chart
Notation
Multifactorial designs are identified by a numbering notation
Number of numbers = how many independent variables
Number of values = how many levels of each independent variable
Number of conditions = product of the numbering notation
A Complex Within-Subjects Experiment
Adams and Kleck (2003)
Two independent variables: gaze direction (direct / indirect), facial muscle
contraction (anger / fear)
Numbering Nomenclature: A “2 by 2 Within” Design
Within-subjects design
Participants made anger / fear judgments of faces and reaction time was
recorded
Adams and Kleck (2003) Results-in book
Main Effects
The effects of each independent variable on the dependent variable
Row means = the averages across levels of one independent variable
Column means = the averages across levels of the other independent
variable
Interactions
When the effects of one level of the independent variable depend on the particular level of
the other independent variable
A significant interaction should be interpreted before the main effects
Graphing the Interaction
A line graph of the simple main effects is useful for examining the interaction
Simple main effect = the value of each cell (or possible combination of
levels of the independent variables)
Interaction Types
Crossover interaction
Lines cross over one another
Antagonistic interaction
Independent variables show opposite effects
Parallel lines indicate no interaction (additivity) Additivity: No Interaction-in book
Antagonistic Interaction- in book
Crossover Interaction- in book
A Complex Between-Subjects 2x3 Experiment
Baumeister, Twenge, & Nuss (2002)
Can feelings of social isolation influence our cognitive abilities?
Manipulated participants‟ “future forecast” (alone, rich relationships,
accident-prone)
Also manipulated the point at which the participant was told the forecast
was bogus (after test/recall, before test/encoding)
Nomenclature: A “3 by 2 Between (groups)” Design
Baumeister et al. (2002) Study Design- in book
Results: Baumeister et al. (2002) - in book
Analyzing Multifactorial Designs
ANOVA (or F-test) = statistical procedure that compares two or more levels of independent
variable(s)
Simple ANOVA = only one IV
Factorial ANOVA = more than one IV
Allows comparison of all effects simultaneously
Ratio of systematic variance to error variance
Analyzing Multifactorial Designs
Ratio of systematic variance to error variance… Basic idea:
1. Calculate the variance using the entire sample
2. Calculate the variance within each group
3. Under the Null Hypothesis (i.e., grouping the sample for each treatment group),
there is no difference between the overall variance and sum of the individual
grouped/within variances because under the Null hypothesis, the various group
means is equal to the overall mean.
4. We then look at the ratio between the sum of the grouped (systematic) variances
and the overall (random or error)variance.
5. The greater the ratio, the less likely the results can be attributed to “chance’
More Complex “Hybrid” Designs
It is possible to combine Between and Within factors in a single study:
Example: Looking at Male-Female differences in self-esteem at ages 5, 7 and 12.
This would be classified as a “2 Between, 3 Within” design.
Quasi-Experimental & Non-Experimental Designs Chapter 8
Quasiexperimental Design
Often, we cannot manipulate a variable of interest
Quasi-independent variables:
Subject variable = individual characteristic used to select participants to
groups Natural treatment = exposure in the “real world” defines how participants are
selected
Types of Quasiexperimental Design
Nonequivalent-control-group designs
Experimental and comparison groups that are designated before the
treatment occurs and are not created by random assignment
Before-and-after designs
Pretest and posttest but no comparison group
Nonequivalent-Control-Group Designs
Random assignment cannot be used to create groups
Confounds related to equivalency of groups cannot be eliminated
Often high in external validity
Particularly ecological validity
Matching
Individual matching = individual cases in the treatment group are matched with similar
individuals
Aggregate matching = identifying a comparison group that matches the treatment group in
the aggregate rather than trying to match individual cases
Regression to the mean can be a problem
What is Regression to the Mean ???
Int J Epidemiol. 2005 Feb;34(1):215-20. Epub 2004 Aug 27.
Regression to the mean: what it is and how to deal with it.
Barnett AG, van der Pols JC, Dobson AJ.
Abstract
BACKGROUND:
Regression to the mean (RTM) is a statistical phenomenon that can make natural variation
in repeated data look like real change. It happens when unusually large or small
measurements tend to be followed by measurements that are closer to the mean.
RESULTS:
The effect of RTM in a sample becomes more noticeable with increasing measurement
error and when follow-up measurements are only examined on a sub-sample selected
using a baseline value.
How to reduce the effects of RTM at the study design stage
1. Random allocation to comparison groups
2. Selection of subjects based on multiple measurements
What is Regression to the Mean ???
CONCLUSIONS:
RTM is a ubiquitous phenomenon in repeated data and should always be considered as a
possible cause of an observed change. Its effect can be alleviated through better study
design and use of suitable statistical methods
Before-and-After Designs aka Pre-Post Designs
Useful for studies of interventions that are experienced by virtually every case in some
population
No comparison group Fixed-sample panel design = one pretest and one posttest
Interrupted-time-series design = examine observations before and after a naturally
occurring treatment
Multiple group before-and-after design = several before-and-after comparisons are made
involving the same independent and dependent variables but with different groups
Repeated-measures panel designs = include several pretest and posttest observations
Memories of 9/11 Example
Sharot, Martorella, Delgado, and Phelps (2007)
Participants viewed word cues while in fMRI scanner
Words belonged to one of two categories: September 2001 and Summer
2001
Participants also rated the word cues on a number of dimensions
Participants were divided into groups (near the World Trade Center or far from the WTC) ex
post facto
Negative correlation between distance and memory rating (near yielded higher memory
ratings)
Researchers also found a different pattern of brain activity between the near and far groups
Scatter Diagram-inbook
Pattern of fMRI activity-inbook
Memories of 9/11: Quasiexperimental Characteristics
Summer condition served as a control condition
Baseline for comparison
9/11 is a natural treatment
Researchers could not manipulate
Random assignment was not possible
Participants did not decide on their treatment condition
Culture and Cognition
Hong, Morris, Chiu, & Benet-Martinez (2000)
Can a bicultural individual be experimentally induced to switch his/her cultural mental set?
Randomly assigned to priming condition to activate mental sets (American / Chinese)
Participant rated the internal / external forces on the behavior of a fish
Participants were selected based on biculturalism
But randomly assigned to priming conditions
Cross-sectional Design
Selects groups of people of different ages and then compares these age groups on
psychological processes
Confounded by:
Cohort effects
Period effects
Longitudinal Design
Same research participants are followed over time
Problems:
Attrition
Secular trends
Cross-sequential Design Time-lag design = a researcher aims to determine the effects of time of testing while
holding age constant
Cross-sequential design = tests two or more age groups at two or more time periods
Avoids problems of both cross-sectional and longitudinal designs
Nonexperiments
Researcher has even less control over the independent variable, and seldom can specific
levels of the independent variable be precisely established or quantified
Serious limitations in terms of internal validity
Ex Post Facto Control Group Design
Experimental and comparison groups that are not created by random assignment
Individuals may decide whether to enter the treatment or control group
Selection bias is a significant issue
Small-N and Single-Subject Designs Chapter 9,
Small-N Designs
Alternative to group designs
Generally involve between 1-9 participants
Systematic procedure for testing changes in a single subject‟s or small number of subjects‟
behavior
Often used in clinical cases
Components of Small-N Designs
1. Repeated measurement of the dependent variable
If pre-intervention measurements cannot be taken, retrospective data
may be used.
2. Baseline phase (A)
Intervention not offered to subject
Acts in place of a “control group”
Repeated measurements of the DV are taken until a pattern emerges
3. Treatment phase(s) (B)
Intervention is implemented
Repeated measurements of the DV are taken
Should be as long as the baseline phase
4. Graphic display
Facilitates monitoring and evaluating the impact of the intervention
Types of Patterns
Stable line
Changes easily detected
Generally few problems with the measure
Trend
Scores increase or decrease over time
May even be cyclical
No Pattern
Possible problems with reliability of measure or client reports
Internal Validity Considerations Repeated measures during the baseline phase help rule out threats to validity
Validity threats should appear in the baseline
Will not control for an extraneous event (history) that occurs between the
last baseline measurement and the first intervention measurement
Measuring Targets of Intervention
DV should be the target of the intervention
Can be measured simultaneously or sequentially
Measures of behavior are often categorized according to:
Frequency = how often behavior occurs
Duration = how long behavior lasts
Interval = time between episodes
Magnitude = intensity of behavioral event
Consider who will collect the data
Choose nonreactive measures
Ensure the measurement process is feasible
Consider the measurement‟s sensitivity
Note that target of the measurement must occur relatively frequently
Analyzing Small-N Designs
Common techniques:
Visual examination of the graph
Statistical technique
Assessing practical (clinical) significance is of primary importance
Determining Practical Significance
Set criteria for success with individual or community
Use clinical cut-off scores
Weigh costs and benefits of producing the change
Visual Analysis
Guiding concepts:
Level = magnitude of the target variable; typically used when the
observations fall along relatively stable lines
Trend = direction in the pattern of the data points
Variability = how different or divergent the scores are within a baseline
or intervention phase
Examination of Trend-inbook
Examination of Variability-inbook
Basic Design (A-B)
Baseline phase (A) with repeated measurements and an intervention phase (B) continuing
the same measures
Fluctuations are difficult to interpret
Cannot rule out other extraneous events, so causality cannot be established
Withdrawal Designs
Intervention is concluded or is temporarily stopped during the study
May pose ethical issues
Carryover effects may limit usefulness A-B-A Design
Includes post-treatment follow-up
Follow-up period should include multiple measures
A-B-A-B Design
Adds second intervention phase that is identical to the first
Replication of treatment phase reduces the possibility that an event or
history explains the change
Multiple-Baselines Designs
Adds additional subjects, target problems, or settings to the study
Controls for the effects of history
Concurrent multiple baseline design
Series of A-B designs are implemented at the same time for at least three
cases
Length of the baseline phase is staggered
May have problem finding available subjects
Nonconcurrent multiple baseline design
Different lengths of time for the baseline period
Subjects are randomly assigned to one of the baseline phases
Alternatively, can be used across different target problems or settings
Multiple-Treatment Designs
Nature of the intervention changes over time
Each change represents a new phase
Yields a more convincing picture of the effect of the treatment program
Can change:
Intensity of the intervention
Number of treatments
Nature of the intervention
Problems of Interpretation
Widely discrepant scores in the baseline
Delayed changes in the intervention phase
Improvement in the target problem scores during the baseline phase
Act of graphing can create visual distortions
Statistical Analysis
Can help avoid the problems associated with visual inspection
Requirements of the parametric statistical tests may be difficult or impossible to meet in a
small-N design
Use of Non-Parametric Statistics
Rank Order Statistics:
Statistics computed from rankings of the observations rather than from the observations
themselves.
Rank Order Statistics:
Statistics computed from rankings of the observations rather than from the observations
themselves.
Imagine that A represents a positive outcome and a B represents a negative outcome. If the treatment has no effect on the outcome and there are a number of “ups & downs”,
you would expect the order of ups and downs to be of random order:
A B A B A B Treatment A B A B A B
Rank Order Statistics:
If the treatment is effective, you would expect that the downs would occur before the
treatment was initiated and the ups would occur after the treatment was initiated.
A B A B A B Treatment A A A A B A
Rank Order Statistics:
Rank Order Statistics evaluate the likelihood that any given order is likely to have occurred
by chance or unlikely to occur by chance.
A B A A B A A B A B A B
A A B B A A B B A B A B
B A B B B A A A B A A A
Rank Order Statistics: Mann–Whitney U test (also called the Mann–Whitney–Wilcoxon (MWW) or
Wilcoxon rank-sum test) (Courtesy of Wikipedia)
A non-parametric statistical hypothesis test for assessing whether one of two samples of
independent observations tends to have larger values in terms of order than the other. It is
one of the most well-known non-parametric significance tests.
There is only an assumption of Ordinal Scale of measurement; No assumptions of Interval
or Ratio Scale are required.
There is no assumption regarding the normality of the distribution of scores.
Rank Order Statistics: Mann–Whitney U test (also called the Mann–Whitney–Wilcoxon (MWW) or
Wilcoxon rank-sum test)
Calculation: First, arrange all the observations into a single ranked series. That is, rank all
the observations without regard to which sample they are in.
For small samples a direct method is recommended. It is very quick, and gives an insight
into the meaning of the U statistic.
Rank Order Statistics: Mann–Whitney U test (also called the Mann–Whitney–Wilcoxon (MWW) or
Wilcoxon rank-sum test)
Choose the sample for which the ranks seem to be smaller (The only reason to do this is to
make computation easier). Call this "sample 1," and call the other sample "sample 2."
Taking each observation in sample 1, count the number of observations in sample 2 that
have a smaller rank (count a half for any that are equal to it). The sum of these counts is U.
Rank Order Statistics: Mann–Whitney U test (also called the Mann–Whitney–Wilcoxon (MWW) or
Wilcoxon rank-sum test)
Illustration of Calculation Methods
Suppose that Aesop is dissatisfied with his classic experiment in which one tortoise was
found to beat one hare in a race, and decides to carry out a significance test to discover
whether the results could be extended to tortoises and hares in general. He collects a
sample of 6 tortoises and 6 hares, and makes them all run his race at once. The order in
which they reach the finishing post (their rank order, from first to last crossing the finish
line) is as follows, writing T for a tortoise and H for a hare:
T H H H H H T T T T T H What is the value of U? Rank Order Statistics: Mann–Whitney U test (also called the Mann–Whitney–Wilcoxon (MWW) or
Wilcoxon rank-sum test)
Illustration of calculation methods
T H H H H H T T T T T H What is the value of U?
Using the direct method, we take each tortoise in turn, and count the number of hares it is
beaten by, getting 0, 5, 5, 5, 5, 5, which means U = 25. Alternatively, we could take each
hare in turn, and count the number of tortoises it is beaten by. In this case, we get 1, 1, 1,
1, 1, 6. So U = 6 + 1 + 1 + 1 + 1 + 1 = 11. Note that the sum of these two values for U is 36,
which is 6 × 6.
Rank Order Statistics: Mann–Whitney U test (also called the Mann–Whitney–Wilcoxon (MWW) or
Wilcoxon rank-sum test)
Illustration of calculation methods
T H H H H H T T T T T H What is the value of U?
What is the value of U? 36
Looking this up on the U statistic table shows that this is significant
Rank Order Statistics: Mann–Whitney U test (also called the Mann–Whitney–Wilcoxon (MWW) or
Wilcoxon rank-sum test)
Illustration of object of test
A second example illustrates the point that the Mann–Whitney does not test for equality of
medians. Consider another hare and tortoise race, with 19 participants of each species, in
which the outcomes are as follows:
H H H H H H H H H T T T T T T T T T T H H H H H H H H H H T T T T T T T T T
Rank Order Statistics: Mann–Whitney U test (also called the Mann–Whitney–Wilcoxon (MWW) or
Wilcoxon rank-sum test)
The median tortoise here comes in at position 19, and thus actually beats the median hare,
which comes in at position 20.
However, the value of U (for hares) is 100
(9 Hares beaten by (x) 0 tortoises) + (10 hares beaten by (x) 10 tortoises) = 0 + 100 = 100
Value of U(for tortoises) is 261
Rank Order Statistics: Mann–Whitney U test (also called the Mann–Whitney–Wilcoxon (MWW) or
Wilcoxon rank-sum test)
Obviously this is an extreme distribution that would be spotted easily, but in a larger sample something
similar could happen without it being so apparent. Notice that the problem here is not that the two
distributions of ranks have different variances; they are mirror images of each other, so their
variances are the same, but they have very different skewness.
Rank Order Statistics: Mann–Whitney U test (also called the Mann–Whitney–Wilcoxon (MWW) or
Wilcoxon rank-sum test)
(10 tortoises beaten by 9 hares) + (9 tortoises beaten by 19 hares) = 90 + 171 = 261
Consulting tables, or using the approximation below, shows that this U value gives evidence that hares
tend to do significantly better than tortoises (p < 0.05, two-tailed).
Generalizability of Small n Studies
Difficult to demonstrate in small-N designs
Requires replication:
Direct replication = same study with different clients
Systematic replication = same interventions in different settings Clinical replication = combining different interventions into a clinical package
to treat multiple problems
Quantitative Analysis Chapter 10:
Case Study: Dunn (2008)
Can you buy happiness?
Participants randomly assigned to:
Money condition($5 or $20)
Spending condition (self or others)
2 x 2 between-subjects factorial design
Types of Statistics
Descriptive = describe variables in a study
Inferential = estimate characteristics of a population from a random sample
Is the effect we observed due to chance alone?
Used to test hypotheses about the relationship between variables
Must consider level of measurement
Frequency Distributions
Frequency Table
Shows the number of cases and/or the percentage of cases who receive each possible
score on a variable
Often precedes the formal statistical analysis
A Frequency Distribution-inbook
Grouping Values in Frequency Distributions
May group the values if:
There are more than 15-20/ category
It would clarify the distribution
Resulting categories:
Should be logical
Should be mutually exclusive and exhaustive
Graphing
Bar charts
Bars separated by spaces
Good for nominal data
Histograms
Displays a frequency distribution of a quantitative variable
Avoiding Misleading Graphs
Begin the graph of a quantitative variable at 0 on both axes
Always use bars of equal width

More
Less
Related notes for PSYB01H3

Join OneClass

Access over 10 million pages of study

documents for 1.3 million courses.

Sign up

Join to view

Continue

Continue
OR

By registering, I agree to the
Terms
and
Privacy Policies

Already have an account?
Log in

Just a few more details

So we can recommend you notes for your school.

Reset Password

Please enter below the email address you registered with and we will send you a link to reset your password.