Study Guides (248,447)
Canada (121,541)
Psychology (1,882)
PSYB01H3 (82)

PSYB01 Final Lecture Notes Ch.7-12.docx

20 Pages
Unlock Document

David Nussbaum

True Experiments II: Multifactorial Designs Chapter 7 Multifactorial Designs  Also called factorial designs  Two or more independent variables that are qualitatively different  Each has two or more levels  Can be within- or between-subjects  Efficient design  Good for understanding complex phenomena Multifactorial Design Example-in book chart Notation  Multifactorial designs are identified by a numbering notation  Number of numbers = how many independent variables  Number of values = how many levels of each independent variable  Number of conditions = product of the numbering notation A Complex Within-Subjects Experiment  Adams and Kleck (2003)  Two independent variables: gaze direction (direct / indirect), facial muscle contraction (anger / fear)  Numbering Nomenclature: A “2 by 2 Within” Design  Within-subjects design  Participants made anger / fear judgments of faces and reaction time was recorded Adams and Kleck (2003) Results-in book Main Effects  The effects of each independent variable on the dependent variable  Row means = the averages across levels of one independent variable  Column means = the averages across levels of the other independent variable Interactions  When the effects of one level of the independent variable depend on the particular level of the other independent variable  A significant interaction should be interpreted before the main effects Graphing the Interaction  A line graph of the simple main effects is useful for examining the interaction  Simple main effect = the value of each cell (or possible combination of levels of the independent variables) Interaction Types  Crossover interaction  Lines cross over one another  Antagonistic interaction  Independent variables show opposite effects  Parallel lines indicate no interaction (additivity) Additivity: No Interaction-in book Antagonistic Interaction- in book Crossover Interaction- in book A Complex Between-Subjects 2x3 Experiment  Baumeister, Twenge, & Nuss (2002)  Can feelings of social isolation influence our cognitive abilities?  Manipulated participants‟ “future forecast” (alone, rich relationships, accident-prone)  Also manipulated the point at which the participant was told the forecast was bogus (after test/recall, before test/encoding)  Nomenclature: A “3 by 2 Between (groups)” Design Baumeister et al. (2002) Study Design- in book Results: Baumeister et al. (2002) - in book Analyzing Multifactorial Designs  ANOVA (or F-test) = statistical procedure that compares two or more levels of independent variable(s)  Simple ANOVA = only one IV  Factorial ANOVA = more than one IV  Allows comparison of all effects simultaneously  Ratio of systematic variance to error variance Analyzing Multifactorial Designs  Ratio of systematic variance to error variance… Basic idea: 1. Calculate the variance using the entire sample 2. Calculate the variance within each group 3. Under the Null Hypothesis (i.e., grouping the sample for each treatment group), there is no difference between the overall variance and sum of the individual grouped/within variances because under the Null hypothesis, the various group means is equal to the overall mean. 4. We then look at the ratio between the sum of the grouped (systematic) variances and the overall (random or error)variance. 5. The greater the ratio, the less likely the results can be attributed to “chance’ More Complex “Hybrid” Designs  It is possible to combine Between and Within factors in a single study:  Example: Looking at Male-Female differences in self-esteem at ages 5, 7 and 12.  This would be classified as a “2 Between, 3 Within” design. Quasi-Experimental & Non-Experimental Designs Chapter 8 Quasiexperimental Design  Often, we cannot manipulate a variable of interest  Quasi-independent variables:  Subject variable = individual characteristic used to select participants to groups  Natural treatment = exposure in the “real world” defines how participants are selected Types of Quasiexperimental Design  Nonequivalent-control-group designs  Experimental and comparison groups that are designated before the treatment occurs and are not created by random assignment  Before-and-after designs  Pretest and posttest but no comparison group Nonequivalent-Control-Group Designs  Random assignment cannot be used to create groups  Confounds related to equivalency of groups cannot be eliminated  Often high in external validity  Particularly ecological validity Matching  Individual matching = individual cases in the treatment group are matched with similar individuals  Aggregate matching = identifying a comparison group that matches the treatment group in the aggregate rather than trying to match individual cases  Regression to the mean can be a problem What is Regression to the Mean ???  Int J Epidemiol. 2005 Feb;34(1):215-20. Epub 2004 Aug 27.  Regression to the mean: what it is and how to deal with it.  Barnett AG, van der Pols JC, Dobson AJ.  Abstract  BACKGROUND:  Regression to the mean (RTM) is a statistical phenomenon that can make natural variation in repeated data look like real change. It happens when unusually large or small measurements tend to be followed by measurements that are closer to the mean.  RESULTS:  The effect of RTM in a sample becomes more noticeable with increasing measurement error and when follow-up measurements are only examined on a sub-sample selected using a baseline value. How to reduce the effects of RTM at the study design stage 1. Random allocation to comparison groups 2. Selection of subjects based on multiple measurements What is Regression to the Mean ???  CONCLUSIONS:  RTM is a ubiquitous phenomenon in repeated data and should always be considered as a possible cause of an observed change. Its effect can be alleviated through better study design and use of suitable statistical methods Before-and-After Designs aka Pre-Post Designs  Useful for studies of interventions that are experienced by virtually every case in some population  No comparison group  Fixed-sample panel design = one pretest and one posttest  Interrupted-time-series design = examine observations before and after a naturally occurring treatment  Multiple group before-and-after design = several before-and-after comparisons are made involving the same independent and dependent variables but with different groups  Repeated-measures panel designs = include several pretest and posttest observations Memories of 9/11 Example  Sharot, Martorella, Delgado, and Phelps (2007)  Participants viewed word cues while in fMRI scanner  Words belonged to one of two categories: September 2001 and Summer 2001  Participants also rated the word cues on a number of dimensions  Participants were divided into groups (near the World Trade Center or far from the WTC) ex post facto  Negative correlation between distance and memory rating (near yielded higher memory ratings)  Researchers also found a different pattern of brain activity between the near and far groups Scatter Diagram-inbook Pattern of fMRI activity-inbook Memories of 9/11: Quasiexperimental Characteristics  Summer condition served as a control condition  Baseline for comparison  9/11 is a natural treatment  Researchers could not manipulate  Random assignment was not possible  Participants did not decide on their treatment condition Culture and Cognition  Hong, Morris, Chiu, & Benet-Martinez (2000)  Can a bicultural individual be experimentally induced to switch his/her cultural mental set?  Randomly assigned to priming condition to activate mental sets (American / Chinese)  Participant rated the internal / external forces on the behavior of a fish  Participants were selected based on biculturalism  But randomly assigned to priming conditions Cross-sectional Design  Selects groups of people of different ages and then compares these age groups on psychological processes  Confounded by:  Cohort effects  Period effects Longitudinal Design  Same research participants are followed over time  Problems:  Attrition  Secular trends Cross-sequential Design  Time-lag design = a researcher aims to determine the effects of time of testing while holding age constant  Cross-sequential design = tests two or more age groups at two or more time periods  Avoids problems of both cross-sectional and longitudinal designs Nonexperiments  Researcher has even less control over the independent variable, and seldom can specific levels of the independent variable be precisely established or quantified  Serious limitations in terms of internal validity Ex Post Facto Control Group Design  Experimental and comparison groups that are not created by random assignment  Individuals may decide whether to enter the treatment or control group  Selection bias is a significant issue Small-N and Single-Subject Designs Chapter 9, Small-N Designs  Alternative to group designs  Generally involve between 1-9 participants  Systematic procedure for testing changes in a single subject‟s or small number of subjects‟ behavior  Often used in clinical cases Components of Small-N Designs 1. Repeated measurement of the dependent variable  If pre-intervention measurements cannot be taken, retrospective data may be used. 2. Baseline phase (A)  Intervention not offered to subject  Acts in place of a “control group”  Repeated measurements of the DV are taken until a pattern emerges 3. Treatment phase(s) (B)  Intervention is implemented  Repeated measurements of the DV are taken  Should be as long as the baseline phase 4. Graphic display  Facilitates monitoring and evaluating the impact of the intervention Types of Patterns  Stable line  Changes easily detected  Generally few problems with the measure  Trend  Scores increase or decrease over time  May even be cyclical  No Pattern  Possible problems with reliability of measure or client reports Internal Validity Considerations  Repeated measures during the baseline phase help rule out threats to validity  Validity threats should appear in the baseline  Will not control for an extraneous event (history) that occurs between the last baseline measurement and the first intervention measurement Measuring Targets of Intervention  DV should be the target of the intervention  Can be measured simultaneously or sequentially  Measures of behavior are often categorized according to:  Frequency = how often behavior occurs  Duration = how long behavior lasts  Interval = time between episodes  Magnitude = intensity of behavioral event  Consider who will collect the data  Choose nonreactive measures  Ensure the measurement process is feasible  Consider the measurement‟s sensitivity  Note that target of the measurement must occur relatively frequently Analyzing Small-N Designs  Common techniques:  Visual examination of the graph  Statistical technique  Assessing practical (clinical) significance is of primary importance Determining Practical Significance  Set criteria for success with individual or community  Use clinical cut-off scores  Weigh costs and benefits of producing the change Visual Analysis  Guiding concepts:  Level = magnitude of the target variable; typically used when the observations fall along relatively stable lines  Trend = direction in the pattern of the data points  Variability = how different or divergent the scores are within a baseline or intervention phase Examination of Trend-inbook Examination of Variability-inbook Basic Design (A-B)  Baseline phase (A) with repeated measurements and an intervention phase (B) continuing the same measures  Fluctuations are difficult to interpret  Cannot rule out other extraneous events, so causality cannot be established Withdrawal Designs  Intervention is concluded or is temporarily stopped during the study  May pose ethical issues  Carryover effects may limit usefulness  A-B-A Design  Includes post-treatment follow-up  Follow-up period should include multiple measures  A-B-A-B Design  Adds second intervention phase that is identical to the first  Replication of treatment phase reduces the possibility that an event or history explains the change Multiple-Baselines Designs  Adds additional subjects, target problems, or settings to the study  Controls for the effects of history  Concurrent multiple baseline design  Series of A-B designs are implemented at the same time for at least three cases  Length of the baseline phase is staggered  May have problem finding available subjects  Nonconcurrent multiple baseline design  Different lengths of time for the baseline period  Subjects are randomly assigned to one of the baseline phases  Alternatively, can be used across different target problems or settings Multiple-Treatment Designs  Nature of the intervention changes over time  Each change represents a new phase  Yields a more convincing picture of the effect of the treatment program  Can change:  Intensity of the intervention  Number of treatments  Nature of the intervention Problems of Interpretation  Widely discrepant scores in the baseline  Delayed changes in the intervention phase  Improvement in the target problem scores during the baseline phase  Act of graphing can create visual distortions Statistical Analysis  Can help avoid the problems associated with visual inspection  Requirements of the parametric statistical tests may be difficult or impossible to meet in a small-N design  Use of Non-Parametric Statistics Rank Order Statistics:  Statistics computed from rankings of the observations rather than from the observations themselves.  Rank Order Statistics:  Statistics computed from rankings of the observations rather than from the observations themselves.  Imagine that A represents a positive outcome and a B represents a negative outcome.  If the treatment has no effect on the outcome and there are a number of “ups & downs”, you would expect the order of ups and downs to be of random order:  A B A B A B Treatment A B A B A B Rank Order Statistics:  If the treatment is effective, you would expect that the downs would occur before the treatment was initiated and the ups would occur after the treatment was initiated.  A B A B A B Treatment A A A A B A Rank Order Statistics:  Rank Order Statistics evaluate the likelihood that any given order is likely to have occurred by chance or unlikely to occur by chance.  A B A A B A A B A B A B  A A B B A A B B A B A B  B A B B B A A A B A A A Rank Order Statistics: Mann–Whitney U test (also called the Mann–Whitney–Wilcoxon (MWW) or Wilcoxon rank-sum test) (Courtesy of Wikipedia)  A non-parametric statistical hypothesis test for assessing whether one of two samples of independent observations tends to have larger values in terms of order than the other. It is one of the most well-known non-parametric significance tests.  There is only an assumption of Ordinal Scale of measurement; No assumptions of Interval or Ratio Scale are required.  There is no assumption regarding the normality of the distribution of scores. Rank Order Statistics: Mann–Whitney U test (also called the Mann–Whitney–Wilcoxon (MWW) or Wilcoxon rank-sum test)  Calculation: First, arrange all the observations into a single ranked series. That is, rank all the observations without regard to which sample they are in.  For small samples a direct method is recommended. It is very quick, and gives an insight into the meaning of the U statistic. Rank Order Statistics: Mann–Whitney U test (also called the Mann–Whitney–Wilcoxon (MWW) or Wilcoxon rank-sum test)  Choose the sample for which the ranks seem to be smaller (The only reason to do this is to make computation easier). Call this "sample 1," and call the other sample "sample 2."  Taking each observation in sample 1, count the number of observations in sample 2 that have a smaller rank (count a half for any that are equal to it). The sum of these counts is U. Rank Order Statistics: Mann–Whitney U test (also called the Mann–Whitney–Wilcoxon (MWW) or Wilcoxon rank-sum test)  Illustration of Calculation Methods  Suppose that Aesop is dissatisfied with his classic experiment in which one tortoise was found to beat one hare in a race, and decides to carry out a significance test to discover whether the results could be extended to tortoises and hares in general. He collects a sample of 6 tortoises and 6 hares, and makes them all run his race at once. The order in which they reach the finishing post (their rank order, from first to last crossing the finish line) is as follows, writing T for a tortoise and H for a hare:  T H H H H H T T T T T H What is the value of U? Rank Order Statistics: Mann–Whitney U test (also called the Mann–Whitney–Wilcoxon (MWW) or Wilcoxon rank-sum test)  Illustration of calculation methods  T H H H H H T T T T T H What is the value of U?  Using the direct method, we take each tortoise in turn, and count the number of hares it is beaten by, getting 0, 5, 5, 5, 5, 5, which means U = 25. Alternatively, we could take each hare in turn, and count the number of tortoises it is beaten by. In this case, we get 1, 1, 1, 1, 1, 6. So U = 6 + 1 + 1 + 1 + 1 + 1 = 11. Note that the sum of these two values for U is 36, which is 6 × 6. Rank Order Statistics: Mann–Whitney U test (also called the Mann–Whitney–Wilcoxon (MWW) or Wilcoxon rank-sum test)  Illustration of calculation methods  T H H H H H T T T T T H What is the value of U?  What is the value of U? 36  Looking this up on the U statistic table shows that this is significant Rank Order Statistics: Mann–Whitney U test (also called the Mann–Whitney–Wilcoxon (MWW) or Wilcoxon rank-sum test)  Illustration of object of test  A second example illustrates the point that the Mann–Whitney does not test for equality of medians. Consider another hare and tortoise race, with 19 participants of each species, in which the outcomes are as follows:  H H H H H H H H H T T T T T T T T T T H H H H H H H H H H T T T T T T T T T Rank Order Statistics: Mann–Whitney U test (also called the Mann–Whitney–Wilcoxon (MWW) or Wilcoxon rank-sum test)  The median tortoise here comes in at position 19, and thus actually beats the median hare, which comes in at position 20.  However, the value of U (for hares) is 100  (9 Hares beaten by (x) 0 tortoises) + (10 hares beaten by (x) 10 tortoises) = 0 + 100 = 100  Value of U(for tortoises) is 261 Rank Order Statistics: Mann–Whitney U test (also called the Mann–Whitney–Wilcoxon (MWW) or Wilcoxon rank-sum test) Obviously this is an extreme distribution that would be spotted easily, but in a larger sample something similar could happen without it being so apparent. Notice that the problem here is not that the two distributions of ranks have different variances; they are mirror images of each other, so their variances are the same, but they have very different skewness. Rank Order Statistics: Mann–Whitney U test (also called the Mann–Whitney–Wilcoxon (MWW) or Wilcoxon rank-sum test) (10 tortoises beaten by 9 hares) + (9 tortoises beaten by 19 hares) = 90 + 171 = 261 Consulting tables, or using the approximation below, shows that this U value gives evidence that hares tend to do significantly better than tortoises (p < 0.05, two-tailed). Generalizability of Small n Studies  Difficult to demonstrate in small-N designs  Requires replication:  Direct replication = same study with different clients  Systematic replication = same interventions in different settings  Clinical replication = combining different interventions into a clinical package to treat multiple problems Quantitative Analysis Chapter 10: Case Study: Dunn (2008)  Can you buy happiness?  Participants randomly assigned to:  Money condition($5 or $20)  Spending condition (self or others)  2 x 2 between-subjects factorial design Types of Statistics  Descriptive = describe variables in a study  Inferential = estimate characteristics of a population from a random sample  Is the effect we observed due to chance alone?  Used to test hypotheses about the relationship between variables  Must consider level of measurement Frequency Distributions  Frequency Table  Shows the number of cases and/or the percentage of cases who receive each possible score on a variable  Often precedes the formal statistical analysis A Frequency Distribution-inbook Grouping Values in Frequency Distributions  May group the values if:  There are more than 15-20/ category  It would clarify the distribution  Resulting categories:  Should be logical  Should be mutually exclusive and exhaustive Graphing  Bar charts  Bars separated by spaces  Good for nominal data  Histograms  Displays a frequency distribution of a quantitative variable Avoiding Misleading Graphs  Begin the graph of a quantitative variable at 0 on both axes  Always use bars of equal width
More Less

Related notes for PSYB01H3

Log In


Join OneClass

Access over 10 million pages of study
documents for 1.3 million courses.

Sign up

Join to view


By registering, I agree to the Terms and Privacy Policies
Already have an account?
Just a few more details

So we can recommend you notes for your school.

Reset Password

Please enter below the email address you registered with and we will send you a link to reset your password.

Add your courses

Get notes from the top students in your class.