PSYC 403 Lecture Notes - Lecture 3: Systematic Review, Meta-Analysis, Statistical Significance
13 May 2018
School
Department
Course
Professor
• Reporting RCT results
o Do a CONSORT flow diagram
o List the reasons why people were excluded
o Explain the randomization
o Explain if people stopped coming
o Who was excluded from the analysis
• Meta-analysis
o Pools effect sizes
▪ Aggregates them across studies
o Useful to see what influences the size of this effect across studies
o What does the body of research say about psychotherapy or a particular
treatment?
o Statistical technique to pool effect size estimates
o Moved toward meta-analysis in psychology
o Smith and Glass
▪ Meta-analysis of 375 controlled therapy studies
▪ Typical therapy client did better than 75% of untreated clients
▪ Combined effect size of 0.68 (medium to large)
▪ No strong evidence for superiority of specific treatments
▪ New statistical technique that showed the effect of psychotherapy
• Definition of empirically supported treatment
o Chambless and Hollon
▪ Their article reviewed what a well-established treatment is
• At least 2 good etee-group design experiments that show
the treatment is better to a medication, psychotherapy placebo,
or to another treatment
o A WLC is not sufficient
• Or a large series of single-case design experiments with good
experimental design and comparison to another treatment
• Must be conducted with treatment manuals or other clear
description
o Need to be able to take that treatment and use it in other
studies
• Characteristics of samples must be clearly defined
o Who are your participants and why are you selecting them
• Effects must be demonstrated by at least two different
investigators or teams
o If soeoe deeloped a teatet they’e likely to fid
that it works
▪ Probably efficacious treatments
• Two experiments show treatment is better to waitlist control
• Or one or more experiments meet criteria above but have not
been replicated by independent investigators
find more resources at oneclass.com
find more resources at oneclass.com
• Or a small series of single case design experiments have been
conducted
o Tolin et al.
▪ Article by Tolon and colleagues who are experienced in RCTs
• No that thee’s so uh eseah hat is the defiitio of E“T
▪ Evaluation based on 2 studies is unreasonably low bar given number of
RCTs
• Most treatments now have multiple studies
• Need to focus more on systematic reviews and meta-analysis
• Consider quality and risk of bias of individual studies and
systematic review
▪ Focus has been on symptom reduction
• Measure functional impairment and quality of life
• What is most relevant to the patient
▪ No guidance on which EST to choose
• “oeties thee’s lots of teatets that ae sho to e
effective then how do you choose?
• Include info on strength of treatment
• Evaluate clinical and statistical significance
• What is the evidence base?
o Originally, almost exclusively CBT treatments
▪ People who developed CBT were the only ones doing research
▪ Psyhodyai ad huaisti theapies ee loge te ad did’t
lend themselves well to RCTs
• Hard to put unconditional regard in a manual
▪ Over time, more treatments developed and tested in context of RCT
• The American Psychological Association Division
o Thee’s ay teatets that hae ee sho to e effetie i teating
depression
o For panic disorders only CBT has strong research support
o The website says some support is controversial-> mixed results
o For social anxiety disorders and public speaking anxiety only CBT has strong
research support
o Only CBT has been studied for lots of disorders
• Assignment from Tuesday on Wonderlich article:
o What mechanisms did ICAT try and target?
▪ Self-directed behavior styles
▪ Interpersonal problems
▪ Self-discrepancy and evaluative standards
o Why did they include people with subthreshold diagnosis?
▪ They’e ey siila
▪ Increases the generalizability (external validity)
o What comorbid diagnosis were excluded and why?
find more resources at oneclass.com
find more resources at oneclass.com
