PSYC19H3 Chapter Notes - Chapter 2: Open Access Journal, Replication Crisis, Self-Control

22 views5 pages
21 Jun 2018
School
Department
Course
Replication Crisis & Importance of Self Control
Replication crisis – inability/difficulty in replicating results of many scientific studies
oCannot replicate the findings of experiments independently
What if results were by chance; questionable research practice
oVery odd, hints that something Is awry because how do you know results are significant
Priming elderly concepts affects walking speed by John Barg 1998
oStudents looked at sentences with words associated with elderly (slow, wrinkles, grey)
oThen measured how long they took to walk down hall
Those primed with stereotype walked more slowly than those who were not
oTried to replicate this in 2000s, but could not, only if had profound that affected
Darrel Bem 2011 published paper in JPSP with experiments showing that ESP existed
oVery weak evidence of ESP, but was still published in top journal
What other bullshit stuff could be published?
oHardly anyone conducted replications on experiments (no one verifies)
oReplications were hard to publish in journals besides open access journal
oMany “significant” findings had low statistical power
Ability to detect effect when it is real
Men on average taller than women // need many participants to test, not 2-5
Abusing Experimenter Degrees of Freedom (P Hack) – make impossible, possible
oUnder-powered designs // very small participant population used
oOptional stopping // analyze data until reach significant value (P values)
Stop only when data is what they want it to be
Optional starting: start over experiments again & again until get data wanted
oDropping conditions // dropping some original conditions to get significant values
Just dropping conditions out of experiment; data determines analysis
oDropping dependent variables // selective reporting & flexibility in operationalizing
oDropping participants // dropping selective people who will affect data
find more resources at oneclass.com
find more resources at oneclass.com
Unlock document

This preview shows pages 1-2 of the document.
Unlock all 5 pages and 3 million more documents.

Already have an account? Log in
Dropping the shortest woman and the tallest man in the sample
oUse of exploratory moderators // include other moderators just to find significance
oUse of exploratory covariates // to inflate alpha
The way you conduct experiment and variables will influence many data points of experiment
oP Hacks change alpha levels drastically depending on how variables are experimented
oPeople who listen to “I’m 64” became a year younger than those who listen to kalimba
…. But nothing will make you younger, time only foes forward…
oMany studies that seem possible may have been influenced by P Hacks
Publication Bias – the published literature is not the complete literature
oOnly a portion of what they studied; publish the biased sample of experiment
Don’t publish the failure findings, only the significant ones
But null findings are important to understand how robust findings are
oJournals very rarely publish null results, but they should more often
Attempt to replicate 100 studies from high-impact psychology journals
oOnly 39% could be replicated, only 25% were social psychology studies
oMay not be representative sample since only some studies, not all
Hard to interrupt numbers, just know it’s a problem
o100% replication may also be odd; because no risks in experiments
Need mistakes in experiments and new ways to experiment = won’t be 100%
Want innovations and creativity; maybe 85% region
Few failed replications here and there may be okay
oSome fields/experiments are so solid that some failed are ignored
oReplications only test robustness of one study
Meta-analysis helps to view robustness of all experiments of that field
But publication bias on meta-analysis can be problematic/make meaningless
Funnel plots to spot problems // study size vs power of study
2
find more resources at oneclass.com
find more resources at oneclass.com
Unlock document

This preview shows pages 1-2 of the document.
Unlock all 5 pages and 3 million more documents.

Already have an account? Log in

Document Summary

Replication crisis inability/difficulty in replicating results of many scientific studies: cannot replicate the findings of experiments independently. What if results were by chance; questionable research practice: very odd, hints that something is awry because how do you know results are significant. Priming elderly concepts affects walking speed by john barg 1998: students looked at sentences with words associated with elderly (slow, wrinkles, grey, then measured how long they took to walk down hall. Those primed with stereotype walked more slowly than those who were not: tried to replicate this in 2000s, but could not, only if had profound that affected. Darrel bem 2011 published paper in jpsp with experiments showing that esp existed: very weak evidence of esp, but was still published in top journal. What other bullshit stuff could be published: hardly anyone conducted replications on experiments (no one verifies, replications were hard to publish in journals besides open access journal, many significant findings had low statistical power.

Get access

Grade+20% off
$8 USD/m$10 USD/m
Billed $96 USD annually
Grade+
Homework Help
Study Guides
Textbook Solutions
Class Notes
Textbook Notes
Booster Class
40 Verified Answers
Class+
$8 USD/m
Billed $96 USD annually
Class+
Homework Help
Study Guides
Textbook Solutions
Class Notes
Textbook Notes
Booster Class
30 Verified Answers

Related Documents