Lecture 12, External validity Errors
Threats to external validity
•Population and ecological setting. Non representative sample, or lab is too
conﬁned and doesn’t transfer to everyday life "
•Pretesting sensitization: when the researcher says something like “thats
interesting”, making you think you are unique in some way, changing your results "
•Hawthorne eﬀect: The novelty of you being in an experiment or study makes you
perform better. Just because you’re involved in the study you feel like you want to do
your best because you’re excited "
•Over generalizing: you might measure reaction time, and state that people who have
good reaction time, also have good movement time. You can’t make that
generalization because you didn’t measure movement time "
•Expectancy (placebo eﬀect): if you believe something is going to work, it is
probably going to work whether its actually good or not because that is what you
•Post hoc error: you assume a false cause and eﬀect relationship "
•Example of this: the Japanese eat little fat in their diet, and have less heart attacks
than Americans. Italians eat lots of fat and have less heart attacks than Americans.
The post hoc error would be saying “eat and drink what you want, its speaking
English that kills you” "
Reliability: refers to the consistency and repeatability of the data. If i measure a desk
one day, it will be the same length another day, so it is reliable "
validity: is what you measured what you thought you measured? How good was your
•anytime you make a measurement there will always be some error "
•any measurement has 2 parts. Xt (true component) and Xe (error component) "
•So the formula is Xo=Xt+Xe "
•Xo is the observed score, Xt id the true score, and Xe is the measurement error "
To deal with this error, you do test and retest "
•you then ﬁnd a correlation between the two tests, to see how reliable your results are "
The other way you can do this is using the split half method "
•Look at quiz 1 for example "
•You would expect quiz 1 pink and yellow to have the same averages because they
are both the same test just rearranged "
•small sample, big population "
•easiest way to do this is with 2 groups, 1 control and 1 experimental. "
•Referred to as an RCT "