Textbook Notes (362,870)
Canada (158,081)
Psychology (998)
PSYCH 291 (32)
Chapter 5

Chapter 5.docx

6 Pages
Unlock Document

University of Waterloo
Richard Eibach

Chapter 5: How do we misinterpret? Common Threats to Validity - Any threat to the validity of a research finding can be placed under one of three broad categories - These categories include the ideas that (1) people are different, (2) people change, and (3) the process of studying people changes people People are different Individual Differences and “Third Variables” - People differ in how much they appreciate the implications of being different - The type of research claim that is most susceptible to this threat is a pseudo-experiment or false experiment - It is a research design in which someone tests a claim about a variable by exposing people to the variable of interest and noting that these people feel, think, or behave as expected - Individual differences provide a very good alternate explanation for the findings observed in pseudo-experiments, especially those involving a small number of research participants - Individual differences can also undermine careful, systematic, scientific observation - E.g tattoos and death rates p131 Selection Bias and Nonresponse Bias - The examples above indicate that individual differences create a potential threat to the internal validity of many research studies - Individual differences can also be threat to the external validity of many research studies - Sampling people from an unrepresentative sample is referred to as selection bias - E.g. U.S. Presidential election - Why were the polls wrong? Because they made use of biased sampling techniques - They sampled their pool of potential voters - Another important form of bias that is likely to have played a role in the Literary Digest error is nonresponse bias - Nonresponse bias is closely related to selection bias except that in the case of nonresponse bias, the respondent themselves are the source of the bias - Surveys that have low response rates may yield information that is highly misleading People Change p134 - Differences in the same person across time and situations can sometimes do the same History and Maturation - Each represent common threats to internal validity - History refers to changes that occur more or less across the board in a very large group of people such as a nation or culture - In contrast, maturation refers to the specific developmental or experiential changes that occur in a particular person, or a particular age cohort, over time - Unlike individual differences, history and maturation still pose serious problems even when a researcher attempts to control for individual differences by assessing the responses fo the same person before and after a treatment - E.g. improving people’s dancing ability by using mental imagery - Researchers can conduct a true experiment in which they randomly assign a dancer to either a mental imagery condition or a control condition - Learning and maturation would almost certainly happen, but they should happen equally to dancers randomly assigned to each of the two conditions - Where threats to validity are concerned, the problems of history and maturation can also refer to relatively short-terms changes in a person’s physical or psychological state – changes that may occur over a couple of weeks, or even a few minutes, rather than a few months - Example of relatively short-term “maturation” comes from an episode of The Beverly Hillbillies Regression toward the Mean - Closely related to maturation also represents a threat to internal validity - This threat to validity is regression toward the mean: the tendency for people who receive high or low scores on a particular measure to score closer to the mean on a subsequent testing - This movement can sometimes cause researchers to misinterpret their data - Changes that are mistaken for experimental treatments occur when research participants do not experience any true changes (such as maturation) but nonetheless score differently than they did originally when they take a retest of a particular measure of personality or performance - E.g. pilots belief about the effects of rewards and punishments on performance - Punishment improves performance, and reinforcement causes drop in performance - In both cases, the changes in the pilots’ behaviour can be readily explained by regression toward the mean - The kinds of studies that are most likely to be influenced by regression toward the mean are pretest-posttest studies that do not involve a control group - If people who are left to their own devices improve more than people who are treated, it would be better to leave people to their own devices The process of studying people changes people p139 - Researchers have dubbed this effect (the increases in productivity that occur when workers know when they are being studied) the Hawthorne effect - Research in psychology suggests that the act of studying people can dramatically change the way that people behave - One of the most dramatic instances we know of is called the mere measurement effect - This effect refers to the tendency for participants to change their behaviour simply because they have been asked how they will act in the future (e.g. asked if they are going to exercise after this study, and if they say yes, and we check back with them tomorrow, we will learn that they did exercise after the study) - Research suggest clearly that your chances of exercising in this situation would be higher than usual, simply because we asked you if you intended to exercise - It is entirely possible that many studies have the unintended consequence of changing research participants’ behaviour in some small way - The mere measurement effect thus speaks to the external validity of research Testing effects - Poses more of a threat to internal than to external validity - This threat to internal validity, called testing effects, is a problem in pretest-posttest designs that have no control group - Testing effects refers to the tendency for most participants to perform better on a test or personality measure the second time they take it - Testing effects may masquerade as treatment effects anytime a researcher tests a group of participants twice without including a pretested control group - Testing effects can even occur for people who score at or above the mean on a psychological measure - Testing effects and regression toward the mean can both occur even in the same study or performance situation (figure 5.2 p142) - To correct the problem of testing effects – is to conduct a true experiment with a pretested control group - Or conduct a true experiment where you eliminate the pretest in order to obtain a posttreatment score that is not in any way influenced by a pretest Experimental Mortality (Attrition) p143 - The failure of some of the participants in an experiment to complete the study is known as experimental mortality, or attrition - Depending on exactly what form it takes, this problem can represent a threat to internal validity, a threat to external validity, or a threat to both - One troublesome form of experimental mortality happens in experiments or longitudinal studies that take a very long time to complete - E.g. 40 participant study for implicit learning - It is noteworthy that the proportion of people who dropped out of the study (the attrition rate) is exactly the same in the two different conditions - Studies in which there is an equal level of attrition across all of the experimental conditions suffer from simple or homogeneous attrition - Any individual differences associated with attrition should thus be constant in the two different conditions - It means that the high attrition rate observed in the study is predominantly a threat to external rather than internal validity - A second form of mortality does represent a serious threat to internal validity - This more troublesome form of mortality is known as differential or heterogeneous attrition - Heterogeneous attrition occurs when the attrition rate in two or more conditions of an experiment are noticeably different - The differential mortality rates erase all the normal benefits of random assignment - After heterogeneous attrition, the people in one condition of the study are systematically different from the people in some other condition of the study - What can be done to reduce or eliminate problems associated with morta
More Less

Related notes for PSYCH 291

Log In


Don't have an account?

Join OneClass

Access over 10 million pages of study
documents for 1.3 million courses.

Sign up

Join to view


By registering, I agree to the Terms and Privacy Policies
Already have an account?
Just a few more details

So we can recommend you notes for your school.

Reset Password

Please enter below the email address you registered with and we will send you a link to reset your password.

Add your courses

Get notes from the top students in your class.