POLS 3650 Lecture 2: Pols 3650 l-2
24 Jan 2019
School
Department
Course
Professor
Pols 3650 (L-2)
Main Dangers in Statistical Research
Introduction
• Statistics are numerical summaries of the data under study
o Cumulative average of all grades in your courses
• They are the main tool in quantitative data analysis
• While statistics can teach us much, there are at least 5 reasons we need to treat all statistical
research with caution
Problem 1: Measurement Error
• Measurement error: assigning an incorrect value
o Not measuring what we are looking for in the study
• Random measurement error is considered less problematic than systematic measurement error or
measurement bias
o Random error: can go in any direction
o Systematic error: always under estimating or over estimating a
• Most important sources of measurement error:
o Simple human mistakes
o Poor operationalization
• Measuring certain concepts with poor indicators
• Measurement error is not always easy to avoid and the potential may just have to be recognized
o Difficult to understand overrepresentation
• Spike in sexual abuse in the workplace may be because of the number of cases may be
going up or the reporting of cases may just be going up – measurement error
• Always investigate whether the indicators match the concepts that the study draws conclusions
about
• In survey research, ask yourself:
o The concepts are operationalized in survey questions
o Would I answer the question in the same way the researchers interpret the answers?
• Ex. Getting asked if you would recommend a bank - trying to see how the experience
was at the bank (measurement error)
o How is the question worded?
• And how is the order of the questions
o In which order have the questions been posed?
o Do respondents have a reason to give a dishonest answer? (is there social desirability bias?)
• People will systematically give the answer that is the most socially acceptable answer
o Are respondents able to answer the question? (could the answers be non-attitudes?)
• People will answer questions about something they do not know about or something
that is not true
Problem 2: Sampling Error
• In inferential statistics, our goal is to generalize from a sample to a population
Document Summary
In survey research, ask yourself: the concepts are operationalized in survey questions, would i answer the question in the same way the researchers interpret the answers, ex. Getting asked if you would recommend a bank - trying to see how the experience was at the bank (measurement error: how is the question worded, and how is the order of the questions. Sampling in the wrong area: don"t rely on random chance, sampling frame underrepresents portion of the population. Samples over the telephone directory became problematic with younger generations (will only get people over 50) In survey research: low response rate: the people who are answering your question are systematically different from those who do not. Winner take all politics hacker 2010: points out that survey data was used opt measure income distribution, the problem is that the rich will not be included in the sample. Problem 3: incorrect inferences: overgeneralization, do not generalize beyond your population of study.
