January 27, 2014
50 MC and T/F questions
3 short answer (choose from 5)
1.5 hours to complete
Bring student ID
Ch 1, 2, 3, 4 (emphasis will be on what we cover in class today)
We will break and come back to discuss assignment #1
Step 9: Analysis
- big step! Covered in some detail later in this course and much in stats
- choose statistical techniques and test relationships between variables
- check for validity and reliability issues – moving onto some of these now
Reliability always concerned with the consistency of measures of a concept, yet several
1. Stability over time:
- consistency in a single measure over time (assuming no change in what is being measured)
- the bathroom scale example
Asking income over time is terrible in terms of reliability because people often guess. Even
though consistent over time, give us different answers.
2. Internal reliability:
- consistency in measurement when using multiple indicators to measure the same concept of a
single point in time
- how to test?
Cronbach’s alpha coefficient > .80 – there is an underlying concept and each of the things you
chose to relate it to does relate
Split-half method – randomly split them in half and do same test. Do we get the same answers,
how close are they, how far apart are they? Splitting them and comparing them
3. Inter-observer reliability
- consistency in measurement across researchers
Ex. Coding open ended questions in a questionnaire or coding in content analysis
Some researchers might interpret was people are saying. When people code things differently,
end up with different interpretations for the researcher (likely has a team of researcher and they
all may code differently) Everyone has to do the same thing. Everyone will train together,
Content analysis – look at the stuff in peoples lives. TV shows maybe evidence male dominance over yng women everytime you see that check a box. Diff. Researchers would have a different
idea of that. If everyone has a different idea, we end up with inter-observer reliability issues
There are various kinds of measurement validity:
Face validity: Does the measure appear ‘on the face of it’ to be valid?
- sort of about common sense
- do other researchers agree?
- much easier to establish with more straight forward concepts
Concurrent validity: does the measure correlate to another measure that is also relevant to the
Ex. Research on ‘job satisfaction’
- ask people to rate how satisfied they are with their jobs (our measure)
- find a measure related to job satisfaction – maybe absenteeism
- do workers who score low on absenteeism score high on job satisfaction?
- if yes, concurrent validity!
Simple comparison. In most case, most of these are all simple comparison.
Construct validity: concepts relate to each other in a way that is consistent with the
Ex. Theory states that child ‘hyperactivity’ is caused by ‘poor parenting’
- we see that ‘hyperactivity’ increases as parenting ‘gets worse’
- we thus suggest that the measures used to gauge ‘poor parenting’ and ‘hyperactivity’ are valid
Using measures to check theory than using theory to check measures that are valid. Are
checking something we don’t know about to check something that we don’t know about.
N.B. Must be very careful with this!! – it might not be non-valid measures. It might be real
differences between the theory and observation
Convergent validity: a measure of a concept correlates with a second measure of the same
concept that uses a different measurement technique
Ex. Stats Canada
- survey based measures of income (asks respondents how much they make)
- info from Revenue Canada on income – if averages are close, measures are considered valid.
Also not valid because people aren’t honest about what they make so not valid in that sense.
Steps 10, 11
Step 10: Findings/conclusions
- is your research question answered or your hypothesis supported?
- are there implications for theory, for social life or social policy?
Step 11: Write-up findings/conclusions
- publish it and let others judge the quality/usefulness of your work, potentially replicate it etc. Critique of Quantitative Methods
Critique: Humans and their actions are fundamentally different from what is studied in the
natural sciences – we shouldn’t be studying them in the same way
Counter: Humans are a (very complex, unique) part of nature – science can and should be used
to understand the human condition
- including consciousness, emotions, meaning, etc.
The measurement process produces an artificial and false sense of precision and accuracy.
Specific, getting nominal definitions and then measurements exact but we are all still people so
as a result end up interpreting all the things we’ve detailed a bit different. Don’t have the
precision we thought we had.
Ex. Problems can arise if ppl interpret the same survey item differently
How is your general health?