March 3, 2014
Is this method for you?
1. Is the topic sensitive or prone to social desirability bias? People want to be politically
appropriate, may mislead you to seem appropriate.
If yes, maybe SO is for you
Ex. Kids don’t admit to bullying on surveys, but we can see bullying if we watch kids in
their normal enviro. (kids much more likely to act naturally when being watched so can
see this kind of behaviour)
2. Can you actually observe the phenomena?
- are you trying to observe sex, crime or other “hidden” topic?
Ex. – may be able to observe shoplifting, but murder? To observe shoplifting likely. To
observe murder, likely rare. Almost impossible. Murder – more likely to go to a prison.
- may be able to observe flirting, but affairs? Can’t observe and chances of happening to
be around an affair, unlikely. Unlikely see couple and know they are having an affair or
just a regular couple.
3. Do you have a lot of time?
- Can be very time consuming
Often requires making repeat observations over an extended period, across many
4. Do you want to generalize your findings?
- these studies are usually rather small scale and specific, external validity is often a
problem. External validity (I think) usually very low.
Advantages over Survey Research
- Major problem in survey research: Gap between stated and actual behaviour
1. Social desirability effect/threatening questions esp with sensitive questions
ex. Children almost never admit to bullying when asked, only being bullied
Acts of ethnic/religious/gender discrimination all understand – these people won’t
admit to things.
2. Problems of memory: especially everyday behaviours – people trying to be honest,
but we’re just not very good at memorizing things.
- How many texts do you send in a day?
- How many hours of TV do you watch? Ppl don’t realize that amnt of time they
watch tv adds up Avoid these problems by not asking just watching, counting, etc.
3. Problems of meaning
Ex. From “asking questions” lecture
- do you refularly take sick days
- how many time a wk do you exercise
Now a problem with SO because researcher defines these terms and applies them
Power relationship is worse for this. Researcher making more decisions.
SO can bridge the gap ...
- Often more reliable measures of events and behaviours
- Greater precision and accuracy regarding
Duration and frequency of events and behaviours
Time-ordering events and behaviours
Do different observers view and code the same behaviours the same way?
May have a different idea of what constitutes a certain beh or what that might look like
Does an observer change the way they record behaviours over time or between participants?
Context change – behaviours look different in different contexts. Settings might change how
things, behaviours whatever look like
Observer fatigue – People get tired looking at the same things. After a while start to tune out,
N.B. Same as inter and intra interviewer reliability. In this case about interpreting observation
schedule or something
How to test?
Cohen’s kappa – measure of agreement between observers or observations – level of
1= perfect agreement 0 = no agreement
- same as with surveys, experiments, etc.
Does this measure what you want to measures? - Ex: What behaviours constitute racial discrim on a playground? With kids hard to figure out
when being jerks because kids or because they are jerks
- Using derogatory names?
- Social rudeness?
- Something more subtle?
- something more subtle?
Is the schedule standardized and administered correctly each time?
Unreliable measures can’t be valid
People change their behaviour when they know they are being studied/ tested
Guinea Pig Effect (Hawthorne effect):
Classic Example: Hawthorne Works (1924-32)
Trying to test methods to increase productivity in American factors – what small changes could
they make without having to change whole factory
- Test 1 increase lighting
Found greater productivity
- Test 2 decrease lighting
So they tried different things...
- Many other tests with similar results
Increased productivity ended with the end of experiments.
Why? Participants knew they were being tested and whenever researchers were there, workers
tried their best
Specific Guinea Pig effects:
1. Role selection
People choose appropriate roles for themselves in most situations
- These can also be specific to the role of participant.
2. Trying to help the researcher
Guessing what the research is about and trying to do what the observer would like
Especially bad if there is more than one observer – may convey different clues, get different
Only a problem if participants know:
- they are participating in a study
- they know you are watching them
But there is also
3. Researcher presence as a change agent
Even if people don’t know they are being watched, researcher just being there changes situation
in some way.
Even if nobody knows you are a researcher or that they are being studied - you are taking someone else’s spot on the bus – if you, the researcher, sitting on bus in
someone elses regular spot. Just you, a stranger on the scene enough for people to think you
- You have to be invisible to avoid this.
- SO almost can’t avoid effect 3
- But can use unobtrusive (non-reactive) measures to avoid effect 1, and 2.
- Especially Covert observation
Participants don’t know they are being watched
Don’t know that they are part of the study
Sound like there might be some ethical issues here?
When okay to secretly watch someone
When is it okay?
When behaviours are:
Ex. Watching people text – because everyone else is doing it to, no harm
When participation is
- No threat to privacy (anonymity/confidentiality is protected)
- When ‘potential gains outweigh potential risks’
-When there is ‘no other means of gaining the same information’ may consider covert operation
- Participant is a volunteer, and had a right to
Refuse to participate
Give informed consent – must detail to participants what the research is about, its purpose, risks
to participants and threats to privacy
- Obviously must violate this for CO!
Fall-back ethically for CO: