March 3.docx

March 3, 2014 Structured Observation Is this method for you? 1. Is the topic sensitive or prone to social desirability bias? People want to be politically appropriate, may mislead you to seem appropriate. If yes, maybe SO is for you Ex. Kids don’t admit to bullying on surveys, but we can see bullying if we watch kids in their normal enviro. (kids much more likely to act naturally when being watched so can see this kind of behaviour) 2. Can you actually observe the phenomena? - are you trying to observe sex, crime or other “hidden” topic? Ex. – may be able to observe shoplifting, but murder? To observe shoplifting likely. To observe murder, likely rare. Almost impossible. Murder – more likely to go to a prison. - may be able to observe flirting, but affairs? Can’t observe and chances of happening to be around an affair, unlikely. Unlikely see couple and know they are having an affair or just a regular couple. 3. Do you have a lot of time? - Can be very time consuming Often requires making repeat observations over an extended period, across many different settings 4. Do you want to generalize your findings? - these studies are usually rather small scale and specific, external validity is often a problem. External validity (I think) usually very low. Advantages over Survey Research - Major problem in survey research: Gap between stated and actual behaviour 1. Social desirability effect/threatening questions esp with sensitive questions ex. Children almost never admit to bullying when asked, only being bullied Acts of ethnic/religious/gender discrimination all understand – these people won’t admit to things. 2. Problems of memory: especially everyday behaviours – people trying to be honest, but we’re just not very good at memorizing things. - How many texts do you send in a day? - How many hours of TV do you watch? Ppl don’t realize that amnt of time they watch tv adds up Avoid these problems by not asking just watching, counting, etc. 3. Problems of meaning 4. Ex. From “asking questions” lecture - do you refularly take sick days - how many time a wk do you exercise Now a problem with SO because researcher defines these terms and applies them Power relationship is worse for this. Researcher making more decisions. SO can bridge the gap ... - Often more reliable measures of events and behaviours - Greater precision and accuracy regarding Duration and frequency of events and behaviours Time-ordering events and behaviours Reliability Problems: Inter-observer reliability Do different observers view and code the same behaviours the same way? May have a different idea of what constitutes a certain beh or what that might look like Intra-observer reliability Does an observer change the way they record behaviours over time or between participants? Context change – behaviours look different in different contexts. Settings might change how things, behaviours whatever look like Observer fatigue – People get tired looking at the same things. After a while start to tune out, interpret differently. N.B. Same as inter and intra interviewer reliability. In this case about interpreting observation schedule or something How to test? Cohen’s kappa – measure of agreement between observers or observations – level of agreement. 1= perfect agreement 0 = no agreement Validity Measurement Validity: - same as with surveys, experiments, etc. Does this measure what you want to measures? - Ex: What behaviours constitute racial discrim on a playground? With kids hard to figure out when being jerks because kids or because they are jerks - Using derogatory names? - Social rudeness? - Something more subtle? - something more subtle? Is the schedule standardized and administered correctly each time? Unreliable measures can’t be valid Reactive effects People change their behaviour when they know they are being studied/ tested Guinea Pig Effect (Hawthorne effect): Classic Example: Hawthorne Works (1924-32) Trying to test methods to increase productivity in American factors – what small changes could they make without having to change whole factory - Test 1 increase lighting Found greater productivity - Test 2 decrease lighting Greater productivity So they tried different things... - Many other tests with similar results Increased productivity ended with the end of experiments. Why? Participants knew they were being tested and whenever researchers were there, workers tried their best Specific Guinea Pig effects: 1. Role selection People choose appropriate roles for themselves in most situations - These can also be specific to the role of participant. 2. Trying to help the researcher Guessing what the research is about and trying to do what the observer would like Especially bad if there is more than one observer – may convey different clues, get different responses Only a problem if participants know: - they are participating in a study - they know you are watching them But there is also 3. Researcher presence as a change agent Even if people don’t know they are being watched, researcher just being there changes situation in some way. Even if nobody knows you are a researcher or that they are being studied - you are taking someone else’s spot on the bus – if you, the researcher, sitting on bus in someone elses regular spot. Just you, a stranger on the scene enough for people to think you are weird. - You have to be invisible to avoid this. Reactive Effects - SO almost can’t avoid effect 3 - But can use unobtrusive (non-reactive) measures to avoid effect 1, and 2. - Especially Covert observation Participants don’t know they are being watched Don’t know that they are part of the study Sound like there might be some ethical issues here? Covert observation  When okay to secretly watch someone When is it okay? When behaviours are: - Public - non-obtrustive Ex. Watching people text – because everyone else is doing it to, no harm When participation is - Harmless - No threat to privacy (anonymity/confidentiality is protected) - When ‘potential gains outweigh potential risks’ -When there is ‘no other means of gaining the same information’ may consider covert operation - Participant is a volunteer, and had a right to Refuse to participate Give informed consent – must detail to participants what the research is about, its purpose, risks to participants and threats to privacy - Obviously must violate this for CO! Fall-back ethically for CO: - Disclo