January 20, 2014
Really know reliability, validity and replicability
Social scientists don’t use experiments very much.
Case study – classic qualitative. Pick one thing, study in depth
Longitudinal – in depth, over time
The traditional means of ‘doing’ science
- Quantitative only
Rare in soc. Because interested in long term things and things we can’t change about people –
sexual orientation, gender, income, race
Because involve manip of ppl aware that it may cause serious harm to the people
Interested in the way something impacts someone’s life.
- Experimental or treatment group: receives a treatment or manipulation of some kind
- Control group: does not get the treatment or manipulation
- Random assignment: participants are placed in the experimental or control group using a
- Pre-test: measurement of the dependent variable before the experimental manipulation
- Post-test: measurement of the dependent variable after the experimental manipulation. After
we manipulated, should see differences between them if worked out.
(Showed classical experimental design – in text)
Spurters exp. – wanted to look at teachers expectations and childrens development. Teachers
expectations have a lot to do with development. Randomly divided into groups.
Exp – spurters, control – non-spurters. Teachers expecting exp grp to develop faster. IQ test on
both grps of children to see no difference between the grps. Spent time watching the teachers.
Tested both children again and found what expected. Group teachers told children were going to
spurt ahead, did. Teachers expected them to develop better so their expectations created those
Two kinds of experiments:
Laboratory experiments: take place in artificial environments – milgram experiment. Every bit
of it controlled – the teacher, authoritarian figure, etc.
Field experiments: are conducted in real life surroundings – Rosenthal and Jacobson Spurters
– had no control over what teachers were doing, may have talked to parents + parents may hv
treated children differently which may hv accounted for results.
Normally very high in internal validity (best possibly method)
Threats to internal validity that are always possible:
Mortality (Attrition) – participants leave the experiment before it is over. Sometimes they don’t want to participate anymore.
Maturation – participants change over time – get older, develop mentally and emotionally, etc.
Missing some element of a true experiment
- ex. No control, no random assignment
- internal validity is harder to establish
Many kinds (your text highlights one):
Natural experiments: naturally occurring phenomena or changes introduced by people who
are not researchers result in experiment-like conditions
Ex. Alberta and Saskatchewan in the 30s and 40s
They were very similar in kinds of people, size of pop. One prov left wing govern the other
elected right. One oriented family values, other social labour movements – different political
ideas. Can look at the way these places changed over time based on political histories. Had no
way to randomly assign them or tell if they were really that similar at start.
Threats to internal validity that are worse if no control, or no random assignment:
History: Some event occurring after the treatment was given may have influenced the
dependent variable. Something that changes way people will respond to your manipulation. If
don’t have a control grp or random assign can’t tell if it was the thing that caused differences or
experiment. Control group is really important.
Testing: the pre-test may have influenced the dependent variable. Changes the way ppl will
think about the experiment. When you ask them questions early, think about the later questions
differently. Ex. Oh shit, this survey is about health and they are bad with health to they change
Instrumentation: changes in the way a test is administered may account for pre-test and post-
Give them a pretest and give them exp. Something on pretest you didn’t like, just the differences
between 2 tests. One online, one phone will all change way ppl answer your questions.
Selection: post-test differences between the control and exp grps may have b