COVERS CHAPTER 12, 13
- Annotated answers are released
- basically these are brief explanations that come w/ ans's to the correct choice from the
THINGS THAT CAN GO WRONG WITH SAMPLE SURVEYS
- not getting Who you want
- ie. the ppl you picked out at random using SRS, or SS, or any sort of random sampling technique
- you are calling a person picked at random from your sampling frame and he either
- refuses to ans. phone
- picks up phone, but refuses to ans. survey q's
- this is an issue b/c when you randomly sample the person to be in sample, you don't want to replace
them w/ someone else.
- call back
- call back certain number of times at the home of the randomly selected household, until you
find someone at home, or someone that is willing to ans.
2. GETTING THE QUESTION(S) RIGHT
- you are doing phone survey about whether or not ppl agree with funding for police and you ask your
(randomly selected individuals) q's like: "police funding costs a lot of money for the government, what do you think about funding? Should it be
more or less?"
"this society wants a safer place, so what do you think about increasing funds for police?"
^- above q's are making ppl ans. one way or another (ie. introducing bias)
- suppose were funding for police, and issue: do you think should be more or less, or amt is right
- you call someone on phone and you say: "police funding costs us a lot of funding, what do u
think of funding about police?" prejudicing about police
- society wants to be safer place, how about incr'ing funding for police? makes ppl ans. one
way or another (introduces bias)
- Avoid depicting, or suggesting a preference for a certain answer via the way the question is asked
- You want to know what people really think, and if you, with the way you ask the question, tend to
favour one answer over another, than they will answer in the way they think you want to hear it
- so really, you are not getting THEIR true opinion
=> have to be as neutral as possible when asking the q.
3. NOT GIVING CHOICES FOR ANSWER
Problem with open-ended responses
- what is open-ended? When you ask them a question, and instead of giving them choices to choose you
_________________________ (ie. let them give their type of answer, and not from a set of choices, as in the "Agree, Disagree" case)
- Problem: inviting them to say what they think makes it harder to make sense of results when you need
to analyze them
- Ask Q's with choices
- ex. Agree, Disagree, Don't Know
- this is one type of clear-cut strategy for getting responses that we can retrieve analysis with relatively
more ease than compared to open-ended:
- ex. this % of ppl agreed, this % disagreed etc.
=> relatively easier to analyze results when you ask ppl question that is clear-cut, in a neutral manner
and YOU give them the possible choices they can select from
- it is difficult to get the answer to in a survey: why people hold the views they do
- there could be multitude of diff. reasons, so difficult to summarize
3. SAMPLING VOLUNTEERS
- the aim is to randomly choose who is in sample
- but instead if you have people who CHOOSE to respond be part of your picked individuals for sampling,
then you have no way of knowing whether they are giving you a trustworthy depiction of the popn
- this is NOT randomly sampling (ie. by using volunteers)
=> ex. don't want to pick ppl JUST those ppl who are listening to show at this time, willing to call in
Benefit of Random sampling - can potentially generalize results from sample to popn in predictable manner
- can make a conclusion like "based on the survey, we think the popn is like this _____"
- the likelihood of being right is high
- Recall: in sampling, the goal is not merely to learn sth about sample, but rather the rest of the world
- this cannot be done with volunteers => you cannot trust the results you retrieve from them
4. SAMPLING BADLY BUT CONVENIENTLY
- this is similar to volunteer issue 3.
- just b/c it is easy to take a certain sample (ex. first 50 ppl to walk into a shopping mall), does NOT mean
that it will give you good results
=> random sampling is the way to go if you want to find any sort of trustworthy results
- unable to sample certain parts of popn
if doing phone survey, then may miss out ppl, despite using the phonebook or directory b/c either:
a) don't have a phone (use cellphone instead)
b) unlisted in directory/phonebook
Resolution: do random digit calling (resolving (b))
- Problem: will miss out ppl who don't have a phone at all! (a)
- if doing a Census, you need to get input from everyone in the location you are getting it for
- ex. the GTA => for this example, you would have considerable difficulty finding the individuals in ur sample who are
homeless, or may not be able to find them at all
- problems will arise if you can't sample certain parts of popn b/c sampling design will not let you do
CHAPTER 13: EXPERIMENTS AND OBSERVATIONAL STUDIES
Example (we will be referring to this for the next couple of slides)
Exercise and Imsomnia
We are assessing relationship b/ween exercise and imsomnia
How do you find out if exercise helps imsomnia?
- look at random group of ppl, &
- find out if they exercise and how much, and
- tell them to rate their imsomnia
- can usually only make conclusions of association, NOT causation from findings from observational
- ex. we found from the ppl who we asked that those who exercise more suffer less from
- We can not be certain to affirm that ppl who suffer from imsomnia should be
recommended to exercise.
- assesses association but NOT causation
- analogous to concept of correlation; assesses association of linear relationship b/ween 2
qvar's, but does NOT imply causation if its value comes out to be high.
=> detects assoc, but not WHY it is (ie. its reason) (ie. 2 var's are assoc why is this)
- can give suggestive results
- ways of going into re'srch into future
- doing this form of study is like sampling volunteers in that you cannot trust drawing out wide-ranging
conclusions from the study
- commonly used
- could help identify var's that have an effect
- but not guaranteed to identify most impt ones
- ex. could identify var's that have effect on others, but not necessarily telling us which one has
the most profound effect.
=> basically just giving us an idea of what may be interesting to look into further in a more controlled
2 Particular Kinds of Observational Studies
RETROSPECTIVE STUDY PROSPECTIVE STUDY
"looking back" "looking forward"
(ex) Exercise & Insom (ex) Exercise & Insom
- look over historical records of patients who - you gather a random bunch of ppl, assess how
suffered from insomnia, and see what their much they exercise and how much imsomnia they
exercise patterns were like claim to be suffering from
=> collect records of ppl who been thro what ur >- so you could ask them once, and then ask for
interested in studying, and see what was recorded their opinions sometime in future
to have happened to them Advantage: relatively quicker, and easier to Advantage: easy to control confounding, bias
retrieve results than PROSPECTIVE, just by (likely b/c YOU are the one administrating the
examining the records assessment)
Disadvantage: concern over confounding, bias that - could be difficult to find subjects of interest (ex.
may have been present that could've potentially patients who underwent surgery for a particular
affected this past data. region of brain may be a minority)
- more slower
=> could take time having to gather up subj's, and
getting enough to avoid getting replies suggesting
it was due to chance
(think how long it would take to gather sufficient
# of randomly selected individuals for the brain
EXPERIMENTS (in the statistical context)
Example: Exercise and Insom:
How do we establish cause-and-effect?
- randomly select subj's into two groups
- one group will exercise, and other group will NOT
- then afterwards, assess insomnia for ALL subj's
Why is this better? How does it level out effects of other variables?
- if randomly chosen groups, then => groups start out relatively equal in terms of ath that may be impt
- so in the beginning, when we randomize, we are assuming that this process made the groups
be relatively equal in EVERYTHING that we are measuring, and so if anything would make a significant
DIFFERENCE to the group, it would be our form of manipulation
- ie. exercise or no exercise By doing experiments,