Textbook Notes
(362,988)

Canada
(158,130)

York University
(12,352)

Psychology
(3,541)

PSYC 2030
(144)

Krista Phillips
(15)

Chapter 9

#
Reading Notes - Chapter 9.
Premium

Unlock Document

York University

Psychology

PSYC 2030

Krista Phillips

Fall

Description

Study Guide – Chapter 9: Survey Research and Subject Recruitment
What are Opportunity and Probability Samples?
here we examine logic and limitations of the methods used to select research participants
Donald Rubin noted all studies lie on a continuum from irrelevant to relevant with respect to
answering a question
Randomized laboratory-type experiments that use opportunity samples of the first available
students in college settings have a restricted sample of participants but usually have a high
degree of control over the variables of interest
By contrast, researchers who do survey studies select potential respondents using special
sampling procedures in order to generalize their descriptive findsings to a specific larger pool (a
population) of people
If survey researchers used opprunity samples, spurious results and misleading generalizations
about the specific population of interest would seriousluy compromise scientific integrity of
their work
There is wide range of topics of interest to survey researchers
Pollsters use survey designs to map out some specified population’s opinions on important
societal issues such as the community’s fears of crime or its choice of political candidates
Similar methods are used sometimes in epidemiological research, forensic research, economic
research, and many other areas in which scientific surveys are conduction
When health officials wanted to find out about national trends in cases of tuberculosis
contracted on the job, they did scientific surveys of hospitals to count employees reported to
have TB
o As federal courts became inundated with mass torts involving asbestos cases, one
solution was to sample asbestos cases from the larger pool within a court’s jurisdiction
o Asses damages in randomly chosen cases from each of five disease categories were then
applied to each larger pool
o More recently, when researchers wanted to study the prevalence of psychological
resilience after a traumatic event they chose a probability sample of New Yorkers to
survey in the 6 months following 9/11
o Researchers reported resilience was present in two thirds of the sample and never fell
belwo one third even among highly exposed individuals with posttraumatic stress
disorder
Instead of trying to question every member of the population which is usually impossible, this
type of research focuses on a segment (sample) that is believed to be a typical of the population
How can researchers be certain that the segment is representative or typical of the population?
Ex. How can they be sure that a specific form of data from a sample is representative of a pop?
They might compare the sample with the most recent census data but it is well known that
census data are problemative because it is impossible to contact every member of the
population
Researchers who use a sample can enver be 100% sure of their generalizations
They can make reasonable guess by first developing an accurate sampling frame that defines the
target population and then relying on a carefully designed blueprint (sampling plan) to select
sample means of probability sampling
Probability sampling – implies randomness enters into selection process at some stage so that
the laws of mathematical probability apply
Probability – refers to the mathematical chance of an event’s occurring o Ex. Likelihood of getting “heads” when you flip a coin once (1 in 2) or getting a 2 when
you throw a die once (1 in 6)
Survey studies can take many different forms, all using sampling plans in whicih osme method of
probability sampling determines the random selection of the households or people to be
contacted
These plans enable the researcher to assume reaspnably but with no 100% guarantee of being
correct
Practical problems may impose limits on representativeness of the sample
Even in most carefully conducted survey, not every household or person in the sample can be
reached and, of those who are actually contacted, not everyone will agree to be interview
In study of psychological resilience after the September 11, random digit-dialing approach was
used to contact members of the sample
When the number of completed and partial interviews was summed and this total was divided
by the sum of all numbers that were either eligible as residential phone numbers or of unkown
eligibility, the response rate was estimated to be 34%
What Is Meant By Bias and Instability in Survey Research?
Surey research done not only by private organziations, but by individual researchers owkring
alone or with ties to private organizations and in the US, at university based insitutes that can
implement face-to-face and telephone interviewing in national probability surveys
This research takes many different forms, all valid survey research is characterized by sampling
plans in wich every element or sampling unit in the population has a known nonzerio probability
of being selected at each draw
2 imporant statistical requirements of a probability sampling plan are a) the sample values are
unbiased and b) that there be stability in the samples
Unbiased – values produced by sample must, on average, coincide with the true values of the
population – but we can never actually be absolutely sure that this requirement has been met in
a given study unless we already know those values
Stability – means there is not much variability (or spread) in the sample values
o it’s estimated by statistical procedures such as variance and the standard deviation
sampling theory is beautiful because it can be applied not only to individual respondents but
also to teams in a population of teams or to products on an assembly line, or to any other
specified population of animate or inanimate units
bias = systematic error
print page 166 – there is a diagram and it’s above paragraph explains it
The Wine Taster
in manufacture of red wine, grapes are crushed and residue is put into huge cats in which
fermentation occurs
wine is then drawn off into barrels, where fermentation continues and product is periodficlly
sampled by the wine taster
wine taster needs to draw only a small sample in order to evaluate the quality of the wine in the
barrel
it’s the same in survey research: the more homogeneous the population, the smaller the sample
that needs to be drawn
Why Do We Not Know “For Sure” the Bias in Sampling? only way to know for sure about bias in sampling results is to examine every single member of
the popoulationa nd the sample at the same time the sampling is done – if pattern replies in
same exactly matched pattern in population, we would know there was no sampling bias in
surveryed sample
practically speaking, if we already knew how everyone in a population was – we wouldn’t be
doing a sample to help us study the population
it’s sometimes thought that election forecasting allows us to detect bias in sample because we
can compare predicted voting results with actual votes however problem in this case is that we
are comparing data obtained at one point in time with the results at another point in time
a well-designed and carefully implemented selection process involving probability sampling can
usually produce data that are remarkably close to the election results
polls = people can change their minds, they can be no shows, usually polls closer to election are
thought to be more accurate but still – polls right up to day of voting disagree
How Is Simple Random Sampling Done?
Simple random sampling – the basic prototype of probability sampling. The simple tells us that
the sample is selected from an undivided population, random means that the sample is to be
cjosen by a process that will give every unti in the specified population the same chance of being
selected at each draw
In order for this to occur, the selection of one unit must have no influence on the slection of the
other units
Assumption of random sampling: we have an understanding of the existence of all the units in
the population
Procedure: draw individuals one at a time until wehave as large a sample as we need
Process of selecting units might consist of computer drawing units at random, using a table of
random digits, or even spinning a roulette wheel or drawing well-mixed capsules from an urn.
Telephone interviewing: random digit dialing is sued to include households with unlisted
numbers – area code and first three digits can be selected according to the geographic area of
interest, and then a computer program is used to randomly select the last four digits
Famous case showing hazards of inadequate randomization: Vietnam War, use of random
lottery to select conscripts for armed forces, birthdays out of urns, layers went: January, feb,
march, april, may – even though urn was shaken for hours, the layers had formed and the
ballots came out in order
Use of table of random digits can help to avoid this
2250 digits list came from a million random digits that were generated by an electronic roulette
wheel programmed to produce a random frequency pulse every tiny fraction of a second
A computer then counted the frequency of 0s, 1s, 2s, and so on – on assumption that an
impartial probability method would produce an approximately equal number of 0s, 1,s 2,s and
so on in the overall table of a million random digits – this equality was confirmed
To see how you might use this table if you wanted to do random selection in a survey, imagine
you want to interview 10 men and 10 women individually after choosing them at random from a
list of 96 men and 99 women
You would begin by numbering the population of men consecutively from 01 to 96 and the
population of women from 01 to 00
You are not ready to use the random digits – to do so, you can put your finger blindly on a
starting position You can start anywhere in the table and then move your finger in any direction, as long as you
do not pck a set of numbers because they look right or avoid a set of numbers because they
don’t look right
Suppose you put your finger on the first give-digit number in row 5 column 1 (12807), you would
read aross the line two difits at a time, selecting the men numbered 12, 80, 79, 99 and so on
until you had randomly chosen the 10 male interviewees
You would do the same for the women but at a different, blindly chosen spot
If you had fewer than 10 persons on each list, you would read only one digit at a time, if you had
between 100-999 persons on your list, you would read three digits at a time
Suppose you chose the same two-digit number more than once, or suppose you chose a two-
digit number not represented by any member of the population
In either case ou would go on to the next two-difit number in the row (unless you were sampling
with replacement)
What if your population was so small that you were forced to skip many numbers in the table
because they were larger than the largest number of people in your population? Ex. 450 peope
in population and you want 50 people at random, you eould skip approx. half of the three digit
numbers in the section of the table you chose (those from 451-999). As a simple solution, you
can subtract 500 from any number in range from 501-999 and will result in fewer unusable
selections
Sampling with replacement – selected units are placed in the selection pool again and may be
reselected on subsequent draws, every unit in population continues to have the same
probability of being chosen every time a number is read
You must select units one at a time to use sampling with replacement. Ex. Suppose unit are days
of the year sealed in tiny capsules in an urn stirred so completely that there are no layers or
nonrandom clusters, you select a capsule read it and put it back in the urn, making sure urn is
well mixed
In sampling without replacement, previously selected unit cannot be reselected and the
population shrinks each time you remove a unit but all the units remaining have the same
likelihood of being drawn on the next occasion
If you scoop a handful of capsules, record each, and then discard those you picked – sampling
without replacement
Either option is acceptable but without replacement is more common because researchesrs do
not want to draw the same individuals twice or more
Another example of sampling w/o replacement is the wine taster, who draws then spits out a
sample of wine
What Are Stratified Random Sampling and Area Probability Sampling?
Simple random sampling is useful when population is known to be homogeneious or when its
precise composition is unknown.
When we know something about the exact composition, a more efficient method of sampling is
to sample from the different substrates of the population
Professional polling organizations typically use this approach to probability sampling, this is,
randomly selecting sampling units (persons or hosueholds), from several subpopoulations
(termed strata or clusters) Ex. If we know population is 60% female and 40% male, ratio of 3 to 2, and tha gender is a
pertinent variable, we can improve our sampling procedure by selecting subsamples
proportionate in size to this 3:2 ratio of females to males
Stratified random sampling = is what’s described above
separate sample is randomly selected from each homogeneous stratum or layer of the
population
the stratum means are then statistically weighted to form a combined estimate for the entire
population
in a survey of political opinions, for example it might be useful to stratify the population
according to party addiliationi, gender, socioeconomic status and other meaningful categoreies
related to voting behavior
this method ensure that we have enough women, men Democrats, Republicans and so on to
draw descriptive or correlational conclusions about each respective subgroup
area probability sampling – popular variant fo this sampling approach because the population is
divided into geographic areas (population clusters or strata). This method is applicable to any
population divisible into meaningful geographic areas related to the varaibles of interest
ex. Depending on the variables of interest, meaningful geographic areas might be people living
in urban neighborhoods, Inuits in igloos or nomads in tents. The assunmption is that iwthin each
of the areas the untis will have the same probability of being chosen
this sampling proceure can be mor complicated than those described above but the method is
cost-effective because the research design can be used repeatedly with only minor modification
suppose a polling organization needs an area probability sample of 300 out of 6000 estimated
hosing units in a city and a good list of all the dwlling sin the entire city does not exist and would
be too constly to prpare. Using a city map the pollsters can instead select asample of dwellings
by focusing on small clusters of blocks.
To do this in the simplest case they divide the entire map of the city into blocks of equal size and
then select 1 of , say, every 20 blocks ofr the sample
If they define the sample as the housing units located within the boundaries of these equal sized
sample blocks, the probability of selection for any unit is the selection of its block set at 1/20 to
correspond to the desired sampling rate of 300/6000.
In other cases, researchers categorize the blocks by taking into account their size or some other
factor of interest and then treat this factor as a stratum to samplr in a specific way
The procedure can become more complicated as the area gets bigger, but the key requirements
are to ensure 1) that all areas will have some chance of selection and b) that the units within the
areas are chosen impartially
For the asme plant to be sued again, all that must be altered are the randomly selected units
within each area
What Did the Literary Digest Case Teach Pollsters?
George Gallup – pioneering survey researcher who founded Gallup survey once noted some of
the methodological lessons learned by survey researchers going back to 1936
Presidential election between Roosevelt and Alfred landon, most people thought Roosevelt
would win easily but literary digest predicted landon would win overwhelming victory –
magazine had predicted every presidential victory since 1916
List was biased in favor of wealthy Republican households because not many owned phones,
drove cars, belonged to clubs – and that’s how people were being reached
Another problem was nonrespondent Lesson to be learned from this is if we week to generalize an entire population the percentage
differences we have found in a sample, the sampling plan and its execution must be properly
implemented in a precise, scientific way and sampling weights must be used to correct for
potential biases
Similar pseudoscientific public opinion piolls are conducted daily byh many news shows that
pose a yes or no or multiple choice question about some current issue and invite the viewers to
register their opinions by phone or onine
The external validiy of the repored results sis so low as to render any generalization useless as in
all likelihood those who respond are not only nonrepresentative of the general population but
also nonrepresentative of even the regular viewing audience
In one case a television staion skipped its polling one night and still received 20 calls voting yes
or no
George Gallup’s method called quota sampling was an early precursor of current methods; it
assigned a quota of people to be questioned and let the questioner build up a sample that was
roughly representative of the population
Interviewers were given ranges of variables and told to identify by sight people who seemed to
fit this quota
o Ec. An interviewer might be told to talk to so many people of ages 21-35, 36-55, etc.
We do not know how much of this interviewing took place on busy street corners and at trolley
stops rather than in house-to-house canvassing, but bias might be introduced simply as a
consewuence of the interviewed individuals’ being more accessible than other
Now we would use random selection procedures instead of leaving the selection of units to the
judgement of the questioner
Literary digest taught gallup that large numbers do not in and of themselves increase
representativeness or predictive accuracy of a sample
Because of that experience, the methodology of survey sampling has been improved in other
ways as further unexpected problems have been encountered and additional lessons learned
After 1948 the gallup survey and other respected polls adopted area probability sampling in
which election districts are randomly selected throughout the nationa dn then randomly chosen
households within these districts are contacted by interviwers
This procuedreu and lessons learned from mistake made in literary digest and its aftermath
brought about futher imrpvements
Gallup survey, based on little more than 8000 respondents, was able to predict with a magin of
error of only 1.7% that Eisenhower would be reelected president
Margin of error: prediction (based on laws of mathematical probability) was that the anticipated
percentages would fall within an interval bounded by plus and minus 1.7 points
poll watchers now expect an error of no more than 2 or 3 percentage points in national
elections, if probability sampling plan is properly implemented
Push Polls
push polls – an insidious form of negative political campaigning that is designed to push
opinions in a particular direction rather than scientifically sample them
push polls use rumors, gossip, lies, innuendoes, and half-truths to manufacture negative voter
attitudes by posing questions like :would oyu be more or less likely to vote for “name of
candidate” if youknew he/she had been arrested/failed ot pay child support/failed to pay
income taxes/falsified his/her resume? If you’re asked questions like these in a telephone interview, ask about the sponsors of the
survey and how the information is being used
The American Association for Public Opinion Research (AAPOR) has campaigned against push
polling, including issuing repeated warnigns to public and the media about the iniquity of theses
pseudoscientific polls
What Are Point Estimates and Interval Estimates?
Margin of error is an example of an interval estimate, and survey researchers are also
interested in making point estimates of population values
Point estimates – tell us about some typical characteristic of the target population. Ex. In
probability survey of a college population, we might want a point estimate of the number of
seniors who plan to continue their education after graduating
Another example : average number of widgets made by assembly line workers, the number of
cases of tuberculosis contracted on the job, and the incidence of psychological resilience among
New Yorkers after 9/11
Interval estimates – tell us how much the point estimates are likely to be in error (e.g, because
of variability in the composition of the population)
Ex. We do a siple random survey of 100 college students out of population of 2500 graduating
seniors at a certain university
each student asked if he/she plans to continue their education after they graduate fro college by
going onto graduate school, business school, medical school, law, dental, etc.
25 replied yes
In order to make a frequency estimate of the population value, we multiply the sample
proportion replying yes (.25) by the total number of students in the population (2500).
We estimate that 625 out of the 2500 graduating seneiors plan to continue their education
Confidence interval – helps us to determine how “approximate” this estimate is. It will indiciate
the probabilitiy that the estimated population value is correct within plus-or-minus some
specified interval
Suppose we want to state with 95% condidence (95 chances in 100) that the estimated
population value of 625 is likely to be correct within plus-or-minus some specified interval (the
95% confidence interval)
In our polling a sample (n) of 100 graduating seniors, we found that .25 (symbolized as prop, for

More
Less
Related notes for PSYC 2030