Textbook Notes (362,988)
Canada (158,130)
York University (12,352)
Psychology (3,541)
PSYC 2030 (144)
Chapter 9

Reading Notes - Chapter 9.

14 Pages
Unlock Document

York University
PSYC 2030
Krista Phillips

Study Guide – Chapter 9: Survey Research and Subject Recruitment What are Opportunity and Probability Samples?  here we examine logic and limitations of the methods used to select research participants  Donald Rubin noted all studies lie on a continuum from irrelevant to relevant with respect to answering a question  Randomized laboratory-type experiments that use opportunity samples of the first available students in college settings have a restricted sample of participants but usually have a high degree of control over the variables of interest  By contrast, researchers who do survey studies select potential respondents using special sampling procedures in order to generalize their descriptive findsings to a specific larger pool (a population) of people  If survey researchers used opprunity samples, spurious results and misleading generalizations about the specific population of interest would seriousluy compromise scientific integrity of their work  There is wide range of topics of interest to survey researchers  Pollsters use survey designs to map out some specified population’s opinions on important societal issues such as the community’s fears of crime or its choice of political candidates  Similar methods are used sometimes in epidemiological research, forensic research, economic research, and many other areas in which scientific surveys are conduction  When health officials wanted to find out about national trends in cases of tuberculosis contracted on the job, they did scientific surveys of hospitals to count employees reported to have TB o As federal courts became inundated with mass torts involving asbestos cases, one solution was to sample asbestos cases from the larger pool within a court’s jurisdiction o Asses damages in randomly chosen cases from each of five disease categories were then applied to each larger pool o More recently, when researchers wanted to study the prevalence of psychological resilience after a traumatic event they chose a probability sample of New Yorkers to survey in the 6 months following 9/11 o Researchers reported resilience was present in two thirds of the sample and never fell belwo one third even among highly exposed individuals with posttraumatic stress disorder  Instead of trying to question every member of the population which is usually impossible, this type of research focuses on a segment (sample) that is believed to be a typical of the population  How can researchers be certain that the segment is representative or typical of the population?  Ex. How can they be sure that a specific form of data from a sample is representative of a pop? They might compare the sample with the most recent census data but it is well known that census data are problemative because it is impossible to contact every member of the population  Researchers who use a sample can enver be 100% sure of their generalizations  They can make reasonable guess by first developing an accurate sampling frame that defines the target population and then relying on a carefully designed blueprint (sampling plan) to select sample means of probability sampling  Probability sampling – implies randomness enters into selection process at some stage so that the laws of mathematical probability apply  Probability – refers to the mathematical chance of an event’s occurring o Ex. Likelihood of getting “heads” when you flip a coin once (1 in 2) or getting a 2 when you throw a die once (1 in 6)  Survey studies can take many different forms, all using sampling plans in whicih osme method of probability sampling determines the random selection of the households or people to be contacted  These plans enable the researcher to assume reaspnably but with no 100% guarantee of being correct  Practical problems may impose limits on representativeness of the sample  Even in most carefully conducted survey, not every household or person in the sample can be reached and, of those who are actually contacted, not everyone will agree to be interview  In study of psychological resilience after the September 11, random digit-dialing approach was used to contact members of the sample  When the number of completed and partial interviews was summed and this total was divided by the sum of all numbers that were either eligible as residential phone numbers or of unkown eligibility, the response rate was estimated to be 34% What Is Meant By Bias and Instability in Survey Research?  Surey research done not only by private organziations, but by individual researchers owkring alone or with ties to private organizations and in the US, at university based insitutes that can implement face-to-face and telephone interviewing in national probability surveys  This research takes many different forms, all valid survey research is characterized by sampling plans in wich every element or sampling unit in the population has a known nonzerio probability of being selected at each draw  2 imporant statistical requirements of a probability sampling plan are a) the sample values are unbiased and b) that there be stability in the samples  Unbiased – values produced by sample must, on average, coincide with the true values of the population – but we can never actually be absolutely sure that this requirement has been met in a given study unless we already know those values  Stability – means there is not much variability (or spread) in the sample values o it’s estimated by statistical procedures such as variance and the standard deviation  sampling theory is beautiful because it can be applied not only to individual respondents but also to teams in a population of teams or to products on an assembly line, or to any other specified population of animate or inanimate units  bias = systematic error  print page 166 – there is a diagram and it’s above paragraph explains it The Wine Taster  in manufacture of red wine, grapes are crushed and residue is put into huge cats in which fermentation occurs  wine is then drawn off into barrels, where fermentation continues and product is periodficlly sampled by the wine taster  wine taster needs to draw only a small sample in order to evaluate the quality of the wine in the barrel  it’s the same in survey research: the more homogeneous the population, the smaller the sample that needs to be drawn Why Do We Not Know “For Sure” the Bias in Sampling?  only way to know for sure about bias in sampling results is to examine every single member of the popoulationa nd the sample at the same time the sampling is done – if pattern replies in same exactly matched pattern in population, we would know there was no sampling bias in surveryed sample  practically speaking, if we already knew how everyone in a population was – we wouldn’t be doing a sample to help us study the population  it’s sometimes thought that election forecasting allows us to detect bias in sample because we can compare predicted voting results with actual votes however problem in this case is that we are comparing data obtained at one point in time with the results at another point in time  a well-designed and carefully implemented selection process involving probability sampling can usually produce data that are remarkably close to the election results  polls = people can change their minds, they can be no shows, usually polls closer to election are thought to be more accurate but still – polls right up to day of voting disagree How Is Simple Random Sampling Done?  Simple random sampling – the basic prototype of probability sampling. The simple tells us that the sample is selected from an undivided population, random means that the sample is to be cjosen by a process that will give every unti in the specified population the same chance of being selected at each draw  In order for this to occur, the selection of one unit must have no influence on the slection of the other units  Assumption of random sampling: we have an understanding of the existence of all the units in the population  Procedure: draw individuals one at a time until wehave as large a sample as we need  Process of selecting units might consist of computer drawing units at random, using a table of random digits, or even spinning a roulette wheel or drawing well-mixed capsules from an urn.  Telephone interviewing: random digit dialing is sued to include households with unlisted numbers – area code and first three digits can be selected according to the geographic area of interest, and then a computer program is used to randomly select the last four digits  Famous case showing hazards of inadequate randomization: Vietnam War, use of random lottery to select conscripts for armed forces, birthdays out of urns, layers went: January, feb, march, april, may – even though urn was shaken for hours, the layers had formed and the ballots came out in order  Use of table of random digits can help to avoid this  2250 digits list came from a million random digits that were generated by an electronic roulette wheel programmed to produce a random frequency pulse every tiny fraction of a second  A computer then counted the frequency of 0s, 1s, 2s, and so on – on assumption that an impartial probability method would produce an approximately equal number of 0s, 1,s 2,s and so on in the overall table of a million random digits – this equality was confirmed  To see how you might use this table if you wanted to do random selection in a survey, imagine you want to interview 10 men and 10 women individually after choosing them at random from a list of 96 men and 99 women  You would begin by numbering the population of men consecutively from 01 to 96 and the population of women from 01 to 00  You are not ready to use the random digits – to do so, you can put your finger blindly on a starting position  You can start anywhere in the table and then move your finger in any direction, as long as you do not pck a set of numbers because they look right or avoid a set of numbers because they don’t look right  Suppose you put your finger on the first give-digit number in row 5 column 1 (12807), you would read aross the line two difits at a time, selecting the men numbered 12, 80, 79, 99 and so on until you had randomly chosen the 10 male interviewees  You would do the same for the women but at a different, blindly chosen spot  If you had fewer than 10 persons on each list, you would read only one digit at a time, if you had between 100-999 persons on your list, you would read three digits at a time  Suppose you chose the same two-digit number more than once, or suppose you chose a two- digit number not represented by any member of the population  In either case ou would go on to the next two-difit number in the row (unless you were sampling with replacement)  What if your population was so small that you were forced to skip many numbers in the table because they were larger than the largest number of people in your population? Ex. 450 peope in population and you want 50 people at random, you eould skip approx. half of the three digit numbers in the section of the table you chose (those from 451-999). As a simple solution, you can subtract 500 from any number in range from 501-999 and will result in fewer unusable selections  Sampling with replacement – selected units are placed in the selection pool again and may be reselected on subsequent draws, every unit in population continues to have the same probability of being chosen every time a number is read  You must select units one at a time to use sampling with replacement. Ex. Suppose unit are days of the year sealed in tiny capsules in an urn stirred so completely that there are no layers or nonrandom clusters, you select a capsule read it and put it back in the urn, making sure urn is well mixed  In sampling without replacement, previously selected unit cannot be reselected and the population shrinks each time you remove a unit but all the units remaining have the same likelihood of being drawn on the next occasion  If you scoop a handful of capsules, record each, and then discard those you picked – sampling without replacement  Either option is acceptable but without replacement is more common because researchesrs do not want to draw the same individuals twice or more  Another example of sampling w/o replacement is the wine taster, who draws then spits out a sample of wine What Are Stratified Random Sampling and Area Probability Sampling?  Simple random sampling is useful when population is known to be homogeneious or when its precise composition is unknown.  When we know something about the exact composition, a more efficient method of sampling is to sample from the different substrates of the population  Professional polling organizations typically use this approach to probability sampling, this is, randomly selecting sampling units (persons or hosueholds), from several subpopoulations (termed strata or clusters)  Ex. If we know population is 60% female and 40% male, ratio of 3 to 2, and tha gender is a pertinent variable, we can improve our sampling procedure by selecting subsamples proportionate in size to this 3:2 ratio of females to males  Stratified random sampling = is what’s described above  separate sample is randomly selected from each homogeneous stratum or layer of the population  the stratum means are then statistically weighted to form a combined estimate for the entire population  in a survey of political opinions, for example it might be useful to stratify the population according to party addiliationi, gender, socioeconomic status and other meaningful categoreies related to voting behavior  this method ensure that we have enough women, men Democrats, Republicans and so on to draw descriptive or correlational conclusions about each respective subgroup  area probability sampling – popular variant fo this sampling approach because the population is divided into geographic areas (population clusters or strata). This method is applicable to any population divisible into meaningful geographic areas related to the varaibles of interest  ex. Depending on the variables of interest, meaningful geographic areas might be people living in urban neighborhoods, Inuits in igloos or nomads in tents. The assunmption is that iwthin each of the areas the untis will have the same probability of being chosen  this sampling proceure can be mor complicated than those described above but the method is cost-effective because the research design can be used repeatedly with only minor modification  suppose a polling organization needs an area probability sample of 300 out of 6000 estimated hosing units in a city and a good list of all the dwlling sin the entire city does not exist and would be too constly to prpare. Using a city map the pollsters can instead select asample of dwellings by focusing on small clusters of blocks.  To do this in the simplest case they divide the entire map of the city into blocks of equal size and then select 1 of , say, every 20 blocks ofr the sample  If they define the sample as the housing units located within the boundaries of these equal sized sample blocks, the probability of selection for any unit is the selection of its block set at 1/20 to correspond to the desired sampling rate of 300/6000.  In other cases, researchers categorize the blocks by taking into account their size or some other factor of interest and then treat this factor as a stratum to samplr in a specific way  The procedure can become more complicated as the area gets bigger, but the key requirements are to ensure 1) that all areas will have some chance of selection and b) that the units within the areas are chosen impartially  For the asme plant to be sued again, all that must be altered are the randomly selected units within each area What Did the Literary Digest Case Teach Pollsters?  George Gallup – pioneering survey researcher who founded Gallup survey once noted some of the methodological lessons learned by survey researchers going back to 1936  Presidential election between Roosevelt and Alfred landon, most people thought Roosevelt would win easily but literary digest predicted landon would win overwhelming victory – magazine had predicted every presidential victory since 1916  List was biased in favor of wealthy Republican households because not many owned phones, drove cars, belonged to clubs – and that’s how people were being reached  Another problem was nonrespondent  Lesson to be learned from this is if we week to generalize an entire population the percentage differences we have found in a sample, the sampling plan and its execution must be properly implemented in a precise, scientific way and sampling weights must be used to correct for potential biases  Similar pseudoscientific public opinion piolls are conducted daily byh many news shows that pose a yes or no or multiple choice question about some current issue and invite the viewers to register their opinions by phone or onine  The external validiy of the repored results sis so low as to render any generalization useless as in all likelihood those who respond are not only nonrepresentative of the general population but also nonrepresentative of even the regular viewing audience  In one case a television staion skipped its polling one night and still received 20 calls voting yes or no  George Gallup’s method called quota sampling was an early precursor of current methods; it assigned a quota of people to be questioned and let the questioner build up a sample that was roughly representative of the population  Interviewers were given ranges of variables and told to identify by sight people who seemed to fit this quota o Ec. An interviewer might be told to talk to so many people of ages 21-35, 36-55, etc.  We do not know how much of this interviewing took place on busy street corners and at trolley stops rather than in house-to-house canvassing, but bias might be introduced simply as a consewuence of the interviewed individuals’ being more accessible than other  Now we would use random selection procedures instead of leaving the selection of units to the judgement of the questioner  Literary digest taught gallup that large numbers do not in and of themselves increase representativeness or predictive accuracy of a sample  Because of that experience, the methodology of survey sampling has been improved in other ways as further unexpected problems have been encountered and additional lessons learned  After 1948 the gallup survey and other respected polls adopted area probability sampling in which election districts are randomly selected throughout the nationa dn then randomly chosen households within these districts are contacted by interviwers  This procuedreu and lessons learned from mistake made in literary digest and its aftermath brought about futher imrpvements  Gallup survey, based on little more than 8000 respondents, was able to predict with a magin of error of only 1.7% that Eisenhower would be reelected president  Margin of error: prediction (based on laws of mathematical probability) was that the anticipated percentages would fall within an interval bounded by plus and minus 1.7 points  poll watchers now expect an error of no more than 2 or 3 percentage points in national elections, if probability sampling plan is properly implemented Push Polls  push polls – an insidious form of negative political campaigning that is designed to push opinions in a particular direction rather than scientifically sample them  push polls use rumors, gossip, lies, innuendoes, and half-truths to manufacture negative voter attitudes by posing questions like :would oyu be more or less likely to vote for “name of candidate” if youknew he/she had been arrested/failed ot pay child support/failed to pay income taxes/falsified his/her resume?  If you’re asked questions like these in a telephone interview, ask about the sponsors of the survey and how the information is being used  The American Association for Public Opinion Research (AAPOR) has campaigned against push polling, including issuing repeated warnigns to public and the media about the iniquity of theses pseudoscientific polls What Are Point Estimates and Interval Estimates?  Margin of error is an example of an interval estimate, and survey researchers are also interested in making point estimates of population values  Point estimates – tell us about some typical characteristic of the target population. Ex. In probability survey of a college population, we might want a point estimate of the number of seniors who plan to continue their education after graduating  Another example : average number of widgets made by assembly line workers, the number of cases of tuberculosis contracted on the job, and the incidence of psychological resilience among New Yorkers after 9/11  Interval estimates – tell us how much the point estimates are likely to be in error (e.g, because of variability in the composition of the population)  Ex. We do a siple random survey of 100 college students out of population of 2500 graduating seniors at a certain university  each student asked if he/she plans to continue their education after they graduate fro college by going onto graduate school, business school, medical school, law, dental, etc.  25 replied yes  In order to make a frequency estimate of the population value, we multiply the sample proportion replying yes (.25) by the total number of students in the population (2500).  We estimate that 625 out of the 2500 graduating seneiors plan to continue their education  Confidence interval – helps us to determine how “approximate” this estimate is. It will indiciate the probabilitiy that the estimated population value is correct within plus-or-minus some specified interval  Suppose we want to state with 95% condidence (95 chances in 100) that the estimated population value of 625 is likely to be correct within plus-or-minus some specified interval (the 95% confidence interval)  In our polling a sample (n) of 100 graduating seniors, we found that .25 (symbolized as prop, for
More Less

Related notes for PSYC 2030

Log In


Don't have an account?

Join OneClass

Access over 10 million pages of study
documents for 1.3 million courses.

Sign up

Join to view


By registering, I agree to the Terms and Privacy Policies
Already have an account?
Just a few more details

So we can recommend you notes for your school.

Reset Password

Please enter below the email address you registered with and we will send you a link to reset your password.

Add your courses

Get notes from the top students in your class.