CRIM 220 Lecture Notes - Lecture 10: Surveymonkey, Computer-Assisted Telephone Interviewing, Social Desirability Bias
7 Jun 2018
School
Department
Course
Professor
DISADVANTAGES:
1. other different, more idiosyncratic responses may be more adequate
2. can generate a response set among respondents
3. responses can be generated out of boredom or fatigue
Ordering Question in a Questionnaire
•Order can also affect the answers given
•Initial questions can affect following questions, known as ‘halo effect’
•Different versions of a questionnaire can be pretested to see which is more suitable
1. Self-Administered Questionnaires
•Least expensive and easiest to complete
•Home delivery and mail can be used in combination
•Completion rate is higher when researchers pick-up the survey
a. group administered surveys
e.g. classroom, prison
•cheaper, quicker than email
•high response rates
•ADVANTAGES: contact with researcher, privacy of self-report
•DISADVANTAGES: anonymity, voluntary participation
b. home delivery and/or pick up
•Transmittal of a questionnaire accompanied by a letter of explanation and a self-
addressed, stamped envelope of returning the questionnaire
•Timing is to be considered (holidays might be inappropriate time to mail
questionnaires as mail can be the highest at these times)
•Researcher drops off survey in person, or picks up after completion
•Blend advantages of group admin band mail surveys
•Privacy, convenience no pressure to rush them to get it done, not always
opportunity to clarify questions, high response rates than mail
•DISADVANTAGES: more expensive than mail, no control over question order, ….
c. mail surveys
ADVANTAGES: privacy and fairly cheap
Response rates: 50% adequate, 60% good, 70% very good
DISADVANTAGES: low response rate, no control over question order or context, literacy
barriers, no clarifying questions
~6-8 weeks in total
•require administrative support and database management
•How to increase response rates? make it simple to return, prepaid, keep length
reasonable, think about timing, warning mailings, incentives, cover letter, reminders
d. email surveys
Format: text questions, embed questionnaire, attachment (don’t work anymore,
attachments are sometimes consider dangerous)
•avoid attachments
•ADV: cheap, convenient, private
•Problems: spam blockers, suspicion of viruses, spyware = delete, requires computer/
internet access & email use, no control over question order, context; literacy
(language barrier, might not understand, what do you mean by certain things?), no
clarifying questions
e. web-based surveys
56
•Different types of computer-assisted self-administered surveys have become more
common and increasingly sophisticated
•Contributed to the proliferation of data collected by questionnaire
•Respondents are asked to click on a link that eventually takes them to a
questionnaire application
•Can be done through Survey Monkey
•cheap and save time
•responses are automatically recorded in computer files
•private-convenient
•control over Q order
•streamlined formatting
•Can create attractive questionnaires
•Great control over who is the sample, can choose the specific characteristics of the
sample
•DISADVANTAGES:
○posting an open-online survey: generalizability? who will do it and who won’t?
○challenges when offering incentives re. unique respondents
○no control over context, literacy, clarifying questions
○people might not want to open it, might think it is a spam
Warning Mailings and Cover Letters
•Certain types of warning mailings may increase the respondent rate
•After researchers generate a sample, they send a postcard to each selected
respondent, with a notification ‘Address correction requested’ printed on the
postcard
•If addressee has moved, the questionnaire is sent to the new location
•If this is not possible, questionnaire is returned with a ‘Addressee unknown’
•Cover letters accompanying the questionnaire may also increase respondent rate
1. Content of the letter is important (it should explain that a survey is going on)
2. Should identify institutional affiliation or sponsorship of the survey
recognizable affiliation
impressive-sounding affiliation
2. In-person interviews
•Face-to-face interviewer Q&A
•Useful when: access problems prevent mail or phone surveys, visual aids and
explanatory necessary, rapport between interviewer & respondent critical
•Response rates typically ~80-85%
•Probes: silent (effective one), encouragement, elaboration, clarification, repetition
•Interviewer can make independent observations
Problems:
○Interviewer effects (social desirability— people will try to answer questions in a way
to please you, interviewer differences— ability to generate rapport, other
57
Interview Survey- using a questionnaire in a systematic way to interview a
large number of people
Document Summary
Disadvantages: other different, more idiosyncratic responses may be more adequate, can generate a response set among respondents. 3. responses can be generated out of boredom or fatigue. Response rates: 50% adequate, 60% good, 70% very good. Disadvantages: low response rate, no control over question order or context, literacy barriers, no clarifying questions. ~6-8 weeks in total: require administrative support and database management, how to increase response rates? make it simple to return, prepaid, keep length reasonable, think about timing, warning mailings, incentives, cover letter, reminders, email surveys. Challenges when offering incentives re. unique respondents. No control over context, literacy, clarifying questions. People might not want to open it, might think it is a spam. Interviewer effects (social desirability people will try to answer questions in a way to please you, interviewer differences ability to generate rapport, other. !57 characteristics like male vs female, training is important as a way of making sure everyone does thing the same way, bias)
