Study Guide for Exam 2
Chapter 6: Surveys and Sampling
1. Determine when and why surveys are used in behavioral research.
-Used to collect descriptive information about a group of people
-Gathers information in a relatively short time
2. Understand the advantages and disadvantages of using interviews versus questionnaires in survey
Advantages of Interviews Advantages of Questionnaries
-allow researcher to develop a close -cheaper
relationship/sense of trust with subject more -more honest responses because it feels more
open and honest responding anonymous
-less influenced by characteristics of
Disadvantages of Interviews Disadvantages of Questionnaires
-difficult for interviewers to be trained to ask -response rate may not be very high
questions in an unbiased manner -question order (may not answer questions in
-efficiency and coordination for phone interviews the order they are written)
3. Understand how probability sampling is used to ensure that a sample is representative of the population.
Simple Random Sampling: ensure that each person in the population has an equal chance of being selected
to be in the sample. Use a random number generator and correlate that to the list of people in the population
Systematic Random Sampling: if the list of names are in a random sequence, we draw a random number
between 1 – 70 and then sample every person on that list with that number
Stratified Sampling: sample from subgroups (strata) since the variables being measured are expected to
differ across these subgroups
Cluster Sampling: break the population into a set of smaller groups for which there are sampling frames and
then to choose some of the clusters for inclusion in the sample
4. Define sampling bias and understand how it undermines a researcher’s ability to draw conclusions about
Sampling bias: sample is not actually representative of the population because the probability with which
members of the population have been selected for participation is not known
-Representativeness requires: one+ sampling frames that list the entire population of interest, selected
individuals must actually be sampled
May result in snowball sampling (one individual leads researcher to more individuals)
Convenience sampling: researcher sampled whatever was readily available without any attempt to make the
sample representative of the population
5. Determine what statistical procedures are used to report and display data from surveys.
Frequency distribution: table that indicated how many people in the sample fall into eat set of categories
grouped frequency distribution: combine adjacent values into a set of categories and then examine
frequencies of each of the categories
histogram: bars touch each other
stem and leaf plot
Central tendency: point in the distribution around which the data are centered (mean, median, mode) Dispersion: spread (variance, standard deviation)
Mean deviation: score on the variable minus the mean of the variable
Sum of squares: mean deviations are squared and summed to produce a statistic
Variance: sum of squares divided by the sample size (N)
Standard deviation: square root of the variance
6. Determine the margin of error of a sample.
It is also known as the confidence interval. The confidence interval is when we say with some certainty that a
population value is likely to fall.
7. Consider the cases under which a scientist might decide to use a survey research design. What
could the scientist learn from this approach, and what would he or she not be able to learn?
A survey research design would inform us about the descriptive information of a group of people. It provides a
snapshot of the opinions, attitudes, or behaviors of a group of people at a given time. Surveys can be used to
draw conclusions about a population of individuals. Surveys cannot assess causal relationships among
8. What are the advantages and disadvantages of using questionnaires, rather than interviews, in survey
Questionnaires are cheaper, free from the experimenter’s bias, and may produce more honest responses
because it seems more anonymous. However, there are several difficulties with questionnaires as well such as
a low response rate since they are further away from the researcher so they may be less inclined to respond.
The people that respond to the questionnaire may have different responses than those who do not
respond. Also, the question order may become distorted which changes priming and changing interpretation.
9. Define probability sampling, and indicate how three types of probability samples differ. What is the
advantage of using probability sampling?
Probability sampling involves procedures that are used to ensure that each person in the population has a
known chance of being selected to be a part of the sample.
Simple random sampling: randomly select from the sampling frame
Advantage: each person in the population has an equal chance of being selected
Systematic sampling: a list of names on the sampling frame is in a random sequence, a random number (n)
is drawn and the nth person on the list is drawn every nth time
Advantage: only need to select one random number
Stratified sampling: randomly sampling from the strata (subgroups) rather than the entire population because
the variables being measured are expected to differ across these subgroups
Advantage: a more proportionate stratified sample
Cluster sampling: divide the population into clusters and randomly select certain clusters to be included in the
sample or to randomly sample from each cluster.
Ex: done in stages, more convenient
10. Describe the descriptive statistics that are most frequently used to analyze quantitative and nominal
variables from a survey. What techniques are used to graphically display such data?
Central tendency: point at which the distribution is centered around (mean, median, mode)
Dispersion: spread of the distribution (variance, standard deviation)
Graphically display data: frequency distribution (bar chart, histogram – touching bars) – indicates how many
and what percentage of people fall into each category
stem and leaf plot – summarizes raw data but original data values can still be seen 11. What is sampling bias? When is it likely to occur, and how does it undermine the ability to draw accurate
conclusions from surveys?
Sampling bias is when the sample is not representative of the population (sampling frame does not include all
individuals in the population and all of the selected individuals were not sampled). It is likely to occur when the
sampling frame is unclear, when members of the sampling frame are not sampled, or there is no sampling
frame. No longer able to make inferences about the characteristics of the population.
Chapter 7: Naturalistic Methods
1. Define naturalistic research and understand why it is important.
Designed to describe and measure behavior as it occurs in everyday life. It can be used to study variables that
cannot be manipulated in experimental settings (impact of earthquakes). It also has ecological validity
2. Determine ecological validity and understand why naturalistic designs have it.
Ecological validity: the extent to which research is conducted in situations that are similar to the everyday life
experiences of the participants
-Naturalistic research measures behaviors that people do every day
3. Understand the advantages and disadvantages of being an acknowledged or unacknowledged participant
or observer in observational research.
Adv. Of Acknowledged Participant Adv. Of Unacknowledged Participant
-observe situations that are difficult to gain -get close to the people being observed and get them to
access to reveal personal or intimate information about themselves and
their social situation (true feelings about employers)
Disadvantage of Acknowledged Par. Disadvantage of Unacknowledged Par.
-reactivity -difficulty remaining objective
-ethical dilemma (not told they were a part of research)
-activities of observer may influence the process being
4. Understand case studies and determine their benefits and drawbacks.
Benefits: by carefully studying individuals who are socially marginal (special situation), we can learn
something about human nature. Investigate neurological bases of behavior
Disadvantages: very limited number of unusual individuals, cannot tell us much about whether the same
things would happen to other individuals in similar situations or why these specific reactions ot these events
-No comparison group
-only provides weak support for drawing of scientific conclusions
5. Demonstrate how behaviors are systematically coded to assess their reliability and validity.
1. Deciding what to observe
a. Systematic observation: specifying ahead of time which observations are to be made on
which people and in which time/places (develop a set of behavioral categories)
2. Deciding how to record observations – strong coding methods to develop interrater reliability
a. Event Frequencies: # of times something occurred
b. Event duration: amount of time spent on event
6. Understand archival research and what types of questions archival research is used to answer.
Analysis of any type of existing records of public behavior (newspaper articles, speeches, letters).
Systematically coding archival records is done through content analysis -Used to analyze past events and draw conclusions from it
7. Describe the goals and techniques of naturalistic research. What are its advantages and disadvantages
over other research methods?
Naturalistic research attempts to describe and measure the behavior in every day life. It has high ecological
validity and can study variables that cannot be experimentally manipulated. However, it does not provide
much information about why behavior occurs or what would have happened to the same people in
8. Discuss the advantages of approaches of observing versus participating, and of being acknowledged
versus unacknowledged in naturalistic observation.
Unacknowledged Adv: Chance to get intimate information
Participant Dis: researcher may change situation, ethical issues
Acknowledged Participant Adv: ethical, observe situations that are difficult to gain access to
Dis: reactivity, researcher’s bias (by friendships)
Unacknowledged Observer Adv: limits reactivity problems
Dis: ethical issues
Acknowledged Observer Adv: able to spend entire session coding behaviors
Dis: reactivity because people know they’re being watched
9. Define ecological validity, and indicate why it is important in research. What types of research designs are
most likely to have it?
Ecological validity is the extent that the research situation is similar to everyday life experiences. Reactivity is
minimized this way and construct validity is increased. Naturalistic research is most likely to have ecological
10. Define systematic observation. What makes systematic observation different from other types of
Systematic observation is the process of deciding what behavioral categories to code. It specifiesahead of
time exactly which observations are to be made on which people and in which times and places. It has higher
reliability and validity than other observational research techniques because it uses inter-rater reliability.
Observational research just uses various sampling techniques (participant/observer) and archival research
uses content analysis.
11. What are archival research methods and when are they likely to be used?
Archival research methods analyze existing records of public behavior as data. The data is coded through
content analysis. It is most likely to be used to study behavior throughout time to draw conclusions.
Chapter 8: Hypothesis Testing and InferentialStatistics
1. Understand what inferential statistics are and how they are used to test a research hypothesis.
Use the sample data to draw inferences about the true state of affairs. It uses probability and statistical
analysis to draw inferences on the basis of observed data through the scientific method
2. Define the null hypothesis.
We assume that the observed data do not differ from what would be expected on the basis of chance and the
sampling distribution of the statistic is used to indicate what is expected to happen by chance (least interesting
outcome). We want to reject the null hypothesis
3. Define alpha. Alpha is the significance level which is set to .05. We can reject the null hypothesis if the observed data’
significance is less than .05. The smaller alpha is, the more stringent the standard is.
4. Understand what the p-value is and how it is used to determine statistical significance.
The p-value is the probability value that shows the likelihood of an observed statistic occurring on the basis of
the sampling distribution. The p-value indicates how extreme the data are so we compare the p-value to alpha.
If the p < .05, we reject the null and we say that the result is statistically significant
If p > .05, we fail to reject the null, we say the result is not statistically significant.
5. Understand why two-sided p-values are used in most hypothesis tests.
Two-sided p-values take into consideration that unusual outcomes may occur in more than one way. They are
more conservative but they allow us to interpret statistically significant relationships even if those differences
are not in the direction predicted by the research hypothesis
6. Define Type 1 and Type 2 errors and understand the relationship between them.
Type 1: we reject the null hypothesis but it is true (Concluded that therapy reduced anxiety even if it did not).
Equal to alpha (.05)
Type 2: failing to reject the null hypothesis when it is actually false (therapy program isn’t working even when it
is). Equal to beta
Type 1 Error Correct
Probability = decision
alpha Probability =
Correct Type 2 Error
decision Probability =
Probability = 1 beta
7. Understand beta and how it related to the power of a statistical test.
Power of a statistical test is the probability that the researcher will be able to reject the null hypothesis given
that the null hypothesis is actually false and should be rejected.
Power = 1 – B (beta)
-Influenced by sample size (N). As N increases, likelihood the researcher finding a statistically signficiant
relationship between the IV and DV increases (therefore, power increases)
8. Understand the effect size statistic and how it is used.
Effect size is the size of a relationship indicated by a statistic. The effect size indicates the magnitude of a
relationship (0 = no relationship between the variables and a larger/positive effect size indicates stronger
Statistical significance = effect size x sample size
-Because the p-value is influenced by the sample size, it is not a very good indicator of the size of the
relationship by itself
-Effect size is an index of the strength of a relationship not influenced by sample size
9. Describe the procedures that scientists use to test their research hypotheses. Be sure to
consider the following terms: null hypothesis, research hypothesis, alpha, beta, Type 1 errors,
and Type 2 errors. 1. Develop research hypothesis (observed data is what is expected on the basis of chance)
2. Set alpha (.05) – probability of conducting a type I error (rejecting the null when the null is true)
3. Calculate power to determine the sample size that is needed (power = 1-beta). Beta is the probability of
committing a Type II error (failing to reject the null when the null is false)
4. Collect data
5. Calculate statistic and p-value