Additional Notes on Data Analysis and Confidence Intervals—A Summary and
Review from Chapters 12 and 13
As discussed in class, once we have obtained our data, we need to somehow make
sense of it in relation to the hypothesis we proposed before conducting our study
(i.e. in the introduction).
There are three main steps to this process:
1. Get to know your data (see chapter 12): Look at your data sets carefully, and
use visual distributions (i.e. frequency distributions) to ‘get a sense’ of what the data
set ‘looks like’. Does there appear to be errors or oddities in what you have
obtained? Are there outliers or anomalies? Do the data make sense, given the
research question and hypothesis you proposed?
Let’s use our dog patting example from class to keep things simple. Recall that we
want to test the effect that stroking a dog has on resting heartrate (our DV). We
hypothesized, based on previous studies, that patting a dog for five minutes would
result in a lower resting heart rate than in the no-dog patting group. We cannot
sample the entire population we are interested in, so instead obtain a sample of
participants who are willing to take part in our study.
We obtain a sample from the population and randomly assign our participants to
one of two groups, dog patting or no dog patting (perhaps we even use a stuffed
animal in our control group). Our IV is therefore the patting condition (pat or no
pat). Everything in the two groups is exactly the same, except for the presence of a
friendly real therapy dog and handler (who doesn’t speak to the participant other
than to give simple directions re handing the dog, or to intervene if something goes
wrong) in our experimental condition.
Here are our individual heartrate scores in each group—
Individual scores from dog patting group:
58 75 Individual scores from no-dog group:
You might ask questions like:
Are there any outliers or anomalies in the data sets?
Do the numbers make sense given that you are measuring heartrate? For example, if
all your participants had scores between 300 and 400, you might suspect that there
was something wrong with your measuring device ☺!
You might construct a frequency distribution or use stem and leaf plots to help you
‘get a visual’ on what your data looks like:
Stem and leaf plot for dog patting group
5 0 8
6 0 2 5 6
7 1 3 5
Here, we have one score, 48, that is perhaps lower than all others. Would this one
score skew your data (i.e. throw your mean off?) Would it be best to eliminate this
score in order to obtain a more representative measure of what the group’s data
looks like? These decisions are made at this stage…
Stem and leaf plot for non dog patting group
5 0 8 8
6 5 8 7 3 4 8
8 0 4
2. Describe/summarize your data in a way that is meaningful (see chapter 12):
Use descriptive statistics appropriate for the type of study you have conducted.
Here, for example, measures of central tendency (i.e. mean, mode, or median)
and/or standard deviation may be used.
--remember that type of central tendency appropriate first depends on the
measurement scale that you used. You can only use means for ratio and interval
If you have used other measurement scales, you might be restricted to calculating
the mode (i.e. only measure of central tendency that can be used for nominal data).
Moreover, in some data sets, the median or mode may be better measures of central
tendency to describe your data (especially if you have outliers and decide that you
should not discard this data).
Because our data is interval data (there is no absolute zero heartrate unless
someone is dead), and we would like to subject our data to inferential statistics (i.e.
hypothesis testing), we would calculate the mean scores within each group. In an
independent groups design experiment, remember that we are interested in
comparing the average scores between the different groups.
Dog Patting group
Mean heartrate= 62.8
Non-dog patting group
58 84 50 58
Mean heartrate =68.8
(please see chapter 12 notes and your text for how to calculate different measures
of central tendency)
Note the individual variability in scores within each group!
We would also calculate the variability in our scores (the amount of spread in the
distribution of scores) using standard deviation (indicates average deviation of
scores from the mean). You do not have to do this calculation on the exam, but have
a look at what is being done within the equation (see lecture notes from chapter 13)
3. Determine the probability that your results are due to random error or
chance (confirm what the data reveal) (chapter 13): This step actually begins at
step 1 above, where we get to know our data and determine whether it makes sense,
and summarize the patterns that we have found in step 2. We then continue by
using statistical techniques to determine the probability that our obtained results
were ‘due to chance’.
Null hypothesis testing (chapter 13):
Remember, when we conduct a study, we are not usually able to test our entire
population of interest. Instead, we sample from the population; the results obtained
may not be representative of what the entire population looks like, so we use
statistical theory to determine the likelihood that the results we have obtained are
simply due to noise, or random error (refer to our discussion of probability in
In other words, what is the likelihood that the result obtained in our study (i.e. the
difference in mean heartrate in our hypothetical study) was simply due to chance
rather than due to patting the real dog for five minutes?
Recall the lecture on probability in class, and the likelihood that we are to obtain
heads or tails when flipping a coin…Are there factors that we could employ that
would increase our likelihood to obtain heads or tails when flipping a coin (i.e.
maybe we bend the coin a certain way).
Similarly, in our study, we are testing whether patting the dog for five minutes
would make a difference in heart rate IN THE POPULATION—would heart rate be
lower in the dog patting group if we were to test EVERYONE IN THE POPULATION (randomly assign them to dog patting or no dog patting and compare the population
Recall that the first step is to determine two different hypotheses:
1. The null hypothesis (POPULATION means (not group means) are exactly
equal)= H 0
2. The research hypothesis (POPULATION means (not group means) are NOT
Because we don’t know what the true research hypothesis scores would be IN THE
POPULATION, statistical inference testing focuses on the NULL HYPOTHESIS. Thus,
statistical inference uses sampling distributions and probability to determine the
likelihood that any differences in the scores were due to chance or random error.
You might want to think about our discussion of sampling distributions, in which
we looked at what would happen if we sampled from the population over and over
again. The true population mean (related to the research question we are asking
and the subsequent hypothesis we are proposing) is not known. So, our frequency
distribution represents the curve that would result from data (i.e. the mean of the
means of the difference in scores) where we sampled from the population over and
over again… Some results would be closer to the population mean and some further away.
The sampling distribution = the hypothetical mean of all the mean scores
So, thus far, we should have done the following:
1. Specify null hypothesis and research hypothesis
2. Specify the significance level that you will use to decide whether to reject the
null hypothesis (alpha level; .05)
3. Decide on most appropriate test to use
In our example:
1. null hypothesis:
Mean heartrate scores IN THE POPULATION would be exactly equal in the dog
patting and non-dog patting conditions.
Mean heartrate scores IN THE POPULATION would not be equal in the dog patting
and non-dog patting conditions (we don’t know what the true population mean is) 2.. significance level in psychology is usually p<=.05
(sometimes we use p<=.01 if we want to be very conservative (i.e. definitely don’t
want to risk type I error)
2. determine most appropriate test. In our example, we have two independent