Textbook Notes (270,000)
CA (160,000)
UVic (500)
PSYC (200)
Chapter 8

PSYC 300B Chapter Notes - Chapter 8: Rfactor, Type I And Type Ii Errors, Multiple Comparisons Problem

Course Code
David Medler

This preview shows half of the first page. to view the full 2 pages of the document.
multiple comparisons: allows for specific comparisons between means to
determine which experimental condition(s) responded best to the treatment
-ANOVA only tells you that groups differ, whereas multiple comparisons
shows where that difference occurs
-maybe 2 groups responded equally well, & participants in both groups
did better than participants in the control group
-maybe only 1 group (experimental condition) was better than all others
-controls for inflated (p) of making a type I error
ex. when you have a k > 2 design & you carry out a series of t-tests
(comparing several pairs of means), if each pair of means sets an
error-rate of p(𝛼) = 0.05, then you have increased the probably that
you will make a type I error because your true p(𝛼) should be less
-makes 4 modifications to the standard t-test ratio for when k > 2:
(i) considers all group means in the design
weights each mean according to its role in the analysis
the sum of these weights must = 0 (w = 0)
(ii) uses MSe for the error term instead of Sx̅-x̅ as estimated by SP2
SP2 is a good estimate of the population error σe2 when k = 2,
but when k > 2, then SP2 is a biased overestimate of σe2
(iii) weights the number of groups in the design
applied to the denominator of the test-ratio
includes only the number of groups in the analysis
can be adjusted for unequal Ns in the groups
(iv) uses df for MSe term (dfe) for the critical value
reflects the true number of estimates of population variance
MSe is the value from the ANOVA source table
Nk = the number of scores in each group
-2 types of multiple comparisons:
(i) pairwise comparisons: compares pairs of means
the maximum number of possible pairwise comparisons
possible is [k(k - 1) ÷ 2]
don't always carry out all possible comparisons
when there are equal Ns use:
compare the outcome to the difference between the 2 group
means being analyzed
(ii) complex comparisons: compares combined pairs of means
average the experimental conditions & compare that
averaged mean to the mean of the control group
error-rate: the probably of making a type I error for a specific analysis
error-rate per comparison (𝛼PC) or protected t: the probability of
making a type I error for each individual comparison
-set by the researchers for each comparison (do not have to be equal)
error-rate per experiment/experimenter-wise error (𝛼EW) or!
error-rate per family/family-wise error (𝛼FW): the probability of a type
I error for an entire family of comparisons (the entire experiment)
-will not exceed p(𝛼) = 0.05
-always involves a uniquely computed table of critical values
planned (a priori) comparisons: a subset of comparisons planned & justified
by the researcher before the data is collected
-can choose which comparisons you are going to make
-must also follow the same guideline as those used for justifying choosing a
directional hypothesis
-used in confirmatory analyses — we are dis-confirming the null
-defined by the research question
-does not require the significant omnibus Fobs value in order to carry out
the comparison
-the total number of comparisons possible never exceeds k - 1
-the error rate is adjusted only for the number of comparisons done
have a larger error rate value for each comparison
-a more powerful alternative, & thus are more likely to reject the null
allows for either pairwise or complex comparisons
examples of planned comparisons:
(a) separate t-tests: can be used for a priori & post hoc comparisons
-for 1 pairwise comparisons, set p(𝛼) = 0.05
-for > 1 pairwise comparison, set p(𝛼PC) individually for each
comparison & sum them together (ex. for 3 comparisons, set
p(𝛼PC) at 0.02, 0.02, & 0.02, so that p(𝛼PC) = 0.06)
-can be either directional or non-directional
(b) Dunn’s test: can apply pairwise &/or complex comparisons
-significant Fobs value is not required
-p(𝛼EW) is based on the number of actual comparisons done
-more conservative & this very reliable
-uses the Bonferroni distribution for tDunn’s table of critical values
-uses MSe as the pooled error term from the ANOVA source table
-tDunn’s critical value is based on:
(i) dfe
(ii) the number of actual comparisons made
(iii) p(𝛼EW) at either 0.05 or 0.01 (0.05 is the default)
-the distribution of Dunns t is a special distribution of a t-statistic
called Bonferroni t
based on Bonferroni’s inequality for p(𝛼EW): the
probability of occurrence of one or more events can never
exceed the sum of individual probabilities
the error rate never exceed the overall value for 𝛼
for each comparison
-steps to applying Dunn’s test to a 1-way between-group design:
(i) apply the test ratio to find the critical value for Dunns t
get the value for tDunn’s(crit) from the table
compute tDunn’s using tDunn’s(crit), MSe, & Nk
computed value for tDunn’s represents the minimum
distance (difference) between 2 means in order for tDunn’s
to be significant
(ii) find the mean difference between each selected
(planned) pairwise comparison
(iii) compare the mean difference to the value for tDunn’s
if the mean difference is > tDunn’s, reject the null
if the mean difference is < tDunn’s, retain the null
post hoc (a posteriori) comparisons: tests between sample means that are not
specified before the data was collected
-adjusts p(𝛼) for all possible comparisons, even if you are not interested in
that comparison, or the comparison doesn’t even make sense
assumes all possible pairwise will be made
-used in exploratory analyses — when the phenomenon has not been
well-researched or the study is novel
-applied to outcomes with no basis for prediction
-allows you to examine interesting or unexpected outcomes that may arise
-by convention, significant Fobs value is required
-total number of comparisons possible is [k(k - 1) ÷ 2]
-error rate is either p(𝛼PC) or p(𝛼EW), depending on the choice of test
-is a less powerful alternative, & is thus harder to reject the null because
the overall value for any individual p(𝛼) is small
-always non-directional
PSYC 300B - Chapter 8: Multiple Comparisons
You're Reading a Preview

Unlock to view full version