# PSYC 300B Chapter Notes - Chapter 8: Rfactor, Type I And Type Ii Errors, Multiple Comparisons Problem

by OC757575

This

**preview**shows half of the first page. to view the full**2 pages of the document.**•multiple comparisons: allows for speciﬁc comparisons between means to

determine which experimental condition(s) responded best to the treatment

-ANOVA only tells you that groups differ, whereas multiple comparisons

shows where that difference occurs

-maybe 2 groups responded equally well, & participants in both groups

did better than participants in the control group

-maybe only 1 group (experimental condition) was better than all others

-controls for inﬂated (p) of making a type I error

•ex. when you have a k > 2 design & you carry out a series of t-tests

(comparing several pairs of means), if each pair of means sets an

error-rate of p(𝛼) = 0.05, then you have increased the probably that

you will make a type I error because your true p(𝛼) should be less

-makes 4 modiﬁcations to the standard t-test ratio for when k > 2:

(i) considers all group means in the design

•weights each mean according to its role in the analysis

•the sum of these weights must = 0 (∑w = 0)

(ii) uses MSe for the error term instead of Sx̅₁-x̅₂ as estimated by SP2

•SP2 is a good estimate of the population error σe2 when k = 2,

but when k > 2, then SP2 is a biased overestimate of σe2

(iii) weights the number of groups in the design

•applied to the denominator of the test-ratio

•includes only the number of groups in the analysis

•can be adjusted for unequal Ns in the groups

(iv) uses df for MSe term (dfe) for the critical value

•reﬂects the true number of estimates of population variance

•MSe is the value from the ANOVA source table

•Nk = the number of scores in each group

-2 types of multiple comparisons:

(i) pairwise comparisons: compares pairs of means

•the maximum number of possible pairwise comparisons

possible is [k(k - 1) ÷ 2]

•don't always carry out all possible comparisons

•when there are equal Ns use:

•compare the outcome to the difference between the 2 group

means being analyzed

(ii) complex comparisons: compares combined pairs of means

•average the experimental conditions & compare that

averaged mean to the mean of the control group

•error-rate: the probably of making a type I error for a speciﬁc analysis

•error-rate per comparison (𝛼PC) or protected t: the probability of

making a type I error for each individual comparison

-set by the researchers for each comparison (do not have to be equal)

•error-rate per experiment/experimenter-wise error (𝛼EW) or!

error-rate per family/family-wise error (𝛼FW): the probability of a type

I error for an entire family of comparisons (the entire experiment)

-will not exceed p(𝛼) = 0.05

-always involves a uniquely computed table of critical values

•planned (a priori) comparisons: a subset of comparisons planned & justiﬁed

by the researcher before the data is collected

-can choose which comparisons you are going to make

-must also follow the same guideline as those used for justifying choosing a

directional hypothesis

-used in conﬁrmatory analyses — we are dis-conﬁrming the null

-deﬁned by the research question

-does not require the signiﬁcant omnibus Fobs value in order to carry out

the comparison

-the total number of comparisons possible never exceeds k - 1

-the error rate is adjusted only for the number of comparisons done

•have a larger error rate value for each comparison

-a more powerful alternative, & thus are more likely to reject the null

•allows for either pairwise or complex comparisons

•examples of planned comparisons:

(a) separate t-tests: can be used for a priori & post hoc comparisons

-for 1 pairwise comparisons, set p(𝛼) = 0.05

-for > 1 pairwise comparison, set p(𝛼PC) individually for each

comparison & sum them together (ex. for 3 comparisons, set

p(𝛼PC) at 0.02, 0.02, & 0.02, so that ∑p(𝛼PC) = 0.06)

-can be either directional or non-directional

(b) Dunn’s test: can apply pairwise &/or complex comparisons

-signiﬁcant Fobs value is not required

-p(𝛼EW) is based on the number of actual comparisons done

-more conservative & this very reliable

-uses the Bonferroni distribution for tDunn’s table of critical values

-uses MSe as the pooled error term from the ANOVA source table

-tDunn’s critical value is based on:

(i) dfe

(ii) the number of actual comparisons made

(iii) p(𝛼EW) at either 0.05 or 0.01 (0.05 is the default)

-the distribution of Dunn’s t is a special distribution of a t-statistic

called Bonferroni t

•based on Bonferroni’s inequality for p(𝛼EW): the

probability of occurrence of one or more events can never

exceed the sum of individual probabilities

•the error rate never exceed the overall value for 𝛼

for each comparison

-steps to applying Dunn’s test to a 1-way between-group design:

(i) apply the test ratio to ﬁnd the critical value for Dunn’s t

•get the value for tDunn’s(crit) from the table

•compute tDunn’s using tDunn’s(crit), MSe, & Nk

•computed value for tDunn’s represents the minimum

distance (difference) between 2 means in order for tDunn’s

to be signiﬁcant

(ii) ﬁnd the mean difference between each selected

(planned) pairwise comparison

(iii) compare the mean difference to the value for tDunn’s

•if the mean difference is > tDunn’s, reject the null

•if the mean difference is < tDunn’s, retain the null

•post hoc (a posteriori) comparisons: tests between sample means that are not

speciﬁed before the data was collected

-adjusts p(𝛼) for all possible comparisons, even if you are not interested in

that comparison, or the comparison doesn’t even make sense

•assumes all possible pairwise will be made

-used in exploratory analyses — when the phenomenon has not been

well-researched or the study is novel

-applied to outcomes with no basis for prediction

-allows you to examine interesting or unexpected outcomes that may arise

-by convention, signiﬁcant Fobs value is required

-total number of comparisons possible is [k(k - 1) ÷ 2]

-error rate is either p(𝛼PC) or p(𝛼EW), depending on the choice of test

-is a less powerful alternative, & is thus harder to reject the null because

the overall value for any individual p(𝛼) is small

-always non-directional

PSYC 300B - Chapter 8: Multiple Comparisons

###### You're Reading a Preview

Unlock to view full version