Chapter 7 – Experimental Design I: Single-Factor Designs
Single Factor (I.V.) with two levels
Factor stands for I.V.
Single-factor designs have one I.V. with two or more levels
-There are 4 different single-factor designs.
-I.V. can be tested either between or within subject designs.
-Between-subjects, it could be either a manipulated or a subject variable.
-If the I.V. is manipulated, the design will be called independent groups designs, if simple random
assignment is used to create equivalent groups or matched groups design, matched on a potentially
confounding var. and then randomly assigned.
-If a subject variable is being tested, the groups are composed of different types of individuals
(male/female, introverts/extrovert, liberal/conservative, etc), this design is called an ex post facto
design (non equivalent groups) because the subjects in the study are placed into the groups “after the
fact” of their already existing subject characteristics.
-In ex post facto designs, random assignment is not possible, subjects are already in one group or
another by virtue of their variable being studied (eg – gender)
-The I.V. tested within subjects is called repeated-measures design – that is, each participant in the
study experiences each level of the I.V. (is measured repeatedly).
Analyzing Single-Factor, Two Level Designs
-To determine whether the differences found between the two conditions/levels of a single factor (I.V.),
two -level design are significant (true) or due to chance, an inferential statistical analysis is required.
-When an interval or ratio scales of measurement is used, it requires two types of t-test. Using a t-test
(inferential stats) assumes the data from the two conditions at least approximate a normal distribution.
A null hypothesis can be tested.
-Two types of t-test for comparing two sets of scores are:
1. Independent Samples t-test is used when the two groups of participants are completely independent
of each other. Occurs when we use random assignment to create equivalent groups, or if the variable
being studied is a subject variable involving two different groups (males vs. females).
2. Dependent samples t-test (related) is used if the independent variable is a within-subjects factor, or if
two groups of people are formed in such a way that some relationship exists between them
(participants in group A are matched on IQ with participants in group B.
T-test Independent Groups - Independent groups design
- Ex post Facto design
T-test Dependent Groups (related)
- Matched groups design
- Repeated-measures design
T-Test examines the difference between the mean scores for two samples and determines (with
probability) whether this difference is larger than would be expected by chance factors alone. If the
difference is significantly large and if potential confounding var. can be ruled out, then the researcher
can conclude with high probability that the differences between the means reflect a real effect.
-Homogeneity of variance means that the variability of each set of scores being compared should to be
similar, so if the standard deviation for one group is significantly larger than the standard deviation for
the other group, there is a problem. If the tests for homogeneity of variance indicate a problem,
inferential analyses other than t test can be used called nonparametric tests.
Single Factor – More Than Two Levels
-Single-factor multilevel designs involves one I.V. with more than 2 levels.
-This also includes both between and within-subject designs of the same four types of designs.
Advantage of multilevel designs– allows the researcher to discover nonlinear effects.
- When there are only 2 levels, you get a linear effect when actually there isn’t one. However,
with more levels, it lets you discover the nonlinear effect.
- It provides more information and often provide for more complex and interesting outcomes
than two-level designs
- Adding levels can also function as a way to test and perhaps rule out (falsify) alternative
explanations of the main result. Eg: Results would show comprehension improvement in two
level study: no context/1 rep (control group) and context-before (experimental group), however
with more levels/conditions, would illustrate other result.
Presenting the Data
1. Sentence Form: an approach that might be fine for reporting the results of experimental studies
with two or three levels. However, it’s tedious.
2. Table Form
3. Graph (Good for multiple I.V., between 2 – 4): A graph always places the dependent variable
D.V. on the vertical (Y) axis and the independent variable I.V. on the horizontal (X) axis.
Types of Graphs (Line vs. Bar Graph)
-If continuous variable: line graph is preferred; bar graph is ok too. Line graph can estimate in-between
effects from one point to another.
-If discrete variable (gender, political party, etc): Use bar graph. A line graph is NOT ok. Discrete Variable: represents a distinct category that is qualitatively different from another; no
interpolation can be done. Eg: Gender are qualitatively different not quantitative.
Continuous Variable: a variable for which an infinite number of values potentially exist. The variable
exists on a continuum. Eg: time, temperature, weight, height, distance, dosages etc.
In some case, variables can be both discrete and continuous. Different colors: red, yellow, green is
considered discrete variable. Shades of colors: shades of grey is considered continuous var. same as
-Interpolating is problematic if a study uses only two levels of I.V., the two levels are far apart, and the
relationship may show linear effect when in fact the relationship is actually nonlinear.
-Also, altering Y-axis (taking out half of the data) can mislead the uniformed consumer of research.
Analyzing Single-Factor, Multilevel Designs
Note: Descriptive statistics are things that describe the characteristics of your data: mean, mode,
median, standard deviation, coefficient correlation.
Whereas, inferential statistics help you determine whether your results are statistically significant or not
because results can sometime be a fluke, or by chance. Thus, we can determine how confidant we are in
the differences of the result.
-In a study, you have to do both descriptive and inferential statistics.
-t-test (kind of inferential stats)
Multiple t-tests: finding differences between each condition, group, or level.
-However, this is not an appropriate method because it can increases the risks of making a Type 1 error
– that is, the more t-tests you calculate, the greater the chances are of having one accidentally yield
significant differences between conditions. In order words, when you say that there are significant
differences between the groups when there really isn’t.
*The chances of making at least one Type 1 error when doing multiple t tests can be estimated by using
the formula; Probability of a Type 1 error = 1-(1-alpha) , wh