19 Apr 2012
School
Department
Course
Professor
Lecture Notes
Hypothesis testing: not an exact science
- Based on data you see
Steps in hypothesis testing
1. Develop null & alternative hypothesis
2. Set alpha
3. Calculate power to determine sample size
4. Collect data
5. Collect statistic and p-value
6. Accept or reject the null
Sampling distribution of means
- Sampling distribution is a frequency distribution of a sample statistic
- Distribution of sample means is usually referred to as the sampling distribution of means, or
the sampling distribution of the mean
Develop two hypotheses
- All hypotheses test are comprised of 2 mutually exclusive hypothesis
- No difference between control and experimental
- Null: both means come from the same population
oAny difference between the means is due to chance
- Null: µcontrol = µexperimental
oThis is the hypothesis that you are able to test
oIf the null hypothesis is probably not true, what must we conclude
- Alternative
oNo direction hypothesis (2-tailed)
µcontrol ≠ µexperimental
odirectional hypothesis (1 tailed)
µexperimental > µcontrol
- cant test this hypothesis directly, but can find support by rejecting the null hypothesis
We cannot just look at means and determine if they differ
- remember random error
- statistical fluctuations in the measured data
- random error is always present
Why are means different?
- The means are different because of random error (ie/ any difference is due to chance) this is
the null hypothesis
- Means are from two different populations (ie. There is a true difference between the means)
alternative hypothesis
- In any situation, we can never know the truth, but we can calculate the probability that any
difference are due to chance (p-value)
Set the alpha
- “cut off” probability to reject the null
- Conventionally alpha = 0.05 or 5%
- There is only a 5% probability that the difference is due to chance, then we reject the null
Collect data
- Calculate statistic and p-value associated with the test
- Compare the p-value to the alpha
- Make decision
oReject the null (if p < alpha)
oFail to reject the null (if p > alpha)
Statistics: does not mean you have to be certain
Reject Fail to reject
Null is
true
Type 1 Right choice: p= 1-
alpha
Null is
false
Right choice:
Power
Type 2
Effect size
- Just because a difference is significant doesn’t mean it’s big
- Refers to how large the difference between your variables
- Greater effect size = greater similarities
Power: probability that we will reject the null when there is a significant difference
- If: we reject the null – increases confidence that our study can detect a difference when it’s
there
- Or if: fail to reject – increases confidence that it’s due to insufficient power to detect the effects
Assuming its there, how powerful does the telescope (power_ need to be
- Power to detect your size? (telescope)
- What size of effect are you trying to detect? (object)
- Sample size to detect effect size (magnify)
- Lots of random error (if its cloudy)
Independent t-tests
- Difference in means that we observe between 2 groups more than we would expect to see
based on chance alone
- T-test: 1 dependent variable and 1 independent variable containing 2 groups
Effect Size
- Is only relevant when effect is significant
- P-value is not a good indicator of relationship size
- Strength of the effect size measure is that it provides the strength of the relationship
- Results are statistically significant if p < alpha
Standard deviation
- Average amount of scores differ from the mean
- Add all scores and % by # of observations
Z-scores
- Standardization of scores
- Observations expressed in standard deviation units from the mean
- Allows for comparison of a single score
- Calculate: [individual-mean] % standard deviation
- Sign: indicates above or below the mean
- # how far from the mean
Correlational research design: describe relationships among measured variables
Scatterplots
- Visual tool used to study the nature of relationship
- Idea of strength, direction, if its linear/nonlinear
- X (predicator) = horizontal
- Y (outcome) = vertical
- Closer to 1.00 or -1.00 are more likely to have a relationship
Interpreting correlation
Strengt
h
r R2
Weak .1
to .3
1 to
10%
Moderat
e
.3
to .5
10 to
25%
Strong > .5 > 25%
Factors that affect correlation
- Restricted range
oWhen sample contains restricted range of scores
oIf range restrictions be cautious in generalizing beyond the range for data available
- Heterogeneous samples
oSub samples (eg/ gender) may artificially increase/decrease overall r
oSolution: calculate r for subsamples (ie. Conduct study separately for males and females)
- Outliers
oCan disproportionally increase or decrease r
oSolution
Compute r with and without outliers
Get more data for outlying values
Recode outliers as having more conservative scores
Transformation
Recode variable into lower level measurement
Non-linear relationships
- Not all relationships are best described by straight line
- Curvilinear relationships: sometimes relationships change direction
Chi-square
- Nominal
- Compares actual proportions to proportions expected by chance
- Non-parametric: no assumptions associated with this test
Checklist
1. Graphs & Scatterplots