19 Apr 2012

School

Department

Course

Professor

Lecture Notes

Hypothesis testing: not an exact science

- Based on data you see

Steps in hypothesis testing

1. Develop null & alternative hypothesis

2. Set alpha

3. Calculate power to determine sample size

4. Collect data

5. Collect statistic and p-value

6. Accept or reject the null

Sampling distribution of means

- Sampling distribution is a frequency distribution of a sample statistic

- Distribution of sample means is usually referred to as the sampling distribution of means, or

the sampling distribution of the mean

Develop two hypotheses

- All hypotheses test are comprised of 2 mutually exclusive hypothesis

- No difference between control and experimental

- Null: both means come from the same population

oAny difference between the means is due to chance

- Null: µcontrol = µexperimental

oThis is the hypothesis that you are able to test

oIf the null hypothesis is probably not true, what must we conclude

- Alternative

oNo direction hypothesis (2-tailed)

µcontrol ≠ µexperimental

odirectional hypothesis (1 tailed)

µexperimental > µcontrol

- cant test this hypothesis directly, but can find support by rejecting the null hypothesis

We cannot just look at means and determine if they differ

- remember random error

- statistical fluctuations in the measured data

- random error is always present

Why are means different?

- The means are different because of random error (ie/ any difference is due to chance) this is

the null hypothesis

- Means are from two different populations (ie. There is a true difference between the means)

alternative hypothesis

- In any situation, we can never know the truth, but we can calculate the probability that any

difference are due to chance (p-value)

Set the alpha

- “cut off” probability to reject the null

- Conventionally alpha = 0.05 or 5%

- There is only a 5% probability that the difference is due to chance, then we reject the null

Collect data

- Calculate statistic and p-value associated with the test

- Compare the p-value to the alpha

- Make decision

oReject the null (if p < alpha)

oFail to reject the null (if p > alpha)

Statistics: does not mean you have to be certain

Reject Fail to reject

Null is

true

Type 1 Right choice: p= 1-

alpha

Null is

false

Right choice:

Power

Type 2

Effect size

- Just because a difference is significant doesn’t mean it’s big

- Refers to how large the difference between your variables

- Greater effect size = greater similarities

Power: probability that we will reject the null when there is a significant difference

- If: we reject the null – increases confidence that our study can detect a difference when it’s

there

- Or if: fail to reject – increases confidence that it’s due to insufficient power to detect the effects

Assuming its there, how powerful does the telescope (power_ need to be

- Power to detect your size? (telescope)

- What size of effect are you trying to detect? (object)

- Sample size to detect effect size (magnify)

- Lots of random error (if its cloudy)

Independent t-tests

- Difference in means that we observe between 2 groups more than we would expect to see

based on chance alone

- T-test: 1 dependent variable and 1 independent variable containing 2 groups

Effect Size

- Is only relevant when effect is significant

- P-value is not a good indicator of relationship size

- Strength of the effect size measure is that it provides the strength of the relationship

- Results are statistically significant if p < alpha

Standard deviation

- Average amount of scores differ from the mean

- Add all scores and % by # of observations

Z-scores

- Standardization of scores

- Observations expressed in standard deviation units from the mean

- Allows for comparison of a single score

- Calculate: [individual-mean] % standard deviation

- Sign: indicates above or below the mean

- # how far from the mean

Correlational research design: describe relationships among measured variables

Scatterplots

- Visual tool used to study the nature of relationship

- Idea of strength, direction, if its linear/nonlinear

- X (predicator) = horizontal

- Y (outcome) = vertical

- Closer to 1.00 or -1.00 are more likely to have a relationship

Interpreting correlation

Strengt

h

r R2

Weak .1

to .3

1 to

10%

Moderat

e

.3

to .5

10 to

25%

Strong > .5 > 25%

Factors that affect correlation

- Restricted range

oWhen sample contains restricted range of scores

oIf range restrictions be cautious in generalizing beyond the range for data available

- Heterogeneous samples

oSub samples (eg/ gender) may artificially increase/decrease overall r

oSolution: calculate r for subsamples (ie. Conduct study separately for males and females)

- Outliers

oCan disproportionally increase or decrease r

oSolution

Compute r with and without outliers

Get more data for outlying values

Recode outliers as having more conservative scores

Transformation

Recode variable into lower level measurement

Non-linear relationships

- Not all relationships are best described by straight line

- Curvilinear relationships: sometimes relationships change direction

Chi-square

- Nominal

- Compares actual proportions to proportions expected by chance

- Non-parametric: no assumptions associated with this test

Checklist

1. Graphs & Scatterplots