Textbook Notes (280,000)
CA (160,000)
York (10,000)
PSYC (4,000)
PSYC 2030 (100)
Chapter 12

PSYC 2030 Chapter Notes - Chapter 12: Odds Ratio, Meta-Analysis, Dependent And Independent Variables


Department
Psychology
Course Code
PSYC 2030
Professor
Krista Phillips
Chapter
12

This preview shows pages 1-2. to view the full 8 pages of the document.
Study Guide: Chapter 12 Understanding P values and Effect Size Indicators
Why Is It Important to Focus Not Just On Statistical Significance?
Besides describing data and measuring relationships many behavioral researchers are usually
interested in making comparisons using statistical tests such as t, F, and chi-square
Null hypothesis significance testing (NHST) -> traditional procedure
Much has been written about common misconceptions regarding statistical significance, it
remains a source of confusion for many people
J.D. Miller noted that many meical specialists mistakenly view “statistical significance” as a proxy
for the “degree of improvement a new treatment must make for it to be clinically meaningful”
Statistical significance tells us nothing about the degree of improvement a new treatment must
make for it to be clinically meaningful, but effect size indivators, properly interpreted, can often
give us insights about the practical significance of an obtained effect
On the other hand, misconceptions and illusions abound about the implications of certain effect
size indicators and can result in people drawing unwarranted conclusions
Box 12.1 Three Families of Effect Sizes
We call the three families of effect sizes a) the correlation (or r-type) family, b) the difference
family and c) ratio family
The emphasis in this chapter is on r-type effect size indicators, such as the point-biserial r and
the phi coefficient
Other member of difference family called risk difference
Also explained later in this chapter is the odds ratio and relative risk (both belong to the ratio
family of effect size indicators)
Advantage of r-type effect size indicators is their usefulness when predictions specifically involve
more than two groups or more than two conditions
Difference-type and ratio-type effect size indicators are not so naturally applicable in such
situations
Common thread throughout rest of text is the general relationship between the p value and the
effect size, as given by the following conceptual equation :
o Significance test = size f effect X size of study
The general relationship means that any test of statistical significance (such as t, F, or chi square)
can be shown to consist of two components a) an indicator of the size of the effect and b) an
indicator of the sampling units (e.g., the total N)
The conceptual equation shows that the value of the significance test is the product of these
two components
o Thus, unless the size of the obtained effet is exactly 0 (which is quite rare), the larger the
size of effect component or the larget the size of study component (e.g., the larger the
N), the larger is the value of the signifianve test and, therefore, the smaller (and usually
more coveted) is the p value
o In other words, focusing our attention only on statistical significance (the p value) would
not tellus whether the effect size, the total N or both were priariy rersponsible for the
level of statistical significance reached

Only pages 1-2 are available for preview. Some parts have been intentionally blurred.

o Furthermore, even if the effect size component of the significance test were mainly
responsible, a question would still linger concerning which aspect of the effect size
indicator was the primary contributing factor
o Another common mistake is to equate “nonsignificance” (frequently defined as p > .05)
as equivalent to estimating an effect size equal to zero
o Later in this chapter, we will describe a useful statistic (the counternull statistic) that can
eliminate this error
o We will also discuss concept of statistical power and illustrate how a power analysis is
done
What Is the Reasoning Behind Null Hypothesis Significance Testing?
Analogy: imagine you’re strolling along Atlantic City Boardwalk when a shady character
approaches and whispers he has a quarter he is willing to sell you for “only five dollars”
o You ask what about this coin makes it worth more than its face value?
o He answers, this is a quarter with a special property, when properly used it could win an
enterprising person a lot of money because it does not always come up heads and tails
with equal regularity
o Intstead, one outcome is far more likely than the other, and a person with a touch of
larceny in his soul could bet on the outcome and win a tidy sum
o Because you haven’t walked away yet he adds – it might sound like a cock-and-bull
story, but flip the coin and see for yourself
If the coin is not what it is said to be, meaning it’s ordinary, then the probability of heads or tails
is always one chance in two
So you accept the challenge and test the probability of tails
You flip nine times and each time it comes up heads
Would you believe him now?
If your answer is yes, would you if you flipped nine times and once it came up heads, 8 tails?
This is the essential question in NHST
You can be as stringent as you like in setting a rejection criterion but you may eventually pay for
this decision by rejecting what you perhaps should not
When you decided to test whether the probability of heads “does or does not” equal the
probability of tails, two hypothesis were implied
One was that the quarter is unbiased (the probability of heads does equal the probability of
tails); the second implicit hypothesis was that the coin is biased (the probability of heads does
not equal the probability of tails)
You can think of the experiment of tossing a coin as a way of trying to determine which of these
hypotheses you cannot logically reject
In statistical terms, the name for the first hypothesis (that the quarter is unbiased) is the null
hypothesis (H0) and the second, that the quarter is biased is the alternative hypothesis (H2)
Null hypothesis : the probability of heads equals the probability of tails in the long run because it
is an ordinary quarter, and therefore getting a head or a tail is the result purely of chance (i.e.
the coin is not biased)
Alternative hypothesis: the probability of heads is not equal to the probability of tails in the long
run because it is not an ordinary quarter (i.e., the coin is biased)
Notice that these two hypotheses are mutually exclusive, that is, when one hypothesis is true,
the other must be false
You're Reading a Preview

Unlock to view full version

Only pages 1-2 are available for preview. Some parts have been intentionally blurred.

Experimenters who do NHST are usually interested in testing the specific H0 (i.e., no difference)
against a general H1 (i.e., some difference)
In a between-subject design with an experimental and a control group, the null hypothesis
generally implies no difference in the success rate between experiment group and the contrl
group (e.e, no difference in survival rates, performance rates, or however else “success rates”
may be defined)
The idea behind NHST is to see whether we can reject H0 and yet be reasonably sure that we
will not be wrong in doing so
This leads to further ida that there are two kinds of decision risks of general concern in NHST,
called Type I error and Type II error
Box 12.2 How Are Probabilities Determined?
One characteristic of probabilities is that if all outcomes are independent (i.e., one outcome is
not influenced by any other), the sum of all the probabilities associated with an even is equal to
1
If you throw an ordinary six-sided die, there are six possibilities and unless the die isloaded, the
probability of any particular outcome is 1/6 or .167
Summing all the independent probabilities gives us .167 X 6 = 1.00
Instead of throwing a die, suppose you have two coins and fli both at the same time
There are four possible combinations of heads and tails: HH,HT,TH, TT
In determining probabilities, the general rule is to count the total number of possible outcomes
and then to count the number of outcomes and then to count the number of outcomes yielding
the event you are interested in
The probability of that event is the ratio of the number you are looking for (the favorable event)
to the total number of outcomes
Ex probability of p of two heads (out of the four possible outcomes) can occur in only one way
(HH) and is therefore 1 divided by 4, so p = .25
The probability of only one head out of these four possible events can occur in two ways, ht or
TH and is therefore 2 divided by four, so p = .5
What Is the Distinction Between Type I Error and Type II Error?
Type I Error -> implies that the decision maker mistakenly rejected the null hypothesis when it is
in fact true and should not have been rejected
Type II Error -> implies the decision maker mistakenly failed to reject the null hypothesis when it
is, in fact false and should have been rejected
the risk (or probability) of making Type I error is called by three different names: alpha (a), the
significance level, and the p value
the risk or probability of making a Type II error is known by one name: beta (B)
to make the most informed decision, researcher who do NHST would, of course, like to know
what each type of risk is in a given case so that they can balance these risks in some way
5% significance level = you do not want to be wrong more than 1/20 times
If probability is less than 1/20, p<.05, will reject null hypothesis and buy the coin
If p > .05, will fail to reject null
Analogy is simplified and not exactly what goes on in NHST, it’s not a relational event – there is
only one variable: the result of the coin toss
You're Reading a Preview

Unlock to view full version