Study terms for test 2
Reliability:Index of the consistency of a measuring instrument in repeatedly providing
the same score for a given participant
Internal consistency reliability: Index of the homogeneity of the items of a measure.
Test re-test reliability: Index of the consistency in scores over time, range of .8=stable
Split half reliability: A measure of consistency where a test is split in two and the scores for each
half of the test is compared with one another. If the test is consistent it leads the experimenter to
believe that it is most likely measuring the same thing. This is not to be confused with validity
where the experimenter is interested if the test measures what it is supposed to measure.
Reliability index: Correlation coefficients quantify test-retest and interrater reliability, ranging
from -1.00 to +1.00 (although negative correlations for reliability are unlikely unless something is
seriously wrong like raters using different rating scales) . A correlation of +1.00 means perfect
reliability, 0.00 means no reliability.
Alternate forms: Form of reliability of a survey instrument, overcome the "practice effect", which
is typical of the test-retest reliability. Change the wording of the survey questions in a
functionally equivalent form, or simply to change the order of the questions in the first and the
second survey of the same respondents. A common quantitative measure of the alternate-form
reliability is the value of the correlation coefficient between the results obtained in two surveys -
with the initial and re-worded questions.
Threats to reliability:
- Internal – Low inference descriptors, multiple researchers, participant researchers, peer
examination, mechanically (hand-written) recorded data
- External – researcher status position, informant choices, social situations/conditions,
analytic construct/premises, methods of data collection and analysis (descriptive)
Rosenthal effect: (Pygmalion effect) - the phenomenon in which the greater the expectation
placed upon people, the better they perform. People will internalize their positive labels, and
those with positive labels succeed accordingly. Often seen in education and social class.
Hawthorne effect: (commonly referred to as the observer-expectancy effect) is a form of
reactivity whereby subjects improve or modify an aspect of their behavior, which is being
experimentally measured, in response to the fact that they know that they are being studied
John Henry effect: A tendency for members of the control group in certain experiments to adopt
a competitive attitude towards the experimental group, thereby negating their status as controls.
The control group will work harder to compete and beat the experimental group. Subject reactivity: Participants responding differently because they know they are being
Experimenter reactivity: Researchers unconsciously influencing participants
Double-blind: Neither the experimenter nor the participant knows what condition the participant
Single-blind: The experimenter does not know what condition the participant is in
Placebo effect: Phenomenon where an otherwise ineffective intervention in a study induces an
improvement in the patient’s condition. Thought to be due to expectation.
• Research tests specific hypotheses - generated from the initial research idea through a
series of steps
• A research idea can generate dozens of research hypotheses depending on how:
- it is translated into a statement of the problem
- the variables are operationally defined
• Tests The Null Hypothesis, the confounding variable hypothesis, and the casual
Null hypothesis: states that there is NO difference between the population means;
compare sample means to test the null hypothesis; population parameters and sample
Causal hypothesis: states that the independent variable has a casual effect on dependent
• Accept casual hypothesis if you:
- Reject null hypothesis
- Rule out potential confounding variable hypothesis (based on appropriate controls)
Statistical validityare the statistical tests accurate?
• Threatened by:
- Unreliable measures
- Violations of statistical assumptions • Strengthened by
- Using well validated measures
- Having approximately equal sample sizes in each group
Threats to internal validity:
• Internal validity: is the independent variable responsible for the observed changes in the
• Threatened by Confounding variables
• Strengthened by adding adequate controls to reduce or eliminate confounding
Contemporary History: refers to the effect external events have on subjects between the various
measurements done in an experiment. These experiences function like extra, and unplanned,
Maturation: refers to how subjects naturally can change over the passage of time (rather than
due to the treatment). For example, the more time that passes in a study the more likely
subjects are to become tired and bored, more or less motivated as a function of hunger or thirst,
Instrumentation: refers to the objectivity, reliability and validity of the research measurements.
Data that is biased (nonobjective) or unreliable threatens a study's internal validity.
Practice effects: Any change in performance that results from previous exposure to the
Regression to the mean: Phenomena whereby an individual is measured on test and
obtains an extremely low or high score one on reassessment tends to move towards a
more average score. Maybe misinterpreted as misrepresenting a real change.
Differential mortality: attrition and selection threat – only effected if multigroup design. Usually
occurs if the groups are only categorized for one reason (ex: male vs. female), but nothing else.
The results of the experiment may be altered by the confounding effects and not because of the
Nonequivalence of groups: quasi-experimental design in which 2 or more groups that may not
be equivalent are compared on the dependent measure after a manipulation in one of more of
Threats to external validity:
• Do the results apply to the broader population? • Threatened by
- Unrepresentative samples
- Generalizing beyond the limits of the sample
• Strengthened by
- Gathering a representative sample (if possible)
- Clearly describing sample, so that other researchers will know the limits of
Randomization: Any procedure that assigns a value in a random manner.
Counterbalance: Control for sequence effects. With complete counterbalancing, all possible
arrangements of conditions are included.
Correlation: degree of linear relationship between two or more variables, correlations can be
used for prediction, cannot prove a theory but can negate a theory, CANNOT establish
causation. Range from -1.00 to +1.00, size indicates strength and direction of relationship.
Correlation Coefficient: index of the degree of linear relationship between two or more variables
Type 1 Error: probability of rejecting the null hypothesis when it is true.
Type 2 Error: probability of NOT rejecting the null hypothesis when it is false.
P- value: the probability of obtaining the statistic (e.g., t or F) or a larger statistic by chance if the
null hypothesis is true.
Effect size: index of the size of the difference between groups, expressed in standard deviation
Statistical power: ability of an inferential s