PSYCH 303 Chapter 5: Identifying Good Measures, Part II

6 views2 pages
9 Feb 2017
School
Department
Course

For unlimited access to Textbook Notes, a Class+ subscription is required.

Chapter 5: Identifying Good Measures
Reliability of measurement- are the scores consistent?
The construct validity of a measure has two aspects: reliability, which refers to how
consistent the results of a measure are, and validity, which concerns whether the
operationalization is measuring what it is supposed to measure
Introducing three types of reliability
Test-retest reliability
The researcher gets consistent scores every time he or she uses the measure
Can apply whether the operationalization is self-report, observational, or physiological
Primarily relevant when researchers are measuring constructs (such as intelligence,
personality, religiosity) that they expect to be relatively stable in most people
Interrater reliability
Consistent scores are obtained no matter who measures or observes
Two or more independent observers will come up with consistent (or try similar)
findings
Most relevant for observational measures
Internal reliability/consistency
A study participant gives a consistent pattern of answers, no matter how the
researcher has phrased the question
Using a scatterplot to evaluate reliability
Scatterplots can show interrupter agreement/disagreement
Perfect agreement is indicated when the points hover close to the sloping line in a
scatter plot
Disagreement is represented when the individual dots scatter wild from a straight line
drawn through them
Using the correlation coefficient r to evaluate reliability
Anatomy of a scatterplot
Researchers can use a single number, called a correlation coefficient, or r, to indicate
how close the dots on a scatterplot are to a line drawn through them
The direction of the relationship is represented by the the slope direction, which can be
positive (sloping up), negative (sloping down), or zero (not sloping at all)
Spread of dots represents the strength of the relationship: the relationship is strong
when dots are close to the line and weak when they're spread out
The numbers below the scatterplot are the correlation coefficients, or r
The r indicted the same two things as the scatterplot: the direction of the relationship
and the strength of the relationship
Test-retest reliability
If r is positive and strong (for test-retest reliability, we might expect 0.50), we would
have a good test-test reliability
If r is positive and weak, we would know that the participants’ score on the test
changes from the first trial to the second
Interrater reliability
If r is positive and strong (r = 0.70 or higher), we would have a good interrater reliability
If r is positive and weak, we could not trust the observers’ ratings
We would retrain the coders or refine our operational definition so it could be more
reliably coded
A negative r would individuate a big problem
Unlock document

This preview shows half of the first page of the document.
Unlock all 2 pages and 3 million more documents.

Already have an account? Log in

Get access

Grade+
$10 USD/m
Billed $120 USD annually
Homework Help
Class Notes
Textbook Notes
40 Verified Answers
Study Guides
1 Booster Class
Class+
$8 USD/m
Billed $96 USD annually
Homework Help
Class Notes
Textbook Notes
30 Verified Answers
Study Guides
1 Booster Class