For unlimited access to Textbook Notes, a Class+ subscription is required.

•Chapter 5: Identifying Good Measures

•Reliability of measurement- are the scores consistent?

•The construct validity of a measure has two aspects: reliability, which refers to how

consistent the results of a measure are, and validity, which concerns whether the

operationalization is measuring what it is supposed to measure

•Introducing three types of reliability

•Test-retest reliability

•The researcher gets consistent scores every time he or she uses the measure

•Can apply whether the operationalization is self-report, observational, or physiological

•Primarily relevant when researchers are measuring constructs (such as intelligence,

personality, religiosity) that they expect to be relatively stable in most people

•Interrater reliability

•Consistent scores are obtained no matter who measures or observes

•Two or more independent observers will come up with consistent (or try similar)

ﬁndings

•Most relevant for observational measures

•Internal reliability/consistency

•A study participant gives a consistent pattern of answers, no matter how the

researcher has phrased the question

•Using a scatterplot to evaluate reliability

•Scatterplots can show interrupter agreement/disagreement

•Perfect agreement is indicated when the points hover close to the sloping line in a

scatter plot

•Disagreement is represented when the individual dots scatter wild from a straight line

drawn through them

•Using the correlation coefﬁcient r to evaluate reliability

•Anatomy of a scatterplot

•Researchers can use a single number, called a correlation coefﬁcient, or r, to indicate

how close the dots on a scatterplot are to a line drawn through them

•The direction of the relationship is represented by the the slope direction, which can be

positive (sloping up), negative (sloping down), or zero (not sloping at all)

•Spread of dots represents the strength of the relationship: the relationship is strong

when dots are close to the line and weak when they're spread out

•The numbers below the scatterplot are the correlation coefﬁcients, or r

•The r indicted the same two things as the scatterplot: the direction of the relationship

and the strength of the relationship

•Test-retest reliability

•If r is positive and strong (for test-retest reliability, we might expect ≥0.50), we would

have a good test-test reliability

•If r is positive and weak, we would know that the participants’ score on the test

changes from the ﬁrst trial to the second

•Interrater reliability

•If r is positive and strong (r = 0.70 or higher), we would have a good interrater reliability

•If r is positive and weak, we could not trust the observers’ ratings

•We would retrain the coders or reﬁne our operational deﬁnition so it could be more

reliably coded

•A negative r would individuate a big problem