Textbook Notes (280,000)
CA (170,000)
UTSC (20,000)
Psychology (10,000)
PSYB01H3 (600)
Anna Nagy (200)
Chapter 5

PSYB01H3 Chapter Notes - Chapter 5: Inter-Rater Reliability, Discriminant Validity, Content Validity


Department
Psychology
Course Code
PSYB01H3
Professor
Anna Nagy
Chapter
5

Page:
of 2
PSYBO1 RESEARCH METHODS CHAPTER NOTES
CHAPTER 5: MEASURING CONCEPTS
Reliability: consistency or stability of a measure of behaviour
Same result is observed with every administration
A reliable measure does not fluctuate from one reading to another
Proportion of true scores in a measure: Variance of true scores (T) / Variance of the
measure (X)
True score: real score on the variable
Measurement error: unreliable measure of intelligence contains considerable
measurement error
Reliability can be increased by making multiple measures
Pearson product-moment correlation coefficient (r)
Test-retest Reliability: assessed by measuring the same individuals at two points in time
-the correlation maybe artificially high because the participants remember how they
responded the first time.
-alternate forms reliability is used to avoid this problem: it requires administering two
different forms of the same test to same individuals at two points in time
Internal consistency reliability: assessment of reliability using responses at only one point in
time
Split-half reliability- the correlation of the total score on one half of the test with the total
score on the other half
-combined measure will have more items and will be more reliable [Spearman-Brown split-
half reliability coefficient]
cronbach’s alpha: provides us with the average of all possible split-half reliability coefficients
-scores on each item are correlated with scores on every other item
-average of all the inter-item correlation coefficients and the number of items in the measure
-item-total correlation- provide information about each individual item.
Interrater Reliability- the extent to which raters agree in their observations
-high interrater reliability is obtained when most of the observations result in the same
judgment.
Construct validity is a question of whether the measure that is employed actually measures the
construct it is intended to measure
Face validity: the content of the measure appears to reflect the construct being measured. Do the
procedures used to measuere the variable appear to be an accurate operational definition of
the theoretical variable?
Content validity- the content of measure is linked to the universe of content that defines the
construct.
Predictive validity- scores on the measure predict the behaviour on a criterion measured at a
future time. The criterion measure is based on the future behaviour or outcomes
Concurrent validity- scores on the measure are related to a criterion measured at the same time. A
common method is to study whether two or more groups of people differ on the measure in
expected ways; another is to study how people who score either low or high on the measure
behave in different situations
Convergent validity- score on the measure are related to other measures of the same construct
Discriminant validity- scores on the measure are not related to the other measures that are
theoretically different.
Reactivity:
find more resources at oneclass.com
find more resources at oneclass.com
PSYBO1 RESEARCH METHODS CHAPTER NOTES
A measure is said to be reactive if awareness of being measured changes an individual’s
behaviour.
A reactive measure tells what the person is like when he/she is aware of being observed.
Scales of measurement:
Nominal- no numeric values; impossible to define any quantitative values and or differences
between/across categories
Ordinal- rank ordering; numeric values limited, intervals between items not known
Interval- numeric properties are literal; assume equal interval between values. No true zero
Ratio- zero indicates absence of variable measured; can form ratios
find more resources at oneclass.com
find more resources at oneclass.com