Class Notes
(808,840)

Canada
(493,410)

University of Toronto Scarborough
(30,817)

Psychology
(7,620)

PSYB01H3
(260)

Anna Nagy
(133)

Lecture 4

# PSYB01 - Lecture 4 Notes

Unlock Document

University of Toronto Scarborough

Psychology

PSYB01H3

Anna Nagy

Fall

Description

Research Methods
September 26th, 2013 - Lecture 4
Measurement Concepts
Learning Objectives
Define reliability of a measure of behavior and describe the difference between test-retest, internal
consistency, and interrater reliability.
Discuss ways to establish construct validity, including predictive validity, concurrent validity, convergent
validity, and discriminant validity.
Describe the problem of reactivity of a measure of behavior and discuss ways to minimize reactivity.
Describe the properties of the four scales of measurement: nominal, ordinal, interval, and ratio.
Reliability of Measures
Consistency or Stability of a Measure of Behavior
True Score
Measurement Error
Pearson Product-Moment Correlation Coefficient
True Score
Theory of Reliability Reliability
Proportion of true scores in a measure
But characteristic of a measure taken across individuals
Look at variance in scores across subjects
Variance of true scores (T)
____________________
Variance of the measure (X)
Don’t know true scores, so can’t calculate reliability of true scores…must estimate
Assume measured scores (X) related to each other to the degree that they share true scores
Estimate of reliability=correlation between two scores of the same measure
How to Estimate Reliability--Pearson r
Correlation coefficients called regression coefficients (reliability coeffiecient) used for ratio and
interval scales
Coefficients range from 0.00 to - 1.00 and 0.00 to +1.00
Estimating Reliability
Estimation of Reliability
Will always range between 0 & 1.
Examples:
Reliability of .6 means that approx. 60% of scores attributable to true score and 40% to error
Reliability of .75 means that approx. 75% of scores attributable to true score and 25% to
error…
Internal Consistency Reliability
Test-Retest Reliability Assessed by measuring the same individuals two points in time (subjects position in relative
distribution of scores should be similar)
Assumes no substantial change in construct or condition over time
Vulnerable to artificiality
Vulnerable to maturation
Parallel forms
- challenging
Use single measurement on one occasion to determine an estimate of reliability
Administered to a large group of people
Reliability Estimates
Internal Consistency Reliability
Inter-Item Correlations
Internal Consistency Reliability
Inter-total Correlations
Split-half reliability Cronbach’s alpha
Equal to the average of all split half correlations
items that do not correlate with other items can be eliminated from the measure to increase
reliability
Reliability of Measures
Interrater Reliability - Correlation between the observations of raters
Inter-rater Reliability
http://www.youtube.com/watch?v=hz65AOjabtM
How to Calculate Interrater Reliability--Percentage agreement
Example: Inter-observer or Interrater Reliability
Number of times two observers agree
_______________________________ x100
Number of opportunities to agree
How to Calculate Interrater Reliability—Correlations between raters
• Calibrate the observers for continuous measures
• Rating of overall noise level in classroom on a scale of 1 to 10
• Have raters provide ratings every five minutes
• Correlation between overall ratings would give estimate of reliability between raters
Research and Critical Thinking: Assessing the impact of parent and teacher agreement on diagnosing atten

More
Less
Related notes for PSYB01H3