Textbook Notes (280,000)
CA (160,000)
Western (10,000)
PSYCH (5,000)
Chapter 4

Psychology 2660A/B Chapter Notes - Chapter 4: Content Validity, Construct Validity, Predictive Validity


Department
Psychology
Course Code
PSYCH 2660A/B
Professor
Natalie J Allen
Chapter
4

This preview shows pages 1-2. to view the full 8 pages of the document.
Chapter 4: Predictors Psychological Assessments
Predictor any variable used to forecast a criterion
Assessing the quality of predictors
Psychometric literally, the measurement (“metric”) of properties of the mind (“psychic”) it is the
standard used to measure the quality of psychological assessments
If a predictor is not both reliable and valid, it is useless
Reliability
o Refers to consistency and stability
o Reliability the consistency, stability, or equivalence of a measure
o A measure should yield the same estimate on repeated use if the measured trait hasn’t been
changed
o Test-retest reliability
Reveals the stability of test scores upon repeated applications of the test
Simplest assessment of a measuring device’s reliability
Measure something at 2 different times and compare scores
Coefficient of stability: the higher, the more reliable the scores (~.70 = acceptable)
The shorter the interval between two tests, the higher the test-retest reliability coefficient
o Equivalent-form reliability
Reveals the equivalence of test scores between two versions or forms of the test
Two forms of a test which measure the same attribute given to an individual
researcher correlates the two
Correlation of equivalence
Least popular it’s hard to make two different tests for the same thing
o Internal-consistency reliability
Reveals the homogeneity of the items comprising a test
Split-half reliability test is given and then scored twice, once for the odd numbers and
once for the even correlate the two
The longer the test, the greater its reliability
Cronbach’s alpha or Kuder-Richardson 20 (KR20) each test item is a minitest the
response to each item is correlated with the response to every other item
Homogeneous test = similar item content = high internal consistency reliability coefficient
Used frequently
o Inner-rater reliability
Reveals the degree of agreement among the assessments of two or more raters
Conspect reliability
Used frequently
Validity
o Refers to accuracy
o Reliability is inherent in a measuring device; validity depends on the use of a test
o Validity a standard for evaluating tests that refers to the accuracy or appropriateness of
drawing inferences from test scores
o It’s not “valid/not valid” – it can range from “not valid” to “highly valid”
o Construct validity
Construct validity the degree to which a test is an accurate and faithful measure of the
construct it purports to measure
Convergent validity coefficients reflect the degree to which the scores from our new
test of intelligence (for ex.) and the existing measures of intelligence converge in
assessing intelligence
Divergent validity coefficients reflect the degree to which scores from two tests that
shouldn’t correlate diverge from each other in assessing unrelated concepts
Tests with high construct validity = most freq used, most widely accepted
Only linkage tested: measure of construct 1  measure of construct 2
o Criterion-related validity
Criterion-related validity degree to which a test forecasts or is statistically related to a
criterion

Only pages 1-2 are available for preview. Some parts have been intentionally blurred.

Concurrent validity used to diagnose the existing status of some criterion
No time interval between collecting the predictor and criterion data
Predictive validity used to forecast future status
Determines whether there is a relationship between predictor scores and criterion scores
based on a sample of employees for whom we have both set of scores
Validity coefficient reveals the degree of association between two variables
Desired range: .30 - .40
o Content validity
Content validity degree to which subject matter experts agree that the items in a test
are a representative sample of the domain of knowledge the test purports to measure
Degree to which a predictor covers a representative sample of the behavior
being assessed
More scientifically relevant
No correlation coefficient, but uses subject matter experts test questions and is revised
until it has a high content validity
Face validity the appearance that items in a test are appropriate for the intended use
of the test by the individuals who take the test
More practically relevant
Content validity > face validity
Content matched tests did not result in higher criterion-related validity coefficients than
did tests that did not match the content of the job
Predictor development
I/O psychologists have developed a broad array of predictor measures designed to help us make
decisions (e.g. hire or not hire) about individuals these predictor measures are classified on two
dimensions:
1. Whether the predictor seeks to measure directly the underlying psychological construct in
question OR whether it seeks to measure a sample of the same behavior to be exhibited on the
job
Behavioral sampling: samples the types of behavior exhibited on the job lacks the
breadth of coverage of a paper-and-pencil test doesn’t measure if they could
eventually learn this task with proper training
2. Whether they seeks to measure something about the individual currently or something about the
individual in the past
Job interviews, letters of recommendation
Measure past, current, and predict future behaviors
Psychological tests and inventories
Inventory method of assessment in which the responses to questions are recorded and interpreted
but are not evaluated in terms of their correctness, and in a vocational interest inventory
o No right or wrong answer (as compared to tests where there are right or wrong answers)
Types of tests
Speed versus power tests
o Speed tests a type of test that has a precise time limit; a person’s score on the test is the
number of items attempted in the time period
Large number of easy questions; always will get them right
Total score is the number of items answered
Reflects speed of work
o Power tests a type of test that usually does not have a precise time limit; a person’s score on
the test is the number of items answered correctly
Difficult questions; cannot get them all right
No time limit (unless for the convenience of tester)
Total score is number of q’s answered correctly
Individual versus group tests
o Individual tests a type of test that is administered to one individual test taker at a time
Uncommon because time consuming
Administrator must play an active role (asking q’s, demonstrations, etc)
You're Reading a Preview

Unlock to view full version

Only pages 1-2 are available for preview. Some parts have been intentionally blurred.

o Group tests type of test that is administered to more than one test taker at a time
Most common
No active participation by administrator is required
Ex. Army alpha and army beta tests
Time and cost efficient
Paper-and-pencil versus performance tests
o Paper-and-pencil tests a method of assessment in which the responses to questions are
recorded on a piece of paper
Most common in business and education
Answers in MC or essay form
Doesn’t test physical ability
o Performance test type of test that requires the test taker to exhibit physical skill in the
manipulation of objects
Manipulate objects/equipment
Ex. Typing test, or test of finger dexterity
o Driving license needs both written and performance test
Sources of information about testing
Mental measurements yearbooks (MMY) a classic set of reference books in psychology that provide
reviews and critiques of published tests in the public domain
o Most important source of info about testing
Test content
Intelligence tests
o Intelligence or cognitive ability = most heavily researched construct in all of psychology
o General mental ability (g) predictive of success in most jobs
General level of intellectual ability
o Criterion-related validity of g is impressive range: .40 - .60
o g is to psychology what carbon is to chemistry
o g is a ubiquitous predictor of a wide variety of performance criteria
o But this is oversimplification of the complexity of intelligence
o The controversy regarding the assessment of intelligence rests primarily on the adequacy of
measuring g only or assessing multiple cognitive abilities in forecasting job behavior
Current research: measuring g = superior predictive accuracy in future job performance
o Sternberg’s triarchic theory of intelligence:
Academic intelligence what intelligence tests typically measure; fluency with words and
numbers
Practical intelligence everyday world competence; not highly related to academic
intelligence
Creative intelligence ability to create novel and appropriate/useful work; writing, art,
advertising
o Concept of practical intelligence is intended to complement, rather than to contradict, the
narrower views of g-based theories of intelligence
Mechanical aptitude tests
o Require a person to recognize which mechanical principle is suggested by a test item
o Highly predictive of performance in manufacturing/production jobs
o Women generally perform worse
Personality inventories
o Do not have right or wrong answers
o Items are scores according to a predetermined key such that responding one way or another to
an item results in a higher or lower score on a particular scale
o Big 5 theory of personality defines personality in terms of five major factors:
1. Neuroticism the person’s characteristic level of stability vs instability
2. Extraversion tendency to be sociable, assertive, active, talkative, energetic, and
outgoing
3. Openness to experience the disposition to be curious, imaginative, and unconventional
4. Agreeableness disposition to be cooperative, helpful, and easy to get along with others
5. Conscientiousness disposition to be purposeful, determined, organized, and controlled
You're Reading a Preview

Unlock to view full version