PSYC 3250 Lecture Notes - Lecture 6: Content Validity, Construct Validity, Reliability Engineering

6 views13 pages
12 Feb 2016
Department
Course
Professor
Validity
Review:
-What is reliability? – consistency of scores.
-Would we call a bathroom scale a reliable measure?
-What if we used the numbers from the bathroom scale to select people for
admission to university?
Validity—Two primary issues in psychological measurement: (does it measure what
its supposed to measure)
1. Determining whether a test measures what it is supposed to measure
(
validity of measurement
)
2. Determining whether that test can be used in making accurate decisions
(
validity for decisions
)
Sources of Validity Evidence
-Content validity
-Construct validity
-Criterion related validity
Content Oriented Validation—Content Domain: (it is what you are interested in)
Assignment: we have a one-facet domain
General: more complex (EX: di/erent types of IQ)
-Every psychological test is a systematic sample from a particular domain of
behaviours
-Content validity describes a judgement of how adequately a test samples
behaviour that is representative of the universe or domain of behaviour
Description of Content Domain: Risk Taking
Unlock document

This preview shows pages 1-3 of the document.
Unlock all 13 pages and 3 million more documents.

Already have an account? Log in
-A content domain has boundaries: A detailed description of the content
domain to be measured allows one to determine whether each test item lies
within the boundaries of the domain
Assessing Content Validity
-Content validity represents a judgement regarding the degree to which a test
provides an adequate sample of a particular content domain
-Basic procedure consists of three steps:
oDescribe the content domain
oDetermine the areas of the content domain that are measured by each
test item
oCompare the structure of the test with the structure of the content
domain
-To demonstrate content validity:
oTests should sample all parts of the content domain
oTests should devote the largest number of items to the larger, more
important areas of the domain
Content Validity: An Example—Humor Appreciation (Carretero-Dios, Perez, & Buela-
Casal, 2009)
-Steps:
oDe<ning the construct of interest by specifying its content domain
(deprived from literature review, and from subject mature experts)
oDeveloping items that are valid operational indicators of the domains
oDetermining to what extends each item is relevant for the component
of the construct it was create for by means of an assessment by judges
Step 1: De<ning the Content Domain (content of the test look like the
denition)
Subject:
1. Incongruity resolution (A fact which appears to not make sense <nally
becomes meaningful thanks to a logical rule)
2. Nonsense humour (A meaningless or ambiguous fact is presented and
generates more incongruity or is partially resolved)
3. Superiority-disparagement (jokes or cartoons that attempt to ridicule
characteristics of a given group or person eg. The region they are from, the
job they do, etc)
4. Sexual Humour
5. Black Humour (jokes or cartoons on dramatic events, catastrophes, or
physical or other damage)
Content Validity: An Example
-7 judges rated each item
Unlock document

This preview shows pages 1-3 of the document.
Unlock all 13 pages and 3 million more documents.

Already have an account? Log in
-Judges were asked which facet of the construct they believed each item
corresponded to
-CVI (Content Validity Index) was calculated as the proportion of judges that
matched the item to its intended facet
- They are looking at the items and based on the denition then they
are classifying the item
-Ruled out any items that had not been classi<ed by at least 5 out of the 7
judges into the intended category
Content Validity: A Second Example
-In the last lecture we talked about reliability
oThe content domain would be all of the concepts we talked about in
that lecture (alternatively, the content domain could be the content in
book—but we’ll stick to the lecture for now)
-Place the item in one of the categories
-We can compare the structure of the test to the structure of the lecture
-We could also look at agreement of judges for each item (ie do judges agree
on which category it fall into?)
Could use this as a rule of thumb for our midterm; where is the majority of
the content.
Is the content of the test and structure of the test agreeing?
-Does the structure of our test match the structure of the content domain?
-If not, what changes need to be made?
Content Validity and Reliability
-Both reliability theory and content validity studies assume that attest
represents a sample from a domain of possible items
Unlock document

This preview shows pages 1-3 of the document.
Unlock all 13 pages and 3 million more documents.

Already have an account? Log in

Get access

Grade+
$10 USD/m
Billed $120 USD annually
Homework Help
Class Notes
Textbook Notes
40 Verified Answers
Study Guides
1 Booster Class
Class+
$8 USD/m
Billed $96 USD annually
Homework Help
Class Notes
Textbook Notes
30 Verified Answers
Study Guides
1 Booster Class