Reliability and validity in assessment
o Reliability—consistency of measurement
Test-retest reliability—related to how a test looks over time if you run it twice. It
should have the same scores if the same individual takes the test twice.
Sometimes we don’t want test-retest reliability to be high, i.e. a depression level
test after treatment.
Alternate-form reliability—certain tests are influenced by practice effects,
therefore we measure the same construct with different content. Reliability
between scores between two different versions of test.
Internal consistency reliability—relates to actual content of test. Patient’s test
scores should be reliable and consistent across the entire test.
Inter-rater reliability-- how reliable two different clinicians will look at a patient
and come up the same results.
o Validity—whether or not the test is measuring what it is supposed to measure
Content validity—test must have enough content on it to completely measure
what it is supposed to measure
Criterion validity—the test converges with another test that measures the same
Concurrent validity—e.g. comparing one test with another concurrent
test. If both tests show the same result, test has criterion validity
Predictive validity—whether or not the test predicts something about
the actual person. E.g. if they score high on the test for depression, do
they look depressed? Does the test predict real world situations?
Construct validity—actually capturing some sort of construct that is unique to a
group of people. I.e. giving people with depression a test and people without
depression a test. People with depression should all score higher than regular
o Clinical interviews—can tell you more about the patient than the actual test, but tests
are needed to give a solid, more scientific proof.
Characteristics of clinical interviews
‘How are you doing?’
‘What are your concerns?’
‘With the problems you are describing, does it implicate any difficulties
in your everyday life?’
‘How are your sleep and eating patterns?’
‘How is work and leisure life?’
‘What is your family history of medical disorders or diseases?’
‘What other medical conditions do you have?’
‘Do you think you have a problem?’
‘How frequent are symptoms?’
‘What medications/treatment are you on?’
‘Any history of hospitalization?’
‘How long have symptoms been present?’ ‘Age?’
‘Cultural factors? Social factors? Economic factors?’
‘Any illegal substances?’
Behavioural observations, appearances.
o Avoidance of questions
o Eye contact (sometimes)
o Behavioural observations should be observed before patient
o Discrepancies in stories of patients
Structured/semi structured interviews
o Screening measures—sometimes used diagnostically by patients
Beck scales—anxiety, depression inventories.
Problems: patient may be having symptoms at a specific time, but if they take
the test again, test-retest reliability is low. I.e. prof having anxiety during
lecture, but not later on when he’s at home. We never rely on these test results
on their own, clinical interview is needed.
o Personality inventories
Personality assessment inventory/personality assessment screener
Minnesota multiphasic personality inventory-2 (MMPI-2)
o Projective measures
Rorschach inkblot test
Thematic apperception test (TAT)
o Specific inventories
Pain inventories (e.g. P-3; Multidimensional Pain Inventory)
General personality inventories
o Omnibus measures
Extensive measures that attempt to cover a wide range of clinical pathology
Typically self-repot measures (patient fills it out on their own).
Contain clinical measures and validity measures
Can directly assess clinical psychopathology from that profile (MMPI-2)
o Personality assessment inventory
PAI is a 344 item self-report questionnaire that attempts to understand an
individual’s personality traits and characteristics
Renders diagnostic considerations based on the DSM IV
Diagnostic considerations involve Axis I and Axis II disorders
Provides clinical and validity scales
Has screening measure to make assessment more efficient, 22 items vs. 344
Test is normed on people with various psychopathology disorders.
People’s results are compared to people with a disorder and test will try to
match the person with a person who has psychopathology. Projective techniques—less scientific, used to explore hypotheses in counselling and not for
o Projective hypothesis
The notion that highly unstructured stimuli are necessary to bypass defenses in
order to reveal unconscious motives and conflicts
Therapists from psychoanalytic paradigms would use this.
o Projective techniques
Tests of personality that involve use of unstructured stimulus materials. Use of
such materials maximize the role of internal factors such as emotions and
motives in perception