PSYB32H3 Chapter Notes - Chapter 4: Criterion Validity, Concurrent Validity, Inter-Rater Reliability

31 views9 pages
Published on 17 Nov 2012
of 9
inter-rater reliability: refers to the degree to which two independent observers or judges agree.
test-retest reliability: measures the extent to which people being observed twice or taking the same test twice score in
generally the same way. Makes sense only when the theory assumes that people will not change greatly between testings
on the variable being measured.
alternate-form reliability: using two forms of tests rather than giving the same test twice, perhaps when there is
concern that people will remember their answers from the first test and aim merely to be consistent. The extent to which
scores on the two forms of the test are consistent.
internal consistency reliability: assesses whether the items on a test are related to one another.
In each of these types of reliability, a correlation is calculated between raters or sets of items. The higher the
correlation, the better the reliability.
Validity is generally related to whether a measure fulfills its intended purpose.
content validity: refers to whether a measure adequately samples the domain of interest.
criterion validity: is evaluated by determining whether a measure is associated in an expected way with some other
measure (the criterion).
- concurrent validity: both variables are measured at the same point in time, the relationships are concurrent.
- predictive validity: criterion validity can be assessed by evaluating the measure’s ability to predict some other
variable that is measured in the future (eg; IQ tests used to predict future school performance).
construct validity: is relelvant when we want to interpret a test as a measure of some characteristic or construct that is
not simply defined.
- A construct is an inferred attribute, such as anxiousness, that a test is trying to measure.
interview: any interpersonal encounter, conversational in style, in which one person, the interviewer uses language as
the principal means of finding out about another, the interviewee.
Characteristics of clinical interviews:
clinical interview: is different from a casual conversation or a poll by the attention the interviewer pays to how the
respondent answers, or does not answer, questions.
The paradigm within which an interviewer operates influences the type of information sought, how it is obtained and
how it is interpreted. A psychoanalytically trained clinician can be expected to inquire about the person’s childhood
history because the analytic paradigm holds that the most significant aspects of a disturbed or normal person’s
developmental history are repressed into the unconscious.
Creating a rapport is important, as well as building trust. It doesn’t always suffice to be an ‘authority’ figure such as a
doctor to have a patient open up. Empathising, repeating what they said to show you’re listening,
Interviews vary in the degree to which they are structured. In practice, most clinicians operate from only the vaguest
outlines. Exactly how information is collected is left largely up to the particular interviewer and depends too on the
responsiveness and responses of the interviewee.
Because of the overwhelming majority of clinical interviews are conducted within confidential relationships, it has not
been possible to establish either their reliability or their validity through systematic research.
Structured Interviews:
At times, mental health professionals need to collect standardized information, particularly for making diagnostic
judgements based on the DSM.
structured interview: one in which the questions are set out in a prescribed fashion for the interviewer.
structured clinical interview diagnosis (SCID) for axis 1 of DSM-IV: a branching interview; that is the client’s response to
one question determines the next question that is asked. It also contains detailed instructions to the interviewer
concerning when and how to probe in detail and when to go on to questions bearing on another diagnosis (See p.103,
figure 4.1).
Structured interviews have also been developed for diagnosing personality disorders and more specific disorders, such
as the anxiety disorders.
psychological tests: standardized procedures designed to measure a person’s performance on a particular task or to
assess his or her personality or thoughts, feelings, and behaviour. If the results of a diagnostic test are inconclusive,
psychological tests can provide information that can be used in a supplementary way to arrive at a diagnosis.
standardization: the same test is administered to many people at different times, and the responses are analysed to
indicate how certain kinds of people tend to respond. Statistical norms for the test can thereby be established as
sufficient data have been collected.
There are three basic types of psychological tests; self-report personality inventories, projective personality tests and
tests of intelligence.
(1) Personality Inventories:
o Personality inventory: the person is asked to complete a self-report questionnaire indicating whether
statements assessing habitual tendencies apply to him/her.
o The best known is MMPI, intended to serve as an inexpensive means of detecting psychopathology. Is
multiphasic because it is designed to detect a number of psych. problems at a time.
In designing the test, first, many clinicians provided statements that they considered indicative of
various mental illnesses. Second, these were rated by patients already diagnosed with particular
disorders and by a large group of individuals considered normal.
o The MMPI-2 has several noteworthy changes. The old version lacked representations of racial minorities,
including African Americans and native Americans; its standardization sample was restricted to white
men and women (60 years after first).
Several items containing allusions to sexual adjustment, bowl and bladder functions, and
excessive religiosity were removed because they were judged in some testing contexts to be
needlessly intrusive and objectionable. Sexist wording was eliminated, along with outmoded
idioms. They added substance abuse, type A behaviour and marital problems. (p.105, table 4.1)
There is a lie scale, designed to detect deliberately faked responses. Individuals who indorse a
large number of statements in the lie scale try to present themselves in a good light.
(2) Projective Personality Tests:
o Projective test: a psychological assessment device in which a set of standard stimuli inkblots or
drawings ambiguous enough to allow variation in responses is presented to the individual. The
assumption is that because the stimulus materials are unstructured, the patient’s responses will be
determined primarily by unconscious processes and will reveal is or her true attitudes, motivations and
modes of behaviour projective hypothesis.
o Rorschach Inkblot test: a person is shown 10 inkblots and asked to tell what figures they see in each. Five
are black, white and shades of grey, two also have red splotched and three are pastel color.
o Thematic apperception test (ACT): Shown a series of black and white pictures and asked to tell a story
related to each.
o Projective techniques are derived from the psychodynamic paradigm. The use of projective tests assumes
that the respondent would be either unable or unwilling to express his/her true feelings if asked directly.
o They were originally used as a stimulus to fantasy that was assumed to bypass ego defences which are
meant to protect us. The content of the persons’ responses was viewed as symbolic of internal dynamics.
o Other uses of the Rorschach test, however, concentrate more on the form of the person’s responses. The
test is considered more a perceptual-cognitive task and the person’s responses are viewed as a sample of
how he or she perceptually and cognitively organizes real-life situations.
o Exner system for evaluating responses; Respondents who see a great deal of human movement in the
Rorschach inkblots tend to use inner resources when coping with their needs, whereas those whose
Roschach responses involve colour are more likely to seek interaction with the environment.
o The system may have more validity in some cases than in others. It appears to have utility in identifying
people with schizophrenia or at risk of developing it.
o The utility of the Rorschach can most likely be attributed to the fact that a person’s responses on the test
are highly related to the communication disturbances that are an important symptom of schizophrenia.
o The Roberts Apperception Test for Children; much like the thematic Apperception test TAT, in this test,
pictures of children and families are presented to the child, who tells a story about each ne.
Whereas many scoring approaches to the TAT are impressionistic and non-standardized, the
Roberts test provides objective criteria for scoring, along with normative data to determine
whether the child’s pattern of responses is abnormal.
Unique to this test are the scales that provide information about a child’s coping skills.
(3) Intelligence Tests:
o Intelligence test: often reffered to as an IQ test, is a standardizes means of assessing a person’s current
mental ability.
o All based on the assumption that a detailed sample of an individual’s current intellectual functioning can
predict how well he/she will erform in school.
o Intelligence tests are also used:
In conjunction with achievement tests, to diagnose learning disabilities and to identify areas of
strengths and weaknesses for academic planning;
To help determine whether a person is mentally retarded;
To identify intellectually gifted children
As part of neuropsychological evaluations (eg; person with dementia to test deterioration over
o Scores on most IQ tests are standardizes so that 100 is the mean and 15/16 is the standard deviation.
Those with a score below 70 are 2 SD below pop mean = sub-average intellectual functioning. Those with
130 = intellectually gifted.
o IQ tests are highly reliable, have good criterion validity.
o Predict later educational attainment and occupational success.
o Hough the correlations between IQ scores and school performance are statistically significant, much more
is unexplained by IQ test scores than is explained (family situation, contributing factors...)
o Interest has also focuses on emotional intelligence, reflected in such abilities as delaying gratification and
being sensitive to the needs of others.
o High levels of emotional intelligence are associated negatively with alexithymia, a condition of reduced
awareness that is a risk factor for a variety of adjustment problems. High levels of emotional intelligence
are associated with greater levels of subjective well-being and reduced proneness to depression.
traditional assessment concentrates on measuring underlying personality structures and traits, such as obsessiveness...
behavioural and cognitively oriented clinicians often use a system that involves the assessment of four sets of variables,
sometimes referred to by the acronym SORC.
S stands for stimuli, the environmental situations that precede the problem.
O stands for organismic, referring to both psychological and psychological factors assumed to be
operating under the skin.
R refers to overt responses. These probably receive the most attention from behavioural
clinicians, who must determine what behaviour is problematic, as well as the behaviours
frequency intensity and form.
C refers to consequent variables, events that appear to be reinforcing or punishing the behaviour
in question.