Textbook Notes (369,205)
Canada (162,462)
York University (12,903)
Psychology (3,584)
PSYC 3525 (5)
Chapter 6

Ch. 6 - Reliability and Validity.pdf

5 Pages
81 Views

Department
Psychology
Course Code
PSYC 3525
Professor
Josee Rivest

This preview shows pages 1 and half of page 2. Sign up to view the full 5 pages of the document.
Description
Ch. 6 - Reliability and Validity Tuesday, October 30, 20121:03 AM PSYC2520:Introduction to Experimental Psychology BeginningBehavioralResearch: A Conceptual Primer (7thEd. 2012) Rosnow& Rosenthal Chapter 6: Reliabilityand Validityin Measurementand Research Whatis the differencebetweenreliabilityand validity? • Validity-- How well the measure or design does what it's supposed to do ○ Constructvalidity-- The degree to which the conceptualization of what is being measured or experimentally manipulated is what is claimed, such as the constructs that are measured by psychological tests or that serve as a link between independent and dependent variables ○ Content validity-- The adequate sampling of the relevant material or content that a test purports to measure ○ Convergent or discriminant validity-- The grounds established for a construct based on the convergence of related tests or behavior (convergent validity) and the distinctiveness of unrelated tests or behavior (discriminant validity) ○ Criterion validity-- The degree to which a test or questionnaire is correlated with outcome criteria in the present (its concurrent validity) or the future (its predictive validity) ○ External validity-- The degree of generalizability of a relationship over different people, settings, manipulations (or treatments), and research outcomes ○ Face validity-- The degree to which a test or other instrument "looks as if" it is measuring something relevant ○ Internal validity-- The soundness of statements about whether one variable is the cause of a particular outcome, especially the ability to rule out plausible rival hypotheses  Threats to internal validity: □ Between & Within Subject Variables  Researcher expectancy  Participant expectancy □ Between Subject Variables  Participant selection  Loss of participants □ Within Subject Variables  Maturation and historical factors  Habituation and fatigue  Statistical regression  Controls for internal validity: □ Constancy □ Systematic variation  Counter-balancing: LatinSquare □ Random variation ○ Statistical-conclusionvalidity-- The accuracy of drawing certain statistical conclusions, such as an estimation of the magnitude of the relationship between an independent and a dependent variable (a statistical relationship that is called the effect size) or an estimation of the degree of statistical significance of a particular statistical test • Reliability-- The extent to which observations or measures are consistent or stable ○ Implies consistency, stability, and dependability ○ Alternate-formreliability-- The degree of relatedness of different forms of the same test Internal-consistencyreliability-- The overall degree of relatedness of all items in a test or all raters in a ○ Internal-consistencyreliability-- The overall degree of relatedness of all items in a test or all raters in a judgment study (also called reliability of components) ○ Item-to-itemreliability-- The reliability of any single item on average (analogous to judge-to-judge reliability, which is the reliability of any single judge on average) ○ Test-retestreliability-- The degree of temporal stability (relatedness) of a measuring instrument or test, or the characteristic it is designed to evaluate, from one administration to another, also called retest reliability Whatare randomand systematic errors? • Random error-- the name for chance fluctuations, or haphazard errors • Raw scores are true scores plus random errors that push the raw scores up or down around the true scores ○ The greater these fluctuations, the less reliable the scores are • Systematic error-- the name for fluctuations that are not random but are slanted in a particular direction (also called bias) • Random errors are likely to cancel one another, on average, over a great number of repeated measurements (i.e., they are likely to have an average of about 0) • Systematic errors do not cancer one another and do affect all measurements in roughly the same way Whatis the purposeof retestand alternate-formreliability? • Test-retest reliability-- The degree of temporal stability (relatedness) of a measuring instrument or test, or the characteristic it is designed to evaluate, from one administration to another, also called retest reliability • Reports of test-retest reliability of an instrument ordinarily indicate not only the interval over which the retesting was done, but also the nature of the sample on which the test-retest reliability is based • A common concern when people take the same test twice is that the test-retest r may be artificially inflated because of their familiarity with the test; this can be prevented by creating an alternate-form of the test • Alternate-form reliability-- The degree of relatedness of different forms of the same test Whatis internal-consistencyreliability,and howis it increased? • Internal-consistency reliability (R)-- the overall degree of relatedness of all items in a test or all raters in a judgment study • Ways to estimate internal-consistency reliability: ○ Spearman-Brown formula-- where is the number of items in the test and is the average intercorrelation of the items ○ K-R 20-- useful if items are scored dichotomously (i.e., 1 for correct and 0 for incorrect) ○ Cronbach's alpha-- not restricted to dichotomously scored items • The internal-consistency reliability will increase with increased test length as long as the items being added are relevant and are not less reliable than the items already in the test • Item-to-item reliability-- The reliability of any single item on average (analogous to judge-to-judge reliability, which is the reliability of any single judge on average) ○ The mean item-to-item correlation (mean r between all items or judges) Whatare acceptabletest-retestand internal-consistencyreliabilities? • The optimal number of items on a test or questionnaire depends on the context in which your instrument is to be used and the objective of the research • Relevant measures may be found in the Directory of Unpublished Experimental Mental Measures (Goldman & Mitchell, 2003) • Internal-consistency reliability is usually expected to be higher than test-retest reliability, unless the test-retest intervals are short • Personality tests such as the Rorschach and MMPI have relatively low criterion validity (r = 0.29, r = 0.30) for predicting psychopathology, and thus caution should be exercised when using these tests Howis thereliabilityof judgesmeasured? Howis thereliabilityof judgesmeasured? • Judge-to-judge reliability-- the reliability of any single judge on average • Spearman-Brown formula for internal-consistency reliability-- where is the number of judges and is the average judge-to-judge reliability Howis reliabilityrelatedto replicationand external validity? • External validity
More Less
Unlock Document

Only pages 1 and half of page 2 are available for preview. Some parts have been intentionally blurred.

Unlock Document
You're Reading a Preview

Unlock to view full version

Unlock Document

Log In


OR

Join OneClass

Access over 10 million pages of study
documents for 1.3 million courses.

Sign up

Join to view


OR

By registering, I agree to the Terms and Privacy Policies
Already have an account?
Just a few more details

So we can recommend you notes for your school.

Reset Password

Please enter below the email address you registered with and we will send you a link to reset your password.

Add your courses

Get notes from the top students in your class.


Submit