Statistical Sciences 2035 Study Guide - Quiz Guide: Internal Validity, Content Validity, Face Validity
7 Oct 2018
School
Department
Professor
The factors that influence reliability
Length of assessment – a greater number of items measured decreases the
influence of chance factors and balances them out. This notion is also supported in
the formula of Spearman-Brown. However, at some point the law of diminishing
return sets in so that a little increase in reliability requires a vast extension in length.
- Methods for reliability estimation - methods that take into account both
stability and equivalence will tend to give lower coefficients than the other
methods because all major types of error variance are included. With any method
involving two assessment occasions, the longer the interval of time between two
occasions, the lower the coefficient will tend to be.
- Type of measurement – Rating scales, observation techniques and interviews are
subjective in response and scoring, and more likely influenced by individual
opinion, tending to provide low reliability coefficients. Objective measures
produce higher reliability coefficients simply because interpretation is not
involved.
- Standardization and administration - to increase the chances of high reliability,
the administration of the test from occasion to occasion, or between different
groups, must be standardized, particularly in terms of time limit, instructions,
freedom from distraction, etc. These factors must remain constant or else the
results from different measurements will reflect variations in these, as well as
expected individual performance variations.
Standard Error of the Measurement
It is a measure of the range of error around any known score using a reliability
coefficient. A series of data obtained by an individual or piece of equipment, etc. on the
repeated administration of a single test, will approximate to a normal distribution. The
spread of this distribution is another way of conceptualizing the reliability of the
assessment. The smaller the spread, the greater the confidence with which any one
observed score can be taken to represent the ‘true’ score. The standard error of
distribution has the formula:
𝑆𝐸𝑚𝑒𝑎𝑠 =𝑆𝐷√1 − 𝑟𝑡𝑡
Document Summary
Length of assessment a greater number of items measured decreases the influence of chance factors and balances them out. This notion is also supported in the formula of spearman-brown. However, at some point the law of diminishing return sets in so that a little increase in reliability requires a vast extension in length. Methods for reliability estimation - methods that take into account both stability and equivalence will tend to give lower coefficients than the other methods because all major types of error variance are included. With any method involving two assessment occasions, the longer the interval of time between two occasions, the lower the coefficient will tend to be. Type of measurement rating scales, observation techniques and interviews are subjective in response and scoring, and more likely influenced by individual opinion, tending to provide low reliability coefficients. Objective measures produce higher reliability coefficients simply because interpretation is not involved.
