Textbook Notes (368,070)
Canada (161,617)
York University (12,801)
Psychology (3,584)
PSYC 3090 (54)
Chapter 7

PSYC 3090 Chapter 7: CHAPTER 7

4 Pages
Unlock Document

PSYC 3090
Krista Trobst

CHAPTER 7 – UTILITY -how useful a test is -practical value of using a test to aid in decision making -cost effectiveness, saving time, how useful compared to others, comparative utility, clinical utility etc What is Utility? Utility: in the context of testing and assessment, the usefulness or practical value of testing to improve efficiency -anything from single test to large-scale testing program that employs a battery of tests Factors that Affect a Test’s Utility 1. Psychometric soundness a. Reliability + validity of a test b. Index of utility tells us about the practical value of the information derived from scores c. Higher the criterion-related validity, higher the utility, but exceptions to this rule d. Valid tests aren’t necessarily useful e. Behaviour of testtakers, can also be test users 2. Costs a. Financial cost b. Cost: refers to disadvantages, losses, or expenses both in economic and noneconomic terms c. May need to allocate funds for i. Particular test ii. Supply of blank test protocols iii. Computerized test processing, scoring, and interpretation from the test publisher or some independent service d. Additional costs may come from i. Payment to professional personnel and staff associated with test administration, scoring and interpretation ii. Facility rental, mortgage, and/or other charges related to the usage of the test facility iii. Insurance, legal, accounting, licensing and other routine costs of doing business e. Cost of not testing or testing with something that is ineffective 3. Benefits a. Benefits justify costs b. Benefit: profits, gains, or advantages, economic or noneconomic Utility Analysis What is Utility Analysis? Utility analysis: broadly defined as a family of techniques that entail a cost-benefit analysis designed to yield information relevant to a decision about the usefulness and/or practical value of a tool of assessment -various methods -may be used for the purpose of evaluating whether the benefits of using a test outweigh the costs How is a Utility Analysis Conducted? -2 approaches 1. Expectancy data a. Need more than a table b. Taylor-russell tables: provide an estimate of the extent to which inclusion of a particular test in the selection system will improve selection, based on different combinations of 3 variables: i. Test’s validity – computed validity coefficient ii. Selection ratio used – numerical value that reflects the relationship between the number of people to be hired and the number of people available to be hired iii. Base rate – the percentage of people hired under the existing system for a particular position c. One limitation of these tables is that the relationship between the predictor and the criterion must be linear d. Difficulty of identifying a criterion score that separates successful from unsuccessful employees e. Naylor-shine tables: obtaining the difference between the means of the selected and unselected groups to derive an index of what the test is adding to already established procedures f. Both tables assist in judging utility, former determining increase over current procedures and the latter by determining the increase in average score on some criterion measure g. Brogden-cronbach-gleser formula: used to calculate the dollar amount of a utility gain resulting from the use of a particular selection instrument under specified conditions h. Utility gain: an estimate of the benefit of using a particular test or selection method i. Productivity gains: refers to an estimated increase in work output Decision theory and test utility -most cited application of statistical decision theory is Cronbach + gleser’s psychological tests and personnel decisions -cronbach and gleser presented: 1. classification of decision problems 2. various selection strategies ranging from single-stage processes to sequential analyses 3. quantitative analysis of the relationship between test utility, s
More Less

Related notes for PSYC 3090

Log In


Join OneClass

Access over 10 million pages of study
documents for 1.3 million courses.

Sign up

Join to view


By registering, I agree to the Terms and Privacy Policies
Already have an account?
Just a few more details

So we can recommend you notes for your school.

Reset Password

Please enter below the email address you registered with and we will send you a link to reset your password.

Add your courses

Get notes from the top students in your class.