Class Notes (834,936)
Canada (508,830)
York University (35,167)
Psychology (4,108)
PSYC 3090 (8)
Lecture 4

PSYC 3090 Lecture 4: LECTURE 4 - OCTOBER 4TH 2016

7 Pages
Unlock Document

PSYC 3090
Krista Trobst

LECTURE 4 – OCTOBER 4 2016H Domain sampling – extent to which specific sources of variation under defined conditions are contributing to the test score Generalizability – based on the idea that a person’s test scores vary from testing to testing because of variables in the testing situation -instead of conceiving of variability in a person’s score as error, Cronbach encouraged test developers + researchers to describe the details of the particular test situation or universe leading to a specific test score -universe described in terms of its facets, including number of items in test, amount of training people received etc Item-Response Theory – provides a way to model the probability that a person with X ability will be able to perform at a level of Y -family of methods + techniques -considerations of item difficulty and discrimination Standard error of measurement: abbreviated as SEM, provides measure of the precision of an observed test score -higher the reliability, lower the standard error -can be used to estimate extent to which observed score deviates from a true score -confidence interval: range or band of test scores that is likely to contain the true score Standard error of difference: measure that can aid a test user in determining how large a difference in test scores should be expected before it is considered statistically significant -3 questions Validity -whether or not it can measure what we want to measure -process of gathering + evaluating evidence about validity -both test developers + test users may play a role 3 categories 1. Content validity – based on subjects covered in test 2. Criterion-related – relationship of scores obtained on the test to scores on other tests 3. Construct validity – measure of validity that is arrived at by executing a comprehensive analysis of a. How scores on test related to other test scores + measures and b. How scores can be understood within some theoretical framework for understanding the construct that the test was designed to measure Face Validity – how valid it seems to be -self-report high in this -perceived lack of face validity may lead to lack of confidence in the test measuring what it purports to measure Content Validity – how well test samples behaviour representative of the universe of behaviour it was designed to measure -quantifying content validity important for employment tests -lawshe developed method where raters judge each item as essential, useful, or not necessary, if more than half agree, has some content validity -content varies across cultures + time, political considerations may also play a role Criterion-Related Validity -criterion is standard against which a test or test score is evaluated -characteristics: relevant, valid, and uncontaminated -concurrent – test score related to some criterion measure obtained at the same time -predictive validity – test score predicts Validity coefficient – correlation that provides measure of the relationship between test scores + scores on the criterion measure -affected by restriction or inflation Incremental validity – degree to which additional predictor explains something about the criterion measure that is not explained by predictors already in use Expectancy data -table shows the percentage of people within specified test-score intervals who subsequently placed in various categories of the criterion (passed, failed) -may show that higher the initial rating, the greater the probability of job success Construct validity -ability of test to measure a construct its designed to measured -high + low scorers should behave as theorized -content + criterion related under this Evidence of construct validity -evidence of homogeneity – how uniform test is in measuring a single concept -evidences of changes with age -evidence of pretest/posttest changes -evidence from distinct groups – vary in predictable way depending on membership in some group -convergent + discriminant evidence -factor analysis: new test should load on a common factor with other tests of the same construct Bias: factor inherent in a test that systematically prevents accurate, impartial measurement -implies systematic variation in test scores Rating error: judgement resulting from intentional or unintentional misuse of rating scale -raters may be too lenient, too severe, or reluctant to give rates at extremes -halo effect: give someone higher Fairness: extent to which a test is used in an impartial just + equitable way Utility -the usefulness or practical value of testing to improve efficiency Factors affecting utility -psychometric soundness – higher the criterion validity, greater the utility -exceptions because many factors affect utility + assessed in many different ways -valid tests are not always useful Costs – financial, non economic -whether benefits outweigh costs -benefits could be profits, gains etc -increase quality in workers’ performance, decrease in time needed to train workers, reduction in accidents + turnover -better environment + improved morale Utility Analysis -family of techniques that entail a cost-benefit analysis desgined to yield information relevant to a decision about the usefulness and/or practical value of a tool of assessment -some straightforward, others more complicated using math -often: which gives us most bang for our buck? -validity coefficient = validity -selection ratio refers to numerical values that reflects the relationship between the number of people to be hired the number of people available to be hired -base rate – refers to percentage of people hired under the existing system for a particular position Naylor-Shine – looks at difference between means of selected + unselected groups to derive index of what the test is adding to already established procedures -validity coeffic
More Less

Related notes for PSYC 3090

Log In


Join OneClass

Access over 10 million pages of study
documents for 1.3 million courses.

Sign up

Join to view


By registering, I agree to the Terms and Privacy Policies
Already have an account?
Just a few more details

So we can recommend you notes for your school.

Reset Password

Please enter below the email address you registered with and we will send you a link to reset your password.

Add your courses

Get notes from the top students in your class.