Chapter 3.docx

8 Pages
121 Views
Unlock Document

Department
Marketing and Consumer Studies
Course
MCS 3030
Professor
lloydhetherington
Semester
Fall

Description
Chapter 3 – The Theory of Measurement Construct Validity Operationalization- measurement of something not directly measurable (translation of an idea or construct into something real and concrete) Validity-the best available approximation of the truth of a given proposition, inference, or conclusion External validity- degree to which study conclusions hold for other people in other places at other times Construct Validity: approximate truth of conclusion that operationalization accurately reflects its construct (generalizing from program or measures to the concept or idea of your program or measures) -assessment of how well your actual programs or measures reflect your ideas or theories -land of theory: what goes on inside mind and attempt to explain to others (about theories, ideas, hunches, hypotheses about the world) -land of observation: consists of what you see happening in the world around you and the public manifestations of the world –find your actual program and your actual measures or observational procedures 1. definitionalist perspective – the way to ensure construct validity is to define the construct so precisely that you can operationalize it in a straight forward manner -ether you have operationalized the construct correctly or you haven’t (either-or type thinking) 2. relationalist perspective- things are not either/or (black or white)–concepts are more or less related to each other Types of Construct Validity Translation validity-focuses on whether operationalization is a good translation of the construct Face Validity- on its face, does the operationalization look like a good translation of the construct? -weakest way to demonstrate construct validity -improved with ‘expert judgment’ Content Validity- operationalization is check against the relevant traits for the construct -difficult to demonstrate all characteristics that constitute construct -this method increases rigor Criterion-related validity-performance of your operationalization is check against some criterion Predictive validity-operationalization predicts something that theoretically it should be able to -low correlation between constructs indicates problems Convergent Validity- measure of constructs that theoretically should be related to each other are, in fact, observed to be related to each other (should be able to show correspondence between similar constructs) -operationalization highly correlates with other measures of the same construct -higher (usually positivcorrelation between items of the same construct indicates greater convergent validity Concurrent validity- operationalization can distinguish between groups that it should theoretically should be able to -higher ability to distinguish between similar groups indicates greater concurrent validity Discriminant validity- measures of constructs that theoretically should not be related to each other are, in fact, observed not to be related to each other (should be able to discriminated between dissimilar constructs) -operationalization does not correlate with other measures of a different construct *note: need both convergent and discriminant validity to establish construct validity Chapter 3 – The Theory of Measurement Threats To Construct Validity 1. Inadequate preoperational explication of constructs-means measures are not adequate Solutions: think through concepts better, use methods to articulate concepts, get experts to critique operationalizations 2. mono-operation bias -single experiment Solution: try to implement multiple versions of your program 3. mono-method bias -single method Solution: try to implement multiple measures of key constructs and try to demonstrate measures you use behave as you theoretically expect them to 4. interaction of different treatments -something else caused the effect 5. interaction of testing and treatment- if testing/measurement makes groups more sensitive/receptive to treatment then testing is in effect a part of treatment, inseparable from effect of treatment Solution: label ‘program’ to refer to the program alone, but in fact includes testing 6. restricted generalizability across constructs -doesn’t apply elsewhere – essentially labeling iss(label not good description for what you implemented) 7. confounding constructs and levels of constructs -variability of the construct 8. social threats to construct validity -hypothesis guessing- people try to prove their own hypothesis (labeling issue) -evaluation apprehension-many people are anxious about being evaluated which can make them perform poorly -humans tend to want to look good/smart -experimenter expectancies- researcher can bias results of study (consciously and unconsciously) -communicate desired outcome for study (look pleased when participants give desired answer) The Nomological Network Nomological network- includes theoretical framework for what you are trying to measure, an empirical framework for how you are going to measure it, and specification of the linkages among and between these frameworks (representation of constructs of interest in a study, observable manifestations, and interrelationships among these) Nomological network principles: -scientifically, make clear what something is/means so that laws can be made -laws in nomological network may relate: -observable properties/quantities to each other -different theoretical constructs to each other -theoretical constructs to observables - at least some of laws in network must involve observables -basic rule: adding new construct to a theory must generate laws (nomologicals)confirmed by observation or reduce number of nomologicals required to predict some observables Chapter 3 – The Theory of Measurement The Multi-trait-Multi-method Matrix multitrait-mulit-multimethod matrix- matrix of correlations arranged to facilitate the assessment of construct validity -assumes each construct (traithas been measured with different methods in a fully crossed design (traits by methods) -matrix consists of nothing but correlations -square symmetric matric so you only need to look at half of it -correlations can be grouped in three kinds of shapes: diagonals, triangles, and blocks -can assess both convergent and discriminant validity using MTMM Specific shapes: - reliability diagonal (monotrait-monomethod): estimates of reliability of each measure in the matrix -as many correlations in reliability diagonal as there are measures - validity diagonals (monotrait-heteromethod): correlations between measures of the same trait measured using different methods -on validity diagonal in each method block - heterotrait-monomethod triangles: correlations among measures that share the same method of measurement -if correlated highly, it’s because measuring different things with same method results in correlated measures -heterotrait-heteromethod triangles: correlations that differ in both trait and method -because these don’t share trait or method its expected them to be lowest in matrix -monomethod blocks- consist of all of the correlations that share the same method of measurement (as many blocks as there are methods of measurement) -heteromethod blocks: consist of all correlations that do not share the same methods Principles of Interpretation: - MTMM interpretation requires the researcher to use judgment -coefficients in the reliability diagonal should consistently be the highest in the matrix -trait should be more highly correlated with itself than with anything else -coefficients in the validity diagonals should be significantly different from zero and high enough to warrant further investigation -evidence of convergent validity. -validity coefficient should be higher than all coefficients in the heterotrait-monomethod triangles. -emphasizes that trait factors should be stronger than methods factors (is not true in all cases -same pattern of trait interrelationship should be seen in all triangles Advantages and Disadvantages of MTMM: Advantages: Disadvantages: -address convergent and discriminant validity -hard to implement simultaneously -no known overall statistical test for validity -addresses the importance of method of -requires judgment call on interpretation measurement -provides rigorous standard for construct validity Chapter 3 – The Theory of Measurement Pattern Matching For Construct Validity Pattern matching - degree of correspondence between two data items - attempt to link two patterns where one is a theoretical pattern and the other is an observed or operational one -pattern is any arrangement of objects or entities. -theory postulates structural relationships between key constructs (can be used as basis for generating patterns of predictions) -implies more and suggests how one might improve on these current (more complex patterns, if matched, yield greater validity for the theory) Advantages and Disadvantages of Pattern Matching: Advantages: Disadvantages: -more general and flexible than MTMM (doesn’t require -requires that you specify your theory of measuring each construct with multiple methods) the constructs rather precisely -treats convergence and discrimination as a continuum -able to structure the theoretical and (concepts are more/less similar so their interrelations would be observed patterns the same way so that more/less convergent/ discriminant, which moves the you can directly correlate them (requires convergent/discriminant distinction away from the simplistic dichotomous categorical notion to one that is more suitably post-you to quantify both patterns and, ultimately, describe them in matrices that have the same positivist and continuous in nature) dimensions) -makes it possible to estimate overall construct validity for set of measures in a specif
More Less

Related notes for MCS 3030

Log In


OR

Join OneClass

Access over 10 million pages of study
documents for 1.3 million courses.

Sign up

Join to view


OR

By registering, I agree to the Terms and Privacy Policies
Already have an account?
Just a few more details

So we can recommend you notes for your school.

Reset Password

Please enter below the email address you registered with and we will send you a link to reset your password.

Add your courses

Get notes from the top students in your class.


Submit