false

Unlock Document

Marketing and Consumer Studies

MCS 3030

lloydhetherington

Fall

Description

Chapter 3 – The Theory of Measurement
Construct Validity
Operationalization- measurement of something not directly measurable (translation of an idea or construct
into something real and concrete)
Validity-the best available approximation of the truth of a given proposition, inference, or conclusion
External validity- degree to which study conclusions hold for other people in other places at other times
Construct Validity: approximate truth of conclusion that operationalization accurately reflects its
construct (generalizing from program or measures to the concept or idea of your program or measures)
-assessment of how well your actual programs or measures reflect your ideas or theories
-land of theory: what goes on inside mind and attempt to explain to others
(about theories, ideas, hunches, hypotheses about the world)
-land of observation: consists of what you see happening in the world
around you and the public manifestations of the world –find your actual
program and your actual measures or observational procedures
1. definitionalist perspective – the way to ensure construct validity is to define the construct so
precisely that you can operationalize it in a straight forward manner
-ether you have operationalized the construct correctly or you haven’t (either-or type thinking)
2. relationalist perspective- things are not either/or (black or white)–concepts are more or less related to
each other
Types of Construct Validity
Translation validity-focuses on whether operationalization is a good translation of the construct
Face Validity- on its face, does the operationalization look like a good translation of the
construct?
-weakest way to demonstrate construct validity
-improved with ‘expert judgment’
Content Validity- operationalization is check against the relevant traits for the construct
-difficult to demonstrate all characteristics that constitute construct
-this method increases rigor
Criterion-related validity-performance of your operationalization is check against some criterion
Predictive validity-operationalization predicts something that theoretically it should be able to
-low correlation between constructs indicates problems
Convergent Validity- measure of constructs that theoretically should be related to each other
are, in fact, observed to be related to each other (should be able to show correspondence between
similar constructs)
-operationalization highly correlates with other measures of the same construct
-higher (usually positivcorrelation between items of the same construct indicates greater
convergent validity
Concurrent validity- operationalization can distinguish between groups that it should
theoretically should be able to
-higher ability to distinguish between similar groups indicates greater concurrent validity
Discriminant validity- measures of constructs that theoretically should not be related to each
other are, in fact, observed not to be related to each other (should be able to discriminated
between dissimilar constructs)
-operationalization does not correlate with other measures of a different construct
*note: need both convergent and discriminant validity to establish construct validity Chapter 3 – The Theory of Measurement
Threats To Construct Validity
1. Inadequate preoperational explication of constructs-means measures are not adequate
Solutions: think through concepts better, use methods to articulate concepts, get experts to critique
operationalizations
2. mono-operation bias -single experiment
Solution: try to implement multiple versions of your program
3. mono-method bias -single method
Solution: try to implement multiple measures of key constructs and try to demonstrate measures you
use behave as you theoretically expect them to
4. interaction of different treatments -something else caused the effect
5. interaction of testing and treatment- if testing/measurement makes groups more sensitive/receptive
to treatment then testing is in effect a part of treatment, inseparable from effect of treatment
Solution: label ‘program’ to refer to the program alone, but in fact includes testing
6. restricted generalizability across constructs -doesn’t apply elsewhere – essentially labeling iss(label
not good description for what you implemented)
7. confounding constructs and levels of constructs
-variability of the construct
8. social threats to construct validity
-hypothesis guessing- people try to prove their own hypothesis (labeling issue)
-evaluation apprehension-many people are anxious about being evaluated which can make
them perform poorly
-humans tend to want to look good/smart
-experimenter expectancies- researcher can bias results of study (consciously and unconsciously)
-communicate desired outcome for study (look pleased when participants give desired
answer)
The Nomological Network
Nomological network- includes theoretical framework for what you are trying to measure, an empirical
framework for how you are going to measure it, and specification of the linkages among and between
these frameworks (representation of constructs of interest in a study, observable manifestations, and
interrelationships among these)
Nomological network principles:
-scientifically, make clear what something is/means so
that laws can be made
-laws in nomological network may relate:
-observable properties/quantities to each other
-different theoretical constructs to each other
-theoretical constructs to observables
- at least some of laws in network must involve
observables
-basic rule: adding new construct to a theory must
generate laws (nomologicals)confirmed by observation or
reduce number of nomologicals required to predict
some observables Chapter 3 – The Theory of Measurement
The Multi-trait-Multi-method Matrix
multitrait-mulit-multimethod matrix- matrix of correlations arranged to facilitate the assessment of
construct validity
-assumes each construct (traithas been
measured with different methods in a fully
crossed design (traits by methods)
-matrix consists of nothing but correlations
-square symmetric matric so you only need to
look at half of it
-correlations can be grouped in three kinds of
shapes: diagonals, triangles, and blocks
-can assess both convergent and discriminant
validity using MTMM
Specific shapes:
- reliability diagonal (monotrait-monomethod): estimates of reliability of each measure in the matrix
-as many correlations in reliability diagonal as there are measures
- validity diagonals (monotrait-heteromethod): correlations between measures of the same trait
measured using different methods
-on validity diagonal in each method block
- heterotrait-monomethod triangles: correlations among measures that share the same method of
measurement
-if correlated highly, it’s because measuring different things with same method results in
correlated measures
-heterotrait-heteromethod triangles: correlations that differ in both trait and method
-because these don’t share trait or method its expected them to be lowest in matrix
-monomethod blocks- consist of all of the correlations that share the same method of measurement (as
many blocks as there are methods of measurement)
-heteromethod blocks: consist of all correlations that do not share the same methods
Principles of Interpretation:
- MTMM interpretation requires the researcher to use judgment
-coefficients in the reliability diagonal should consistently be the highest in the matrix
-trait should be more highly correlated with itself than with anything else
-coefficients in the validity diagonals should be significantly different from zero and high enough to
warrant further investigation
-evidence of convergent validity.
-validity coefficient should be higher than all coefficients in the heterotrait-monomethod triangles.
-emphasizes that trait factors should be stronger than methods factors (is not true in all cases
-same pattern of trait interrelationship should be seen in all triangles
Advantages and Disadvantages of MTMM:
Advantages: Disadvantages:
-address convergent and discriminant validity -hard to implement
simultaneously -no known overall statistical test for validity
-addresses the importance of method of -requires judgment call on interpretation
measurement
-provides rigorous standard for construct validity Chapter 3 – The Theory of Measurement
Pattern Matching For Construct Validity
Pattern matching - degree of correspondence between
two data items
- attempt to link two patterns where one is a theoretical
pattern and the other is an observed or operational one
-pattern is any arrangement of objects or entities.
-theory postulates structural relationships between key
constructs (can be used as basis for generating patterns of
predictions)
-implies more and suggests how one might improve on
these current (more complex patterns, if matched, yield
greater validity for the theory)
Advantages and Disadvantages of Pattern Matching:
Advantages: Disadvantages:
-more general and flexible than MTMM (doesn’t require -requires that you specify your theory of
measuring each construct with multiple methods)
the constructs rather precisely
-treats convergence and discrimination as a continuum -able to structure the theoretical and
(concepts are more/less similar so their interrelations would be observed patterns the same way so that
more/less convergent/ discriminant, which moves the you can directly correlate them (requires
convergent/discriminant distinction away from the simplistic
dichotomous categorical notion to one that is more suitably post-you to quantify both patterns and, ultimately,
describe them in matrices that have the same
positivist and continuous in nature) dimensions)
-makes it possible to estimate overall construct validity for
set of measures in a specif

More
Less
Related notes for MCS 3030

Join OneClass

Access over 10 million pages of study

documents for 1.3 million courses.

Sign up

Join to view

Continue

Continue
OR

By registering, I agree to the
Terms
and
Privacy Policies

Already have an account?
Log in

Just a few more details

So we can recommend you notes for your school.

Reset Password

Please enter below the email address you registered with and we will send you a link to reset your password.