Study Guides (283,762)
CA (135,460)
UBC (4,352)
PSYC (707)
PSYC 217 (23)
Midterm

PSYC 217 Study Guide - Midterm Guide: Inter-Rater Reliability, Repeated Measures Design, Concurrent Validity

11 Pages
201 Views
Fall 2012

Department
Psychology
Course Code
PSYC 217
Professor
Catherine Rawn
Study Guide
Midterm

This preview shows pages 1-3. Sign up to view the full 11 pages of the document.
COZY CHAPTER 8 EXPERIMENTAL DESIGN
Confounding and internal validity
- Confounding variable=variable that varies along with independent variable cannot determine which of the
variables is responsible for the observed effect
- *good experimental design- eliminating possible confounds and that results in alternative explanations
- Internal validity= when results can confidently be attributed to effect of IV
- External validity= extend to which findings may be generalized
Posttest-only design
- 1. Obtain 2 equivalent groups of participants
- 2. Intro the IV
- 3. Measure effect of IV on DV
Pretest-posttest design
- Makes it possible to ascertain that the groups were, in fact equivalent at the beginning of exp
- Usually not necessary if participants have been randomly assigned
o Min 20-30 per condition to make random assignment effective
- Advantages
o With small sample sizes, group not equal pretest to asses if groups equal to begin with
o Pretest necessary to select participants in exp
o Pretest can measure extent of change
o Pretest can assess effects of mortality (dropout factor in exp)
- Disadvantages
o Time-consuming
o Awkward to administer
o *sensitize participants figure out your hypothesis
Can be disguised
o Can use Solomon four-group design
Combo of both posttest-only and pretest-posttest design
Assigning participants to experimental conditions
- 1. Independent groups design
o *diff participants assigned to each of conditions using random assignment
Prevent systematic biases
Groups will be equivalent in terms of participant characteristics (income, age)
- 2. Repeated measures
o *same individuals participate in both conditions
o Advantages
Fewer participants needed (when cost is issue)
Extremely sensitive to finding statistically significant differences betw groups
In independent groups, get variability in results because people differ random error that we
can’t explain with repeated measures, individual differences can be seen and explained
*much easier to separate systematic individual differences from the effect of
independent variable
o Disadvantages
Order effects= *the order of presenting the treatments affects the DV
1. Associated with passage of time
o Practice effect= improvement in performance because of repeated practice
o Fatigue effect= deterioration in performance as research participant becomes
tired
2. Effect of 1st treatment carries over to response of 2nd treatment
o Contrast effect= response to 2nd cond altered because the 2 cond are contrasted
to one another
2 ways of dealing with:
1. Counterbalancing= all possible orders of presentation are included in exp
o Possible to determine extent to which order is influencing results
Latin squares- control for order effects without having all possible orders
o *limited set of orders constructed to ensure that
Ea cond appears at each ordinal position
Ea cond precedes and follows ea cond one time
Note*- trials counterbalancing not issue because order effect is wanted
2. Time interval betw treatments
o Rest period counteract fatigue effect
o Attending to unrelated task betw treatments reduce contrast effects
o Problem: participants have to commit to exp longer
- Choosing betw independent/repeated measures
o Repeated measures advantages:
1. Reduce # participants
2. Control over participant differences and greater ability to detect effect of IV
o *partially determined by generalization issues
o BUT any exp that produces relatively permanent change in individual cannot be used in repeated
measures
- 3. Matched pairs design
o Goal= first match ppl on a participant characteristic
o Matching variable either DV or variable strongly related to DV
o Achieve same equivalency of groups
o 1. Obtain measure of matching variable from ea individual
o 2. Rank ordered
o 3. Highest 2 participants together, next highest 2 together etc.
o 4. Members of each pair randomly assigned to conditions
o *impt with small sample sizes detect statistically significant effect of IV because possible to account
for individual differences in responses
o Disadvantages
Time consuming
Costly
Only when matching var strongly related to DV and know the relationship exists prior to
conducting study
CHAPTER 7 (P. 124-131) ASKING PPL ABOUT THEMSELVES: SURVEY RESEARCH
Constructing questions to ask
- Defining research objectives
- Attitudes and beliefs questions
- Facts and demographics (age, gender)
- Behaviours
- Question wording
o Unfamiliar technical terms
o Vague/imprecise terms
o Ungrammatical sentence structure
o Phrasing that overloads working memory
o Embedding question with misleading info
- Simplicity avoid jargon and technical terms
- Double-barreled questions= asks 2 things at once
- Loaded questions= written to lead ppl to respond in one way (eg. emotionally charged words)
- Negative wording
- “yea-saying” and “nay-saying”= problem is that respondent may be expressing true agreement but alternatively
may simply be agreeing with anything you say
o Word questions so that consistent agreement is unlikely
Responses to questions
- Closed vs open-ended questions
o Closed- limited # response alternatives given
More structured
Easier to code
Response alternatives same for everyone
More likely to be used when dimensions of variables well defined
o Open- free to answer in any way they like
Require time to categorize and code responsesmore costly
Most useful when researcher needs to know what people are thinking and how they naturally
view their world
- # of response alternatives
o Rating scales
o Graphic rating scale=requires a mark along a continuous 100mm line that is anchored with descriptions
at each end
o Semantic differential scale- on a series of bipolar adjectives using 7-pt scales
Concepts rated along 3 basic dimensions:
1. Evaluation
2. Activity
3. Potency
- Nonverbal scale for children
- Labelling response alternatives
o Wording of alternatives attempts to force raters to make finer distinctions among generally very good
students
o High-frequency scale- most alternatives indicate high frequency of an acitivty
o Low-frequency scale
CHAPTER 5- MEASUREMENT CONCEPTS
Reliability of measures
- Reliability= *consistency/stability of a measure of behaviour
o Any measure = true score + measurement error
o Achieved when use careful measurement procedures
o Making multiple measures
o Use correlation coefficients
Pearson product-moment correlation coefficient *reliability coefficient (high +ve)
Reliability
Reliable measure is consistent
Reliability coefficient: correlation coefficient 0.00-1.00
Test-retest reliability
Internal consistency reliability
interrater reliability
*take measure 2 times
1. *Split-half reliability: correlation of
*extent to which raters agree in their

Loved by over 2.2 million students

Over 90% improved by at least one letter grade.

Leah — University of Toronto

OneClass has been such a huge help in my studies at UofT especially since I am a transfer student. OneClass is the study buddy I never had before and definitely gives me the extra push to get from a B to an A!

Leah — University of Toronto
Saarim — University of Michigan

Balancing social life With academics can be difficult, that is why I'm so glad that OneClass is out there where I can find the top notes for all of my classes. Now I can be the all-star student I want to be.

Saarim — University of Michigan
Jenna — University of Wisconsin

As a college student living on a college budget, I love how easy it is to earn gift cards just by submitting my notes.

Jenna — University of Wisconsin
Anne — University of California

OneClass has allowed me to catch up with my most difficult course! #lifesaver

Anne — University of California
Description
COZY CHAPTER 8 – EXPERIMENTAL DESIGN Confounding and internal validity - Confounding variable=variable that varies along with independent variable cannot determine which of the variables is responsible for the observed effect - *good experimental design- eliminating possible confounds and that results in alternative explanations - Internal validity= when results can confidently be attributed to effect of IV - External validity= extend to which findings may be generalized Posttest-only design - 1. Obtain 2 equivalent groups of participants - 2. Intro the IV - 3. Measure effect of IV on DV Pretest-posttest design - Makes it possible to ascertain that the groups were, in fact equivalent at the beginning of exp - Usually not necessary if participants have been randomly assigned o Min 20-30 per condition to make random assignment effective - Advantages o With small sample sizes, group not equal pretest to asses if groups equal to begin with o Pretest necessary to select participants in exp o Pretest can measure extent of change o Pretest can assess effects of mortality (dropout factor in exp) - Disadvantages o Time-consuming o Awkward to administer o *sensitize participants figure out your hypothesis  Can be disguised o Can use Solomon four-group design  Combo of both posttest-only and pretest-posttest design Assigning participants to experimental conditions - 1. Independent groups design o *diff participants assigned to each of conditions using random assignment  Prevent systematic biases  Groups will be equivalent in terms of participant characteristics (income, age) - 2. Repeated measures o *same individuals participate in both conditions o Advantages  Fewer participants needed (when cost is issue)  Extremely sensitive to finding statistically significant differences betw groups  In independent groups, get variability in results because people differ random error that we can’t explain with repeated measures, individual differences can be seen and explained  *much easier to separate systematic individual differences from the effect of independent variable o Disadvantages  Order effects= *the order of presenting the treatments affects the DV  1. Associated with passage of time o Practice effect= improvement in performance because of repeated practice o Fatigue effect= deterioration in performance as research participant becomes tired  2. Effect of 1 treatment carries over to response of 2 treatment nd o Contrast effect= response to 2 cond altered because the 2 cond are contrasted to one another  2 ways of dealing with:  1. Counterbalancing= all possible orders of presentation are included in exp o Possible to determine extent to which order is influencing results  Latin squares- control for order effects without having all possible orders o *limited set of orders constructed to ensure that  Ea cond appears at each ordinal position  Ea cond precedes and follows ea cond one time  Note*- trials – counterbalancing not issue because order effect is wanted  2. Time interval betw treatments o Rest period counteract fatigue effect o Attending to unrelated task betw treatments reduce contrast effects o Problem: participants have to commit to exp longer - Choosing betw independent/repeated measures o Repeated measures advantages:  1. Reduce # participants  2. Control over participant differences and greater ability to detect effect of IV o *partially determined by generalization issues o BUT any exp that produces relatively permanent change in individual cannot be used in repeated measures - 3. Matched pairs design o Goal= first match ppl on a participant characteristic o Matching variable either DV or variable strongly related to DV o Achieve same equivalency of groups o 1. Obtain measure of matching variable from ea individual o 2. Rank ordered o 3. Highest 2 participants together, next highest 2 together etc. o 4. Members of each pair randomly assigned to conditions o *impt with small sample sizes detect statistically significant effect of IV because possible to account for individual differences in responses o Disadvantages  Time consuming  Costly  Only when matching var strongly related to DV and know the relationship exists prior to conducting study CHAPTER 7 (P. 124-131) – ASKING PPL ABOUT THEMSELVES: SURVEY RESEARCH Constructing questions to ask - Defining research objectives - Attitudes and beliefs questions - Facts and demographics (age, gender) - Behaviours - Question wording o Unfamiliar technical terms o Vague/imprecise terms o Ungrammatical sentence structure o Phrasing that overloads working memory o Embedding question with misleading info - Simplicity avoid jargon and technical terms - Double-barreled questions= asks 2 things at once - Loaded questions= written to lead ppl to respond in one way (eg. emotionally charged words) - Negative wording - “yea-saying” and “nay-saying”= problem is that respondent may be expressing true agreement but alternatively may simply be agreeing with anything you say o Word questions so that consistent agreement is unlikely Responses to questions - Closed vs open-ended questions o Closed- limited # response alternatives given  More structured  Easier to code  Response alternatives same for everyone  More likely to be used when dimensions of variables well defined o Open- free to answer in any way they like  Require time to categorize and code responsesmore costly  Most useful when researcher needs to know what people are thinking and how they naturally view their world - # of response alternatives o Rating scales o Graphic rating scale=requires a mark along a continuous 100mm line that is anchored with descriptions at each end o Semantic differential scale- on a series of bipolar adjectives using 7-pt scales  Concepts rated along 3 basic dimensions:  1. Evaluation  2. Activity  3. Potency - Nonverbal scale for children - Labelling response alternatives o Wording of alternatives attempts to force raters to make finer distinctions among generally very good students o High-frequency scale- most alternatives indicate high frequency of an acitivty o Low-frequency scale CHAPTER 5- MEASUREMENT CONCEPTS Reliability of measures - Reliability= *consistency/stability of a measure of behaviour o Any measure = true score + measurement error o Achieved when use careful measurement procedures o Making multiple measures o Use correlation coefficients  Pearson product-moment correlation coefficient *reliability coefficient (high +ve) Reliability Reliable measure is consistent Reliability coefficient: correlation coefficient 0.00-1.00 Test-retest reliability Internal consistency reliability interrater reliability *take measure 2 times 1. *Split-half reliability: correlation *extent to which raters agree in their total score on half of test with score observations Correlation of score at T1 with score on other half (1pt in time) at T2: scores should be similar (~0.80) Cohen’s Kappa Note: doesn’t take into account each Note: correlation might be artificially individual item’s role in measure’s high because individuals remember st how they responded the 1 time  2. Cronbach`s alpha: correlation of ea alternate forms reliability= item on test with every other item administering 2 diff forms of same test to same individuals at 2 pts in time Note: more itemshigher reliability Note: some variables expected to 3. Item-total correlations: correlation change from one test period to the of ea item score with total score next (mood) based on all items Construct validity - *adequacy of the operational definition of variables - 1. Face validity= *evidence for validity is that the measure appears to reflect the construct being measured o Very subjective, intuitive process o Not sufficient - 2. Content validity= *content of measure linked to the universe of content that defines the construct - 3. Predictive validity= *scores on the measure predict behaviour on a criterion measured at a time in the future - 4. Concurrent validity= *scores on the measure are related to a criterion measured at the same time - 5. Convergent validity= *extent to which scores on the measure in question are related to scores on other measures of the same construct/similar constructs - 6. Discriminant validity= *measure is not related to variables with which it should not be related to Reactivity of measures - *awareness of being measured changes an individual’s behaviour Variables and measurement scales Scale Description Example Distinction Nominal Categories with no numeric scales Males/females Impossible to define any Categories/groups simply differ from Introverts/extroverts quantitative values and/or one another differences betw/across categories Ordinal Rank ordering numeric values 2-, 3-, and 4-star Intervals betw items no limited restaurants known No particular value attached to Ranking TV programs by intervals betw numbers on scale popularity Interval Numeric properties are literal Intelligence No true 0 Assume equal interval betw Aptitude test score variables difference betw #s on Temp (C or F) scale meaningful Ratio 0 indicates absence of variable Rxn time Can form ratios (someone measured Weight weights twice as much as In behavioural sciences, physical Age another person) measures Frequencies of behaviours Importance of measurement scales: conclusions one draws about the meaing of particular score on variable depend on which type of scale was used CHAPTER 3- ETHICAL RESEARCH Milgram’s obedience exp - Teacher/learner (confederate- Mr. Wallace) - Learner asked to learn word pairs, then testedevery time make an error, teacher instructed to give shock- each shock more intense - Levers labelled ‘slight shock’ to ‘danger: severe shock’ then red Xs above 400V - Learner never really received shock but teacher tested on whether would continue to obey experimenter by administering even higher levels of shock - ~65% participants continued to deliver shocks all the way to 450V - Challenged beliefs about ability to resist authority - Ethics? The Belmont Report: Ethical Principles and Guidelines for the Protection of Human Subjects of Research - Assessment of risks and benefits - 1. Beneficence= *need for research to maximize benefits and minimize any possible harmful effects of participation o Risk-benefit analysis o Consider  Potential risks  Cost of not conducting study  Direct benefits  Material benefits  Other less tangible benefits o Risks  Physical harm (medical procedures)  Stress  Loss of privacy and confidentiality  Can make anonymous  Sometimes need to able to identify individual participants occurs when individuals studied on multiple occasions over time  Concealed observation of behaviour - 2. Autonomy (informed consent) o *participants are treated as autonomous, capable of making deliberate decisions about whether to participate o Should be provided with all info that might influence their decision of whether to participate o Informed consent form  Purpose of research  Procedures including time involved  Risks and benefits- reasonable foreseeable factors tha may be expected to influence willingness to participate  Prospective research benefits  Any compensation  Confidentiality  Assurance of voluntary participation and permission to withdraw  Foreseeable consequences of declining/withdrawing  Contact info for further questions o Autonomy issues  1. Lack of autonomy (some ppl unable to make own decisions)  Agreement by guardians for minors= assent  2. Coercion- any procedure that limits an individual’s freedom to consent o Withholding info and deception  Make sure that have good reasons not to have any informed consent  Deception= *active misrepresentation of info  eg. Milgram’s exp – 2 types of deception:  1. Deception ab
More Less
Unlock Document

Only pages 1-3 are available for preview. Some parts have been intentionally blurred.

Unlock Document
You're Reading a Preview

Unlock to view full version

Unlock Document

You've reached the limit of 4 previews this month

Create an account for unlimited previews.

Already have an account?

Log In


OR

Don't have an account?

Join OneClass

Access over 10 million pages of study
documents for 1.3 million courses.

Sign up

Join to view


OR

By registering, I agree to the Terms and Privacy Policies
Already have an account?
Just a few more details

So we can recommend you notes for your school.

Reset Password

Please enter below the email address you registered with and we will send you a link to reset your password.

Add your courses

Get notes from the top students in your class.


Submit