Study Guides
(238,408)
Canada
(115,131)
Dalhousie University
(353)
Psychology
(63)
PSYO 2000
(2)
Simon Gadbois
(2)
Midterm
EXAM 2.docx
Unlock Document
Dalhousie University
Psychology
PSYO 2000
Simon Gadbois
Fall
Description
Sensitivity measures should be sensitive enough to detect differences in a characteristic that are important to the
investigator
Specificity measures should be specific to the characteristics, group, etc.
too general?
Accuracy based on precision of the measurements
good accuracy if averaged values = standard
standards are rare in bio science (behavior)
accuracy = specificity + sensitivity (signal detection theory)
function of systematic error/bias
extraneous/confounded variables are a source
precision consistency, reliability of the data
reliability consistency precision
validity does a variable represent what it intended
systematic error
errors solution
observer/experimenter = blinding
subject/participant = blinding / unobtrusive measures
instruments/apparatus = calibration
blinding – experimenter and participants do not know the assignment of treatments and control groups
random error
influence of the precision of measurements (consistency and reliability)
unpredictable fluctuations
participants
experimental conditions
methods
measures
TRUE RANDOMNESS
sources of random error
observer/experimenter reliability YOU
participant/subject reliability doing what they need to be doing
instrument/apparatus reliability
how to asses precision
measures of variability – descriptive stats
standard deviation – of repeated measures
coefficient of variation (sd/mean) X 100
measures of concordance
correlation coefficient consistency of results of paired measurements
index of concordance
need enough data
other tools kappa, cronbach alpha
reliability/consistency
consistent results over repeated measurements
reliability refers to the precision of your measures
assessment methods
testretest reliability/consistency – stability of test scores over time repeated studies
alternative (parallel) forms use of two lists in a memory test may not be easy
if scores are different something is not equal (in some ways the same)
same level of difficulty – the frequency (high frequency in kidscat, dog, low in adults)
how abstract the word is
word length – in processing time
effective value, emotional value
recognition/recall
internal consistency how consistent are the measures across items intended to measure the same concept
splithalf reliability/consistency
interrater reliability between
consistency of recording and scoring betweenALL OBSERVERS (with an interobserver reliability measure such as in
index of concordance, kappa coefficient, Kendall coefficient)
intraobserver/rater within
consistency each observer, individuality, records, interprets, or identifies SIMILAR behaviors or events in the SAME
WAY (with an intraobserver reliability measure)
impression change
validity and reliability
a measure can have high reliability but no or low validity
cannot be more valid than it is reliable
internal validity measures what it is suppose to
associated with the criteria for analytic experiments
no confounded variables
controlled variables
appropriate control groups
random selection sampling
random assignment (randomization)
external validity generalization potential
determines the application and implications of an experiment
psychologists have low ecological validity
generalize data to…
species
environment
culture
age groups
conditions
criteria for external validity have the potential to influence internal validity
population selection –converging evidence from different population and a representativeness of the sample
example abusive drink different in France and Canada (different population)
sometimes you want a certain population type
operational definitions agreement on definitions
parameter values – the values you select for each variable in your experiment should be well defined
applies to control variables and independent variables
demand characteristics – cues in a research produce that influence the behavior of subjects are absent or minimized
implies what experiment is about
subtypes of validity
face how well the test appears to measure what it is designed to measure
it is a plausible measure of the variable we want to estimate non scientific
contenthow adequately the measure address the representatives of the measured event
expert opinion can determine this
construct –a measure of how well a test asses some underlying (theoretical) characteristic
depends on the operational definitions
criterion –how the assay agrees with others supposed to evaluate the same event on the same criterion
based on criteria you use and the type of evaluation
concurrent –how well an assay estimates a criterion in relation to another event or group of subjects at the same point
parallel in time
convergent –the methods of measurement diverge upon one
discriminant –methods of measurement diverge upon one another and the divergence is expected
predictive –how well an assay predicts a even on a time criterion
measures predicts the future states
external applies well to other people, settings, conditions, etc.
diagnosis and diagnostics: clinical psychology and neuroscience
diagnosis validity idea that trying to diagnosis or detect something, use a different kind of reasoning – sensitivity and
specificity
content validity refers to symptoms and diagnostic criteria
concurrent validity – associated correlates or markers and response to treatment in therapy
predictive validity refers to diagnostic stability over time
discriminate validity can it discriminate between disorders –comorbidity
predictor variable test result
outcome –presence or absence of disease
values can be
dichotomous –positive or negative / yes or no
categorical –scale, 4++++, 3+++,2++, 1+, 0
continuous milligrams of glucose ore deciliter
observational vs diagnostic
observational association between a predictor and outcome variable
diagnostic detection or discrimination between yes and no, more than an association
test phase what constitutes a positive test results
example of the necessity of a pilot study, it is crucial in diagnostic studies
validation phase assess the tests sensitivity and specificity
use a new sample of subjects
detailed steps
1. asses the need for a new diagnostic tool
2. selection of subjects and samples
3. identify a reasonable gold standard for comparison
4. design standardized and blinded trails
5. estimate sample size for accuracy
6. find subjects
7. report accuracy sensitivity and specificity data
true positive= positive and present false positive= positive and absent
false negative= negative and present
true negative= negative and absent
sensitivity = TP/TP+FN
specificity = TN/FP+TN
the best test has low FP and low FN
reality accuracy is a trade off between specificity and sensitivity
implications
decisions: asses the implications or impact of the two possible errors in sensitivity – you have determined a cutoff point
example brain tumors
if the surgery is dangerous and potentially lethal – you want to avoid FP
if the cancer is dangerous and potentially lethal you don’t want to miss it you want to avoid FN
Bayesian theorem  links the degree of belief in a proposition before and after accounting for evidence – prior
knowledge influences decision
Likelihood ratios involves the odds
that a person with a disease would have a particular test result divided by the likelihood that a person
without the disease would have that result
very powerful when combined with the prior probability of a disease – base rate judgments depend on this
Bayesian analyst
applies mostly to categorical and continuous data
d’values, criterion, and ROC curves
covered previously (signal detection theory)
applies mostly to dichotomous data
prevalence how rare the disease is in the population
if low, test needs to be specific
example HIV andAIDS

if high, test needs to be sensitive
example high blood pressure
prior probability that a specific patient has the disease –estimate before testing’’
LOW –prior probability of heart disease in young, healthy, nonsmoker
HIGH – old, high BMI, smoker drinker
called the base rate
predictive value or posterior probability
positive test
likelihood of a true positive /likelihood of TP + FP
or
Sensitivity X Prior probability
[Sensitivity X Prior probability]
+ [(1specificity) X (1 prior probability)]
 negative test
likelihood of TN/ likelihood of TN + FN
or
Specificity X (1 Prior probability)
[Specificity X (1Prior probability)]
+ [(1sensitivity) X Prior probability]
errors in diagnostic studies
random error>>>chance>>>quantify it with CI for specificity and sensitivity
systematic error>>>biases
sampling biases measurement bias
reporting bias
binary response system
1=yes
0=no
s=signal/stimulus
statistical errors
no error
type II error beta
type I error alpha
hypothesis testing
H not reject false = no error
accept false= type II error
true reject =type II error
accept true= no error
summary
type one errors akin to false alarms or false positives
type 2 errors are akin to misses or false negatives
experimental designs
factorial – more complex
when more than one independent variable is used (independent variable= factor) more then A and B, ex.
ABCD
bivalent or multivalent
each variables has 2 or more levels
ANOVA(or analysis) variance are needed for the data
three main types
within – all subjects go through all the conditions (one group of subjects)
also called repeated measures designs or randomized block designs
for ttests “ttests for dependent groups”, or paired ttest “related ttest” or “correlated ttest”
type 1: the same subject is observed under all treatment conditions
type 2: the same subject is observed before and after a treatment (pretestposttest design)
type 3:subjects are matched on a subject variable (organismic variable or individual differences variable)
then randomly assigned to the treatments
this is a between subject design requiring a within subject analysis of variance advantages
each level of the independent variable is applied to al subjects
can evaluate how each level of independent variable affects each subject
each subject is its own control
excellent for assessing experiments on learning, transfer of training, practice effects
may help increase statistical sensitivity or statistical power –want more power, increase the number of subjects
subjects are not divided into groups, all subjects are involved in all conditions
disadvantages
practice effects –if not the focus of study, it becomes a problem
solution appropriate counterbalancing procedures can counteract practice effects
also make the treatment order an independent variable
differential carryover effects lingering effects of one or more treatment conditions
often a issue with drug studies
solution recovery periods (intervals)
violation of statistical assumptionscovered in inferential statistics course
solution – use a more strict significant level
practice and carryover effects
practice/learning: increase in performance via practice
sometimes considered a carryover effect
not necessarily a problem in some experiments
carryover effects
fatigue –decrease performance with time
contrast treatments are compared by subjects
habituation (decrease) or sensitization (more reactive to it)– psychological effect
adaption – physiological effect tolerance in drugs
between – all different animals /participants
match the groups so they are all similar
 type 1: completely randomized design
type 2: matched groups designs
for ttests “ttest for independent groups” or “unpaired ttest
combination of within and between
mixed (between and within) factorial design
within and between components
also called splitplot designs
analysis: SPANOVA
nested factorial design
try to avoid
correlational experimental
quasiexperimental – may not be able to randomize subjects, no control group
expost facto – medical world (clinical) large number of studies and participants. Did not plan it, and naturally
occurring events and then plan it after it happens
developmental designs –to see change in individuals in each group
descriptive stats – important in describing the data
measures of central tendency
arithmetic mean –average
 If there s a zero cannot use them. Use them for ratios
harmonic mean geometric mean
mode
median
measures of variability – need to specify what measures you are using because numbers are different
standard deviation
standard error
confidence intervals
least significant difference
coefficient of variation
range
inferential statistics –confirming or not the hypothesis
using ttests, ANOVAetc.
visual data its not about numbers, its about what it looks like compared to the means and averages
significant differences?
Effect size very important –how strong the effect is within that group
Interactions
Interactions between variables: there is an interaction between two variables if the effect of one independent variable
changes with different values of the second independent variable
the sleep study with melatonin and phototherapy treatments is a good example of a factorial design with two
independent variables (factors) potentially interacting
general information
the factious examples given below will now assume that a completely randomized factorial design was
chosen
different participants are in each treatment condition
each group of participants is independent of every other group
2X3 factorial design
perfect parallel line –can conclude that there is no interaction
mild interaction – trend is the same
the lines not being perfectly parallel (but close) suggest a mild interaction
possibly uncertain if it is significant
strong interaction – treatment helping one group a lot
impact of one factor over the other
variance (ANOVA) will determine the significance of the interaction
problem of interpretation – the main issue
dealing with 2 independent variables
this is why you limit yourselves to 34 variable factors
be parsimonious in choices – more likely to be straightforward
complex designs – complex interactions
have to be able to explain the data –significance difference and interactions
if two lines are straight across one main effect (therapy), and no interaction
two lines are bent in the same way – two main effects, and no interaction
two lines are on top of each other one main effect (drug), no interaction
two lines are mirrored – two main effects with an interaction two lines are mirrored and not touching 2 main effects and an ordinal interaction
two lines are mirrored and cross each other – 2 main effects and a disordinal interaction
antagonistic interaction
cancel out the main effect
significant interaction, no main effects (masked by the interaction)
significant interaction with equal means – no main effect
significant interaction, no main effects, disordinal interaction
additive
interactive
limit the number of levels for each factor
limit the number of factors
Specialized designs
Quasi experimental designs and ex post facto design
Pre experimental designs with no control groups any random assignment
Acceptable for pilot research – if not avoid
Experimental – control groups and random assignment ( not always) – when you take a random sample from the
population it is hard to do it perfectly random (subjective idea)
Quasi experiment often control groups but no random assignment
Quasi independent variables
Ex post facto – special case of between subject design – extract an association or correlation between variables and not
causation
Often take advantage of naturally occurring events
lack of control over variables
issues of quasi
do not fill in the requirements for a true experimental (analytic) experiment
internal validity
external validity
most likely issue
random selection is not possible or not respected (over looked as criterion)
example: within subject design
useful when statistical power is an issue
useful when few measurements are possible
useful when availability of participants is not possible
typically random assignment of participants is not possible
ethical
a common mistake – pooling fallacy
jet lag study
jet lag travel east to west traveling as opposed to non jet lad north to south
melatonin treatment – to induce sleep
measuresrating scale on quality of sleep (onset, duration, restfulness)
issue finding participants often involved in jet lag traveling willing to be part of this experiment
control group? Hard to find (maybe who travel in the same time zone)
participants only based on:
duration and distance of travel number of time zones
departure and arrival times
direction – east to west or west to east sleep pattern strategy before leaving trip (sleep deprivation or over sleeping)
solution
use a within subject treatment
all subjects go through of the experimental conditions
potential problem – order of effects
counter balancing

main effect is there an effect of just one on its own
ex post facto design
researcher arrives after the fact
nature has implemented the treatment group
not controlled by experimenter
(dead or alive) must have 2 things
different environments
different dispositions
combination of the two
people participants can not be ethical
so find some who are already sick for example and study their last 10 years, or someone who is dead
prospective
cause to effect
healthy people (in a low and high stress job for example) and wait 10 years and compare (longitudinal)
look at the frequency and compare
cohort design
retrospective
effect to cause
sample 100 people with cancer and 100 health people
investigate their past
compare what chronic stress
case control designs or criterion group designs
characters of EPF designs
participants selected after the fact
no random assignment
cofounded variables typical to the groups investigated
selection of participants must be to the criterion applied
convince sampling is problematic
some have confounding variables
internal validity issues
no random sampling
no random assignment
confound variable typical to the groups
selecting of subjects becomes a complicated and strict criterion and must be applied
convenience sampling is problematic – may be confound
external validity issues
choice of groups
MATCHING solutions
Matching – making sure that the 2 groups do not differ one nay other except the one selected to study (low or high)
Matching on: diet, smoking, drinking, pescprtion, etc.
subject to subject matching
distribution for distribution matching – descriptive statistics
MEASURE the variables solution the idea try to identify the possible confounding variables and measure them
can determine if the variables are confounded or not and predict their effect
multivariate methods
retrospective EPF design
very common in health research
backward information
comparative advantages
smaller number of participants
shorter time period
less money
same internal and external issues
specific issues
present context motivating the investigation
detection bias
with additional historical issues in diagnosis including awareness
likely made by different people
differential assessment on groups
=less extensive records in the health group
some specific issues with retrospective EPF designs
present context motivating the investigation
detection bias
o with addition of historical issues in diagnosis (related to progress in the ability to diagnosis
some disease) – including awareness
diagnosis was likely made by different parties (same criteria?)
differential assessment or perspective on stress between – healthy and cancer group
less extensive records in the healthy group
solutions
same options as for the prospective EPF designs
o measure
o matching
in all cases matching or identification of measureable variables is more difficult in this case as the design
requires you to search back in time e
you rely on the memory of the subjects, their family and friends, and their physicians, including the
accuracy and completeness of the medical records
change
studying changes in time of structures and processes
ontogenetic change developmental research
change in the individual neuron, brain, animal, person
historical change cross generation research
changes during across generations
phylogenetic change evolutionary research
changes in species and populations, speciation, natural selection
specific needs for specific areas
this section introduced different research strategies
more exists depending on your specific field of study
ex. longitudinal or cross sectional
longitudinal disadvantages
test conditions become well known by child
are performance and behavioral changes die to experience or maturation
can take a lot of time and money
cross sectional disadvantages individual differences can account for some effects (not age) and normal differences in developement
solution hybrid version – cohort sequential design or cross sequential
comparative evolutionary
common in
comparative and evolutionary neuroscience
comparative and evolutionary psychology
ethology, behavioral etc.
cross cultural
linguistics
guiding principal – compare and contract
theoretical foundations – natural selection and or culture
comparative synchronic (static in time)
evolutionary – diachronic (history)
combining within and between manipulation
mixed designs or slip plot designs
most common cases of mixed within and between subject manipulation
nested designs or hierarchical designs
more economical than mixed designs
less information (some interactions cannot be evaluated)
nested designs: between
example comparing fish from clean and contaminated water
objectives
the nested design allows us is to test 2 things
difference between control and study areas (quasi experimental)
the variability of the sites within areas (sites 1,2,3 in control, and 4, 5,6, in the study)
if we fail to find a significant variability among the sites within areas then a significant difference between areas would
suggest that there is an environmental impact
in other words, the validity is due to differences between areas and not to variability among the sites
but in this kind of research (environment and species) it is likely that you will find validity within the sites
(within an area)
but, even if you find significant difference between the site you can still test to see whether the difference
between the areas is significantly larger than the viability among the sites with areas
nested designs: mixed
time is
example comparing spring and fall movements of snakes
it depends what you are trying to accomplish
nest tasks
nest groups
nesting locations, sites, areas
nesting times (years, seasons, months, days etc.)
time series designs a lot of precision
follow the change in the system over time
show how dynamically the system changes
observations and time
basic time
goal: compare before and after the treatment by cell or collapsing
interrupted time
similar but with naturally occurring events example The “baseline” (before T )5 On T th5 event (naturally occurring) or treatment (experimental induction) is
introduced. Observations or measures inT to6 are9influenced.
Variation is to give a treatment early, but then cancel it and see the effects
follow the typical ABAdesign (that will be described in the next chapter)
Atreatment B no treatment A
non equilivant control group design
similar to pervious designs
two independent groups
one control and one experimental
make sure both groups are similar and comparable –matching is a solution
pretest/posttest designs
true experimental designs, similar to a within subject design
two levels pre and post
=problem and solution
carryovers and possibility o counterbalance
how to increase internal validity?
control groups (necessary)
random assignment of subjects to conditions (if possible)
design 2
mixed design (2X2) – within treatment and no treatment is the between factor
group one pre test treatment and posttest
group two – pre test and post test
design 3
other problem – effect of having experienced there test two solutions
eliminating the pre test completely simple two group experiment
better solution four group design partial removal tests this helps you evaluate the effect of pretest (if any)
group 1 – pre test and treatment and posttest
group 2 – pretest and posttest
groups 3 – treatment and posttest
group 4 – posttest
(pretest removed for groups 3 and 4)
measuring patterns and dynamic processes: behavior and neurophysiology
observational methods
quantitative
quantified and systematic observations: objective (frequency, duration, etc.)
our focus
qualitative
qualified and interpretive: subjective (descriptive (verbal) accounts)
not addressed in this course
measuring..
events (dynamic): electrophysiological activity, behavior, hormonal release, etc.
states (static): signs, symptoms, postures, traits, etc. (binary information yes/no, on/off, presence/absence,
etc.)
both at the same time
types of observations research
naturalistic (field, can be in a lab)
systematic (field or lab)
describing dynamic processes: behavior or physiology
studying the causes or the triggers of specific processes studying the process itself (events, actions): spatial and temporal structure, expression, etc.
studying the consequences (effects, reactions) of the process on the organism, the environment, and
(other) individual, cells, tissue, etc.
finding a pattern will help with prediction
the s
More
Less
Related notes for PSYO 2000