Mechanism: process by which behavior occurs
Function: purpose for the behaviours
George Romanes ranks animal intelligence from humans down, using anecdotes of animals being smart
Interest in learning to show intelligence emerges
Ethnologists: not concerned with learning but instinct
To be an instinct a behavior must: be species specific, have a signal stimulus that activates the behavior,
must have fixed action patterns. Instincts studied by ethnologists are similar to reflexes, from receptor
to nervous system, to muscles.
Reflex arc: Sensory neuron interneuron (decision) motor neuron (action)
Habituation or sensitization occurs at interneuron
Dual process theory: different types of underlying neural processes are responsible for increases and
decreases in response to stimuli (conflicting of habituation or sensitization).
Lorenz (genetics, instincts) vs Lehrman (learning, experience)
Domjan- Learning: enduring change in behavior mechanisms as a result of prior experience with stimuli.
Certain species are more apt to different types of learning (bats and taste aversion).
Habituation: decrease reaction to stimulus over time (not due to fatigue or sensory adaptation)
(stimulus response system)
Sensitization: repeated stimulation increasing reaction (ex. something scary) (state system)
Homeostasis: resting point, balance- behaviours are to maintain homeostasis (lizards basking in the sun
to maintain body heat)
Solomon: emotions attempt to reach homeostasis
There are opposing ends of the spectrum (misery, joy)
Opposition process theory: after joy is misery, biphasic, body tries to regulate. Primary reaction
is initial and secondary is the opposing that follows and lasts longer. Ex. Drunk- hangover. Buying
something exciting and buyer’s remorse. Runners high after pain. Primary emotion becomes weaker
over time and the second emotion becomes stronger. Relating to a drug, tolerance increases hangover
becomes worse. Adaptive to regulate homeostasis. Then people do the action to avoid the negative
Classical Conditioning (Pavlov): Pavlov was selling dog digestive enzymes as a cure-all. Then he discovered that digestion starts in the
mouth where saliva breaks down food. He noticed dogs would salivate at the sight of food or even the
experimenter. Used a metronome and not a bell.
Stimulus Unconditioned US, Conditioned CS, Response Unconditioned UR, conditioned CR
Excitatory: pairing of CS with US, creating conditioned response, excite new response to US
CS can be appetitive (good, food, sex) or aversive (bad, nausea)
Sign tracking: discrete stimulus signal appetitive US, animal approaches CS even if in opposite direction
Taste aversion: works in one trial even if long delay between US + CS, weaker in dietary specialists as
they only eat one thing.
Light, sound and saccharine:
Shock- they avoid light and sound
Nausea- they avoid the taste
Inter-trial interval: time between one conditioning trial and the next
Inter-stimulus interval: time between CS and US used to be said CS had to be right after US, now
Inhibitory CS: appears only when there is no US UR and CR are not similar in any way, association takes a
longer time to develop, must begin with excitatory. How to test if it works- directed behavior (if you can
measure in two directions ex. Temp), summation (magnitude does CS+ response get disrupted by CS-),
retardation acquisition (turn CS- into CS+ and see if it takes longer to learn).
Anticipation can promote homeostasis: body begins to come down for a drug in anticipating the drug. B
process is conditioned (tolerance).
What makes optimal CS and US? Biological constraints on learning, need to understand the subject and
the function the behavior serves in the real world. Good stimuli has novelty, salience and relevance
Behavioural systems theory: behaviours evolved to serve a purpose for the animal and they come in
clusters. The form of the conditioned behavior CR will relate to the behavioural system activated by the
US-CS pairing. Ex. Triggering quails sexual response- triggers search behavior, focused searching,
approaching females, crowing, strutting, head-bobbing. CS far from US will trigger a different
component of the behavioural system.
Contiguity: short interval good learning, long interval low responding
Novelty (latent inhibition) Salience (overshadowing)
Relevance/belongingness (selectivity of association)
Rescorla’s contingency experiment- not just contiguity that is important but CONTINGENCY, US must
depend on CS. Every time a US occurs a CS is preceding to it.
Excitatory conditioning p(USI CS)> p(USI no CS)
If food showed up without a metronome and metronome happens when there is food sometimes, then
metronome is not important. Food presentation needs to depend on the metronome.
Blocking: create simple excitatory connection between A(light) and US. Then add B(tone) also paired
with the same US. To learn that light and tone produce food. No learning happens due to association
about light and food. Important factor is surprise. No conditioning to second CS as US is predicted by
Unblocking: surprise animal by causing the second CS to have more US (upshift) or less US (downshift)
Quantify the learning experience, takes into account multiple CS. Measure of surprise, and show the
increase of learning across trials. Based on the concept of surprise and is better than initial contingency
model ( couldn’t take into account blocking). Conditioning depends on the degree to which the US is
processed. Each US has a certain amount of associative strength that will support conditioning for which
the CS compete. Includes that over time CS can take on behaviours of US.
1. If The strength of the actual US is greater than a subject’s expectation all CSs that are paired
with US will receive excitatory conditioning
2. If the strength of the US is less than the strength of the subject’s expectation all CS’s paired with
US will receive inhibitory conditioning
3. If they are the same no learning occurs, conditioning has happened
4. If there is a large difference, lots of learning occurs
5. The more salient the CS will condition faster than less salient CS
6. If 2 or more CSs are presented together the subject’s expectation will be equal to their total
strength with excitatory and inhibitory
Light 1 pellet of food
At each conditioning trial the expectation value of the CS will increase
The difference between what animal expects and the strength of the US becomes smaller
Fastest growth in conditioning is when the numbers are really different- lots of surprise, lots of
*asymptote is reached light and tone food According to rule 6 when 2 CSs presented T is a new stimulus so “0”, Light expectation of 1,
actual US is 1 and no learning occurs.
Surprise= experience- expected
Changes in every trial, score of surprise must be different each trial
Need a value of salience of CS and US
We may need to keep track of many cs
V- conditioned strength of CS (expected by subjects
k- constant salience
∆V- difference in conditioned strength
Final number Can’t be above US.
Multiple CS V = V +V
t a b
Some problems with the RW model: can’t account for sensory pre-exposure, latent inhibition.
Does not account for unblocking. Changing value of US causes animal to learn about second CS.
Unblocking in upshift- increase US Rescorla model makes sense if more US makes more room
for another CS
Unblocking in downshift- lower US
Does not include temporal factors- contiguity not crucial but does play a role
Predicts extinction but not the way it actually works: spontaneous recovery can happen.
Rescorla Wagner model is a stimulus-stimulus association model. New neural pathway forms between
CS and US. This contrasts with stimulus response learning (classical conditioning) that connects CS to CR.
How to test which is correct: Rescorla’s US devaluation experiment conditioning, makes value of food
less (feeding the rats when US is food), links between stimulus are made.
How to test if it is stimulus stimulus, decreasing value of US, will CS still occur?
Recent models of classic conditioning:
Attention model of conditioning- how much animal pays attention to stimulus (how salient stimulus is
can change) CS processing model. Adaptive to pay attention to CSs that are predictive to something
important but not adaptive to pay attention to something that isn’t predictive. Also based on surprise, if
it was surprising subject pays attention. Attention depends on what happened in previous trial.
Temporal factor models- designed to explain the effects of time, not explained in Rescorla model. CS US
interval is one important temporal value, more critical interval appears to be ratio of inter-stimulus
interval and inter-trial interval. Easier to prepare for a reaction when there is time with no CS. The
interval of time emphasizes the importance of the CS.
Comparator hypothesis: model of performance, not learning. Comparing excitatory connection between
CS and other connections. When strength of CS is weaker than background cues there is no learning. Measuring conditioned responses:
How do we know it has learned? Wait until experiment is over and do trials with cs and no us, or
measure something about the behavior of the animal after cs and us which suggests it is reacting to cs
us pairing. Cs confounded by us so we must find what it is learning about cs during the trials, doing a test
trial of presenting cs without us at a random interval during association process. Buuuut if you keep
putting the cs with no us in then it affects the strength of association so should be used sparingly.
Magnitude, probability, latency (speed) of behavior in test trials.
Learning measure requires behavior of subject compared with a control subject. Think of control as a
verb not a noun, what are you trying to control? controlling every other variable that could affect your
study other than the one being tested. The one difference between the groups should be the variable.
Need a control group for every variable being controlled. Control group depends on what is being
Random control- present cs and us at random during the experiment, (animal may still learn)
Explicitly unpaired control: cs and us are presented far apart to prevent their association, same cs and us
are used, can control for effects of habituation, sleepiness, and response to cs or us. (may learn inhibitor
connection between cs and us)
Excitatory conditioning- teaches an association
Inhibitory- removes association, does not predict to US. Subject learns to predict absence of US. Best
way of doing it is causing animal to expect US (excitatory) then not have the CS appear, showing that
CS+ and CS- lead to know US.
Higher order conditioning: pairing CS to create another CS .
CS 1 US
CS 2 CS 1
Sensory preconditioning: make association between two unimportant things, make association with one
of the things and US, the other thing gains value also Operant (instrumental) conditioning: response learning, consequence to actions, responses to our
behavior. Voluntary behavior that occurs because it was previously instrumental in producing certain
consequences. This is also called “goal-directed” behavior as it is toward an outcome.
Edward Thorndike: suggests anecdotal evidence does not find how smart animals are, they can be
stupid. Focuses instead on measurable behavioral responses, puts cats in puzzle boxes. Found that these
experiments could show how they learn. He suggested the association is between behavior and
response. The consequence of a successful response it increases the likelihood of the animal repeating
the behavior. “the law of effect”. If the response is followed by an annoying event the behavior is
Discrete trial: one trial, rat goes through the maze, how fast they do and what they do
Free-operant tests: rat in Skinner box, animal is in the test for a long amount of time, animal determines
when it wants to engage in the behavior (pressing the lever), measured in the environment is rates of
behavior, how fast or how often the lever is pressed
Operant: unit of behavior, measurable
Operant response: unit of behavior sufficient for operating on the environment, it doesn’t matter what
it looks like as long as it effects the environment, rat presses lever with nose, tail, paw, all considered
operant response of pushing the lever
Operant conditioning: learning in which behavior is strengthened if followed by reinforcement or
weakened if followed by punishment.
Learning through shaping: to create an initial association, reward the animal just for getting close to
what you want it to do, taking advantage that behavior is highly variable. (learning by successive
approximation). Can rearrange existing behaviors or create new behaviors. Teaching behavior by
reinforcing closer and closer approximations to the desired behavior.
4 types of instrumental conditioning categorized to:
1. Nature of outcome: controlled by the behavior, the result, appetitive or aversive.
2. Relationship contingency: between response and outcome positive R produces O, negative
contingency R eliminates/prevents O.
1. Positive reinforcement: most common, leads to increase in behavior, behavior leads to a good
2. Positive punishment: leads to decease in behavior, behavior leads to a bad outcome,
consequence that is aversive
3. Negative reinforcement: avoidance, behavior leads to absence of aversive outcome, leads to
increase in behavior (stop shocking the rat when animal presses the bar, sweater when its cold
outside to avoid chill) 4. Omission training: behavior leads to good outcome not occurring. Leads to decrease in behavior.
(getting grounded), usually subject gets a reward for other behaviors, Differential
Reinforcement of Other behaviors.
All instrumental conditioning involves
The behavior that we are interested in triggering and increasing or decreasing the frequency of
Usually an arbitrary motor response- rat pushing a lever
Limits on the types of behaviors that can be modified, relevance or belongingness of the things
trying to be associated is an issue in instrumental conditioning as well as classical—rats use their hands
in food, use their hands to push a lever to receive food