PSYC 310 Lecture Notes - Lecture 18: Sucrose, Supervised Learning, Opiate
7 May 2018
School
Department
Course
Professor
PSYC 318
Behavioural Neuroscience II
January 31st, 2018
Lecture 8/24: Neural circuits underlying drug addiction
• Midterm 1 is Wednesday in class
o Conference review session Tuesday the night before for any last-minute
questions
o Material discussed Monday will be included on midterm
o 40 multiple choice questions
• Finishing paper from last class: orbitofrontal cortex supports behavior and learning using
inferred but not cached values
o Author argues that people point to the OFC as a region where people calculate
universal currencies: neuro-economical idea that marketers could image people
in fMRI scanner to get a sense of how their brain evaluates things in their
environment
o Animals can reach evaluation of a stimuli based on histories of reinforcement
o Or they can use a more model-based approach to calculating values: will this
work for me in the future?
o Paper 8 research question: is the orbitofrontal cortex (OFC) involved in
calculating the value of things, regardless of whether the value derives
from model-based or model-free thinking?
• Using inferred value to block new learning
o Model-free system of brain is not going to be active: no reinforcement
o But cognitive part of animal learns that stimuli are associated with each other:
learning information in the absence of value
o As you condition the animal, if B predicts food there is now direct reinforcement
with cue B and food
▪ Model-free system attaches value to cue B because its worked for me in
the past
▪ B has value regardless of which system
▪ What happens when you show cue A?
• Difference in how they’re calculating value: model-free system
says it has seen cue A before and it has never shown association
with something of value
• Model based cognitive system asks where it has seen A before,
what it reminds me of (cue B) which reminds me of food, maybe
this presentation of A is linked with food?
▪ Animals do have expectations of food in presentation with cue A, you see
this in normal animals
▪ But if you inactivate OFC on this day the animals don’t respond to cue A
• OFC is needed for the brain to associate values of things in a
model-based manner (based on knowledge of world, associative
structure of stimuli in task)
o But not for model-free evaluations
▪ If this is true we should see other effects when we inactivate the
OFC
find more resources at oneclass.com
find more resources at oneclass.com
• Blocking study: learning about certain cues will be blocked
because those cues are somewhat redundant: a compound
cue (AX) results in food delivery
o Animals learn this, check food port whenever they see
AX
• Where is the learning being attributed to? Which cue?
• Knowledge of the predictive value of cue ‘A’ can potentially
block new learning. Is the OFC required for learning effects
that are dependent on model-based valuations?
o In the absence of OFC activity, animal is surprised
when food comes and learning comes to cue A and X
o When you show X later, did the animal learn about it
because he was surprised by food when OFC was
blocked? Or was there no learning about X because
animals expected food when AX was presented
(previously associated A with it)
• Blocking sessions with OFC inactivated
o All rats learn that 3 food pellets are dropped in the food port during the
presentation of compound cues AX and CY. OFC inactivation did not
influence this learning, which indicates it is not needed to learn simple
cue-reward associations
o Cues X and Y are completely new to the rats. Cue C has been seen
before but has no association with food. Cue A was associated with a cue
(B, not presented here) that was later associated with food
o Where Is the learning being attributed to?
▪ In control animals, was knowledge of learning about cue X blocked
because A had expectations associated?
▪ Inferred predictive value of cue A blocked learning about cue X
• Cue X was considered to be somewhat redundant because
cue A was known to predict cue B which was known to
predict food drops
▪ In animals with OFC inactivated, food delivery was surprising
because OFC was not available to say that A might predict food:
they learned equally about both cues and in the future if presented
either they checked food port on both cues
▪ OFC inactivation during compound cue training prevented rats from
inferring the value of cue A. And since no value was inferred for cue
A, its presence did not block learning about the predictive value of
cue
• Conclusions of paper
o Neural activity in the OFC is needed to utilize models of the world
(associative structure of task and stimuli) to guide behavior, create value
absed on models
▪ Model-based info to guide behavior
▪ Either calculates model-based valuations or is in chain of circuitry
needed to pass along this info to other brain regions
find more resources at oneclass.com
find more resources at oneclass.com
o Neural activity in the OFC is needed for model-based info to influence new
learning
▪ Model-based info leads to expectations, shapes learning, but OFC
is needed for this behavior.
NEXT PAPER:
OFC function in drug addiction
• Machine learning
o We have computers that can read handwritten checks, decode speech,
identify faces, play games better than humans, etc.
o Where does the success come from?
▪ Initially, in late 90s, there wasn’t much machine learning involved
▪ Computers fast at thinking which worked well for the end of the
chess game but not for the beginning: at the beginning there were
too many things to think through, whereas chess experts had set
choreographed maneuvers
▪ They hard-coded values of all the opening moves: preprogrammed
thousands of starting move values, set standard opening moves
▪ No learning involved here, just hard-coding and quick thinking
o People then began to have more success with actual machine learning:
not relying on processing speed or ability to preprogram, but a deeper
learning:
o How do children actually learn things, like to read?
▪ In general, you show children letters individually and tell them what
they are, across all different iterations of those letters (its different
forms, handwritings, fonts, etc.)
▪ This is called supervised learning, you had a supervisor or
instructor telling you the answer for a long period of time
▪ Idea to program that process into a machine: first dramatically
demonstrated with reading handwritten checks
• Give computer a few thousand handwritten checks and label
them, telling computer what the right answer to each is
• Computer figures out based on 10,000 instances, what the
number 5 looks like
• What features on this text consistently correlate with the
number 5, vs. the number 4 or 3
• Develop system of identifying features that are consistent in
each number across many different checks with an algorithm
• Was largely a success
▪ Supervised learning: give computer data, answers, then going
forward see how accurate the computer is
• Machine learning terminology – supervised learning
o Supervised learning: by looking at a bunch of labeled data, can you figure
out a system that you could have used to label that data. Then you ask if
the system works with new data
find more resources at oneclass.com
find more resources at oneclass.com
Document Summary
If this is true we should see other effects when we inactivate the. Ofc: blocking study: learning about certain cues will be blocked because those cues are somewhat redundant: a compound cue (ax) results in food delivery, animals learn this, check food port whenever they see. Which cue: knowledge of the predictive value of cue a" can potentially block new learning. Ofc inactivation did not influence this learning, which indicates it is not needed to learn simple cue-reward associations: cues x and y are completely new to the rats. Cue c has been seen before but has no association with food. Inferred predictive value of cue a blocked learning about cue x: cue x was considered to be somewhat redundant because cue a was known to predict cue b which was known to predict food drops. And since no value was inferred for cue.
