Study Guides (238,472)
Canada (115,151)
NROC61H3 (9)
Ito Lee R (2)

NROC61 Midterm notes (Lecture 1-5)

17 Pages
Unlock Document

University of Toronto Scarborough
Ito Lee R

Lecture 1: Pavlovian Conditioning ➣Pavlovian Conditioning •Studies of Pavlov, also known as classical condition •Form of associative learning, by repeated pairing of neutral stimulus with stimulus with reflexive response -Neutral stimulus will acquire to evoke response of reinforcer stimulus ➣Components in Pavlovian Conditioning •Unconditioned stimulus (US): Food /meat powder •Unconditioned response (UR): Unlearned reflex reaction to US (e.g. salivation) •Conditioned stimulus (CS): Usually was a neutral stimulus, be pairing with US (e.g. bell) •Conditioned response (CR): CS alone elicit the UR ➣John Watson – Little Albert •Study with 11 months old boy Albert with fear responses •Rat (CS) will pair with the unpleasant loud noise (US), which evoke fear/crying (UR) response -Then rat alone itself began to elicit same fear response (CR) -Eventually Albert will generalize the fear to other white furry things ➣Other properties of classical conditioning •Acquisition: Repeated pairing of CS-US, initial stage of learning •Extinction: Presentation of CS alone without US, occurrence of UR will decrease/disappear •Spontaneous recovery: Extinction features disappear, recovery of condition response -CR re-appears, but strength of response is not as high as acquisition •Stimulus generalization: Strength of response will similar if share same properties between 2 stimuli -This properties have an adaptive significance •Stimulus discrimination: More further apart, more easier to discriminate between 2 stimuli ➣Adaptive Significance •Pavlovian conditioning allows animals to prepare for upcoming event •i) Signal for Food: Can elicit many anticipatory responses (salivation/flow of gastric juices) -Taste aversion learning: can induce aversion to food that causes gastric illness •ii) Signal for Fearful events: -Allow animals to prepare for dangerous encounter, by activate “fear system” -Rodent main responses to cue associated with electric shock is freezing -Freezing is a predominant response, in face to predator attack •iii) Reproductive significance: Zamble et al. experiment -Experiment in male rat in exposure to receptive female with access prevented (US) -The placement in distinctive plastic chamber (CS) is preceded to exposure to receptive female -Ejaculation latency was shorter when expose to CS Todranks et al. experiment -Pairing opposite sex pictures (CS) with related odours (US) -Odour can shift attractiveness of pictures, disliked odours with paired picture is rated as less attractive Hollis et al. -Male blue gourami fish has 10s CS signalled appearance of egg-bearing female -Start courtship display sooner in conditioned group, compared to control group •iv) Territorial defense -Expose male gourami to red light stimulus (CS), with presentation of male rival (US) behind barrier -Red light itself elicit aggressive display in fish ➣Excitatory and inhibitory Conditioning •Excitatory conditioning: or known as conditioned excitation -CS signals for occurrence of US/ reinforcer -Occur with US which is either appetitive or aversive -Still consider as excitatory conditioning even the CR is inhibition of behaviour •Inhibitory conditioning: known as conditioned inhibition -CS signals the absence of US/reinforcer -e.g. Previous paired flashing light (CS) with food (US), now Light + Metronome with no food -Stop salivating in light + metronome trial, metronome becomes signal of no food •Conditioned excitor: Responds to CS as expects an US •Condition inhibitor: More difficult to detect, test by 2 tests -1) Summation: Present A at the same time as excitatory stimulus B -A is an inhibitor when responding is reduced -Sum of responding to A+B < responding to B -2) Retardation: Train A as excitor, acquisition of CR will be retarded if has inhibitory strength ➣ Stimulus- Stimulus vs. Stimulus-response association •Stimulus-stimulus (S-S) association: -CS activates “neutral and behavioral” representation of US, in absence of actual US -States the association formed between 2 neutral stimuli •Stimulus- response (S-R) association: -Association made between CS and unconditioned response (UR) -More reflexive association than S-S association ➣ US revaluation: animal are trained on CS-US association, then value of US is devalued -If connection is CS-UR: Alteration should have effect -If connection of CS-US: Alteration have an effect -e.g. Rats learned pairing between Tone (CS) and Food (US) -Then Food (US) will pair with aversive Lithium Chloride on re-exposure to Tone (CS) -Rat made less approach to CS, which support S-S theory ➣ Sensory Preconditioning: •Experimental group: Tone (CS1) + Light (CS2) → Tone (CS1)+ Shock → Light (CS) → Fear •Control group: Light (CS2) → Tone (CS1) + Shock (US) → Light → No fear •When light presented alone in test session, induce a fear response in experimental group, not control -Evidence of S-S association, in absence of US ➣ Second-order conditioning •Result of experimental group 2 will indicates S-S learning / S-R learning -S-S learning predict there’s no response to light CS, since tone-shock association is extinguished -but rat continues to respond to light: Direct association of Light (CS2) and Fear (UR) •S-S / S-R association depends on whether stimulus or response is more salient -S-S association: Stronger in early phases of training -S-R association: more powerful in later phase of training ➣ The relationship between UR and CR •Pavlov proposed CS acts as substitute for US – Stimulus substitution theory (CR=UR) -Support by Autoshaping and Conditioned aggression -Autoshaping: Pigeon pecks at CS light after repeated pairings between CS and US -Conditioned aggression: Male Gourrami fish is highly territorial, start attacking th CS •CR is not always same as UR: •Preparatory response theory: CR is the behaviour system related to US -Conditioning with food and sex have “approach” response to CS •Opponent process theory: CR is the process opposes the UR -e.g. Drug tolerance: repeated administration of drug morphine will reduce effectiveness of drug -Measure tolerance by measure latency to lift up its paws from hot plate -Tolerance: result of conditioning taking place between environmental cues -Tolerance developed only in environment administered the drug -Opponent responses (CR) serves to counteract the drug action, lower pharmacological reactions ➣ Neural basis of Pavlovian conditioning •Electrolytic lesions: Apply of electrical current to brain region of interest -Often produce large lesions that were not confined to region of interest •Aspirative lesions: Remove brain regions through suction •Chemical/ Excitotoxic lesions: Using glutamate receptor agonists (e.g. NMDA, quinolinic acid) -Selectively target the cell bodies of neurons, which reside within interested brain region -Can generate small, discrete lesions in smallest brain area through ➣ Neural basis of Pavlovian approach behavior •Acquisition of autoshaping allow conditioned process to be isolated from unconditioned processes •Rats develop conditioned approach response which selectively approach CS+ before food reward •Rats trained to learn presentation of visual stimulus (CS+) on video screen (D), paired with food -Second visual stimulus is presented on other side (CS-), not paired with reward -Rats selectively approach CS+ before the food is delivered •Everitt and Robbins’ experiment: -Tried to established network of structures involved in learning by excitotoxic lesions -Pre-acquisition lesion: allow study of “learning” (lesion before learning) -Post-acquisition lesions: allow study of “expression” of learned behaviour •Brain region involved in acquisition: -Nucleus accumbens core, central nucleus of amygdala, anterior cingulated, orbitoprefrontal cortex •Brain region involved in “learning” behaviour: -Orbitoprefrontal cortex, central nucleus of amygdala -Rest of the brain regions are involved in both learning and performance of associations ➣ Dopamine and Pavlovian approach •Parkinson et al. experiment with 6-hydroxydopamine (OHDA) lesion -Lesions in nucleus accumbens on acquisition and expression behavior •6-hydroxydopamine (OHDA): -Induce siginificant noradrenaline and dopamine depletion in NAc & Prefrontal cortex -Neurotoxin which selectively destroys dopaminergic & noradrenaline terminals -Enters nerve terminals via dopamine & noradrenaline reuptake transporters -Also affect serotonin (5-HT) depletions and catecholamine levels (in dorsal striatum) •NAc dopamine depletion: Impair acquisition and expression of discriminated approach behaviour -Have greater impact on acquisition than expression •Basolateral amygdala (BLA): Central to formation of stimulus-reward associations -Lesion of BLA impair reinforcer re-valuation and second-order conditioning •Sham+/- = Control group, which rats without injection of poison undergo identical procedure Lecture 3: Laws of Association  Laws of Associability •Factors which affect the strength of conditioning between CS and US: 1) Temporal contiguity 2) Nature of CS & US 3) Causal relevance 4) Contingency ➣1) Temporal contiguity: •Two stimuli must occur close together in order to form association •Delay conditioning: When CS occurs right before US, works best among different conditioning -Conditioning ↓, when interval between CS and US exceeds •Trace conditioning: CS and US are separated by big interval -Reflect on contextual conditioning, rather than CS (e.g. extinction) •Simultaneous conditioning: CS and US are presented simultaneously •Backward conditioning: US is presented before the CS, least favourable for conditioning •As CS presented more closer to US, more ↑ strength of association ➣ 2) Novelty (Novel stimulus facilitate learning) •Prior exposure to US/CS interferes with formation of new learning -e.g. Latent inhibition / CS pre-exposure effect, reduce speed of new learning •Latent inhibition/ CS Pre-exposure effect: Tend to ignore familiar stimuli Phase 1: Presentation of CS alone → Phase 2: Conditioning training with CS & US -Causes habituation of attentional response to stimulus -Pre-exposure of CS will retard (slows) the subsequent conditioning -Repeated pre-exposure to CS will limit the attention/processing to stimulus -e.g. Less likely to associate familiar stimuli with aversive experiences •Latent inhibition paradigm: Used as animal model of schizophrenia •US Pre-exposure effect: US may conditioned with contextual background cue ➣3) Salience (intensity ) of CS & US: •The more intense of 2 stimuli, more rapidly and easily to be associated •Might because of intense stimuli tend to be more biologically important ➣4) Frequency: More frequency stimuli are presented together, more stronger they’reassociated •e.g. Thunder and lighting ➣5) Similarity: Some stimuli are more readily associated than others •Internal stimuli go together, external stimuli go together •Might due to the structure of sensory systems within brain •e.g. taste and illness, sounds and shock ➣6) Causal relevance: Strength of association depends on informational value of CS-US association •Will attribute particular effect with it’s most probable cause -Attribution is based on past experience/ evolutionary history -e.g. aversion to bitter taste than to sweet, fear of snake than of flower •Selective conditioning: Example of Pavlovian conditioning, which being sensitive to causal relevance ➣7) Contingency: Learning requires that CS predicts the occurrence of US •Rescorla: Proposed that coincidence of CS and US (Contiguilty) is not sufficient for learning -CS must have Contingent relationship, which CS must predicts US -Rescorla proposed the Contingency theory •Conditioning will be affected by probability of: -CS being followed by US [p(US/CS)] & US in absence of CS [p (US/ No CS) ] •Positive contingency: When p(US/CS) > p (US/ no CS) -CS signals ↑ probability that US will occur, produce excitatory conditioning -Consistent probability of US paired with CS, reliable to predict the rewarding US •Negative contingency: When p(US/CS) < p(US/ no CS) - CS signals ↓ probability that US will occur, absence of reward and cause inhibition -Produce inhibitory conditioning •Zero contingency: When p (US/CS) = p (US/no CS) -CS predicts neither ↑ or ↓ in probability of US, everything is random -CS is ignored, no learning takes place •Rescorla conditioned suppression: Lever press for food → CS paired with shock US -4 conditions with different probability of occurrence, all same # of CS-US pairing -Same contiguity in all groups, but conditioning time slot is interspersed with US presented alone -Measure the CR (freezing) + Rate of bar pressing during tone interval •Best learning when 0 in p(US/no CS)  Rescorla & Wagner Theory •Assume learning occurs only if the US is surprising / unpredicted (Violation of expectation) •US expectancy is important in learning CS-US association •Learn the most in the first few trials, CS become perfect predictor of US when asymptotic level is reached •Equation in R-W model: ∆V=α (λ - V) -V =associative strength/ predictive value α = Intensity of CS, constant between 0-1 - (λ - V) =Suprisingness of US λ = Dependent on US magnitude (large λ = large US) •US is predicted on trial depends ALL CSs that present on trial: ∆V=α (λ - ∑V) -∑V = Summed V values of all stimuli present on given trial •Context can also act as US predictor •R-W model CANNOT explain latent inhibition using the equation  Leon Kamin: Blocking experiment •Blocking: Occur when one of the elements of compound CS is already fully conditioned -Stimulus A was conditioned to asymptote (V = A, in maximum associative strength ) -No learning in stimulus B (VB=0 ) , since A already fully predicts the US, US is no longer surprising •Unblocking: Overcome blocking by increase intensity of US in phase 2 -Able to give λ bigger value by increase intensity, which B will be able to gain associative strength •Conditioned inhibition: Occur in 2 phases: -Phase 1: Stimulus A is conditioned to predict occurrence of US -Phase 2: Conditioning trial with compound stimuli. Without US (λ=0) -CS as a inhibitor will have a negative value, acquires negative strength  Mackintosh Model •Assume when CS become more better predictor, the more the subject will attend to it • α ↑as conditioning proceeds, ↓when CS becomes weak predictor of US •Explain latent inhibition effect, α↓ when CS presented without US in pre-exposure phase -Pre-exposed CS will have lower α value, which impair the rate of learning • Hall and Pearce experiment: -Group1: Tone + Weak shock before, then conditioning with stronger shock in 2 phase nd -Group2: Receive different shocks, but not presented till 2 phase -Mackintosh theory: Hypothesis that group 1 will be learn the association quickly -Hall and Pearce finding: Phase 1 cause slower learning of tone in phase 2 -α↓ as became a good predictor of US  Pearce- Hall model •Proposed less attention will be paid when learned what the CS is predicts -α↓ of CS as became a good predictor of US (As conditioning proceeds) Summary: •Mackintosh and Pearce-Hall model: -Overcome with shortcomings of RW model, noting subjects need to pay attention to CSs -Stated attention is dependent on previous learning, hence attention changes Dopamine and reward prediction error •At least 3 dopamine pathways in the brain: 1) Mesolimbic (VTA) 2) Mesocortical(VTA) 3) Nigrostriatal dopamine systems (SNr) •Hypothesis of Dopamine: -It acts as neurochemical substrate of prediction error -Signals difference between experienced and anticipated reward ( λ-V) -Shows bi-directional coding of reward prediction errors ➣ Evidence of hypothesis: Reward Prediction Error Hypothesis •Wolfram Schultz’s Lab: Take single-unit recording from midbrain dopamine neuron -Involved monkeys in training GO/No-GO task, which press/withhold bar press if the light is green/red -Midbrain dopamine neurons: VTA and SNr cell group alike -Show phasic excitatory responses (activation) after primary food reward -Unpredicted reward: Biggest activation Expected: Depression in neuronal activity •Waetli, Dickson, Shultz: Used Kamin’s blocking procedure -Studied on single neuron responses in substantia nigra + Ventral tegmental areas Pavlovian Training: Stimulus A: associated with juice reward (A+) Stimulus B: Not associated with juice reward (B-) Compound training: Stimulus A+X: associated with juice Stimulus B +Y: Associated with juice -Used spout licking as behavioural measure (CR), only present in B+Y, not in A+X -Stimulus A followed by fully predicted reward did not induce any DA response -Hence both Stimulus A+X did not have activation of DA, already predicted by A -In B+Y: Robust DA response, prediction error occurs, novel Y becomes valid reward predictor Test sessions: (X & Y presented alone) -In X + no reward: Did not elicit any DA response, no prediction error -Positive prediction error when reward is presented after X presented -In Y+ no reward: Negative prediction error, since Y predicts presentation of reward -Elicit depression of dopamine response • DA response to unpredicted primary reward is varies with reward magnitude -Higher the probability of reward, smaller the response Lecture 3: Instrumental Learning • Adaptation requires sensitivity to causal consequences of its actions •Instrumental conditioning: -Requires subject to discover certain response in stimulus situation leads to reinforcement  Pavlovian vs. Instrumental conditioning •Both Pavlovian and instrumental conditioning have many common features -e.g. Both include acquisition/extinction/ spontaneous recovery/ stimulus generalization •Grindley G.C: Train
More Less

Related notes for NROC61H3

Log In


Don't have an account?

Join OneClass

Access over 10 million pages of study
documents for 1.3 million courses.

Sign up

Join to view


By registering, I agree to the Terms and Privacy Policies
Already have an account?
Just a few more details

So we can recommend you notes for your school.

Reset Password

Please enter below the email address you registered with and we will send you a link to reset your password.

Add your courses

Get notes from the top students in your class.