Class Notes (922,037)
CA (542,706)
UTM (24,991)
PSY (4,350)
PSY384H5 (12)
Lecture 10

PSY384H5 Lecture 10: Lecture 10

7 Pages
73 Views

Department
Psychology
Course Code
PSY384H5
Professor
Elizabeth Johnson

This preview shows pages 1-2. Sign up to view the full 7 pages of the document.
Lecture 10
March 15th, 2012
Most verbal communication occurs in context where the listener can see the speaker as
well as hear him/her
Visual information
Helps the listener cope with sub optimal listening conditions
oHearing impaired lip readers
oSegmentation studies with infants
oBlind infants may develop speech differently
blind kids don’t produce labials since they can’t see the people’s faces
where they look at the mouth to learn how to say labials
Can cause illusions in perception of auditory speech
McGurk effect
Hearing lips and seeing voices
Auditory “ba” + visual “ga” = heard “da”
oVisual and auditory information fuse to create new perceived utterance
Some individuals  more susceptible than others to hear the McGurk effect
What we SEE overrides what we HEAR
Study tested 3 age groups
oPreschool (3-4 years)
oPrimary (7-8 years)
oAdults (18-40 years)
Results
All age groups far less accurate with visual and auditory input
Responses were dominated by one modality, this tended to be the auditory modality for
children and visual modality for adults
Speech perception is inherently multimodal
Visual information aids speech perception
oSometimes visual information over-rides acoustic information (audio “ba” + visual
“va” = heard “va”)
oSometimes visual and audio information fuse to create a new perceived
utterance (e.g. audio ‘ba’ + visual ‘ga’ = heard ‘da’)
For example segmental contrast difficult to hear often easy to see
oF/TH
oM/N
While wide individual differences in lip reading exist, evidence to date suggest that all
sighted individuals use visual speech information to some degree
Experiment 2
Researchers had adults try to lip read silent videos
Then presented them with audio tape with speech in noise (audio with NO video)
Half adults heard recording as same person, other half heard different person than who
they saw in the video
Silent lip reading aids in listening in noise
Adults can even integrate visual and auditory speech information that is not temporally
coordinated (Almost like McGurk effect but not completely)
Authors concluded that ‘our brain can transfer familiarity with the way a person talks into
familiarity with the sound of his/her voice’
Prosodic information can be gleaned from visual speech
Word stress
Sentence intonation (Question Vs. Statement)
Pitch changes associated with lexical tone
oEg. Mandarin and Cantonese
Even before they begin speaking infants detect characteristics of visual speech
oMatching audit to one of 2 talking faces
oTelling languages apart based on videos alone
oSusceptible to McGurk effect
this is kind of in contrast to the other study with the preschool kids etc, but
they aren’t completely different
**Question: What implications do these findings have for theories of speech?
Auditory based theories of speech perception cannot account for things like the McGurk
effect
**Question: What is the relationship between audio and visual speech?
Amodal accounts of multimodal speech perception claim that in an important way
speech information some whether instantiated as acoustic or optical energy
oThis does not necessarily mean that speech information is equally available in
both modalities.
oAmodal basically means that even if u know about the mcgurk effect u cant stop
urself for still falling for it
Automatic integration supports Amodal accounts.
Late integration theories argue that auditory visual streams of information are analyzed
separately
Top down effects of lexical status support late integration theories
**Question: Then why is it easy to talk on the phone?
Visual information  strong influences listeners perception of auditory speech
Classic theories of speech perception have difficult time explaining this
Somatosensory information also plays a role in speech perception;
Syllables heard simultaneously with cutaneous air puffs were more likely to be heard as
aspirated (for example, causing participants to heard ‘b’ as ‘p’)
This was observed in the absence of training

Loved by over 2.2 million students

Over 90% improved by at least one letter grade.

Leah — University of Toronto

OneClass has been such a huge help in my studies at UofT especially since I am a transfer student. OneClass is the study buddy I never had before and definitely gives me the extra push to get from a B to an A!

Leah — University of Toronto
Saarim — University of Michigan

Balancing social life With academics can be difficult, that is why I'm so glad that OneClass is out there where I can find the top notes for all of my classes. Now I can be the all-star student I want to be.

Saarim — University of Michigan
Jenna — University of Wisconsin

As a college student living on a college budget, I love how easy it is to earn gift cards just by submitting my notes.

Jenna — University of Wisconsin
Anne — University of California

OneClass has allowed me to catch up with my most difficult course! #lifesaver

Anne — University of California
Description
Lecture 10 th March 15 , 2012 • Most verbal communication occurs in context where the listener can see the speaker as well as hear him/her Visual information • Helps the listener cope with sub optimal listening conditions o Hearing impaired lip readers o Segmentation studies with infants o Blind infants may develop speech differently  blind kids don’t produce labials since they can’t see the people’s faces where they look at the mouth to learn how to say labials • Can cause illusions in perception of auditory speech McGurk effect • Hearing lips and seeing voices • Auditory “ba” + visual “ga” = heard “da” o Visual and auditory information fuse to create new perceived utterance • Some individuals  more susceptible than others to hear the McGurk effect • What we SEE overrides what we HEAR • Study tested 3 age groups o Preschool (3-4 years) o Primary (7-8 years) o Adults (18-40 years) Results • All age groups far less accurate with visual and auditory input • Responses were dominated by one modality, this tended to be the auditory modality for children and visual modality for adults Speech perception is inherently multimodal • Visual information aids speech perception o Sometimes visual information over-rides acoustic information (audio “ba” + visual “va” = heard “va”) o Sometimes visual and audio information fuse to create a new perceived utterance (e.g. audio ‘ba’ + visual ‘ga’ = heard ‘da’) • For example segmental contrast difficult to hear often easy to see o F/TH o M/N • While wide individual differences in lip reading exist, evidence to date suggest that all sighted individuals use visual speech information to some degree Experiment 2 • Researchers had adults try to lip read silent videos • Then presented them with audio tape with speech in noise (audio with NO video) • Half adults heard recording as same person, other half heard different person than who they saw in the video • Silent lip reading aids in listening in noise • Adults can even integrate visual and auditory speech information that is not temporally coordinated (Almost like McGurk effect but not completely) • Authors concluded that ‘our brain can transfer familiarity with the way a person talks into familiarity with the sound of his/her voice’ Prosodic information can be gleaned from visual speech • Word stress • Sentence intonation (Question Vs. Statement) • Pitch changes associated with lexical tone o Eg. Mandarin and Cantonese • Even before they begin speaking infants detect characteristics of visual speech o Matching audit to one of 2 talking faces o Telling languages apart based on videos alone o Susceptible to McGurk effect  this is kind of in contrast to the other study with the preschool kids etc, but they aren’t completely different **Question: What implications do these findings have for theories of speech? • Auditory based theories of speech perception cannot account for things like the McGurk effect **Question: What is the relationship between audio and visual speech? • Amodal accounts of multimodal speech perception claim that in an important way speech information some whether instantiated as acoustic or optical energy o This does not necessarily mean that speech information is equally available in both modalities. o Amodal basically means that even if u know about the mcgurk effect u cant stop urself for still falling for it • Automatic integration supports Amodal accounts. • Late integration theories argue that auditory visual streams of information are analyzed separately • Top down effects of lexical status support late integration theories **Question: Then why is it easy to talk on the phone? • Visual information  strong influences listeners perception of auditory speech • Classic theories of speech perception have difficult time explaining this Somatosensory information also plays a role in speech perception; • Syllables heard simultaneously with cutaneous air puffs were more likely to be heard as aspirated (for example, causing participants to heard ‘b’ as ‘p’) • This was observed in the absence of training Study • These participants are in a room and have to say if they hear “ba” or “pa” o Randomly at times they’d feel a puff of air on their hand or wherever, which would actually alter their response • Worked equally well regardless of whether puffs were presented on hand or on neck Conclusion • concluded that the neural processing of speech is more broadly multimodal than previously believed What about gestures and head movements? Head Movements and Auditory Speech Perception (Munhall et al., 2004) • Hypothesis o The head movements speakers produce when talking may carry or highlight linguistic information Study • 20 Japanese sentences recorded • Stimuli presented with naturalistic animated head • Head movements were strongly linked to pitch and amplitude of spoken utterances • Participants tested with a SPIN task Results • Normal head movement was the most • No head movement had the second most • Double head movement (moved really quickly) • Auditory only has the least correct Conclusion • Head movements can lead to enhanced comprehension of the speech signal Audio-visual speech Take Home • Visual information strongly influences the listener’s perception of auditory speech • Classic theories of speech perception have a difficult time explaining this • Use of visual information may be yet another important difference between o Natural speech recognition o Artificial speech recognition What is the slip of the tongue? • Spoonerism and slip of the tongue are not the same thing o Spoonerism is a subset of slip of tongue o Ex. ”blushing crow” for “Crushing blow” o “work is the curse of the drinking classes” for “drink is the curse of the working classes” **Question: Why do psychologists think speech errors are so interesting? • Language units are represented discretely but speech is continuous • Language is infinitely productive because we can combine finite components with specified grammatical rules • These finite components are not readily apparent in the speech gestures or sounds we make • Speech errors provide evidence for the psychological reality of the finite components o Ex. Phonemes ( allow us to see how these components are combined) Types of spoonerisms Anticipation, Preservation or Reversal • ‘mall tan’ instead of ‘a tall man’ reversal • ‘red rag’ instead of a ‘red bag’  perseveration • ‘bew nall’ instead of ‘a new ball’  reversal • ‘green gawn’ instead of a ‘green lawn’  preservation • ‘beal boat’ instead of a ‘real boat’  anticipation What sort of mental organization do “slips” of the tongue reveal? • When words are exchanged, they are usually exchanged with words of the same grammatical category (noun for noun, verb for verb, etc). • Segmental errors constrained by rules of grammar that dictate how sounds can be combined o (‘stips of the lung’ but not ‘tlip of the sung’)  the errors actually form a particular type of pattern • A single feature can be changed while all other features remain as intended o Ex. Clear blue sky Glear plue sky o Switch the place of articulation of initial sounds in words • Slips of tongue are frequent • Ferber reported 50 in 45 minutes in naturalistic radio conversation **Question: Why is slip of the tongue research so controversial? • Most are based on live recordings and also perceptual bias, and u can only write down what u actually hear • Production and perception cannot be separated when collecting slips o Not possible to determine whether a given slip was committed by the speaker or by the listener, o Only way of collecting spontaneous slips would seem to be by means of tape recordings, which should be listened to repeatedly, preferable by more than one person. Ferber 1991. **Question: What precautions can be taken to ensure validity of slips
More Less
Unlock Document


Only pages 1-2 are available for preview. Some parts have been intentionally blurred.

Unlock Document
You're Reading a Preview

Unlock to view full version

Unlock Document

Log In


OR

Don't have an account?

Join OneClass

Access over 10 million pages of study
documents for 1.3 million courses.

Sign up

Join to view


OR

By registering, I agree to the Terms and Privacy Policies
Already have an account?
Just a few more details

So we can recommend you notes for your school.

Reset Password

Please enter below the email address you registered with and we will send you a link to reset your password.

Add your courses

Get notes from the top students in your class.


Submit