Class Notes (860,433)
CA (521,022)
UTM (24,396)
PSY (4,265)
PSY384H5 (12)
Lecture 10

PSY384H5 Lecture 10: Lecture 10

7 Pages

Course Code
Elizabeth Johnson

This preview shows pages 1 and half of page 2. Sign up to view the full 7 pages of the document.
Lecture 10 th March 15 , 2012 • Most verbal communication occurs in context where the listener can see the speaker as well as hear him/her Visual information • Helps the listener cope with sub optimal listening conditions o Hearing impaired lip readers o Segmentation studies with infants o Blind infants may develop speech differently  blind kids don’t produce labials since they can’t see the people’s faces where they look at the mouth to learn how to say labials • Can cause illusions in perception of auditory speech McGurk effect • Hearing lips and seeing voices • Auditory “ba” + visual “ga” = heard “da” o Visual and auditory information fuse to create new perceived utterance • Some individuals  more susceptible than others to hear the McGurk effect • What we SEE overrides what we HEAR • Study tested 3 age groups o Preschool (3-4 years) o Primary (7-8 years) o Adults (18-40 years) Results • All age groups far less accurate with visual and auditory input • Responses were dominated by one modality, this tended to be the auditory modality for children and visual modality for adults Speech perception is inherently multimodal • Visual information aids speech perception o Sometimes visual information over-rides acoustic information (audio “ba” + visual “va” = heard “va”) o Sometimes visual and audio information fuse to create a new perceived utterance (e.g. audio ‘ba’ + visual ‘ga’ = heard ‘da’) • For example segmental contrast difficult to hear often easy to see o F/TH o M/N • While wide individual differences in lip reading exist, evidence to date suggest that all sighted individuals use visual speech information to some degree Experiment 2 • Researchers had adults try to lip read silent videos • Then presented them with audio tape with speech in noise (audio with NO video) • Half adults heard recording as same person, other half heard different person than who they saw in the video • Silent lip reading aids in listening in noise • Adults can even integrate visual and auditory speech information that is not temporally coordinated (Almost like McGurk effect but not completely) • Authors concluded that ‘our brain can transfer familiarity with the way a person talks into familiarity with the sound of his/her voice’ Prosodic information can be gleaned from visual speech • Word stress • Sentence intonation (Question Vs. Statement) • Pitch changes associated with lexical tone o Eg. Mandarin and Cantonese • Even before they begin speaking infants detect characteristics of visual speech o Matching audit to one of 2 talking faces o Telling languages apart based on videos alone o Susceptible to McGurk effect  this is kind of in contrast to the other study with the preschool kids etc, but they aren’t completely different **Question: What implications do these findings have for theories of speech? • Auditory based theories of speech perception cannot account for things like the McGurk effect **Question: What is the relationship between audio and visual speech? • Amodal accounts of multimodal speech perception claim that in an important way speech information some whether instantiated as acoustic or optical energy o This does not necessarily mean that speech information is equally available in both modalities. o Amodal basically means that even if u know about the mcgurk effect u cant stop urself for still falling for it • Automatic integration supports Amodal accounts. • Late integration theories argue that auditory visual streams of information are analyzed separately • Top down effects of lexical status support late integration theories **Question: Then why is it easy to talk on the phone? • Visual information  strong influences listeners perception of auditory speech • Classic theories of speech perception have difficult time explaining this Somatosensory information also plays a role in speech perception; • Syllables heard simultaneously with cutaneous air puffs were more likely to be heard as aspirated (for example, causing participants to heard ‘b’ as ‘p’) • This was observed in the absence of training Study • These participants are in a room and have to say if they hear “ba” or “pa” o Randomly at times they’d feel a puff of air on their hand or wherever, which would actually alter their response • Worked equally well regardless of whether puffs were presented on hand or on neck Conclusion • concluded that the neural processing of speech is more broadly multimodal than previously believed What about gestures and head movements? Head Movements and Auditory Speech Perception (Munhall et al., 2004) • Hypothesis o The head movements speakers produce when talking may carry or highlight linguistic information Study • 20 Japanese sentences recorded • Stimuli presented with naturalistic animated head • Head movements were strongly linked to pitch and amplitude of spoken utterances • Participants tested with a SPIN task Results • Normal head movement was the most • No head movement had the second most • Double head movement (moved really quickly) • Auditory only has the least correct Conclusion • Head movements can lead to enhanced comprehension of the speech signal Audio-visual speech Take Home • Visual information strongly influences the listener’s perception of auditory speech • Classic theories of speech perception have a difficult time explaining this • Use of visual information may be yet another important difference between o Natural speech recognition o Artificial speech recognition What is the slip of the tongue? • Spoonerism and slip of the tongue are not the same thing o Spoonerism is a subset of slip of tongue o Ex. ”blushing crow” for “Crushing blow” o “work is the curse of the drinking classes” for “drink is the curse of the working classes” **Question: Why do psychologists think speech errors are so interesting? • Language units are represented discretely but speech is continuous • Language is infinitely productive because we can combine finite components with specified grammatical rules • These finite components are not readily apparent in the speech gestures or sounds we make • Speech errors provide evidence for the psychological reality of the finite components o Ex. Phonemes ( allow us to see how these components are combined) Types of spoonerisms Anticipation, Preservation or Reversal • ‘mall tan’ instead of ‘a tall man’ reversal • ‘red rag’ instead of a ‘red bag’  perseveration • ‘bew nall’ instead of ‘a new ball’  reversal • ‘green gawn’ instead of a ‘green lawn’  preservation • ‘beal boat’ instead of a ‘real boat’  anticipation What sort of mental organization do “slips” of the tongue reveal? • When words are exchanged, they are usually exchanged with words of the same grammatical category (noun for noun, verb for verb, etc). • Segmental errors constrained by rules of grammar that dictate how sounds can be combined o (‘stips of the lung’ but not ‘tlip of the sung’)  the errors actually form a particular type of pattern • A single feature can be changed while all other features remain as intended o Ex. Clear blue sky Glear plue sky o Switch the place of articulation of initial sounds in words • Slips of tongue are frequent • Ferber reported 50 in 45 minutes in naturalistic radio conversation **Question: Why is slip of the tongue research so controversial? • Most are based on live recordings and also perceptual bias, and u can only write down what u actually hear • Production and perception cannot be separated when collecting slips o Not possible to determine whether a given slip was committed by the speaker or by the listener, o Only way of collecting spontaneous slips would seem to be by means of tape recordings, which should be listened to repeatedly, preferable by more than one person. Ferber 1991. **Question: What precautions can be taken to ensure validity of slips
More Less
Unlock Document
Subscribers Only

Only pages 1 and half of page 2 are available for preview. Some parts have been intentionally blurred.

Unlock Document
Subscribers Only
You're Reading a Preview

Unlock to view full version

Unlock Document
Subscribers Only

Log In


Don't have an account?

Join OneClass

Access over 10 million pages of study
documents for 1.3 million courses.

Sign up

Join to view


By registering, I agree to the Terms and Privacy Policies
Already have an account?
Just a few more details

So we can recommend you notes for your school.

Reset Password

Please enter below the email address you registered with and we will send you a link to reset your password.

Add your courses

Get notes from the top students in your class.