Lecture 10 - 03-25-13
continued from last lecture...
Knowledge is represented via a vast network of connections and associations between all of the
information you know. Information held in nodes. Nodes are connected similar to what we've
talked about in the past. Other evidence for the knowledge representation in a network comes
from sentence-verification task. This has been a way to show knowledge in a network is a
plausible idea. Basically there's a few different ways to look at it; one is answering true or false
statements i.e. robins are birds, robins are animals, seeing how fast people respond. There's
different types of true statements. There's ones that are directly associated or others that are a
few links away. "Cats have hearts", with this kind of idea, we're generally thinking that animals
have hearts as a more representative idea, cat is an animal, so an animal has a heart. "Cats have
claws" requires one link. It's a much more direct link because not all animals have claws.
People looked at the time to answer these questions. The associative path in these questions,
how they hypothesized will determine their speed of answering true or false. The associative
path takes a little more time if they're not directly linked. We're trying to say that maybe
information is organized in this manner and this is some type of proof that it could be.
Nodes can represent concepts. Links such as hasa or isa can associate each concept. If we want
to say yeah, Max is my dog, and my friend Max has a dog, those are different statements and
ideas and tell different things to us. In one, the Max is represented as a dog. The idea came
about that the way if each node represents a concept rather than connecting information, that
hasa and isa link represents a linkage of information. It's not efficient however based on the
number of links there could be. It then becomes this thing where you need something to
oversee these links. Has to be something that instructs what that link is, becomes inefficient.
A more complex network (Anderson ACT) was around a notion of a proposition, smallest unit
that can be true or false. The proposition is what happens between the agent and the object.
We can say dog, bone. Dog is agent and bone is object. The relationship/proposition is chew;
true or false; eat, true or false, that's the relationship. With this model, it becomes easier to try
to represent this huge network of information we have in our knowledge. We can add to it; with
the addition of time and place. We have time and location nodes that are set between the agent
and the object, with that we can differentiate things. Jacob is agent and object is pigeons,
relationship is feeds, time is last spring, location is Trafalgar Square. All this is to say that you can
build these types of networks, there's a node representing this information, a node for agent
and object, another for time and location which gives us the relationship of when they
happened and other types of information. While it still fits, you can imagine that it would be
quite large if every single information has this node. You think about something, lights up a
node, surrounding nodes activate - serial processing. There's something that's more common
nowadays and thought of as an actual representation of what's going on.
Localist representations - each node is equivalent to one concept. Connectionist networks
Information in connectionist doesn't have a concept node activating others. Instead it's a bunch
of nodes that represent one thing, huge network of nodes representing one piece of
information. They're not unique to only that network. E.g. we can think of the class, A, B, C, D
node; car, D, E, F node, overlap; some of these nodes can be used for other information which
becomes complex because you cannot pinpoint where the node is held. We call this parallel
distributed processing (PDP). There's no central authority that has to say this is how you learn.
Rather, the way learning happens is through connection weights or strengths between different
concepts/information. There's a lot of algorithms in play. Learning algorithms - how weights are
changed, make them stronger or weaker. E.g. having cells that fire at the same time or having
nodes that activate at the same time. When that occurs, they tend to become stronger or more
linked. "cells that fire together wire together". Error signals cause a node to decrease its
connections to input nodes that led to the error (back propagation). They can send an error
signal backwards "gave me the wrong information", changes the strength of the connections
between the first and second and throughout the whole network, and that can reshape how
pieces of information are fit together in the network. How we take our information: we believe
it's stored in memory somewhere. We also say there is reliance of knowledge. How that works is
that it's in this network: whether it's a single node representing a concept or distributed
processing which is more plausible; we can model them now, we can also use them as a way to
try to figure out how other parts of our brain is functioning and how we can have interaction
between knowledge and our systems. The brain divides and conquers. Generally when one
action is happening, there are a bunch of parts lighting up and taking control of it and certain
parts that fit into that model.
Chapter 9: Language
Organization of language
biological roots of language
language and thought
Language is the one thing that makes humans what they are because it relishes in
communication. That's also true for animals but it's not as potent as it is among humans.
The organization of language
If we start to break it down and say what's happening with language, we're thinking of
something, have an idea, using our knowledge or something in memory, taking that from our
thoughts and making them into sounds. And then conversely when someone's talking to us,
we're hearing sounds and turning those into thoughts, there's some hierarchal structure that we
have of organization to be able to do this.
Sentences - sequences of words; things we deal with when we're writing and speaking words - smallest free form
morpheme - smallest unit of meaning;
phoneme - smallest unit of sound
Hierarchal, with each level composed of sublevels. Phonemes are the sounds being made. We
have 40 sounds in English in which we combine to make words. That still leads to infinite
amount of information we could produce.
We have air flowing from our lungs pushed down from mouth and nose, modulated by tongue
and teeth to make sounds. We used to the vocal folds.
Whether vocal folds vibrate - zdbv
Or not - stpf
Manner of production
whether air is fully stopped or merely restricted - bpdt/zsvf
Place of articulation
where in the mouth the air is restricted
closing of lips - bp
top teeth against bottom lip - vf
tongue behind upper teeth - dtzs
Many words have no clear boundaries yet speech segmentation is effortless. When we're talking
in the sentence, we can hear a clear distinction in the words when there is in fact none. We
can't necessarily segment the sentence into pieces of information that makes sense (esp. in
foreign languages). How do we learn languages in the first place? It's just one big sound wave.
What are the properties and how are we able to segment the words? This guy is falling! - The sky is NOT falling! (example from the text)
Coarticulation the blending of phonemes at word boundaries. When we pronounce a letter like
S in the following concepts, the S and the N form different sounds versus S and D, we still can't
necessarily - even based on sound itself - to distinguish what the phoneme is.
"My name is Dwayne."
"My name is Noam Chomsky"
S is slightly different
D and N are slightly different
Perception of language is constructed --> use prior knowledge to fill in missing information. In
one study they had people listen to the sentence of "the state governors met with their
respective legi*latures convening in the capital city." Couldn't tell that the word legislature was
disrupted (by static). No consensus that the static disrupted that particular phoneme which
means that we don't need to hear it to fill in that information, to have the perception that we
have heard it. We call this phonemic restoration effect. When we take that word out of context
and into a list of words, people can easily ID what the phoneme is, or you can do it in a way
where they can't understand what the word is. As soon as you put it into a context of a
sentence, the word can be heard again. We have some ability to use context to help fill in the
There's a study looking at PA versus BA sounds. These two are pretty similar and not easy to
distinguish. E.g. in a noisy room, there may be difficulties distinguishing. Some researchers
slightly altered these sounds so that BA became PA and PA became BA, they altered them so
that there was a slight change in sound until they fully became in each extreme each of the
sounds. You ask people what sound is being said, in one extreme, ok it's PA, when it reaches the
other extreme, people start to make errors (going to BA), some people would say PA some
would say BA. People continued to hear BA up until a point where it changed to PA. Even though
there was a slight variation, an equal variation, the participants heard one or the other which
starts to show us that we probably have some built in mechanism to cushion variability of what
inputs we're getting. We're talking about categorical perception. We see something as in one
category or another. We find that this happens when especially slight changes within a group
versus between a group. Method of articulation or voicing; or in this case the B and P have very
similar ways of being pronounced which is why they're hard to distinguish. They're both being
pronounced using air, lips, b, p. With that slight variation - noisy room or not getting full input -
you can't necessarily tell one versus another. Yet we have the ability to put these in categories.
We can move on to things like how we put these sounds together in terms of language. We're
talking a lot in terms of the English language. Researchers must determine if something that
they're looking at is English phenomena or another language, it shouldn't matter what language,
should be universal. Also important to study differences and what they might be.
We may not see tl sequence as acceptable in English. Only some are acceptable in a language.
We also adjust for certain phoneme sequences. S sound becomes a z in words like bags. These
are rules that govern how things might be adjusted and what's acceptable in certain languages. Words
We also know things like phonology of the word - sequences of phonem