| Home | Table of Contents |
General Intelligence :
The NLP (Natural Language Processing) module handles the mapping between natural languages (eg English) and the knowledge representation.
There are 3 aspects of natural language that a complete NLP module must successfully deal with:
Two excellent textbooks on NLP are [Jurafsky & Martin 2000] and [Nugues 2006].
People who dislike old-fashioned AI tend to think that the traditional approach of translating natural language sentences into "logical form" is doomed. Usually their argument is that such an approach is too inflexible to deal with variabilities in natural language texts.
Techniques in computational pragmatics should convince you that the logical approach is very sound. In particular, look at the research of Jerry Hobbs [Hobbs et al 1993] [Hobbs in progress], "Interpretation as Abduction".
In logic, abduction means finding explanations. An explanation { E } for a proposition P is simply a set of statements such that KB U { E } entails P; where KB is the background knowledge.
When applied to interpretation, the abduction process searches for explanations for natural language sentences. An example in Hobbs' paper is where "The Boston office called" can be resolved to "John, who works at the Boston office, called". This is a case of metonymy resolution. (Metonymy is the substitution for a thing with one of its attributes). From this example one can see that the abduction is a very powerful mechanism.
The second issue has to do with the way an intelligent agent should be conscious of its knowledge about language. We should distinguish two things:
The key feature is that the system should be aware of the internal mechanisms of language. Such a distinction is very subtle: the difference between encoding linguistic knowledge as ad hoc rules and encoding linguistic knowledge as a special kind of knowledge-about-the-world. See more explanations below.
[ Note: there may be inaccuracies in this example, that would differ significantly from the real implementation. ]
Consider a very simple sentence:
"John loves Mary"
The crucial thing is that we represent everything under a logic-based framework. First we represent the sentence as "raw data" (ignoring tenses to simplify matters):
lexeme_John(e1)
lexeme_loves(e2)
lexeme_Mary(e3)
follows(e2, e1)
follows(e3, e2)
where:
e1, e2, and e3 are the "words".
The e's are entities (logical constants).
follows() means a word follows another word.
Up to now, all we have is a sentence as raw text (without meanings).
Next step is to recognize the parts of speech (nouns, verbs, adjectives, etc). We use logical rules to do this.
An example logical rule is:
lexeme_John(X) -> parse_as(e, X) & noun(e)
which simply means that the word "John" is a noun. X is a variable. e is a new entity, which becomes e4 when the rule is applied.
We also use logical rules to parse syntax. We can do "VP <- verb + noun" with this rule (I have simplified things a bit here):
verb(X1) & noun(X2) & follows(X2, X1) -> parse_as(e, X1, X2) & VP(e)
which creates a new entity e5 which is a VP.
Assume that eventually we'll have a parse of the sentence S, e6. Notice that up to now, it's all syntax parsing.
Next we do semantic parsing, using a special kind of logical rules that couple syntax to semantics. For example:
lexeme_love(X1) -> concept(e) & means(e, concept_love)
which creates a new entity e7 with the meaning of "love".
The final result is something like:
concept_John(e10)
concept_love(e11)
concept_Mary(e12)
e11(e10,e12)
sentence(e6)
conveys(e6, e11)
[ Notice that in e11(e10,e12) we have used the "reification" trick. ]
You may ask, why go through all these troubles to do parsing in logic, when it can be done more efficiently using special programs? The answer is that this gives us much more flexibility. By learning logical rules, the intelligent agent can learn new languages like humans do. Flexibility is more important than raw power!
That said, it is entirely conceivable that we use a special-purpose NLP module in conjunction with this method.
The entire scheme is outlined in Hobbs' paper-in-progress [Hobbs in progress].
In Hobbs' words, to interpret a sentence means to:
It seems that all we need is a comprehensive knowledge representation scheme, and a set of rules to govern the abduction process. The abductive mechanism is powerful enough to handle things like co-reference resolution, interpretation of compound nominals, resolution of syntactic ambiguities and metonymy, and schema recognition.
Hobbs also discusses the integration of syntax, semantics, and pragmatics into a single logical framework. This is very helpful when we integrate the NLP module with the rest of the AI.
"Emergent grammar" is first proposed by [Hopper 1987]. In his words,
"The notion of Emergent Grammar is meant to suggest that structure, or regularity, comes out of discourse and is shaped by discourse as much as it shapes discourse in an on-going process. Grammar is hence not to be understood as a pre-requisite for discourse... Its forms are not fixed templates but are negotiable... Emergent Grammar points to a grammar which is not abstractly formulated and abstractly represented, but always anchored in the specific concrete form of an utterance."
We make this idea more concrete by formulating it in formal logic. Also we will follow the infant's way of learning language.
In the earliest stage of language learning, one learns the mapping of words to objects / simple concepts:
What I call "linguistic mapping" is simply a kind of association, special in that it maps a language element to a concept. Another way to think of the mapping is that "APPLE" means apple. Thus, it puts language in the same footing as giving someone a rose means "I love you" etc.
In the next stage we learn complex grammatical rules to construct longer sentences such as:
The linguistic correspondence is of the same nature as before, only that now we have a sentence and a complex idea. The grammatical rules tell us how to use the words like THIS, APPLE, IS, RED, etc.
In the traditional way of natural language processing, the parser takes a sentence and outputs a logical formula with the same meaning. This is known as a semantic parser (as opposed to syntactic parser). In a semantic parser the grammar is augmented with semantic rules of how to construct logic formulas. For example, in the sentence "John loves Mary", the verb phrase (VP) "loves Mary" would be augmented with the λ-expression "λx Loves(x,mary)", which, when applied to the variable john would yield the desired logic formula Loves(john,mary).
Such a complicated way of specifying the semantics is due to the fact that predicate logic has a very different structure from our natural language. An even more complicated example, from [Russell & Norvig 2003], is:
"Every agent smells a wumpus" means forall a, a is an agent => exists w, w is a wumpus ^ Smells(a,w)
As you can imagine the grammatical rules for handling these can be very awkward.
(If you don't understand this part, I suggest reading [Russell & Norvig 2003]'s chapter 22: "Communication". It will enable you to understand this page much better.)
Our approach is basically the traditional one with 2 additional measures:
I advocate not to use Frege-style quantifiers (ie "for all" and "exist"), instead use some "generalized quantifiers" such as "some", "a few", "many", "most", etc. This makes the translation of NL to logical form much easier.
For example, "All ravens are black" in predicate logic is kind of unnatural:
forall x, Raven(x) => Black(x)
With general quantifiers we can have something like:
Black( (all*raven) )
which follows the same pattern of compositionality as our natural language. This makes semantic parsing very straightforward because we can represent the noun phrase (NP) "all ravens" as (all * raven), whereas in predicate logic the quantifier "all" is inseparable from the formula.
I am still working on the knowledge representation scheme. Maybe I will adopt an existing language such as Cyc's. For more about this see knowledge representation.
The purpose of the flat representation is so that the intelligent agent can be made aware of meta-linguistic knowledge. For example, knowing that rose is a symbol for beautiful women, etc.
So we need rules of the form:
certain sentences means certain logical formulas
For example:
"John loves Mary" means Loves(john,mary)
And we require that such rules be in the intelligent agent's knowledgebase, ie be represented as facts themselves. Therefore it requires us to treat logic formulas as first-order objects in our new logic. So
Means("John loves Mary",Loves(john,mary))
is itself a formula in the new logic.
With these two measures, language learning would be an emergent process integral to the intelligent agent.
Natural languages often have multiple ways (surface structures) of saying the same things (deep structures). By using an internal representation, we can reduce such polymorphisms, by establishing a canonical form.
In the simplest case — single-word transliteration — the NLP takes a word and returns a concept corresponding to that word, for example the predicate Pineapple(). Concept are defined in Semantic Memory.
Usually, a complete sentence is needed before individual concepts can be mapped to words. Very often the context in which the sentence appears is also required.
The context is provided by whatever is currently stored in Working Memory (WM). An entire sentence may be stored in WM for parsing. Also, the context of the sentence (such as the meaning of previous sentences [as opposed to mere words]) can be stored in WM. Thus the NLP can make use of context to resolve linguistic ambiguities.
"Apple pie" is an idiom. Each idiom should have its own representation (eg the predicate ApplePie()) in the Natural Language Processor.
"Mushroom" maps to a concept; "pie" maps to another. But "mushroom pie" should map to a concept similar to "apple pie" which is idiomatic. Perhaps we can have a heuristic such as "if x is similar to y then the meaning of xz is should be inferred from the substitution of yz with x=y".
The same problem in reverse. It may be easier because there is no need to resolve ambiguities, but there is the different challenge of expressing things in the desired way.
Here we need a way of manually examining internal representations.
The user examines an internal representation that may or may not be labeled with words. For example, for the representation of pineapple, the user inverts the concept definition and see visual images of pineapples. For the representation of fighting, the user sees predicates such as Hit(), Kick(), Punch(), Slap(), in the concept definition.
Present several examples of an object / event, with proper word labels.
You may provide some instant feedback with this form:
[Hobbs, Stickel, Appelt, & Martin 1993] "Interpretation as Abduction", Artificial Intelligence, Vol. 63, Nos. 1-2, pp. 69-142. [ps] [pdf]
[Hobbs in progress] "Interpretation as Abduction", work in progress. [pdf]
[Hopper 1987] Emergent grammar. Berkeley Linguistics Society 13: 139-157.
[Jurafsky & Martin 2000] Speech and Language Processing: An Introduction to Natural Language Processing, Computational Linguistics and Speech Recognition, Prentice Hall
[Nugues 2006] An Introduction to Language Processing with Perl and Prolog: An Outline of Theories, Implementation, and Application with Special Consideration of English, French, and German, Springer
[Russell & Norvig 2003] Artificial Intelligence: A Modern Approach, Prentice Hall
[Sommers & Englebretsen 2000] An Invitation to Formal Reasoning — The Logic of Terms, Ashgate publishing, UK
| Home | Table of Contents |
19/Oct, 23/Aug/2006 (C) GIRG [ Notice about intellectual property on this page ]