Integrating Language and Cognition in Grounded Adaptive Agents

EOARD Grant # 053060

 


 

 

Background:

   In the future, machine agents will be able to communicate among themselves and with the flexibility of human language. For example, robots will be able to learn language and world understanding from direct interaction with humans. Cognitive systems research focuses on the development of natural and artificial information processing systems (e.g. internet agents, adaptive agents, robots) capable of perception, learning, decision making, communication and action. They are designed to assist humans in a variety of situations including everyday tasks, such as service/household robotics, and highly-specialized situations, such as in autonomous systems for defence.

   

   Recent research in linguistics cognitive systems (Cangelosi et al. 2005) has focused on the close integration of language and other cognitive capabilities (i.e. integration of communication with perception, categorization, action). This approach is based on the important process of "grounding" the agent's lexicon directly into its own internal representations. Agents learn to name entities, individual and states whilst they interact with the world and build sensorimotor representations of it. For example, Steels (2003) studied the emergence of shared languages in group of autonomous cognitive robotics that learn categories of object shapes and colours. Cangelosi and collaborators analysed the emergence of syntactic categories in lexicons supporting navigation (Cangelosi 2001) and object manipulation tasks (Marocco et al.2003; Hourdakis &Cangelosi 2005) in populations of simulated agents and robots.

  

The use of this grounded approach to the design of linguistic cognitive systems is vital for overcoming the known difficulties in intelligent agents whose linguistic abilities are purely based on abstracts symbolic representations. This is the case of search engines that only rely on text corpora and therefore cannot solve lexical ambiguities that require consideration of contextual and extra-linguistic knowledge. Grounded systems that have access to the cognitive and sensorimotor representations of words can, instead, succeed in solving these ambiguities. Equally important, is the reverse: learning abstract categories and situations, which are not directly observed in the world, can only be grounded in language and communications among agents.

   

   Current grounded agent and robotic approaches have their own limitations, in particular for the scaling up of the agent's lexicon since they can only use few tens of lexical entries (see Steels 2003) and can deal with a limited set of syntactic categories (e.g. nouns and verbs in Cangelosi 2001). This is mostly due to the use of computational intelligent techniques (e.g. neural networks, rule systems) subject to combinatorial complexity (CC). The issue of scaling up and CC in cognitive systems has been recently addressed by Perlovsky (2001). In linguistic systems, CC refers to the hierarchical combinations of bottom-up perceptual and linguistic signals and top-down internal concept-models of objects, scenes and other complex meanings. Perlovsky proposed the Modelling Field Theory (MFT) as a new method for overcoming the exponential growth of CC in computational intelligent techniques currently used in cognitive system design. MFT uses fuzzy dynamic logic to avoid CC and computes similarity measures between internal concept-models and the perceptual and linguistic signals. More recently, Perlovsky (2004) has suggested the use of MFT specifically to model linguistic abilities. By using concept-models with multiple senrorimotor modalities, a MFT system can integrate language-specific signal with other internal cognitive representations.

   

   Perlovsky's proposal to apply MFT in the language domain is highly consistent with the grounded approach to language modelling discussed above. That is, both accounts are based on the strict integration of language cognition. This permits the design cognitive systems that are trully able to "understand" the meaning of words being used be autonomously linking the linguistic signals to the internal concept-models of the word constructed during the sensorimotor interaction with the environment. The combination of MFT systems with the grounded agent simulations will permit the overcoming of CC problems currently faced in grounded agent models and scale up the lexicons in terms of high number of lexical entries and syntactic categories.

 

    The potential impact of this research for the development of intelligent systems is great, also in the field of defence interests. Cognitive systems are essential for integrated multi-platform systems capable of sensing and communicating. In future systems, robots and autonomous agents will be able to learn language and world understanding from humans. In the area of internet/text search engines the capability of truly "understanding" the language query and corpora being used will permit the design of more efficient search and data-mining systems. In the area of intelligent agents for defence, the design of cognitive systems able to develop autonomously their own grounded lexicons will be beneficial in collaborative and distributed tasks. (e.g. multi-agent exploration and navigation in unknown terrains, etc.)