Our robot

 IBL: Instruction-based Learning for Mobile Robots

A project funded by the UK Engineering and Physical Sciences Research Council jointly managed by:

  • Dr. Guido Bugmann at the Centre for Interactive Intelligent Systems of the University of Plymouth (GR/M90023).
  • Dr. Ewan Klein the Institute for Communicating and Collaborative Systems (ICCS) of the University of Edinburgh (GR/M90160).
  • With the kind support of Nuance .

      Summary Members Publications Video Clips Corpus Software Links  


    This project explores a still-untapped method of knowledge acquisition and learning by intelligent systems: the acquisition of knowledge from Natural Language (NL) instruction. This is very effective in human learning and will be essential for adapting future intelligent systems to the needs of naive users. The aim of the project is to investigate real-world Instruction Based-Learning (IBL) in a generic route instruction task.  Users will engage in a dialogue with a mobile robot equipped with artificial vision, in order to teach it how to navigate a simplified maze-like environment. This experimental set-up will limit perceptual and control problems and also reduce the complexity of NL processing. The research will focus on the problem of how NL instructions can be used by an intelligent embodied agent to build a hierarchy of complex functions based on a limited set of low-level perceptual, motor and cognitive functions. We will investigate how the internal representations required for robot sensing and  navigation can support a usable speech-based interface. Given the use of artificial vision and voice input, such a system can contribute to assisting visually impaired people and wheelchair users.

    Back to top


    Dr. Johan Bos
    Dr. Guido Bugmann
    Dr. Ewan Klein
    Theocharis Kyriacou
    Dr. Stanislao Lauria

    Dr. Kenny Coventry (advisor)

    Back to top


    Bugmann G., Lauria S., Kyriacou T., Klein E., Bos J. and Coventry K. (2001)  " Using Verbal Instruction for Route Learning: Instruction Analysis ", Proc. TIMR 01 – Towards Intelligent Mobile Robots, Manchester 2001. Technical Report Series, Department of Computer Science, Manchester University, ISSN 1361 – 6161. Report number UMC-01-4-1 .

    Stanislao Lauria, Guido Bugmann , Theocharis Kyriacou, Ewan Klein (2001) " Instruction Based Learning: how to instruct a personal robot to find HAL. ", Proc. 9th European Workshop on Learning Robots, EWLR-9, Sept 8th-9th 2001,  Prague, Czech Republic, pp. 15-24.

    Stanislao Lauria, Guido Bugmann , Theocharis Kyriacou, Johan Bos, Ewan Klein (2001)  "Personal Robot Training via Natural-Language Instructions" (unedited preprint PDF 727 KB), IEEE Intelligent Systems, 16:3, pp. 38-45.

    Stanislao Lauria, Guido Bugmann , Theocharis Kyriacou, Ewan Klein (2002)   "Mobile Robot Programming Using Natural Language" (PDF, 936 KB)   Robotics and Autonomous Systems, 38 (3-4): 171-181 (ISSN 0921-8890)

    Guido Bugmann, Stanislao Lauria, Theocharis Kyriacou,Johan Bos, Ewan Klein (2002)  "Instruction-Based Learning for Mobile Robots" (PDF, 292 KB)   Poster Proceedings of ACDM'2002 (Parmee I. & Wyatt D.I. eds), University of Exeter, pp. 27-29.

    Theocharis Kyriacou, Guido Bugmann, Stanislao Lauria (2002)  "Vision-Based Urban Navigation Procedures for Verbally Instructed Robots" (PDF, 892K)   Proceedings of the 2002 IEEE/RSJ Intl. Conference on Intelligent Robots and Systems (IROS'02) EPFL, Lausanne, Switzerland • October 2002.  pp.1326-1331. (ISBN 0-7803-7545-9)

    S. Lauria T. Kyriacou G. Bugmann, J. Bos, E. Klein " Converting Natural Language Route Instructions into Robot-Executable Procedures (PDF 186K) " Proceedings of the 2002 IEEE Int. Workshop on Robot and Human Interactive Communication (Roman'02), Berlin, Germany, pp. 223-228. (ISBN 0-7803-7545-9)

    Mareile Hillevi Knees (2002) "Designing an Anaphora Resolution Algorithm for Route Instructions", M.Sc. in Cognitive Science and Natural Language. Division of Informatics. University of Edinburgh.

    Bugmann G. (2003) "Challenges in Verbal Instruction of Domestic Robots (PDF 201K)" Proceeding of the ASER'03, the 1st International Workshop on Advances in Service Robotics. Bardolino, Italy, March 13-15, 2003, pp. 112-116. (ISBN 3-8167-6268-9)

    Bugmann G. (2003) Final report to the EPSRC (IGR GR/M90023) (PDF 43K)

    Bugmann G., Klein E., Lauria S. and Kyriacou T. (2004) "Corpus-Based Robotics: A Route Instruction Example" (PDF 183KB)
    in Proceedings of IAS-8, 10-13 March 2004, Amsterdam, pp. 96-103. (ISBN 1 58603 4146)

    Kyriacou T., Bugmann G. and E., Lauria S. (2004) "Vision-Based Urban Navigation Procedures for Verbally Instructed Robots" (PDF 412KB) To appear in Robotics and Autonomous Systems

    Back to top

    Video Clips: (If you media player has problems with slow connections, download the clip before playing it)

    Single command:
    The user asks the robot to take the second right.
    example.mpg (15MB).

    Complete route:
    The robot follows the instructions to go from Boots to the Museum.
    U13_GA_CM_short1.mpg (3.5MB) 
    (phases when the robot is static, during image processing, are removed to shorten the movie)

    Weird World Exhibition, Plymouth City Museum, March 2003
    - In a quite noisy environment:
    IBLSchool.AVI (12MB),
    - An important visitor
    IBLSainsbury.AVI (7MB)

    Back to top


    See the README1 file that is part of the distribution.

    Back to top


    All components of the IBL software are freely available for research purpose. 
    Please contact Dr Guido Bugmann for access details

    Back to top

    Links to related projects:

    VITRA: Visual Translator (Full documentation of the project completed in 1996).
    SPACE: Spatial Cognition Engine    at the German Research Center for Artificial Intelligence (DFKI)
    Situated Artificial Communicators" (SFB 360) (NL instruction of an assembly robot) at the Technische Fakultšt Universitšt Bielefeld
    Spatial Language Group Plymouth
    SPIDER: Open working group dealing with the control of robots by natural language (in German)
    Hermes Project (Human sized German Robot)
    CARL project in Portugal
    WITAS project in Sweden
    Morpha German initiative in the area of man-machine interaction.
    Models for human interaction with mobile service robots a project of the Stockholm HMI Centre.
    Juno Japanese Dialogue System for an Office Robot used on Jijo-2 Office Robot
    BEELINE system - An Australien PhD 1996
    BIRON - Bielefeld Robot Companion
    robot,  Nara, Japan
    CODY Project Bielefeld
    GRAVIS projet Bielefeld

    EURON 2003 Summer School on Multimodal Human Robot Interaction

    Back to top

    This page has been accessed  times since 3 November 1999.