One of the [...] former masters had made a tick tock man, all levers and gearwheels and cranks and clockwork. Instead of a brain, it had a long tape punched with holes. Instead of a heart, it had a big spring. Provided everything in the kitchen was very carefully positioned, the thing could sweep the floor and make a passable cup of tea. If everything wasn't carefully positioned, or if the ticking, clicking thing hit an unexpected bump, then it'd strip the plaster off the walls and make a furious cup of cat. Then his master had conceived the idea of making the thing live, so that it could punch its own tapes and wind its own spring.

[Terry Pratchet, The thief of time, 2002]

COG and I (or what I've been doing at the MIT AI-Lab from 2001 to 2002)

The Problem

It is believed that one of the distinguishing skills of homo sapiens sapiens is that of learning from imitation while it is less clear whether other primates are capable of true imitation. Imitation encompasses a set of different competences such as recognizing other people's actions, recognizing the goal of a particular action and the objects and/or subjects involved. This project attempts to implement a similar set of abilities in a humanoid robot following a biologically motivated perspective.


Motivation and biological basis

Our inspiration comes from one of the most fascinating discovery of the neurophysiology of the last decade: that is mirror neurons. This is a class of neurons found in the monkey's frontal cortex (area F5). A particular mirror neuron is activated both when the monkey executes an action and when it observes the same action performed by somebody else: hence the name mirror. The importance of the discovery lies in the possibility to relate mirror neurons to gesture recognition, language, and learning by imitation. Grossly simplifying, it is believed that mirror neurons evolved from a phylogenetically older system used by the brain to control the interaction between hand and graspable objects. We suggest that a similar process, during ontogenesis, might also explain how the brain develops to endow a "mirror representation". During a first stage the understanding of object properties is connected to the growing motor repertoire (e.g. precision grip, full palm grasp, etc). This leads to a pragmatic description of objects in the spirit of the Gibsonian concept of affordances. Successively, in a second stage, the understanding is extended to include the observation of somebody else's actions: i.e. how a foreign manipulator acts onto a particular known object. Also, we hope that by building a biologically plausible model of this process we can advance the understanding of how similar processes are "implemented" in the brain.



Our approach is based on real manipulation where we expect the robot to interact freely with humans and objects, and to learn during continuous interaction with the environment. The experimental setup is the humanoid robot COG. We employed poking and prodding as a first and simpler step towards full-blown manipulation. The robot interacts with a small set of objects with different physical properties with respect to pushing. For example, objects can roll along a preferred direction and move a long way (think of a toy car) or simply slide along when pushed and come to a quick stop (think of a stuffed animal). In a first set of experiments we showed how the robot was able to learn (statistically) about this limited set of objects and poke them appropriately if they were presented again. Also, we demonstrated goal-directed mimicry behavior. In the latter, the robot poked an object so to imitate the "type of poking" previously executed by a human. These experiments were developed on top of a general attentional mechanism, tracking, and reaching skills.



Research Support

This work is funded by DARPA as part of the "Natural Tasking of Robots Based on Human Interaction Cues" project under contract number DABT 63-00-C-10102, and by the Nippon Telegraph and Telephone Corporation as part of the NTT/MIT Collaboration Agreement.