Artificial Intelligence for Human-Robot Interaction: Papers from the 2014 AAAI Fall Symposium
Make Way for the Robot Animators!
Bringing Professional Animators and AI Programmers
Together in the Quest for the Illusion of Life in Robotic Characters
Tiago Ribeiro and Ana Paiva
INESC-ID & Instituto Superior Técnico, Universidade de Lisboa, Portugal
We are looking at new ways of building algorithms for synthesizing and rendering animation in social robots that can
keep them as interactive as necessary, while still following
on principles and practices used by professional animators.
We will be studying the animation process side by side with
professional animators in order to understand how these algorithms and tools can be used by animators to achieve animation capable of correctly adapting to the environment and
the artificial intelligence that controls the robot.
Figure 1: Two example scenarios featuring a touch-based
multimedia application, sensors, and different robots.
Robotic characters are becoming widespread as useful
tools in assistive, entertainment, tutoring and manufacturing
applications. This is leading to the need of having better animation built into these robotic systems. Traditional character animation relies on having animators design animations
which are then faithfully played back on the character.
Now we pose the question: how can we have the animation process integrated with the artificial intelligence agent
(AI) that drives the robot? We want the animation to be
adaptable to both the agent’s internal state and its external
environment, while keeping the quality of the animation in
line with what an animator expects. The main question here
is not just whether or not the users will like the robot or find
it believable; instead, we want to have animators collaborating with the programmers in a way that the robot’s animation
during interaction is satisfying to the actual animators.
On this issue of breaking out the traditional animation
process into one that can be integrated with an AI, we
highlight two different types of animation:
Pre-designed character animation is when artists design
an animation for a specific character topology using professional animation software, such that the animation can then
be played back in a character with that same topology;
Procedural character animation is when a software
algorithm is able to synthesize the character’s animation
in real-time, by having some higher-level control software
tweaking the parameters of the algorithm.
The purpose of our work is to merge both approaches by:
1. Finding out how to build character animation algorithms
that can be tweaked by the AI that controls the character during interaction with users and the external physical
2. Having these algorithms make use of or blend with predesigned animation;
3. Bringing animation artists into the process of achieving
the previous points.
Our belief is that robot animation should be supported by
principles and practices of professional character animation.
Motivation and Goals
Animating an interactive robot is quite different from animating a virtual character. The biggest differences we point
out from our experience are the fact that the robot shares
the same physical space that humans do, and as such, follow on the same laws of physics. While animating a robot,
animators must take into account gravity, inertia, and the
actual torques that the robot’s motors can achieve. Because
they are physical and can interact with users through various
types of interfaces and sensors, robots also seem to lead to a
much higher expectation from the users, and appeal more to
some interactive non-verbal behaviours like gazing, rapport
or proxemics than a virtual character would.
The influential works by Bates (Bates 1994) and Reily
(Reilly 1996) were some of the first to seek inspiration from
the arts, in particular from the well known ”The Illusion
c 2014, Association for the Advancement of Artificial
Copyright Intelligence (www.aaai.org). All rights reserved.
of Life”, which describes over 60 years of experience in
creating characters at the Walt Disney Animation Studios
(Thomas and Johnston 1995). More recently, van Breemen
also pointed user-interface robots to have the same problem of early day’s animations: they miss the illusion of life’
(Breemen 2004). He defined robot animation as ’the process
of computing how the robot should act such that it is believable and interactive’. When pioneering 3D character animation for Pixar Animation Studios, Lasseter argued that the
traditional principles of animation have a similar meaning
across different animation medium (Lasseter 1987).
Several authors have since then looked into developing
frameworks for expressive behaviour in interactive characters. However, most have approached the synthesis of animation in specific modalities like gesturing, and animation
of humanoid characters (Cassell and Bickmore 2001; Badler
et al. 2002; Thiebaux et al. 2008). Many authors follow
the SAIBA/BML framework (Kopp, Krenn, and Marsella
2006), which is currently very popular in the Intelligent Virtual Agents (IVA) community, but again, authors have used
it mainly oriented at characters or robots that are humanoids
(Niewiadomski et al. 2011; Heloir and Kipp 2009).
In previous work we have shown an EMYS robot continuously interacting with both users and the environment in
order to provide a more lifelike experience (Ribeiro, Vala,
and Paiva 2012; Pereira, Prada, and Paiva 2012). Our latest
work on IVAs demonstrates the use of Thalamus to develop
component-based agents that separate and reuse high-level
intention generation modules and behaviour planning modules in different applications and embodiments, based in the
SAIBA and BML standards (Ribeiro et al. 2014a). In particular, we have been developing the Skene module which
provides high-level semi-automated gazing and expressive
behaviour based on utterances annotated with expressive instructions (Ribeiro et al. 2014b).
In the field of robot animation, Van Breemen’s work was
one of the first to automate one of Disney’s principles of
animation (Slow-In/Slow-Out) (Breemen 2004). Other authors, including ourselves, have also attempted to understand how the principles of animation can be used in robot
animation (Shusong and Jiefeng 2008; Takayama, Dooley,
and Ju 2011; Gielniak and Thomaz 2012; Ribeiro and Paiva
2012). Other fields have already succeeded in creating technological tools for artists to use. The Unreal Development
Kit1 enables artists and game programmers to work side-byside; tools like Pure Data2 give musicians and performers the
ability to program their own instruments and performances;
even in the field of animation, Houdini3 is used in mainstream movies to generate assets and special effects.
More recently we presented Nutty Tracks, an animation
engine that allows to animate any robotic character using
the animation tools commonly used by animators (Ribeiro,
Paiva, and Dooley 2013). Nutty supports both the design and
blending of pre-designed animation with procedural animation by specifying parameters in real-time. It brings anima-
Figure 2: The initial model for expressive robotic characters
we are building on. (Ribeiro et al. 2014b)
tion data to a symbolic level, in order for AI systems to be
able to control such parameters during an interaction.
Current generic robot animation and interaction systems require deep mathematical and programming skills, which
makes it difficult for artists to collaborate with the programmers. This problem poses to us as the challenge of bringing
these two worlds together. The connection will surely lie on
designing an animation system that can connect with an AI
and learn with the animator’s experience while they define
and tweak how such AI generates, selects and blends the
Our challenge requires some studies involving professional animators in order to understand which are their requirements for using an animation system as we propose.
An initial study should hint us on how their creative process works, and what type of tools and techniques they
would expect to use in different situations. From there we
will be developing algorithms and tools in Nutty Tracks
that can meet the animator’s needs and learn from their
experience Next we will develop an interactive scenario
with their collaboration, following an iterative design approach (Meerbeek, Saerbeck, and Bartneck 2009) based
on tools like Thalamus and Skene (Ribeiro et al. 2014a;
2014b). An initial model of how the different components
of such scenario can integrate is illustrated in Figure 2. This
model represents the current state of the architecture we are
using in different projects to develop and drive robotic IVAs.
The Nutty Tracks component however, is still bare of learning mechanisms and proper artist-oriented controllers and
The evaluation of our work will both consider how users
respond to that scenario with and without the new animation
technology, and also how the animators feel about the final
result. We expect our work to lead to a new field in robotics,
which we call Expressive Robotics. This field will, in turn,
lead to new techniques in animation and also new standards
and trends in human-robot interaction and robotic systems
Shusong, X., and Jiefeng, H. 2008. Robot behavior expression by illusion of life. 2008 IEEE Conference on Robotics,
Automation and Mechatronics 994–998.
Takayama, L.; Dooley, D.; and Ju, W. 2011. Expressing Thought : Improving Robot Readability with Animation
Thiebaux, M.; Rey, M.; Marshall, A. N.; Marsella, S.; and
Kallmann, M. 2008. SmartBody : Behavior Realization
for Embodied Conversational Agents. Information Sciences
Thomas, F., and Johnston, O. 1995. The illusion of life:
Disney animation. Hyperion.
Badler, N.; Allbeck, J.; Zhao, L.; and Byun, M. 2002. Representing and Parameterizing Agent Behaviors. Analysis.
Bates, J. 1994. The Role of Emotion in Believable Agents
Joseph Bates. Emotion.
Breemen, A. V. 2004. Bringing robots to life: Applying
principles of animation to robots. CHI2004 4:1–5.
Cassell, J., and Bickmore, T. 2001. BEAT : the Behavior Expression Animation Toolkit. Knowledge Creation Diffusion
Gielniak, M. J., and Thomaz, A. L. 2012. Enhancing interaction through exaggerated motion synthesis. Proceedings
of the seventh annual ACM/IEEE international conference
on Human-Robot Interaction - HRI ’12 375.
Heloir, A., and Kipp, M. 2009. EMBR: A realtime animation engine for interactive embodied agents. 2009 3rd International Conference on Affective Computing and Intelligent
Interaction and Workshops 1:1–2.
Kopp, S.; Krenn, B.; and Marsella, S. 2006. Towards a
common framework for multimodal generation: The behavior markup language. In Intelligent Virtual Agents, 205–217.
Lasseter, J. 1987. Principles of Traditional Animation Applied to 3D Computer Animation. 21(4):35–44.
Meerbeek, B.; Saerbeck, M.; and Bartneck, C. 2009. Iterative design process for robots with personality. In Symposium on New Frontiers in Human-Robot Interaction, 94–
Niewiadomski, R.; Obaid, M.; Bevacqua, E.; Looser, J.; Le,
Q. A.; and Pelachaud, C. 2011. Cross-media agent platform.
Pereira, A.; Prada, R.; and Paiva, A. 2012. Socially present
board game opponents. In Advances in Computer Entertainment, 101–116.
Reilly, W. S. N. 1996. Believable Social and Emotional
Agents. Ph.D. Dissertation, Carnegie Mellon University.
Ribeiro, T., and Paiva, A. 2012. The Illusion of Robotic
Life Principles and Practices of Animation for Robots. In
Human-Robot Interaction, number 1937, 383–390.
Ribeiro, T.; Di Tullio, E.; Corrigan, L. J.; Jones, A.; Papadopoulos, F.; Aylett, R.; Castellano, G.; and Paiva, A.
2014a. Developing Interactive Embodied Characters using
the Thalamus Framework: A Collaborative Approach. In
14th International Conference on Intelligent Virtual Agents.
Ribeiro, T.; Pereira, A.; Di Tullio, E.; Alves-Oliveira, P.; and
Paiva, A. 2014b. From Thalamus to Skene: High-level behaviour planning and managing for mixed-reality characters.
In Intelligent Virtual Agents - Workshop on Architectures
and Standards for IVAs, (in press).
Ribeiro, T.; Paiva, A.; and Dooley, D. 2013. Nutty tracks:
symbolic animation pipeline for expressive robotics. ACM
SIGGRAPH 2013 Posters 4503.
Ribeiro, T.; Vala, M.; and Paiva, A. 2012. Thalamus:
Closing the mind-body loop in interactive embodied characters. In 12nd International Conference on Intelligent Virtual
This work was partially supported by the European Commission (EC) and was funded by the EU FP7 ICT-317923
project EMOTE and partially supported by national funds
through FCT - Fundação para a Ciência e a Tecnologia, under the project PEst-OE/EEI/LA0021/2013.