DEIRA is an agent framework that has been employed as the basis for
embodied conversational agents in the deomains of horse racing
commentary, robosoccer commentary and virtual storytelling. DEIRA was
originally started as a design project by Almer Tigelaar, Thijs Alofs,
François Knoppel and Danny Oude Bos and has had modularity at the core
of its design. In the initial domain of horse racing commentary it has
won both the Jury and Public Awards at the GALA competition at
IVA 2007, making use of SAPI 5.1 TTS engines and a flexible commentary
generation system. Since then, it has been generalized to support easy
implementation of agent systems for different domains.
The original challenge for which DEIRA was developed was the GALA
competition at IVA2007, for which an agent that reported on a virtual
horse race in real time needed to be created. Based on information
provided by horse race enthusiasts, a modular system was designed to
meet the information needs and to try to match their expectations for
such a reporter.
DEIRA uses an event driven modular design to transform the raw
incoming data to agent behavior. The input module for a domain detects
the interesting events and enters these into a queue that manages
these events as events keep coming in, giving priority to the more
important ones and removing invalidated events. In turn, the
processing modules then embellish the event with data that is
ultimately used by the output module to drive an external application
to show the intended behavior.
To produce engaging commentary, a powerful productive context-free
grammar is used, which for the horse racing domain generates over a
thousand different sentences. The specification of the grammar is done
in a simple text file, with conditional applicability of the rules,
which allows for specifying behavior for arbitrary emotional states of
the agent and for specific event parameters.
With the emotional state of the agent being central to the processing
and output modules, DEIRA maintains an emotional state over several
emotional dimensions, based on variable predispositions and the
emotional effects of the events generated by the input module.
Maintaining an emotional state allows the speech to be adapted in
speed, pitch, volume and any parameter the employed TTS engine
supports as well as selecting animations appropriate for the agent as
it comments on the events that are taking place.
After having proved its value within the domain of (virtual) horse
racing commentary DEIRA has been extended to incorporate the Virtuele
Voetbal Commentator, which was designed to provide commentary for
Robosoccer matches. With the VVC being a student project, in part
inspired by DEIRA, adaptation was straightforward and done within a
timeframe of 8 hours.
The Narrator part of the Virtual
Storyteller (VS) was linked to DEIRA in a similar fashion, with
DEIRA delivering the stories generated by the VS. The very different
domain of storytelling posed very different demands on how events were
dealt with. The ease with which these demands were met shows the
versatility of the system as a framework. In the future, the
facilities provided by the modular structure of DEIRA can be further
exploited to improve the delivery of the stories.
Besides the development of DEIRA for the Virtual Storyteller, the
framework is freely accessible and available under the LGPLv3 license.
Students of HMI are more than welcome to put DEIRA to their use in any
Embodied Conversational Agent application they wish to develop.
|
Original DEIRA horse race commentary:
This div will be replaced
DEIRA telling a story:
This div will be replaced
DEIRA robosoccer commentary:
This div will be replaced
Former HMI-members: |
