|
Project Name: Augmented Multi-party Interaction
Abbreviation: AMI Start date:
January 1, 2004 End date:
December 31, 2006 Project Description:
"AMI is concerned with new multimodal technologies to support human
interaction,
in the context of smart meeting rooms and remote meeting assistants.
The project
aims to enhance the value of multimodal meeting recordings and to make
human
interaction more effective in real time. These goals will be achieved
by developing
new tools for computer-supported cooperative work and by designing new
ways to
search and browse meetings as part of an integrated multimodal group
communication,
captured from a wide range of devices."
[See also the
official web page where this text came from].
The AMI project is about multi modal, multi party interaction. IDIAP
(Martigny, Switzerland), TNO and the University of Edinburgh host
special meeting
rooms that contain an extensive setup of microphones and camera's. In
these meeting
rooms the interactions between people during meetings are recorded.
These recordings
are annotated for many modalities. Examples of these
annotations are speech
transcription, emotion, gestures, dialogue acts and posture. This
information is used
for (at least) two purposes: as training material for recognition
algorithms and as
evidence on the basis of which theoretical models of human multi party
interaction
can be developed.
The AMI project is the context for many different activities of the
HMI research group. The following list
gives a few examples of ongoing work.
- The Virtual Meeting Room
is a 3D virtual environment displaying the AMI
meeting room with
avatars as meeting participants. The most obvious use of this
environment is to literally replay (fragments of) the recorded
meetings. More interesting though are applications
such as simple validation of interaction models and/or recognition
results ("does the
predicted behaviour look sensible when displayed in the VMR?"), the
ability to restructure
meetings (e.g. displaying all arguments around a certain decision in
one coherent
discussion, smoothing over the transitions between the original
fragments of the meeting).
- By Gaze Experiment another application of the VMR is
demonstrated.
Machine learning
algorithms are used to predict current speaker from gaze directions or
head orientations.
The results are compared to the performance of humans who get exactly
the same input as the
machine learning algorithms. To this end human judges are presented
with a screenshot of a
Virtual Meeting Room setting that shows four neutral avatars with
correct head orientation.
For each screenshot they have to say who they think is the current
speaker. In this setup,
all information that is not available to the machine learning
algorithms, such as facial
expressions, is suppressed for the human judges as well.
- Annotation tools:
HMI has also been involved in the development of better tools for
manual annotation of
aspects such as dialogue acts, named entities, gaze targets or
gestures.
- Research into Addressing:
In the context of dialogue act recognition work is being done on
addressing behaviour. How
do people indicate at whom they address a certain utterance? Can the
addressee be
automatically detected?
- Pose and Gesture Detection
Within HMI a computer vision platform has been developed. One aspect
of this platform
involves 3D pose recognition from single camera images. The output of
this pose recognition
is now used for the development of gesture recognition in meeting
recordings.
For more information about the AMI project, follow one of the links
below or contact one of the HMI researchers mentioned on the next
column.
|
The following HMI-member(s) is/are coordinator of this Project
Dirk Heylen
Here you can find the publications
The following ShowCases are associated with this Project
|
