Reactive Virtual Trainer (RVT) is an Embodied Conversational Agent
capable of presenting physical exercises that are to be performed by
human, monitoring the user and providing feedback at different
ECA, parameterized animation, fitness, multimodal coordinationSome themes present in this work are :
This work is done in a Bachelor Referaat assignment by Frank Zijlstra.
The Virtual Trainer is likely to be used in a home entertainment
environment. Only one webcam will be available and lab conditions
should not be required for the analysis to work. To provide feedback
timely, the analysis should work in real time.
To find out how tempo affects motion dynamics, we analyze optical
motion capture recordings of subjects performing excercises.
Using a silhouette based approach, these requirements can be met. The
silhouette is extracted from a video recording. In this silhouette,
body parts are identified in real time.
The body part are matched to body parts in a reference silhouette to
provide feedback on how to correct the pose.
By playing the users preferred music during the exercise, exercising
could be made more enjoyable. We interpret the music to find the beats
so that the exercise, and thus the movement of the trainer, can be
aligned to the tempo of the music. For this we can borrow the beat
detection algorithm from our Virtual
Conductor, which implements methods described in the
publications of Anssi Klapuri. This algorithm will detect the tempo
and beat in the music played during the exercise.
Motion captured fitness exercises can be found at the HMI mocap database.
We make use of the multi-modal synchronization language
BML to specify the
metronome beats, speech and clapping motion.
Using our animation framework, we can define exercises as by
describing movement paths of limbs. Movements can be accentuated by
adapting amplitude or velocity profile.
Parameterized animation (1Mb).
|Z.M. Ruttkay, J. Zwiers, H. van Welbergen and D. Reidsma Towards a Reactive Virtual Trainer, in Proceedings of the 6th International Conference on Intelligent Virtual Agents, J. Gratch, M. Young, R.S. Aylett, D. Ballin and P. Olivier (eds), Lecture notes in Computer Science, volume 4133, Springer Verlag, Berlin, ISBN 978-3-540-37593-7, ISSN 0302-9743, pp. 292-303, 2006 [ BiBTeX ]  |
|A. Nijholt, D. Reidsma, Z.M. Ruttkay, H. van Welbergen and P. Bos Nonverbal and Bodily Interaction in Ambient Entertainment, in Proceedings workshop on Fundamentals of Verbal and Non-verbal Communication and the Biometrical Issue, A.M. Esposito, M. Bratanic, E. Keller and M. Marinaro (eds), NATO Security through Science Series, E: Human and Societal Dynamics, volume 18, IOS Press, Amsterdam, ISBN 978-1-58603-733-8, pp. 343-348, 2007 [ BiBTeX ]  [Official URL] |
|H.H. Vilhjálmsson, N. Cantelmo, J. Cassell, N.E. Chafai, M. Kipp, S. Kopp, M. Mancini, S.C. Marsella, A.N. Marshall, C. Pelachaud, Z.M. Ruttkay, K.R. Thorisson, H. van Welbergen and R.J. van der Werf The Behavior Markup Language: Recent Developments and Challenges, in Proceedings of the 7th International Conference on Intelligent Virtual Agents, C. Pelachaud, J.-C. Martin, E. Andre, G. Collet, K. Karpouzis and D. Pelé (eds), Lecture Notes in Computer Science, volume 4722, Springer, Berlin, ISBN 978-3-540-74996-7, ISSN 0302-9743, pp. 99-120, 2007 [ BiBTeX ]  |
|Z.M. Ruttkay and H. van Welbergen Elbows higher! Performing, observing and correcting exercises by a Virtual Trainer, in Proceedings of the Eight International Conference on Intelligent Virtual Agents, Lecture Notes in Artificial Intelligence, volume 5208, Springer Verlag, London, ISBN 978-3-540-85482-1, ISSN 0302-9743, pp. 409-416, 2008 [ BiBTeX ]  |
|H. van Welbergen Using motion capture data to generate and evaluate motion models for real-time computer animation, in Proceedings of Measuring Behavior 2008, 6th International Conference on Methods and Techniques in Behavioral Research, A.J. Spink, M.R. Ballintijn, N.D. Bogers, F. Grieco, L.W.S. Loijens, L.P.J.J. Noldus, G. Smit and P.H. Zimmerman (eds), Noldus Information Technology bv, Maastricht, ISBN 978-90-74821-81-0, ISSN A.J. Spink, M.R. Ballintijn, N.D, pp. 26-27, 2008 [ BiBTeX ]  |
- GATE [GATE: Games for Advanced Training and Entertainment]
- HUMAINE [Human-Machine Interaction Network on Emotion]