Virtual Kung Fu Practice

Title:Virtual Kung Fu Practice
Institute:TNO ICT
Place:Anna van Buerenplein 1, 2595 DA Den Haag The Netherlands
Type:final project
Start date:1 maart 2017
End date:not present
HMI ContactDirk HeylenJan van Erp


In a previous cycle of the virtual reality industry (i.e. before Oculus / HTV Vive), virtual reality has been used extensively for training purposes (e.g. military), and has been investigated before for use in training physical skills actions, such as e.g. in Tai Chi [1], [3]. Also, using virtual reality to assist in Martial Arts training in a more game-like manner is also tried out [2]. Also, martial arts seems well suited for current VR headsets, e.g. in a boxing game like ‘The Thrill of the Fight - VR Boxing’.

In Martial Arts, practitioners often exercise certains drills without an actual opponent, e.g. called the ‘forms’ in Wing Chun Kung Fu and ‘kata’ in Karate. Purpose of these exercises is built up muscle memory of the movements, and to help perform the various movements correctly by e.g. using a mirror or by having the teacher correct the form. Still, executing the form correctly may be difficult, as there is no opponent to block or hit. It is often unclear which movements to use to counter which attacks, and what counterattacks to use in what situation.

For real practice, martial arts requires an opponent. This sometimes limits the opportunity to train to the practice hours at the local gym / dojo. Also, practice with physical contact can be painfull, specially for beginners. For practice at home, wooden dummies are used, allowing the practitioner to perform various movements with physical feedback. Still, wooden dummies are quite expensive and take up considerable space, and are thus mostly used by the more serious practitioners.

Virtual reality may offer a training environment that bridges the gap between the various current practice methods like training with an actual opponent, doing the solo forms or training on a wooden dummy. A main research question is thus: can Virtual Reality be used to offer an enhanced training method for martial arts practitioners?
This assignment consists of setting up a virtual reality environment and testing the setup with test subjects. Access to martial artists is available through a local Wing Chun school, with a high-level trainer and various students willing to participate both in setting up the experiment and participating as test subjects.

For the technical setup, a video-based system is envisioned. Off-line recordings of opponents are foreseen, similar to the setup in [3] but much more easy to accomplish using current-day capture equipment. Possible challenge here is dealing with the closeness of the opponent, as objects close by are often causing eye convergence/accomodation problems in virtual reality. Another challenge is offering a self view. HTC Vive offers built-in virtual hands, which may be sufficient for the testing. An alternative may be using a HMD-mounted camera to offer a self-view.

Some state of the art
[1] is a paper on using a virtual reality environment to help in training Tai Chi moves. The paper analyses the effects of seeing the teacher in various ways, e.g. super-imposed on your body or in four directions around you. The paper does not find any positive effect of using VR compared to regular training, but mentions this might be because of the technical limitations of the setup.

[3] also focusses on Tai Chi, is actually using [1] as main state of the art. [3] shows positive effects of using virtual reality, but still has its limitations. Quality of the video in VR and frame rate in VR are stated as significant shortcomings of the technology, 2 areas that have significantly improved in the past 10 years.

[2] is more about a game, playing while exercising, and less about actually learning and training.

[1] Chua, Philo Tan, et al. "Training for physical tasks in virtual environments: Tai Chi." Virtual Reality, 2003. Proceedings. IEEE. IEEE, 2003.

[2] Hämäläinen, Perttu, et al. "Martial arts in artificial reality." Proceedings of the SIGCHI conference on Human factors in computing systems. ACM, 2005.

[3] Bailenson, Jeremy, et al. "The effect of interactivity on learning physical actions in virtual reality." Media Psychology 11.3 (2008): 354-376.