The virtual guide is an agent in the Virtual Music Centre that can help users find their way in the building. It includes a multimodal Dutch dialogue system that accepts speech or text input as well as pointing gestures (mouse clicks) from the user. Output comes in the form of the virtual guide speaking and making gestures.
The dialogue system is mainly suited to be used in a virtual world with objects and agents that can be talked about or pointed at. It consists of several components including a natural language parser, a fusion agent that merges parsed text or speech input with pointing gestures, a dialogue act recogniser that acts on a parsed Dutch phrase or sentence using the dialogue history, a reference resolver based on salience factors that links noun phrases to objects in the virtual world, and an action stack that the dialogue manager fills by matching the user’s dialogue acts to action templates.
Click to play a video
