Adding Speech to Dialogues with a Council of Coaches

Currently in COUCH the user interacts with the coaches by selecting one of several predefined multiplele-choice responses on a tablet or computer interface. Although this is a reliable way to capture input from the user, it may not be ideal for COUCH its target user group of older adults. Spoken dialogues might improve the user experience of COUCH for older adults. In this project we adapted the COUCH system to support spoken interactions. The NLSpraak speech recognizer is used for understanding the older adult’s speech (in Dutch), while the Google cloud Text-to-Speech API is used for synthesizing the virtual coaches. In addition to incorporating ASR abd TTS, we investigated smart ways to organise the dialogue to facilitate adequate recognition in noisy and uncertain settings while keeping the conversation going. Finally, we also evaluated the user experience and the quality of dialogue progress in various settings and thereby the suitability of state of the art speech recognition for an application such as COUCH.