Output multimodal interaction: the case of augmented surgery

Overview & Synopsis

Output multimodal interaction involves choice and combination of relevant interaction modalities to present information to the user. In this paper, we present a framework based on reusable software components for rapidly developing output multimodal interfaces by choosing and combining interaction modalities. Such an approach enables us to quickly explore several design alternatives as part of an iterative design process. Our approach is illustrated by examples from a computer-assisted surgery system that runs in a specific environment (i.e. an operating room) and so needs adapted multimodal interaction. Our approach supports the exploration of several output multimodal interaction design alternatives with the surgeons.