Exploring Visual and Multimodal Interaction in NCL Authoring

This paper introduces two innovative tools for enhancing interactive multimedia authoring using the Nested Context Language (NCL): (i) a visual extension that supports more traditional interactions with mouse and keyboard and (ii) a multimodal extension that incorporates gesture recognition and voice commands. These tools were implemented as Visual Studio Code extensions and aim to streamline the editing process, making it more intuitive and accessible. We present an evaluation of the usability and acceptance of both tools with developers in an experiment with three tasks for creating and manipulating spatial regions in hypermedia documents. By exploring the potential of multimodal interfaces, this work sets the stage for more efficient and user-friendly document editing.