THEIA-XR: Interaction Modality Assignment (D4.2)

Deliverable 4.2 provides the interaction modality framework with criteria-based recommendations for information presentation. It is rather a methodology for the development of multimodal user interfaces, but at the same time, it offers the question of how modern Human-Machine Interfaces (HMI) can be improved. It discusses different interaction modalities and synthesizes the results into work materials guiding the decision-making process of appropriate information flow between the operator and system. In doing so, work packages 5 and 6 can benefit from the results. It includes a short analysis of general interaction models and modalities in literature. D4.2 evaluates their respective capabilities, advantages, disadvantages, and operator-specific requirements. The result is a methodology for a creative yet systematic ideation of multimodal interaction. For this purpose, the necessary working materials were developed. It consists of a template for the description of operating challenges, inspiring modality cards, two handbooks and solution templates for a comparable variant documentation. Moreover, the use of an interlinked process was described. With the attached detail cards, the focus is on promoting multimodal thinking in the field of HMI elements and their use for off-highway commercial vehicles and machines.