Intelligent Multimodal Human-Machine Collaboration System for Safety and Security in the Cockpit

Failures or interruptions in avionics systems, in which the flight remains controllable, negatively affect the quality of piloting techniques. To increase the efficiency of the pilot-aircraft interface, the principles of multimodal collaboration are used in the Intelligent Flight Control Systems, i.e., various additional contacts for control: visual contact, voice, facial expressions, body, arm, head movements, gestures, gaze, etc. The article provides an algorithm for quantitative and qualitative evaluation of avionics systems to maintain flight and navigation parameters within acceptable limits based on efficiency, significance, reliability, accuracy, and availability criteria. The relationship between the levels of a person's interaction with the environment (according to the psychoanalytic model such as intuition, ego, will-mind, reason, and sensory perceptions) and the character of actions of information display and control system is studied. To solve the identified problems and ensure safety and security for the pilot through the multimodal interaction human-aircraft, the design and principle of functioning of a new intuitive information display and control system is developed. For realization of the system using of Artificial Intelligence technologies (weak Artificial Intelligence and Augmented Intelligence) is proposed. An example of collaborative decision-making (CDM) based on the Expert Judgment Method in the situation "Only one source of power remains in the aircraft" is presented.