Integrated assembly, measurement, and adjustment method of reconfigurable flexible fixture for aircraft panels based on augmented reality and human-computer interaction

Owing to the characteristics of reconfigurable flexible fixtures (RFFs) for aircraft panels, the automation of their assembly is limited by technology and cost. As a result, manual assembly remains the predominant method. During the manual assembly, workers are affected by several challenges, such as difficulty in understanding the process documents and drawings, distinguishing the assembly positions of the similar components, inefficient data transmission, and determining the adjustment direction of contour board locators (CBLs). These issues arise because of inadequate digital assistance to workers. This paper proposes an integrated assembly, measurement, and adjustment (AMA) method for RFFs based on augmented reality (AR) and human–computer interaction (HCI) to assist workers. First, an information model based on core process elements for the assembly process is constructed. This model clarifies the correlations among multi-source data. Then, measured data is transformed into six-dimensional (6D) parameters to assist in the adjustment of CBLs. Based on the model and 6D parameters, the multiple visual assembly guidance is established by AR virtual-reality fusion technology. Subsequently, the HCI technology is introduced, to adaptively provide guidance via the hand-free head pointer. Workers use the AR head-mounted devices (HMDs) as a medium to interact with the laser tracker, which enables to quickly and accurately obtain the measurement data. Finally, AR and HCI technology are combined to establish an integrated process of AMA of RFFs. This method significantly improves the collaboration between workers and information during the assembly process of RFF. Field-assembly validation demonstrates that, compared with conventional methods, the proposed method achieves a positioning accuracy of ± 0.12 mm and enhances assembly efficiency by 32.87 %. • Based on AR and HCI, a method for assembling reconfigurable fixtures is proposed to enhance worker-information collaboration. • Constructed an information model based on core process elements, providing a comprehensive relationships among elements. • Develop a six-dimensional parameter-based adjustment method for the large-size high-accuracy contour board locators. • Adaptive multimodal visual AR assembly guidance is realized using the head pointer and virtual–reality fusion technology. • With AR head-mounted devices, workers can interact remotely with measurement devices to obtain data quickly and accurately.