Construction machinery work support system using mixed reality

Although the number of fatal and injured accidents in the construction industry has decreased in recent years, they are still high compared to other industries, and ensuring safety at construction sites is an important issue. Construction sites are complex and dangerous working environments, and workers from different industries often have to work together. In addition, changes in work methods and procedures may occur due to design changes, bad weather, delays in work in other trades, and shortages of equipment and labor. In order to ensure safety and resume work, it is necessary to report the change in work method to the construction supervisor and adjust and discuss the changed work method. Meanwhile, the construction industry is also promoting the digitalization of work and the use of smart devices as DX (digital transformation). One of these is to provide rapid and accurate technical support even in the event of a wide-area disaster by grasping the local situation from a remote location without having to go directly to the disaster site. In addition, in order to improve work efficiency, attempts are being made to apply remote presence to work that previously required attendance. In this study, we developed a human-machine system that visually displays instructions from a site supervisor to workers using mixed reality (MR), that is, an MR work instruction presentation system, with the aim of quickly and accurately conveying complex instructions to workers while working with construction machinery, which has been difficult to convey while working. By using this system, arrows and construction drawings can be superimposed on the worker's field of vision, making it possible to easily and accurately convey detailed instructions such as the location to crush with construction machinery, the position to grab, and the transport route of the transported items. In this study, we built a construction machinery work support system using Microsoft's HoloLens and conducted subject experiments using an actual machine.As a method of presenting work instructions using MR for the transport and stacking work of blocks by construction machinery using this system, we proposed four methods: A) work instructions using only drawing processing, B) work instructions using drawing processing and text, C) work instructions using drawing processing and voice, and D) work instructions using animation.In the experiment, block transport and assembly work was performed using the four proposed work instruction presentation methods, and the optimal instruction presentation method was examined based on the work time, mental burden using NASA-TLX, and a questionnaire survey. As a result, it was confirmed that using this system to give work instructions shortens the time required to operate construction machinery and is also effective in reducing the mental burden on NASA-TLX. In addition, it was found that of the four proposed methods of presenting work instructions, the most effective methods were the method using drawing processing and voice, and the method using animation. These results are believed to be useful for the development of a human-machine system that visually displays instructions to workers.