A Large Language Model-based Trajectory Prediction Algorithm for Industrial Accident Prevention

Trajectory prediction for industrial vehicles, such as forklifts, must account for complex industrial environments, including load capacity, variations of terrain, and confined spaces. Unlike trajectory prediction for pedestrians or personal vehicles, the trajectories of industrial vehicles are influenced by these physical factors, necessitating models that integrate such considerations for accurate predictions. This study proposes a volume-based trajectory prediction method by applying prompt engineering to the input of large language models (LLMs). Specifically, the study includes the size of the bounding box and terrain characteristics in the LLM input, providing physical and spatial constraints as prompts, to derive predictions that take into account the characteristics of industrial vehicles. The data required for this study was insufficient, so necessary videos were collected and processed using computer vision techniques to build the dataset. Experimental results demonstrate that, unlike previous studies, the proposed method achieves more reasonable trajectory predictions for both pedestrian and vehicle objects. In comparative experiments using our prepared dataset, the proposed approach outperformed baseline models, reducing the Average Displacement Error (ADE) by 0.051 meters. This study highlights the potential of LLMs to handle numerical data in trajectory prediction tasks and underscores the feasibility of multimodal applications combining computer vision and LLMs in industrial applications.