Rail vehicle assembly job shop scheduling via hierarchical multi-agent deep reinforcement learning

The dynamic and distributed nature of rail vehicle assembly is characterised by large components across multiple production lines. Traditional batch-dispatching approaches lack the responsiveness needed for efficient line reconfiguration. Although the Dynamic Distributed Flexible Job-Shop Scheduling Problem (DDFJSP) shares structural similarities with rail vehicle assembly and offers a relevant theoretical model, existing solutions often rely on dispatching rules or batch-order processing, which are inadequate for highly customised assembly environments. To address these limitations, this paper introduces a Hierarchical Multi-Agent Deep Reinforcement Learning (HMADRL) framework for near-real-time optimisation. Our approach enables immediate order dispatch upon arrival by formalising layered Markov Decision Processes that simultaneously optimise job assignment across interconnected production lines through flexible manufacturing units and intra-workstation operation sequencing. The framework represents the assembly as a heterogeneous graph, integrating structural and operational attributes, and employs a hierarchical graph attention mechanism to capture both operation-level correlations and job-level dependencies. Computational experiments show that HMADRL achieves the lowest average makespan across all test classes. The average makespans of the best-performing non-HMADRL baselines are 1.25–7.24% higher, while the metaheuristic requires substantially greater cumulative online computation. The framework exhibits strong generalisation across diverse processing-time distributions, confirming its adaptability to complex manufacturing domains beyond rail assembly.