Comprehensive performance–reliability analysis of ROS-based local planners

In the classical ROS 1 navigation stack, local planners compute velocity commands by using global-plan guidance together with local costmap information, enabling mobile robots to follow planned routes while responding to obstacles, kinematic constraints, and local environmental changes. This study presents a multidimensional performance–reliability analysis of five ROS-based local planners, namely Dynamic Window Approach (DWA), Timed Elastic Band (TEB), Trajectory Rollout, Neo Local Planner, and Hybrid Local Planner. A full-factorial experimental design was conducted with two parameter configurations, four environment categories (empty, static, dynamic, and mixed), and a fixed start–goal pose set. In total, 4000 navigation trials were performed utilizing move_base framework while keeping the global planner, costmaps, robot model, sensors, and simulation infrastructure constant. The evaluation combines performance-oriented indicators, including navigation time, deviation time, executed path length, and average speed, with reliability-related indicators, including static collision rate, dynamic collision rate, local planner failure rate, timeout rate, and successful attempt rate (SAR). In this framework, SAR is used as the empirical task-completion reliability indicator, while the remaining reliability-related indicators characterize different failure modes and safety-related outcomes. The metrics are first normalized according to their desirable directions, and the resulting values are visualized using parallel coordinates plots for joint comparison. The results indicate that no local planner is universally superior across all criteria. Based on the overall SAR across all configuration–environment combinations, Hybrid Local Planner achieved the highest empirical task-completion reliability (69.4%), followed by DWA (55.8%), Trajectory Rollout (38.8%), Neo Local Planner (34.0%), and TEB (25.0%). Although the SAR results provide a compact summary of task-completion reliability, the experimental findings show that local planner selection should consider not only task-completion success, but also failure modes, safety, motion continuity, efficiency, and path-tracking behavior. The proposed framework provides a reproducible basis for multidimensional performance–reliability evaluation.