Multi-objective hierarchical reinforcement learning for efficient and safe robotic manure cleaning in dairy barns

Autonomous mobile robots for manure removal in dairy barns must resolve conflicting objectives of cleaning efficiency, environmental impact, and animal welfare under highly cluttered and dynamic barn conditions. We introduce a Multi-Objective Hierarchical Reinforcement Learning (MO-HRL) framework that integrates a Global Evolutionary Scheduler (GES) for long-horizon cleaning task planning with an Adaptive RL-based Executor (ARE) for real-time, risk-aware robot motion control. The GES solves a multi-objective optimization problem over the robot's travel distance, ammonia emissions, and cleaning yield via an evolutionary algorithm. The ARE employs a constrained Markov decision process enriched with historical high-loss weighting, adaptive action masking, and a final execution shield to guarantee smooth, non-injurious contacts with cows. In Webots simulations across three barn layouts (with ten, twenty, and thirty free-roaming cows, respectively), MO-HRL maintains high task completion (≥93.03%) with low comfort-distance violations (≤6.67%), achieves a peak throughput of 1.98kg/min, and shortens completion time by up to 6× versus PPO (up to 8× in Small). Ammonia emission reduction reaches up to 0.144 kgm−3. These results indicate that hierarchical, risk-aware RL can translate long-horizon scheduling and real-time safety constraints into tangible gains for rapid, safe, and sustainable barn sanitation.