Leveraging Part‐Based NeRF for Robot Self‐Modeling and Control

ABSTRACT Accurate and adaptive models of a robot's morphology and kinematics are crucial for reliable planning, precise control, and seamless interaction within dynamic environments. Traditional modeling approaches demand substantial human intervention and are prone to degradation over time because of factors such as unforeseen damage or wear. Recent research has increasingly focused on task‐agnostic data‐driven self‐modeling methods, which enhance the robot's ability to operate flexibly across various contexts and tasks. This paper proposes a novel self‐modeling framework that leverages neural radiance fields (NeRF) for part‐level reconstruction and aims to provide both the morphology self‐model and the kinematic self‐model. The framework enables robots to predict their pose at different joint configurations. With the self‐model built by our framework, the robot can generate trajectories by itself to accomplish simple tasks or solve inverse kinematic problems.