Dynamic Motion State Estimation and Control via RNNs and Sim-to-Real Transfer

Dynamic motion plays a vital role in biological and technological domains, contributing to animal survival, human well-being, and the practical viability of robotic systems. The quantification of movements is crucial for both passive observation and active manipulation of movements. For example, inertial sensor-based motion tracking allows to objectively assess a patient's rehabilitation progress, while exoskeletons can actively support the patient's gait. Despite the large number of different problems in motion analysis and control, the current state of the art almost exclusively comprises methods that solve individual specific problems and require considerable technical expertise to select and configure suitable methods. Methods for inertial sensor-based knee angle measurement, for example, differ fundamentally from those for detecting movements of the shoulder joint or entire extremities. Furthermore, the parameterization of the algorithms usually depends on the sensor attachment or the movement performed -- similarly to the parameterization of an algorithm for controlling dynamic movements of an exoskeleton or robot that requires precise identification of the system dynamics model. Both for motion analysis and control, data-driven methods offer great potential for reducing this complexity and effort. However, they usually require a large amount of experimental data from real-world systems in order to achieve sufficient generalization capabilities and robustness even beyond simulation environments. This thesis proposes and validates a recurrent neural network-based approach, which enables the efficient solution of motion analysis problems on the one hand and motion control problems on the other, and provides easy-to-use plug-and-play solutions in the form of trained recurrent neural networks. Due to extensive domain randomizations, the neural networks trained on simulation data alone can be transferred to the real-world problem without additional data or adjustments. Training across very different simulation scenarios--including those traditionally treated as separate problems--provides a single pluripotent solution that can be used without problem-specific prior knowledge. The approach is used to derive specific methods for the state estimation task of inertial motion tracking of kinematic chains and for the task of reference tracking control in unknown nonlinear dynamics. The resulting solutions enable for the first time: 1) a data-driven method for observability analysis of complex state estimation problems of inertial motion tracking; 2) an inertial motion tracking method, validated on a real system, that overcomes four key challenges simultaneously and enables magnetometer-free inertial motion tracking with a reduced number of sensors, with sensor-to-segment calibration capabilities and motion artifact reduction; 3) a broadly applicable solution for inertial motion tracking that eliminates the need to select specific methods by training a single, problem-agnostic recurrent neural network that generalizes even over the morphology of the kinematic chain; 4) a data-efficient learning control method that enables reference tracking in a system with unknown nonlinear dynamics without expert knowledge and that is validated on a variety of simulated dynamics; 5) a method that allows a pneumatically driven robotic arm to learn agile movements within a few trials despite its inherent compliance, while effectively rejecting external disturbances. The progress achieved makes inertial motion tracking less restrictive, enabling its use without expert knowledge and beyond established applications. At the same time, they contribute to the development of autonomous plug-and-play approaches for model- and prior-knowledge-free motion learning in systems with unknown nonlinear dynamics, e.g., for self-calibrating, inherently compliant robots or learning exoskeletons and neuroprostheses.