Modeling and Controlling Cyber-Physical Systems Based on Hybrid Turing Machine and Reinforcement Learning

This paper proposes a Hybrid Turing Machine (HTM) framework for modeling and controlling cyber-physical systems (CPS), which integrates discrete symbolic computation with continuous physical evolution. In HTM, each symbolic configuration is associated with real-valued dynamics governed by neural flow functions, and transitions are triggered by guard predicates based on system evolution. A pointer-based memory mechanism induces partial observability, which is addressed by a recurrent neural network (RNN) that encodes controller and device behavior into latent continuous states. We prove that computing the maximum total reward or synthesizing an optimal policy in HTM is incomputable, motivating an approximate control approach. To this end, we abstract HTM execution traces into a Partially Observable Markov Decision Process (POMDP) using latent discrete state clustering and RNN-based dynamic modeling. Reinforcement learning is then applied to train a policy over the latent POMDP, which is lifted back to the HTM configuration space for execution. Experimental results on a multi-UAV scheduling task show that our framework enables interpretable hybrid control and consistent decision-making under limited observability.