SCE-DP: System-Context-Conditioned Diffusion Policy Under Simulated Visual–Control Deployment Shifts

Generalization under deployment shifts remains a major challenge for visuomotor robot manipulation, especially when changes in camera placement and control behavior are unknown to the policy. This paper introduces the System-Context-Conditioned Diffusion Policy (SCE-DP), a history-conditioned manipulation policy designed to adapt its latent context to simulated changes in camera extrinsics, action scale, and bounded control delay without explicit recalibration or test-time model-weight updates. SCE-DP encodes a short history of issued commands and their subsequent visual and proprioceptive responses into a latent system context, which conditions the diffusion-based action denoising process. Auxiliary system-parameter regression and one-step response prediction further encourage the context to capture deployment-relevant system properties and their behavioral consequences. We evaluate SCE-DP on five independently trained ManiSkill manipulation tasks across held-out in-range configurations, a withheld camera-delay composition, and mild extrapolation settings. SCE-DP improves the five-task macro-average over domain-randomized Diffusion Policy by 14.5 percentage points on held-out configurations and 15.7 points on the withheld camera-delay composition. Across four shifted evaluation suites, it achieves an average success rate of 64.3%, compared with 48.9% for the domain-randomized baseline, while preserving nominal performance. Command-only, response-only, previous-action, recovery-source, and matched explicit-parameter controls indicate that the gain is not explained solely by command statistics, redundant command input, or recovery-data composition. These results show in simulation that command–response history is an effective source of online latent system context under coupled visual and control shifts; real-robot generalization remains to be established.