Joint Sparse Optical Flow Estimation and Keypoint Detection via Dual-task Imperative Learning

Contemporary deep learning approaches for optical flow estimation continue to face persistent challenges in model interpretability, generalization capacity, and deployment efficiency, significantly constraining their practical implementation. This limitation becomes particularly critical in applications such as visual odometry (VO), where precise sparse point tracking supersedes the conventional emphasis on dense optical flow accuracy. Moreover, the lack of a joint framework combining keypoint detection and optical flow estimation limits sparse optical flow performance. To address these fundamental issues, we propose a novel dual-task imperative learning framework that synergistically optimizes sparse optical flow estimation (iFLOW) with adaptive keypoint detection (iPOINT). Our methodology implements an Expectation-Maximization (EM) paradigm where iFLOW and iPOINT undergo alternating optimization through a Gauss-Newton reasoning engine. This innovative architecture leverages convolutional feature advantages under the generalized feature invariance principle. The resulting imperative learning mechanism imbues our framework with enchanced interpretability and cross-domain adaptability while maintaining computational efficiency. Through comparative evaluations against classical and learning-based baselines, our ultra-compact models (0.05M parameters for iFLOW, 0.09M for iPOINT) demonstrate remarkable performance across multiple metrics (End-point Error, F1-all, VO trajectory accuracy) despite requiring only 200 training image pairs.