Delta: A Lightweight Monocular Depth Estimation Model via Knowledge Distillation

Deploying accurate monocular depth estimation models on mobile devices is challenging due to computational constraints. We present Delta, a lightweight model achieving near real-time inference (average 186 ms on an iPhone 13) through an efficient knowledge distillation framework leveraging the state-of-the-art teacher model, Depth Anything V2. Delta employs a MobileViT-XS encoder to capture global context with minimal overhead, paired with a custom lightweight MiniDPT decoder. Training utilizes a composite loss function that integrates scale-invariant logarithmic, gradient matching, feature matching, and attention matching components for robust knowledge transfer. A proof-of-concept mobile application demonstrates the model's inference speed. Comprehensive evaluations affirm its suitability for applications such as augmented reality and navigation. Source code, ONNX models, and demo application: https://github.com/GitGud-f/delta