ADVE: Anchor-Delta Video Embedding for Efficient Semantic Scene Understanding

We propose ADVE (Anchor-Delta Video Embedding), a training-free method for efficient semantic video representation that reduces vision encoder invocations by 60.33–96.67% while maintaining a mean cosine similarity of 0.9923 relative to ground-truth CLIP embeddings on real-world video. Current video understanding systems independently encode every frame through expensive vision encoders such as CLIP, ignoring the high temporal redundancy between consecutive frames. ADVE instead encodes only anchor frames fully and approximates the embeddings of all subsequent delta frames using spatial graph deltas—compact representations measuring how detected objects moved relative to each other. The core approximation, Ê(ft) ≈ f(Eanchor, ΔG(t)), is validated on a controlled synthetic benchmark and the MOT17 pedestrian tracking dataset. On MOT17, ADVE achieves 0.9923 mean cosine similarity and 0.9490 minimum cosine similarity—outperforming the best fixed-interval baseline (Keyframe-30: 0.9762 mean, 0.9054 minimum) on every accuracy metric—while consuming 65% less GPU memory (330 MB vs. 950 MB). Real-time inference is demonstrated at 53.5 FPS on a consumer GPU (NVIDIA RTX 4050), enabling deployment on edge hardware where existing methods cannot fit.