Improved Open-World Fingerprinting Increases Threat to Streaming Video Privacy but Realistic Scenarios Remain Difficult

Recent work on video stream fingerprinting has begun to explore its effectiveness in large open-world scenarios, in which the vast majority of test samples are from unmonitored videos that are unknown to the model at training time, showing that it is more difficult than earlier small open-world results have suggested. However, the evaluated approach employed deep learning techniques with potential shortcomings for the open-world task. We build on that work to evaluate more advanced techniques drawn from the literature on open set recognition, out-of-distribution detection, and robustness to adversarial examples, hypothesizing that they can improve effectiveness. We find that combinations of techniques can improve effectiveness, cutting the open-world false positive rate by up to 92% at a recall of 0.5. However, precision would likely still be problematic at the full-scale of the largest platforms hosting hundreds of millions or more videos. Additionally, we find that introducing two other dimensions of realism - when training and test sets are streamed from different vantage points, and when monitoring shorter videos or traffic flows - can greatly increase open-world false positives, making the full-scale open-world task even more difficult. Accordingly, we call for more work to focus on larger and more realistic open-world scenarios to continue to gain a better understanding of the effective envelope for fingerprinting.