Towards an End-to-End (E2E) Adversarial Learning and Application in the Physical World

The traditional process for learning patch-based adversarial attacks, conducted in the digital domain and later applied in the physical domain (e.g., via printed stickers), may suffer reduced performance due to adversarial patches’ limited transferability between domains. Given that previous studies have considered using film projectors to apply adversarial attacks, we ask: Can adversarial learning (i.e., patch generation) be performed entirely in the physical domain using a film projector? In this work, we propose the Physical-domain Adversarial Patch Learning Augmentation (PAPLA) framework, a novel end-to-end (E2E) framework that shifts adversarial learning from the digital domain to the physical domain using a film projector. We evaluate PAPLA in scenarios, including controlled laboratory and realistic outdoor settings, demonstrating its ability to ensure attack success compared to conventional digital learning–physical application (DL-PA) methods. We also analyze how environmental factors such as projection surface color, projector strength, ambient light, distance, and the target object’s angle relative to the camera affect patch effectiveness. Finally, we demonstrate the feasibility of the attack against a parked car and a stop sign in a real-world outdoor environment. Our results show that under specific conditions, E2E adversarial learning in the physical domain eliminates transferability issues and ensures evasion of object detectors. We also discuss the challenges and opportunities of adversarial learning in the physical domain and identify where this approach is more effective than using a sticker.