Formguard: Continuous Privacy Testing for Websites Using Automated Interaction Replay

Websites commonly use third-party scripts for purposes such as advertising, analytics and payment processing. In recent years, several popular third-party scripts fell victim to supply-chain attacks where users' login credentials and credit card details were stolen. These devastating attacks sometimes remain hidden for several weeks until they are discovered. In this paper, we present Formguard, a continuous testing tool that detects web-based supply-chain attacks in an automated manner. Formguard allows website owners to record complex interactions such as logging in, signing up or checking out a product on their websites. These recordings can then be periodically replayed, while monitoring the HTTP requests and WebSocket messages, accesses to input fields, and information on the embedded scripts. The periodic and automated testing allows for faster detection of malicious supply-chain attacks and potential compliance issues that are impossible to detect with noninteractive security scanners. While Formguard specializes in detecting digital skimming attacks, it can also perform various privacy tests against different aspects of a website including embedded scripts, HTTP headers and cookies. We evaluate Formguard through two case studies. First, a long-term robustness test on 75 websites shows that even complex recordings remain replayable for several months, suggesting minimal maintenance workload for website owners. Second, we use Formguard's crawl mode to study access to and exfiltration from login and registration forms on 100,000 websites, revealing access to password fields on over 10 K sites by third-party scripts. Finally, we discuss the challenges of automated testing for modern web forms, providing insights that may benefit researchers and practitioners.