On the Brittleness of Legacy Web UI Testing: A Pragmatic Perspective

Automated web UI testing addresses the labor- and time-intensive demands of manual testing. Despite its importance, numerous barriers impede the continuous automation of end-to-end web UI testing: brittle legacy tests, suboptimal automation frameworks, and web app design unsuited for effective UI testing. While many studies investigate these challenges, most overlook the root causes of brittleness from a pragmatic perspective. This dissertation first reassesses these causes by examining the brittleness of testing real-world web apps. Building on that, we extend existing frameworks to mitigate flakiness stemming from impractical automation frameworks. We also transform traditional web UI testing by decoupling test cases from web applications, using natural language as an intermediary. Finally, we introduce a pragmatic benchmark to steer research toward more realistic web applications. Through these studies, we aim to shed light on current challenges and encourage a more pragmatic perspective on UI testing.