Fishbowl: AlloyDB’s Extensible Database Hammering Framework

Ensuring the reliability and stability of Database Management Systems (DBMS) is paramount to their success. Rigorous testing and stabilization are critical prerequisites for deploying enterprise-grade DBMS solutions. This paper introduces Fishbowl, an automated testing tool developed specifically to rigorously test Google's cloud-native database products, including AlloyDB and CloudSQL. Fishbowl's architecture prioritizes extensibility, enabling the generation of diverse, enterprise-grade workloads and failure scenarios. With over two years of operational experience, Fishbowl has proven instrumental in identifying and resolving critical issues, directly contributing to the high quality of Google's database offerings. This paper details Fishbowl's architecture, components, and demonstrates its impact through a selection of real-world case studies.