A Verification Protocol for Silent Tenant-Isolation Failures in Dynamically Routed Database-per-Tenant Applications

Applications that give each customer, or tenant, its own database must route every request to the right database at run time. In Spring applications the routing lookup commonly returns a default data source when it finds no entry, and when that default is the control-plane database, tenant work runs there without an error. This paper proposes a protocol of six verification classes, each stating a property, its instrumentation, and an observable read from the infrastructure, such as the database that answered, rather than from the application's response. With 16 threads resolving connections while the routing table was replaced a million times per second, rebuilding it in place with the framework's fallback sent 939 of 120,000 resolutions to the control-plane database without an error. Atomic replacement sent none for tenants in the table, but a tenant missing from it stayed off the control plane only when a missed lookup raised an error. Of twelve seeded faults, which re-create nine of eleven defects found by code review and add one synthetic fault, the protocol detected twelve and the existing functional suite one; the same tests asserting on responses detected six.