Righteous: Automatic Right-Sizing for Complex Edge Deployments

Edge deployments perform complex deep learning inference and analysis in the wild in highly resource constrained environment. They are positioned everywhere from our largest cities to the bottom of our oceans, and often necessitate significant financial resources and labor to create and deploy. These properties make correctness of edge deployments simultaneously extremely important and difficult to verify a priori. In the past decade, a series of IoT and cloud testbeds have emerged to facilitate this testing. They provide users with access to resources and, less often, sensors that can be used to emulate workloads before deployment. While developers can use these resources to verify the correctness of their configurations, often users would like to “right-size” their deployments - that is, to find a minimal resource configuration that guarantees correctness - to decrease cost and prevent over-provisioning. The current suite of cloud and IoT testbeds does not provide this capability. We present Righteous, an automatic deployment right-sizing tool for edge deployments. Righteous treats configuration as a hyperparameter optimization problem, testing hyperparameter combinations to find a near-optimal configuration as quickly as possible. Righteous uses a new optimization algorithm, informed Pareto Simulated Annealing (iPSA) to find near-optimal configurations faster than other leading approaches. We use Righteous in conjunction with the PROWESS testbed to optimize a drone swarm deployment workload. Our results demonstrate that Righteous configurations use up to 3.5X less resources than those identified by leading hyperparameter tuning and resource allocation techniques, and does so up to 76.3X faster.