Exploration of Failures in an sUAS Controller Software Product Line

Small uncrewed aerial systems (sUAS) are growing in their use for commercial, scientific, recreational, and emergency management purposes. A critical part of a successful flight is a correctly tuned controller which manages the physics of the vehicle. If improperly configured, it can lead to flight instability, deviation, or crashes. These types of misconfigurations are often within the valid ranges specified in the documentation; hence, they are hard to identify. Recent research has used fuzzing or explored only a small part of the parameter space, providing little understanding of the configuration landscape itself. In this work we leverage software product line engineering to model a subset of the parameter space of a widely used flight control software, using it to guide a systematic exploration of the controller space. Via simulation, we test over 20,000 configurations from a feature model with 50 features and 8.88 × 1034 products, covering all single parameter value changes and all pairs of changes from their default values. Our results show that only a small number of single configuration changes fail (15%), however almost 40% fail when we evaluate changes to two-parameters at a time. We explore the interactions between parameters in more detail, finding what appear to be many dependencies and interactions between parameters which are not well documented. We then explore a smaller, exhaustive product line model, with eight of the most important features (and 6,561 configurations) and uncover a complex set of interactions; over 48% of all configurations fail.