Automatic Assessment of Software Quality Indicators

Software plays a vital role in research, but we do not know much about it's quality. There are no well defined indicators or even measurements, despite the software often being written by domain researchers with basic software engineering skills. In this talk we want to introduce a set of key performance indicators (KPIs) provided by a working group of the German Helmholtz Association and present the results of a discussion we had in context of the HERMES project regarding them. The goal of the suggested KPIs is to support assessment and evaluation of the quality of Research Software and datasets. While the set of indicators is not fixed yet, the working group already published a first draft of the metrics they are about to use at ArXiv. This involves an adopted maturity model (COBIT) with indicators for different dimensions. The Hermes project regards KPIs as important metadata for the given artifacts, especially for Research Software. As Hermes is all about publication of software (and datasets) with rich, automatically collected metadata, it was our natural reaction to look into the planned indicators.