IN-CORE - Software Engineering Practices on a Large Scale Project

The Interdependent Networked Community Resilience Modeling Environment (IN-CORE) is a state-of-the-art computational environment developed by the NIST Center of Excellence for Risk-Based Community Resilience Planning to advance community resilience measurement science and provide a decision support system. IN-CORE integrates various models, including physics-based modeling of infrastructure, network models, data-based socio-economic models, and resilience metrics, to help communities prepare for, mitigate, and rapidly recover from hazard events. IN-CORE has over 28 GitHub repositories, managing various components of the system, including Helm charts used for deployment in a Kubernetes cluster, several Python libraries for modeling, visualization, and community contribution, as well as the Java-based IN-CORE web services. Each software component has continuous build and integration that happens through a series of GitHub actions that help automate the software development process, from Unit testing and Linting to automatic GitHub releases to GitOps for software deployment. While all projects can benefit from this automation, it is particularly beneficial on a large-scale project with many components to help manage the complexity of the entire process, help with quality control, and ensure smooth delivery of releases and deployments. On the modeling side, the IN-CORE development team at NCSA has developed a process for managing the intake of new models from researchers that is dependent on where the researcher is in their development process. For the case where a researcher has code that is ready, they can submit it for review by a research software engineer (RSE). An RSE is assigned to work with the researcher to review their code and finish the implementation in IN-CORE. If a researcher is at an earlier stage of development, they can submit for consultation so that an RSE can help them with library selection, requirements gathering and other implementation details to help get their code ready for inclusion in IN-CORE. Regardless of the readiness, this often involves an iterative process of working with the researcher and communicating with the research team through online meetings, Slack and email. In some cases, there may be scientific questions that need to be resolved and a separate Science Committee (SciCom) is involved to help resolve any issues before the implementation can continue. The software development life cycle is more complex on larger projects with many moving parts. Having good processes in place can help manage this complexity.