Reusable Execution Environments in NFDIxCS

In the digital age, research data is no longer defined solely by observations of physical processes. As computers became widespread, software began playing a key role throughout the data life-cycle—from measurement equipment to data management and analysis. Moreover, in many areas of computer science, software has become the primary research subject. Yet, scientific practices have not kept up. This gap has led to a reproducibility crisis in machine learning-based science, which is mainly attributed to missing data and source code. However, availability alone does not overcome the problem: Proper setup and compilation can be prohibitively time-consuming, or not be feasible in face of unclear or missing dependencies. Instead, we need reusable execution environments (REEs) that allow running an application in an environment identical or similar to the original system, built upon specifications that accurately describe the target software and its dependencies. This is challenging: (a) To re-create the same environment that was used by the software, REEs must embed all bootstrap software or rely on well-known long-term archives. (b) Precise specification languages are indispensable for REEs, but it is imperative that researchers aren't burdened with overly complex standards. (c) Appropriate run-time abstractions must be defined so REEs can be executed on future systems, using virtualization for unavailable computer architectures. (d) Software can additionally depend on hardware specifics (e.g., instruction sets) so re-creating an identical environment is not always economically feasible. In NFDIxCS, we have identified an ensemble of REEs to be integrated into the Research Data Management Container (RDMC): • Source-Defined Systems achieve highest reproducibility by building the software environment from source, using emerging technologies such as GNU Guix or Nix. For case studies, we developed REEs with Guix and created portable Docker based binaries. Critically, package sources are archived in the Software Heritage Archive. However, the complex specification language makes running pre-existing binaries difficult. • Containers (e.g., Docker/Podman) offer a simpler approach to REEs, providing lightweight, isolated environments encoded via command-line operations. Requiring less complete specification, they simplify usage at the cost of reproducibility. To address this, our REE prototype records HTTP(s) communication (e.g., downloads) to rebuild the same environment. • Virtual Machines (e.g., VirtualBox/VMware) lower barrier of entry by virtualizing the full system, letting users create environments as usual. The resulting machine state can then be replayed on any host. However, user actions do not constitute a useful specification. This approach is heavyweight and requires available virtualization support. • Software Bill of Materials (SBoM), only gives a detailed list of software components and dependencies without providing an executable. Tools like EasyBuild can rebuild from an SBoM, but its main use is comparing original and replication environments – assessing the differences is crucial, especially on clusters with specialized instructions where no universal binaries exist. Though these REEs address the outlined challenges to varying degrees, integrating them into the RDMC is key to meeting FAIR principles and handling practical issues such as exascale data.