Sardine: A modular framework for developing data acquisition and near real-time analysis applications

We present Sardine, a software framework built with .NET to control experimental setups through the reliable execution of dynamic networks of independent modules, where each module can interface with a hardware device (e.g., camera, motor) or represent an operation over data (e.g., image filter, data stream). The Sardine framework delivers robust fault-tolerant hardware control and eliminates downstream delays by implementing dedicated data processing queues for each module. Any .NET class can be seamlessly adapted into a Sardine module, enabling effortless integration with existing codebases. Sardine's modular architecture ensures flexibility to accommodate changes in experimental paradigms, simplifying the adaptation of essential features, such as swapping hardware components or redefining the logic of their interactions. The core of Sardine is a novel aggregation system that connects modules in two layers, designed to streamline complex workflows such as those in microscopy applications. The first layer (link layer) enables concurrent operations between modules, such as synchronizing a camera module with a stage controller or a laser module, while maintaining a dependency tree to ensure devices operate as intended. For example, in an imaging experiment, Sardine oversees individual module malfunctions, such as a camera failure or stage misalignment, gracefully handling errors and dynamically restoring functionality to impacted network segments to minimize disruption. The second layer (data layer) facilitates the transmission of information across modules by associating them with operations that produce, consume, or transform data. In an imaging context, this could involve processing raw image data from a camera module, passing it through a real-time analysis module for feature detection, and forwarding the results to a visualization module for immediate feedback. This two-layer architecture ensures seamless data flow and robust error handling, making Sardine ideal for dynamic and complex experimental setups. Sardine also integrates logging, a metadata collection system, and tools to create graphical applications.