The Kieker Observability Framework for Analyzing and Visualizing the Software Architecture of Existing Software Systems

The DFG-funded SustainKieker project is a sustainability and reusability research project for the Kieker Observability Framework, intended as an example for RSE research. The research software Kieker started 20 years ago with goals to monitor and analyze software systems. It has been attracting users in both academia and industry, with active development until today. In this demo, we show how Kieker may be used for observing existing software systems (with the TeaStore as example) and how the recovered architecture is visualized with the ExplorViz visualization tool.The demo comprises of two software systems and two microservice containers, where one software system comprises of one or more microservice containers, and each microservice occupies an entire Docker container: (1) a JMeter service, (2) the TeaStore microservices reference application, (3) the ExplorViz visualization tool, and (4) the Kieker-to-OpenTelemetry Transformer. First, JMeter runs as a single microservice instance, responsible for generating and sending requests to the TeaStore. Second, the TeaStore implements the front-end and back-end of an e-commerce system as microservices, which becomes active as soon as JMeter requests arrive. Each of the TeaStore microservices is defined as a Docker image, and each service instance is launched inside a Docker container. The TeaStore uses Kieker's monitoring agent to monitor itself, generating Kieker monitoring records. This is done by embedding a Kieker agent to all TeaStore service instances. Third, ExplorViz visualizes the target application architecture with the city metaphor. Each microservice of TeaStore is visualized as a single city block. ExplorViz, however, only consumes OpenTelemetry spans, which is semantically equivalent to Kieker monitoring records, but they use different data structures. OpenTelemetry is an industry standard for observability data. Here, we introduce the fourth software: Kieker-to-OpenTelemetry Transformer. To mitigate the incompatibility of the two data types, we built a Kieker-to-OpenTelemetry Transformer on top of Kieker's analysis framework. The transformer runs as a single microservice container, receiving Kieker records from the TeaStore and sending transformed OpenTelemetry spans to ExplorViz. Using this approach, The ExplorViz framework does not need to know about the Kieker monitoring record type, because the Kieker-to-OpenTelemetry transformer generates correct OpenTelemetry spans. With our demo, we have successfully presented an implementation to establish the interoperability between Kieker monitoring records and OpenTelemetry spans, that obviates their internal structural differences. Our next goals include (1) using the Kubernetes microservices framework as our default platform, instead of the Docker Compose, for enhanced scalability. We will also (2) embark systematic benchmarks to measure the scalability of the current approach, and (3) continue with other interoperability approaches such as using domain-specific languages that automatically generate an on-premise Kieker-to-OpenTelemetry transformer. SustainKieker is funded by the Deutsche Forschungsgemeinschaft (DFG – German Research Foundation) (grant no. 528713834) and UK Research and Innovation (UKRI) through the UKRI Metascience Research Grants programme (Reference S26368).