Deliverable D5.3 - Intermediate report on KPI and KVI validation

This report covers the steps taken to implement testing solutions in the 6G-SANDBOX project as of August 31st 2024. An initial test scope is defined, as well as a main set of KPIs that will be put under test. Two main types of tests are conducted: those testing network cores, or end to end tests that test a full network setup, RAN included. Two Keysight tools are employed for this task, LoadCore for the former and IxChariot for the latter. All 4 6G-SANDBOX platforms (Athens, Berlin, Malaga, Oulu) were put under test, with multiple different trial networks defined and tested at each platform. This report introduces the concept of the Reference Trial Network, which was used during this process to achieve calibration between the platforms. The overall concepts of Inter-Platform and Intra-Platform calibration are also defined, both of which leverage the concept of the Reference Trial Network to provide 6G-SANDBOX users the ability to contextualize any and every measured result. Measurement results from multiple different test cases are presented. Each platform presents an analysis of these results, what they mean for the platforms performance and potential explanations for any outliers. The performance of each Reference Trial Network implementation is also compared in greater detail. The testing process detailed within was successful, in terms of result production and the drawing of large- and small-scale conclusions. A noticeable delta in the performance of the Reference Trial Network between the 4 platforms hasled to the realization that future iterations of the Reference Trial Network need to be defined more granularly, especially in terms of the hardware and virtualization specifications. The difficulties encountered in the implementation of the testing solutions mean that the future process of automating the testing process will need to be approached more carefully and critically than previously expected. This report is an important stepstone for 6G-SANDBOX and will greatly inform and guide the path of continuing testing work in the project.