Deciphering the “Art” in Modeling and Simulation of the Knee Joint: Model Benchmarking

Given the strong ties to data sharing and the responsible use of resources, reproducibility of modeling and simulation practice is of paramount importance in science. Computational models in orthopedics provide insight into healthy and injured joint mechanics and can inform clinical decision-making. The KneeHub project investigated the influence of modelers' decisions and thus their "art" in simulation and modeling; five teams developed and calibrated knee models using the same experimental data. Model benchmarking evaluated the predictive ability of the models under loading scenarios that were not considered in the development and calibration process. The objective of this study was to evaluate the accuracy of predictions of knee-specific joint biomechanics for benchmark scenarios of simulating a resected anterior cruciate ligament (ACL) using models of one knee and a combined pivot shift loading using models of another knee. The models predicted the major trends in kinematics and kinetics; however, differences were observed in comparison to experimental data and between teams. Model-to-experiment root-mean-square (RMS) errors were up to 6.6±2.4 mm in anterior-posterior (AP) translation, 13.5±12.9 deg in internal-external (IE) rotation, and 5.3±3.4 deg in varus-valgus (VV) rotation; errors were largest in internal-external rotation, and standard deviations reflected differences between teams. While calibrated models were tuned to a similar set of conditions (albeit with different decisions), the optimized stiffness and reference length/strain of ligament structures may not fully reproduce the contributions of these structures to joint kinematics that were measured experimentally in the benchmark scenarios. As researchers often extend models beyond the conditions used to calibrate them, quantifying model accuracy and limitations with benchmarking represents a crucial step toward reproducibility and can help establish best practices for credible modeling in our community.