An Evaluation Metric for Automatic Medical Report Generation Based on Tree Structure

Existing evaluation metrics for medical report generation (e.g., BLEU, ROUGE) focus on lexical overlap but fail to capture structural accuracy and clinical relevance. We propose the Tree Structure Aware Matrix (TSAM), a novel metric that models medical reports as hierarchical trees to evaluate semantic precision. TSAM decomposes reports into location-attribute-description triplets, constructs tree structures, and computes similarity through path matching. Experiments on the MIMIC-CXR dataset (<tex xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">${3, 8 5 8}$</tex> reports) and a real-world ultrasound dataset (<tex xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">${3 0, 5 9 7}$</tex> reports) demonstrate TSAM's superiority: it achieves sensitivity to keyword deletions ranging from 25.95% to 98.40%, outperforming traditional metrics. Human evaluation alignment analysis further confirms TSAM's clinical relevance.