Explainable machine learning audit of attribute rating relationships in Football Manager 2023

BACKGROUND: Football Manager data contain simulated player attributes generated within a proprietary game environment. They can reveal relationships within the rating system but cannot establish real-world talent-identification validity. OBJECTIVE: To audit how baseline attributes predict an internal Football Manager 2023 (FM2023) rating threshold, compare four untuned models under repeated cross-validation, and examine model-specific attribute-contribution stability across categories and positions. METHODS: We analyzed 2,282 anonymized male player profiles aged 15-18 years. The outcome was an FM2023 overall score of at least 10. Logistic regression, random forest, radial support vector machine (SVM), and a single-hidden-layer neural network were evaluated with three repeats of stratified five-fold cross-validation. We report receiver operating characteristic area under the curve (ROC-AUC), precision-recall AUC (PR-AUC; average precision), F1 score, sensitivity, specificity, balanced accuracy, Matthews correlation coefficient (MCC), and Brier score. Logistic-regression contributions were calculated on the logit scale; position-specific top-five stability was evaluated with 1,000 bootstrap samples per modeled position. RESULTS: Logistic regression had the highest mean ROC-AUC (0.779 ± 0.033) and PR-AUC (0.380 ± 0.047), whereas the neural network had the highest mean F1 score (0.284 ± 0.055). At the prespecified 0.50 threshold, logistic-regression sensitivity was 0.132 and specificity was 0.983. Technical, physical, and mental attributes appeared among the ten highest-ranked contributions. Mean absolute contribution per feature was slightly lower for technical than physical attributes (difference -0.0043). Position-specific rankings differed, but several top-five features had bootstrap inclusion frequencies below 50%. CONCLUSIONS: Baseline simulated attributes showed moderate discrimination for an internal FM2023 rating threshold, but sensitivity was low and position rankings were not uniformly stable. The results describe the internal structure of this simulation dataset and do not validate real-world coaching, scouting, or player-selection decisions.

Authors

Institutions

Publication Details

Journal
PLoS ONE
Published
2026-11-17
DOI
https://doi.org/10.1371/journal.pone.0363141
Primary Topic
Sports Analytics and Performance
Type
article
Field-Weighted Citation Impact
0.00

Funders

Controls
|||
ALL TIME
JAN
FEB
MAR
APR
MAY
JUN
JUL
AUG
SEP
OCT
NOV
article

Explainable machine learning audit of attribute rating relationships in Football Manager 2023

Zhang, Zhuyue Zhang, Lin Qiu
1 links
PLoS ONE
Sports Analytics and Performance
article

Explainable machine learning audit of attribute rating relationships in Football Manager 2023

Zhang, Zhuyue Zhang, Lin Qiu
article en

Abstract

BACKGROUND: Football Manager data contain simulated player attributes generated within a proprietary game environment. They can reveal relationships within the rating system but cannot establish real-world talent-identification validity. OBJECTIVE: To audit how baseline attributes predict an internal Football Manager 2023 (FM2023) rating threshold, compare four untuned models under repeated cross-validation, and examine model-specific attribute-contribution stability across categories and positions. METHODS: We analyzed 2,282 anonymized male player profiles aged 15-18 years. The outcome was an FM2023 overall score of at least 10. Logistic regression, random forest, radial support vector machine (SVM), and a single-hidden-layer neural network were evaluated with three repeats of stratified five-fold cross-validation. We report receiver operating characteristic area under the curve (ROC-AUC), precision-recall AUC (PR-AUC; average precision), F1 score, sensitivity, specificity, balanced accuracy, Matthews correlation coefficient (MCC), and Brier score. Logistic-regression contributions were calculated on the logit scale; position-specific top-five stability was evaluated with 1,000 bootstrap samples per modeled position. RESULTS: Logistic regression had the highest mean ROC-AUC (0.779 ± 0.033) and PR-AUC (0.380 ± 0.047), whereas the neural network had the highest mean F1 score (0.284 ± 0.055). At the prespecified 0.50 threshold, logistic-regression sensitivity was 0.132 and specificity was 0.983. Technical, physical, and mental attributes appeared among the ten highest-ranked contributions. Mean absolute contribution per feature was slightly lower for technical than physical attributes (difference -0.0043). Position-specific rankings differed, but several top-five features had bootstrap inclusion frequencies below 50%. CONCLUSIONS: Baseline simulated attributes showed moderate discrimination for an internal FM2023 rating threshold, but sensitivity was low and position rankings were not uniformly stable. The results describe the internal structure of this simulation dataset and do not validate real-world coaching, scouting, or player-selection decisions.

PLoS ONEVol. 21(11)
Soochow University (CN)
National Office for Philosophy and Social Sciences, National Social Science Fund of China
Sports Analytics and Performance
AI Navigator

Ask Laika to Summarize, Analyze, and Connect papers live on the map.

Summarize Papers & Methodologies

Extract key findings, datasets, and comparative methods across publications.

Benchmark Rankings & Visual Analytics

Rank top research institutions, authors, funders, topics, and journals by Field-Weighted Citation Impact (FWCI) and paper volume with instant charts.

Connect Distant Disciplines

Bridge topological clusters on the map to find hidden collaborative intersections.

Laika access coming soon