Evidence-Weighted Multi-Criteria Decision Support for Subjective Quality Assessment Under Sparse and Unbalanced Information

Decision making for complex products often depends on subjective user experience, while competing alternatives may be supported by strongly unequal numbers of observations. This study develops an evidence-weighted multi-criteria decision-making methodology for such sparse and unbalanced information. A seven-category rating scale is aggregated at the respondent level and transformed affinely to [0, 1]. Domain-specific variance components are estimated by restricted maximum likelihood and used for empirical Bayes partial pooling, so that sparse estimates are moderated without excluding valid alternatives. AHP preference weights are kept conceptually separate from evidence weights, inherent product quality is separated from non-inherent ownership attributes, and uncertainty is propagated through Monte Carlo simulation. The empirical demonstration comprised 140 unique questionnaire records for 21 agricultural tractor brands with sample sizes from 1 to 50. Compared with direct averaging, the empirical Bayes ranking was highly preserved (Spearman ρ=0.992) while unsupported extremes were reduced. A controlled simulation showed lower RMSE for empirical Bayes than for the arithmetic mean, median, and fixed shrinkage under both Gaussian and bounded non-Gaussian data generation, with the largest benefit at n=1. Sensitivity analyses showed high ranking stability to the upper-level quality weight, perturbations of AHP weights, and bounded score transformations. The framework therefore provides reproducible uncertainty-aware decision support without treating weak evidence as either absent or equally strong as data-rich evidence.