Divide and conquer? A combination of judgments method for comparing DSSs. Pairwise comparison vs. holistic paradigms

• A new modeling of DSSs based on symbolic encoding is proposed. • Symbolic encoding allows general analysis (across all performance distributions). • Pairwise and holistic paradigms of expert judgment are compared. • Pairwise paradigm outperforms holistic paradigm when the probabilities of expert judgments are proportional to performance. Despite the prevalence of Decision Support Systems (DSSs) in the field of decision-making, there is a paucity of research dedicated to the evaluation and comparison of these systems. This paper put forward a novel approach to symbolically encoding a DSS, which enables the generalization of comparisons between DSSs for any distribution of performances of the alternatives. The only hypothesis required in the proposed methodology is that the probability of choosing each alternative is proportional to its latent performance. The approach developed is demonstrated with its application to compare two paradigms commonly employed in DSS: holistic versus pairwise. Using a set of three alternatives, the present study provides mathematical proof that a DSS based on the pairwise comparison paradigm achieves higher expected performance than a DSS based on the holistic evaluation paradigm. This result challenges the emerging preference for holistic evaluation of alternatives and suggests that this result may apply to any number of alternatives.