Preferential knowledge for multi-criteria decision making: Stability and consistency of decision rules and weights

• Human-provided decision rules differ between individuals and change in time substantially. • Inter-personal differences account to about three-fourth of the intra-personal ones. • Criteria weights were shown to drift at rates 9–27% in 48 days. • Decision rules are generally consistent when based on preferentially ordered criteria. • However, achieving full consistency of decision tables is hard and seeks for algorithmic support. This research investigates the variability of decision-making preferences, represented in terms of decision rules and criteria weights, in the context of the qualitative multi-criteria method DEX (Decision EXpert). We study the differences between decision rules acquired from different subjects (inter-personal differences) and from the same subjects at different times (intra-personal differences). We also assess the consistency of so-acquired rules and the ability of subjects to estimate the importance (weights) of criteria. The methodological approach consisted of two surveys among students, carried out about one and a half month apart. Four thematic areas were addressed in the questionnaires: selection of study programs, student success, car purchase decisions, and choices regarding everyday shopping venues. In both survey periods, participants were required to assess the importance of these criteria and to define decision rules according to the DEX method. The findings provide insights into the stability of decision-making processes among participants and in time. The results indicate a high variability of decision rules, both inter- and intra-personal. Intra-personal drift is lower than inter-personal differences, but not by much (three-quarters of the latter). The consistency of rules varied between small decision tables with clearly ordered criteria, where it was almost perfect, and large decision tables with less apparent preferential relations. Defining fully consistent decision tables turned out to be hard, indicating the need for automated consistency-checking tools. Criteria weights also drifted in time at the rate about 9% (user-provided weights) and 10–27% (weights assessed algorithmically from decision rules). The main contributions of this study are identified and quantified magnitudes of decision rules variability and consistency.