Traversal direction changes priorities in multiple criteria decision analysis

Hierarchical structuring is routine in multi-criteria decision analysis (MCDA), and the direction in which decision-makers traverse the hierarchy is usually treated as a matter of analyst convenience. Weights can be elicited top-down, from broad objectives to their sub-criteria, or bottom-up, from the detailed criteria upwards. We test whether this choice really is inconsequential. In a within-participant experiment, 1,070 participants in three countries evaluated the same four alternatives under both approaches. We compare the approaches on the weights they produce and on how well their scores match participants' own informed preferences. Direction had a small effect on fit (d_z = -0.174): top-down scores were slightly closer, but bottom-up fitted better for 43% of participants. Direction had a large effect on the weights (d_z = +0.669): bottom-up elicitation moved weight towards the branches divided into more sub-criteria. Participants rated the two procedures as equally useful, intuitive, easy and enjoyable, yet chose the better-fitting one more often than chance. None of the psychological measures we collected predicted which approach suited a given person, and neither approach reproduced participants' informed top choice more than 40% of the time, against a 25% chance benchmark. Traversal direction therefore matters less for the decision reached than for the explanation of priorities that comes with it. Since the weights are what an MCDA study records, communicates and defends, we argue that they should be reported together with the hierarchy and elicitation level that produced them.