Optimal Item Placement for Information Retrieval in Stochastic Paired Comparison Models Under Special Comparison Structures

Paired comparison models are examined from the perspective of the placement of objects within specific comparison structures. For both pairwise comparison matrix-based models and stochastic models, previous studies have examined which comparison structures maximize the amount of information that can be recovered from incomplete comparisons. In this paper, we investigate how the amount of extracted information can be increased in stochastic paired comparison models—primarily the Bradley–Terry model—by way of exploiting prior information about the ranking of the objects, if such information is available. We examine several comparison structures to identify the optimal placement of objects within each structure with respect to information recovery and evaluability. The investigated structures are the star graph, the union of two star graphs, and the union of two edge-disjoint spanning trees. Parameters are estimated using the maximum likelihood method. The applied evaluation metrics are the Euclidean distance, Pearson, Spearman, and Kendall correlations, called similarity metrics. Moreover, the rate of evaluable datasets and an inconsistency index is also computed. We found that, in almost all cases, all four similarity metrics identified the same placement as optimal. Our results show that, for the star graph, placing an object of medium strength at the center and comparing all other object to it maximizes the amount of information recovered from the comparisons. For the union of two star graphs, placing objects that occupy middle positions in the ranking at the centers also outperforms the commonly used best–worst centered placement. However, the union of two edge-disjoint spanning trees provides, on average, even better information recovery based on all investigated metrics. We also examined the proportion of evaluable datasets and found it to be higher when medium-strength objects were placed at the centers. Finally, we compared the findings obtained from the stochastic models with those from pairwise comparison matrix-based models and observed strong agreement between the two approaches.