Probabilistic ordinal regression methods for multiple criteria sorting admitting certain and uncertain preferences

Abstract

We propose a family of probabilistic ordinal regression methods for multiple criteria sorting. They employ an additive value function model to aggregate the performances on multiple criteria and the threshold-based procedure to derive the class assignments of alternatives. The Decision Makers (DMs) can provide certain and uncertain assignment examples concerning a subset of reference alternatives, expressing the confidence levels using linguistic descriptions. On the one hand, we introduce Bayesian Ordinal Regression to derive a posterior distribution over a set of all potential sorting models by defining a likelihood for the provided preference information and specifying a prior of the preference model. This distribution emphasizes the potential differences in the models’ abilities to reconstruct the DM's classification examples and thus is robust to the DM's potential cognitive biases in her judgments. We also develop a Markov Chain Monte Carlo algorithm to summarize the posterior distribution of the preference model. On the other hand, we adapt Subjective Stochastic Ordinal Regression to sorting problems. It builds a probability distribution over the space of all value functions and class thresholds compatible with the DM's certain holistic judgments. The ambiguity in representing incomplete and potentially uncertain preference information by the assumed sorting model is quantified using class acceptability indices. We investigate the performance and robustness of the introduced approaches through an extensive experimental study involving real-world datasets. We also compare them against novel methods based on mathematical programming that handle potential inconsistencies in uncertain preferences in the traditional way by minimizing the misclassification error or the number of misclassified reference alternatives.