AI Insight
This paper introduces Shapley compositions, a novel method for explaining multiclass probabilistic predictions from machine learning models. Unlike traditional approaches that compute Shapley values separately for each class, this method uses Aitchison geometry from compositional data analysis to treat the output probability distribution as a coherent whole on the simplex. The authors prove that their approach uniquely satisfies key axiomatic properties (linearity, symmetry, and efficiency) when properly extended to the multiclass setting.
Why it matters
This work provides a mathematically rigorous foundation for explaining AI models that output probability distributions across multiple classes, which is common in real-world applications like medical diagnosis, image classification, and risk assessment. By respecting the compositional nature of probability distributions, the method offers more coherent and interpretable explanations than existing one-vs-rest approaches.
arXiv:2408.01382v3 Announce Type: replace
Abstract: Originating in game theory, Shapley values are widely used for explaining a machine learning model’s prediction by quantifying the contribution of each feature’s value to the prediction. This requires a scalar prediction as in binary classification, whereas a multiclass probabilistic prediction is a discrete probability distribution, living on a multidimensional simplex. In such a multiclass setting the Shapley values are typically computed separately on each class in a one-vs-rest manner, ignoring the compositional nature of the output distribution. In this paper, we introduce Shapley compositions as a well-founded way to properly explain a multiclass probabilistic prediction, using the Aitchison geometry from compositional data analysis. We prove that the Shapley composition is the unique quantity satisfying linearity, symmetry and efficiency on the Aitchison simplex, extending the corresponding axiomatic properties of the standard Shapley value. We demonstrate this proper multiclass treatment in a range of scenarios.
Source: Explaining a probabilistic prediction on the simplex with Shapley compositions