Calibrated simplex-mapping classification

We propose a novel methodology for general multi-class classification in arbitrary feature spaces, which results in a potentially well-calibrated classifier. Calibrated classifiers are important in many applications because, in addition to the prediction of mere class labels, they also yield a confidence level for each of their predictions. In essence, the training of our classifier proceeds in two steps. In a first step, the training data is represented in a latent space whose geometry is induced by a regular (n − 1)-dimensional simplex, n being the number of classes. We design this representation in such a way that it well reflects the feature space distances of the datapoints to their own- and foreign-class neighbors. In a second step, the latent space representation of the training data is extended to the whole feature space by fitting a regression model to the transformed data. With this latent-space representation, our calibrated classifier is readily defined. We rigorously establish its core theoretical properties and benchmark its prediction and calibration properties by means of various synthetic and real-world data sets from different application domains.

Train (line 1): Calculate the simplex vertices p 1 , . . . , p n and train the regression model based on a training data set D, the user-defined parameters α, β, k α , k β , and d for the training data transformation f and the hyperparameters h of the regression model. In this manner, obtain a point prediction f (x) -possibly along with a probabilistic prediction q( · |x) -for every x ∈ X . For the calculation of the simplex vertices p 1 , . . . , p n , we provide two methods. First, the explicit method via (66) and second, an iterative method. For the latter, we choose p 1 to be the first canonical basis vector and determine the remaining elements of p 2 , . . . , p n one after another based on the already specified elements such that (58.b) and (68.b) are fulfilled. The resulting simplex vectors from the two methods differ only by a joint rotation so that there is no substancial effect on our method apart from minor numerical differences. All results in this paper are based on simplex vertices from the iterative method.
PredictClassLabel (line 7): Predict the class label of (possibly unseen) feature space points x ∈ X based on the point prediction f (x) of the trained regression model and on the simplex vertices p 1 , . . . , p n .
PredictClassLabelProbability (line 13): Predict the probability of class label y ∈ Y for unseen feature space points x ∈ X based on the probabilistic prediction q( · |x) of the trained regression model, on the simplex vertices p 1 , . . . , p n , and on a fixed sample size N . We presume that q( · |x) is normally distributed with mean µ(x) and variance σ(x) 2 .
Compress (line 24): Transform the latent space point z ∈ Z to reference-simplex counterpart w = C(z) ∈ −S • based on the user-defined parameter τ and the simplex vertices p 1 , . . . , p n .
Inflate (line 28): Transform the reference-simplex point w ∈ −S • to latent space counterpart z = I(w) ∈ Z based on the user-defined parameter τ and the simplex vertices p 1 , . . . , p n .
In particular, our implementation is intended to be used with scikit-sklearn [2]. We therefore do not prescribe the specific form of the regression model, but instead allow the user to plug in an arbitrary scikit-sklearn estimator.

2:
Calculate simplex vertices p 1 , . . . , p n either explicitly according to (66) using (65) and (67) or iteratively as described in appendix S2.  Figure 10. An overview of the functionality of the provided implementation [1] as outlined in Algorithm 1. After the initial training stage, there are two possible use cases, namely classification and visualization, as described in detail in Section 2. The first use case refers to the prediction of class labels and of class probabilities for unseen feature space points, whereas the second use case allows to perform a transformation from the latent space to a reference simplex and back for visualization purposes.