Teodora Popordanoska

Every proper score decomposes into two fundamental components: proper calibration error and refinement, utilizing a Bregman divergence. We propose a method that allows consistent, and asymptotically unbiased estimation of all proper calibration errors and refinement terms. In particular, we introduce Kullback-Leibler calibration error, induced by the commonly used cross-entropy loss. As part of our results, we prove a relation between refinement and f-divergences, which implies information monotonicity in neural networks, regardless of which proper scoring rule is optimized.

Paper    Code

We tackle the challenge of defining and estimating calibration error for object detection. In particular, we adapt the definition of classification calibration error to handle the nuances associated with object detection, and predictions in structured output spaces more generally. Furthermore, we propose a consistent and differentiable estimator of the detection calibration error, utilizing kernel density estimation. Our experiments demonstrate the effectiveness of our estimator against competing train-time and post-hoc calibration methods, while maintaining similar detection performance.

Paper    Code    Video    Poster

We propose a low-bias, trainable calibration error estimator based on Dirichlet kernel density estimates, which asymptotically converges to the true Lp calibration error. Due to its favorable computational and statistical properties, it enables evaluating the strongest notion of multiclass calibration, called canonical (or distribution) calibration. Empirically, we demonstrate its utility in canonical calibration error estimation and calibration error regularized risk minimization.

Paper    Code    Video    Poster

We investigate the relationship between model calibration and volume estimation. Accurate volume measurement, e.g., of a tumor or an organ, is of real importance for many medical applications. We demonstrate both mathematically and empirically that if the predictor is calibrated per image, we can obtain the correct volume by taking an expectation of the probability scores per pixel/voxel of the image.

Paper    Code