Conformal Prediction Sets Quantify Information Gain: A Theoretical Perspective
The paper gives conformal set size a formal information-theoretic meaning.
Kevin Zhang and Stephen Bates introduce generalized information measures based on the size and coverage of conformal prediction sets. They show Shannon mutual information can be written exactly as an integral over these measures. In classification, they prove that reductions in conformal set size from added information track information gain, within calibration and model-error terms. Their experiments cover 11 classification settings and find that set-size reduction can rank features differently from Shannon mutual information. ArXiv · AI/CL/LG's note
Kevin Zhang and Stephen Bates introduce generalized information measures based on the size and coverage of conformal prediction sets. They show Shannon mutual information can be written exactly as an integral over these measures. In classification, they prove that reductions in conformal set size from added information track information gain, within calibration and model-error terms. Their experiments cover 11 classification settings and find that set-size reduction can rank features differently from Shannon mutual information. ArXiv · AI/CL/LG's note
score 4