CADI: Restoring Output Diversity in Large Language Models via Calibration and Self-Distillation

Large language models (LLMs) often concentrate their outputs on a small set of canonical responses, causing repeated sampling to explore only a narrow subset of valid outputs. Inference-time methods can improve diversity, but their gains depend on persistent prompting or decoding strategies rather than being internalized in model parameters, while training-based methods alter the model itself but typically require carefully designed diversity-specific rewards. In this paper, we propose CADI, a two-stage framework that restores access to valid alternatives suppressed by concentrated model preferences. First, diversity calibration uses simple candidate-selection tasks to reduce overly concentrated model preferences while preserving the model's original preference structure. Second, online guided self-distillation contrasts plain and guidance-conditioned response sets generated by the current model, and distills the guided behavior only when it achieves a Pareto improvement over unguided generation. Experiments on NoveltyBench and Artificial Hivemind show that CADI consistently outperforms strong training-based baselines in semantic diversity and utility across two LLM backbones, while largely preserving response quality and general capabilities. Our code is available at https://anonymous.4open.science/r/CADI-D4B3.