- Rating
- 3(+1)
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
The paper addresses limited semantic diversity when sampling responses to open-ended prompts. CADI first calibrates candidate preferences, then distills guided responses selected for diversity, utility, and quality into a model that needs no guidance at inference. An idealized coverage analysis motivates the approach; experiments on two open-ended benchmarks and appendix studies report positive results.
Critical strengths
-
The simple, base-anchored calibration objective is well motivated. It reduces concentration on held-out candidate sets, suggesting the effect extends beyond the exact training choices (p. 27, Table 5).
-
The evaluation considers semantic coverage, utility, and quality across two backbones, with additional larger-model and reasoning-trajectory checks (p. 7, Table 1; Appendix C). This gives the paper a meaningful empirical signal.
Critical weaknesses
-
The gain in useful diversity at comparable quality is not yet secure. Some advantages over strong baselines are small, one backbone shows a quality trade-off, and the main results lack paired uncertainty and independent scorer checks (page 7, Table 1; pages 18–19). These checks would clarify how much of the claimed benefit is robust.
-
Stage 2 adds a modest improvement over Stage 1 while requiring external guidance and substantial extra generation and training. Without total resource costs and a matched-budget comparison, its practical value is hard to judge.
-
The proposed mechanism and generalization claim are broader than the direct evidence: the proof uses an idealized semantic distribution, and the main open-ended evaluations cover a limited set of prompts (pages 3–6). Appendix results help, but a category-level analysis and quality-matched decoding comparison would better define where the method works.
Questions for authors
-
Which gains over the strongest baselines remain under paired uncertainty estimates and an independent semantic/quality check of the existing outputs (page 7, Table 1)?
-
Which models and definitions score diversity, utility, and quality during Stage 2, and do these evaluators overlap with those used at test time (pages 5–6, 18–19)?
-
What are Stage 2's total generation, compute, and API costs, and does its gain remain worthwhile against a matched-budget baseline (pages 7, 18, 28)?
-
How do gains vary by prompt category, and do they persist against a temperature/decoding sweep at matched quality?
-
While this does not constitute a specific weakness or a point for questioning, I believe the Llama3.1/Qwen2.5 used in the paper are already outdated. Besides, benchmarks such as GSM8K, MMLU and HumanEval seems to be easy for modern LLMs. I would like to see more recent models and challenged benchmarks, though this is not a requirement.
5 comments
Q3. Stage 2 Computational Costs and Matched-Budget Controls
Thank you for this suggestion. We provide the realized Stage 2 training costs and additional controlled comparisons to assess whether its incremental gains justify the computational overhead.
(1) Training Efficiency and Resource Costs. Stage 2 generates eight unguided and eight guidance-conditioned responses per screened prompt. We employ early stopping based on held-out validation performance, selecting checkpoints at step 10 for Llama and step 15 for Qwen. The following table summarizes the actual generation volume, average screening acceptance rates, and resource costs.
| Backbone | Selected Step | Total Generations | Avg. Acceptance Rate | GPU-hours | API Cost |
|---|---|---|---|---|---|
| Qwen2.5-7B | 15 | 28.3k | 13.58% | 5.4 GPU·h | $27.30 |
| Llama-3.1-8B | 10 | 9.8k | 26.23% | 3.8 GPU·h | $14.34 |
Total generations include both unguided and guidance-conditioned responses as recoreded in our log. Training was conducted on NVIDIA H100 GPUs. Guidance can also be generated by locally hosted models or the model being trained, we used an external API primarily to accelerate training and report its cost for transparency.
(2) Matched-Budget Distillation Controls. To assess whether simpler training strategies can recover similar improvements, we compare Full CADI with hard-target SFT and fixed-teacher context distillation. Both controls start from the same Stage 1 checkpoint and use the same accepted training trajectories, with comparable optimization budgets and identical evaluation protocols. Hard-target SFT applies cross-entropy supervision to accepted responses, while fixed-teacher distillation uses soft token targets from the frozen Stage 1 model with query-specific guidance.
| Method | Llama Δ Distinct₁₀ | Llama Δ Utility₁₀ | Qwen Δ Distinct₁₀ | Qwen Δ Utility₁₀ |
|---|---|---|---|---|
| Full CADI | +0.18 | +0.13 | +0.15 | +0.18 |
| Hard-target SFT | +0.12 | +0.03 | +0.09 | −0.14 |
| Fixed-teacher distillation | +0.09 | +0.05 | +0.08 | +0.04 |
All improvements are relative to Stage 1. Hard-target SFT recovers 60–67% of Full CADI's Distinct₁₀ gains but substantially less of its Utility₁₀ improvements, even reducing Utility₁₀ on Qwen. Fixed-teacher distillation recovers 50–53% of the Distinct₁₀ gains and 22–38% of the Utility₁₀ gains.
Q4. Prompt-Category Analysis and Decoding Robustness
Thank you for this valuable suggestion. We provide additional category-level analysis and quality-matched comparisons across decoding configurations to assess the robustness of CADI's improvements.
(1) Performance Across Prompt Categories. We further analyze the results on NoveltyBench-Curated across four prompt categories, using the existing per-prompt evaluation results under the default decoding configuration (T=1.0, top-k=10, top-p=1.0). The table reports improvements of Full CADI over Base.
| Prompt Category | Qwen Δ Distinct₁₀ | Qwen Δ Utility₁₀ | Llama Δ Distinct₁₀ | Llama Δ Utility₁₀ |
|---|---|---|---|---|
| Randomness | +1.000 | +0.856 | +1.267 | +1.107 |
| Factual Knowledge | +1.286 | +0.965 | +1.286 | +0.824 |
| Creative Writing | +0.667 | +0.750 | +1.444 | +0.784 |
| Subjectivity | +0.700 | +0.517 | +1.767 | +0.543 |
Following NoveltyBench's four-category taxonomy[1], we group the six fine-grained labels in the released dataset by merging creation and naming into Creative Writing, and recommendation and subjective opinion into Subjectivity.
CADI consistently improves both Distinct₁₀ and Utility₁₀ across all four categories on both backbones. The largest Distinct₁₀ gains occur in Factual Knowledge on Qwen (+1.286) and Subjectivity on Llama (+1.767), while Utility₁₀ improves across all categories. These results suggest that CADI's benefits extend across different types of open-ended prompts rather than being concentrated in a particular task category.
(2) Decoding Robustness at Matched Quality. We also evaluate CADI across six decoding configurations per backbone, varying temperature and top-k/top-p truncation. As detailed in Appendix C.3 (Tables 4–5) of the revised manuscript, CADI achieves the highest Utility₁₀ in 11 of the 12 settings and improves both Distinct₁₀ and Utility₁₀ over Stage 1 in all settings.
More importantly, under untruncated decoding (top-k disabled, top-p=1.0), we adjust sampling temperatures to compare Base and CADI at nearly identical response quality, using 10 responses per prompt. On Qwen, CADI (T=0.85) improves Distinct₁₀ by 1.31 and Utility₁₀ by 0.738 over Base (T=0.8), at comparable Quality₁₀ (7.593 vs. 7.570). Similarly, on Llama, CADI (T=0.8) improves Distinct₁₀ by 0.73 and Utility₁₀ by 0.612 over Base (T=1.0), at nearly identical Quality₁₀ (7.497 vs. 7.456).
Q5. Evaluation on More Recent Models and Challenging Benchmarks
Thank you for this valuable suggestion. We agree that evaluating more recent models and challenging benchmarks would further strengthen our conclusions. Accordingly, we have extended our experiments to Gemma-4-12B-it[2], comparing Base, CADI Stage 1, Full CADI, and DARLING. Beyond MMLU-Pro, we additionally evaluate SWE-bench Verified[3] and BIG-Bench Extra Hard (BBEH)[4] to assess general capabilities on more challenging tasks. The results are also included in Section 4.3 (Table 2) of the revised manuscript.
| Method | Distinct₁₀ ↑ | Quality₁₀ ↑ | Utility₁₀ ↑ | Heterogeneity ↑ | MMLU-Pro ↑ | SWE-bench Verified ↑ | BBEH ↑ |
|---|---|---|---|---|---|---|---|
| Base | 2.74 | 9.38 | 3.77 | 12.79 | 77.2 | 44.2 | 53.0 |
| DARLING | 3.48 | 8.45 | 4.24 | 14.35 | 73.9 | 43.2 | 51.9 |
| CADI Stage 1 | 3.73 | 9.13 | 4.53 | 15.28 | 77.8 | 44.4 | 52.6 |
| Full CADI | 3.89 | 9.16 | 4.68 | 15.98 | 77.4 | 45.2 | 52.8 |
On Gemma-4-12B-it, Full CADI improves Distinct₁₀ by 1.15 and Utility₁₀ by 0.915 over Base, while reducing Quality₁₀ by only 0.224. It also outperforms DARLING in semantic diversity, utility, and response quality. Importantly, CADI largely preserves general capabilities across all three benchmarks, achieving the highest SWE-bench Verified score of 45.2%. These additional results further support the effectiveness of CADI on a more recent backbone and challenging evaluation tasks.
[1] Zhang, Yiming, et al. "NoveltyBench: Evaluating Language Models for Humanlike Diversity." Proceedings of the Second Conference on Language Modeling (COLM), 2025.
[2] Gemma Team, et al. "Gemma 4 Technical Report." arXiv, 2026, arXiv:2607.02770.
[3] Chowdhury, Neil, et al. "Introducing SWE-bench Verified." OpenAI, 13 Aug. 2024.
[4] Kazemi, Mehran, et al. "BIG-Bench Extra Hard." Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), Association for Computational Linguistics, 2025, pp. 26473–26501.
Thank you again for your time and thoughtful consideration. We would be happy to address any remaining concerns or provide further clarification. We sincerely hope that our responses help resolve your concerns and encourage you to reconsider your assessment.
We sincerely thank you for the thoughtful evaluation and constructive feedback. We greatly appreciate your valuable suggestions and provide detailed responses to each of your concerns below.
Q1. Statistical Reliability and Independent Validation
Thank you for this valuable suggestion. We provide additional prompt-paired uncertainty estimates and independent validation to assess the reliability of CADI's improvements.
(1) Prompt-Paired Confidence Intervals. Following your suggestion, we computed 95% confidence intervals for Full CADI relative to the strongest external training baseline for each metric using 10,000 prompt-paired bootstrap replicates. The complete results are also presented in Figure 4 of the revised manuscript.
Full CADI achieves higher mean scores than the strongest external baselines across all four metrics on Qwen, and across Distinct₁₀, Utility₁₀, and Heterogeneity on Llama. The paired 95% confidence intervals remain entirely above zero for Utility₁₀ on Qwen and for Distinct₁₀ and Heterogeneity on Llama, providing statistical support for these improvements. The remaining positive mean differences have intervals overlapping zero, while Llama's Quality₁₀ is lower than DQO's.
(2) Independent Semantic and Quality Validation. To independently verify the results in Table 1, we re-evaluate the existing NoveltyBench outputs of Full CADI and the strongest training baselines under matched decoding settings and a fixed budget of 10 responses per prompt. We use GPT-6 as an independent evaluator. Under the same generation budget and decoding settings, we independently assessed valid semantic coverage and mean class quality, and recomputed utility accordingly.
On Qwen, CADI improves Valid Coverage@10 by 0.43, Mean Class Quality by 0.49, and Utility by 0.37 over DARLING. On Llama, CADI achieves 0.97 higher valid semantic coverage and 0.34 higher Utility, with only a 0.14 decrease in mean class quality. These results provide complementary evidence that CADI's semantic coverage and utility advantages persist under an independent evaluation protocol.
Q2. Stage 2 Evaluation Metrics and Evaluator Overlap
Thank you for raising this important question. Stage 2 follows the NoveltyBench evaluation protocol, using the released DeBERTa-v3-large-generation-similarity classifier for semantic clustering and Skywork-Reward-Gemma-2-27B-v0.2 for response quality scoring.
Specifically, diversity measures the number of distinct semantic classes, quality averages the scores of the first response in each observed class, and utility rewards the early discovery of distinct, high-quality responses using a discount factor of 0.8. The complete mathematical definitions are provided in Appendix C.2. For each training prompt, we compare eight unguided and eight guidance-conditioned responses, retaining only those satisfying the Pareto margins specified in Section 3.2 and Appendix C.1.
We acknowledge that these evaluators are also used for NoveltyBench testing, although the training and evaluation prompts are disjoint. To address potential evaluator-specific bias, we additionally conducted an independent GPT-6-based evaluation of existing CADI and DARLING outputs, as detailed in our response to Q1. CADI retains higher valid semantic coverage and utility on both backbones under this independent evaluation, providing complementary evidence that its improvements are not solely attributable to the original evaluators.