acceptodds
Under review as a conference paper at ICLR 2027

Learning When Evidence-Sensitive Guidance Helps: Calibrating Counterfactual On-Policy Distillation for Memory-Grounded Question Answering

Abstract

Memory-grounded question answering requires language models to use supporting evidence from past interactions. Counterfactual on-policy distillation provides evidence-sensitive supervision by contrasting teacher predictions with and without that evidence. However, the contrast alone does not determine how strongly the teacher target should be adjusted. We propose Mirror-Calibrated Counterfactual On-Policy Distillation (MCC-OPD), which uses a mirror KL budget to adapt the strength of evidence-sensitive teacher corrections. At each student-generated prefix, the method retains the full-context teacher distribution as its base and limits the counterfactual correction by the KL divergence of an equal-strength move in the opposite direction. It selects the largest feasible correction coefficient and trains the student with reverse KL, without answer-level rewards or a learned value model. We establish convexity of the target divergence, enabling a scalar search for the calibrated coefficient. We also analyze outcome-based calibration through local student utility and compare the two calibration strategies empirically. Trained on only 140 LoCoMo questions, MCC-OPD outperforms strong baselines including full-context OPD, CROP, SA-OPD, and ExOPD in LLM-judge accuracy across LoCoMo, REALTALK, MuSiQue, and LongMemEval, with the largest absolute gain of 6.54 percentage points over CROP on LoCoMo. Matched ablations further show that calibration improves LLM-judge accuracy over uncalibrated counterfactual distillation on all four benchmarks.

Primary area:
foundation or frontier models, including LLMs
Keywords:
on-policy distillation, knowledge distillation, large language models, counterfactual guidance, evidence-grounded reasoning, long-term memory

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_3bveOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes MCC-OPD for memory-grounded question answering. It contrasts teacher predictions with and without supporting evidence, then uses a mirror KL budget to control the correction strength. The student learns from the calibrated targets through reverse-KL distillation. The paper proves that the budget can be enforced by scalar bisection and evaluates the method on four benchmarks after training on 140 LoCoMo questions.

Critical strengths

The method gives a simple way to adjust evidence-sensitive supervision without answer-level rewards or a learned value model. The convexity argument provides a clear justification for the scalar search. The experiments include matched component ablations and transfer to three benchmarks not used for training. These comparisons provide useful evidence that the approach can improve answering beyond the training benchmark.

Critical weaknesses

The ablations do not establish whether the mirror budget is better than simply weakening the evidence contrast. The clearest calibration gain is on MuSiQue. On the other three benchmarks, the paired Judge confidence intervals cross zero. These intervals also condition on fixed checkpoints. The evidence therefore leaves uncertainty about the benefit of adaptive calibration and its consistency across independent training runs.

Questions for authors
  1. How does mirror calibration compare with a fixed coefficient c<1c<1, or an uncalibrated target with α\alpha tuned on the same validation set? This would help separate the value of prefix-specific calibration from simply using a weaker contrast.

  2. Are independent training repeats available for MCC-OPD and the matched uncalibrated baseline? Do the calibration gains persist when uncertainty from training is included, rather than only resampling evaluation examples at fixed checkpoints?

  3. Were the benchmark corrections and exclusions finalized before comparing model outputs? Are the revised data and evaluation scripts available? Does the method ranking remain similar on the subset with unchanged questions, references, and evidence?

  4. Does an independent check of the answers changed by calibration support the reported Judge gains? A second judge or blinded human assessment would be useful, especially on REALTALK, where F1 and EM decrease.

  5. How sensitive are the calibration coefficients and task scores to KK in the shared token partition? Since the remaining vocabulary is grouped into one tail cell, could the aggregation hide evidence-sensitive changes that would affect the mirror budget?

4 comments
AuthorsOpenReview ↗

Response to Reviewer 3bve: Matched Controls, Repeats, Blind Judge & Unchanged-Set Checks

Thank you for your sharp methodological questions. We completed all five checks: validation-selected α\alpha / constant-c (+0.208 pp, 5 seeds) and single-run fixed-KL (+1.90 pp) controls, 20 training runs yielding 80 benchmark evaluations, unchanged-item checks, blind second-Judge audits, and 5-seed K-sweeps.

Q1. Mirror calibration vs. weaker contrast (c<1c \lt 1) and fixed KL.

Constant c<1c \lt 1 gives Q~(c)∝QF1+cαQC−cα\widetilde{Q}(c) \propto Q_F^{1+c\alpha}Q_C^{-c\alpha}, identical to uncalibrated FC with αeff=cα\alpha_{\mathrm{eff}}=c\alpha:

  • Validation-selected global contrast (5 seeds): We retrospectively selected MCC α=2\alpha=2 and FC α=1.5\alpha=1.5 by final-checkpoint F1 on the same 77 validation questions, using one run per candidate in α∈{1,1.5,2}\alpha \in \{1, 1.5, 2\}, then evaluated these fixed settings across 5 seeds (equivalent to c=0.75 at α=2\alpha=2). MCC beats FC (43.465±0.241 vs. 43.257±0.365, +0.208 pp) with lower SD and gains on 3/4 benchmarks (+0.18 LoCoMo, +0.91 REALTALK, +0.90 MuSiQue, −1.16 LongMemEval).
  • Fixed-KL control (matched single-run, α=1.5\alpha=1.5): Against a validation-tuned fixed KL threshold (from 3 training-derived candidates), parameter-free mirror calibration gains +1.84/+1.92/+2.63/+1.21 Judge points (+1.90 pp mean), winning 15/16 metrics.

Alongside App. C.1/Table 4 (MuSiQue +2.95/+2.95/+2.76/+2.63 with positive fixed-checkpoint intervals; other 3 Judge intervals cross zero), these controls confirm gains beyond weaker contrast and fixed KL caps.

Q2. Independent training repeats.

MCC/FC × two α values × 5 seeds gives 20 training runs, each evaluated on four benchmarks for 80 evaluations; both same-α\alpha settings, plus Q1’s validation-selected pair (+0.208 pp), retain positive 5-seed macro Judge gains:

Judge scores are percentages; macro equally weights four benchmarks, with mean±sample SD across five seeds. Gains are percentage points.

α\alphaMCC JudgeUncalibrated FC JudgeMCC−FC
1.543.352 ± 0.73943.257 ± 0.365+0.095
243.465 ± 0.24143.305 ± 0.459+0.160

MCC leads in 3/5 runs per α\alpha. At α=2\alpha=2, calibration nearly halves seed SD (0.241 vs. 0.459) and improves 3 benchmarks (+0.49 REALTALK, +0.10 MuSiQue, +0.16 LongMemEval, −0.12 LoCoMo; F1/EM/BLEU-1 −0.14/−0.21/−0.18). At α=1.5\alpha=1.5, MuSiQue improves in 5/5 seeds (+1.18 pp).

Q3. Unchanged examples and data availability.

On the 4,391 strictly unchanged items (also excluding 40 decomposition-edited MuSiQue items), MCC stays #1 among all 11 student methods on all four Judge scores, winning 12/16 metrics, with a +2.316-point macro Judge gain over Full OPD versus +2.318 on the full set. Also excluding item-specific Judge guidance (4,264 items), MCC remains #1 on all four Judge scores, with a +2.528-point macro gain over Full OPD. While some edits followed output inspection, all were source-verified and uniformly applied across methods; data, scripts, and logs are ready for release.

Q4. Blind independent Judge check.

Blind re-evaluation of all 80 results with MiMo v2.6 Pro (fixed rubrics, hidden labels, 2 votes + tie-breaker) gives +0.097/+0.151 macro gains at α=1.5/2\alpha=1.5/2, matching DeepSeek’s +0.095/+0.160. On REALTALK changed answers across 5 seeds, newly-correct exceed lost-correct predictions: 123 vs. 99 (+24) at α=1.5\alpha=1.5 and 130 vs. 96 (+34) at α=2\alpha=2, confirming net semantic correctness gains alongside F1/EM.

Q5. Partition size K and tail cell.

Sweeping K∈{32,64,128}K \in \{32, 64, 128\} (5 seeds each, α=1.5\alpha=1.5, separate trajectories) shows stable behavior:

KMacro Judge Mean ± SDMean Coeff.Activation
3242.86 ± 0.060.9116529.74%
6443.35 ± 0.740.9113029.48%
12843.48 ± 0.520.9119229.40%

The tail preserves the remaining vocabulary's total probability mass but can hide intra-tail shifts. Activation rates use valid tokens; Prop. 1 guarantees the budget on the chosen partition (not cross-partition invariance or full-vocabulary equivalence). Across tested K, coefficients and activation rates stay close, and K: 64→12864 \to 128 shifts mean accuracy by just +0.13 pp.

Given these results across all five questions, we respectfully invite you to consider updating your score.

AuthorsOpenReview ↗

Follow-up: Joint Judge × Subset Check and Full 11-Method Rank Consistency

We jointly varied the evaluation subset and Judge to assess average calibration gains, and separately checked the consistency of all 11 student-method rankings under DeepSeek Judge.

  1. Joint five-seed check across two Judges and three subsets. Using identical answers, references, rubrics, and subset definitions across DeepSeek and blind MiMo v2.6 Pro, we computed five-seed paired same-α\alpha MCC−FC macro Judge gains (pp, equal benchmark weights per run):
Subset (Items)α=1.5\alpha=1.5 DeepSeekα=1.5\alpha=1.5 MiMoα=2\alpha=2 DeepSeekα=2\alpha=2 MiMo
Full set (4,591)+0.095+0.097+0.160+0.151
Unchanged (4,391)+0.092+0.122+0.166+0.167
Strict (4,264)+0.100+0.094+0.266+0.269

Unchanged excludes revisions to any recorded field, and Strict further removes items with item-specific Judge guidance (seed-level variation is reported in our first response). All 12 mean gains are positive, and at α=2\alpha=2 the average calibration gain widens on the strictest unedited, unguided subset under both independent Judges.

  1. Consistency of the full 11-method ranking. Holding the submitted main-table checkpoints fixed under DeepSeek Judge (distinct from the five-seed comparison above), we computed Spearman rank correlations against the full-set ordering of all 11 student methods across the four benchmarks:
SubsetSpearman ρ\rho Range
Unchanged0.984 – 1.000
Strict0.964 – 0.995

These near-unity correlations show that MCC retains its #1 standing and the overall ranking of the compared student methods remains highly consistent.

AuthorsOpenReview ↗

Follow-up before the discussion deadline

Dear Reviewer,

Thank you again for your time and constructive feedback. We would like to follow up on our rebuttal and supplementary response, where we have provided additional experiments, analyses, and clarifications addressing your concerns.

We would greatly appreciate your thoughts on these updates and whether any questions remain. If the additional evidence resolves your main concerns, we would be grateful if you would consider raising your score. We are happy to provide any further clarification that would help your assessment.

Thank you for your continued consideration.

Best regards,
The Authors

I read the rebuttal and appreciate the extensive new experiments, which address some of my questions. However, the gain of mirror calibration over a tuned weaker contrast remains small and within seed variability, so the benefit of the paper's central component is not yet clearly established. I maintain my score.

Reviewer_doKLOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
1
Soundness
2
Read review
Paper summary

The paper proposes MCC-OPD, an on-policy distillation method for memory-grounded question answering that uses mirror KL divergence to calibrate the strength of counterfactual corrections to the teacher’s predictions. Counterfactuals are defined here by removing relevance evidence about the question from the context. The key idea is that the existing method uses the counterfactual difference to modify teacher supervision, but they introduce a mirror KL divergence budget to improve this modification. They prove that this target divergence is convex in correlation strength, hence they can find an optimal coefficient for correlation strength. Experiments using only 140 LoCoMo questions to train and comparing results on four benchmarks outperforms strong baselines on Qwen3 model family.

Critical strengths
  1. The experiments have been performed across many baselines and have shown that their method outperforms.
  2. Several ablation studies have been performed to strengthen the results.
  3. The paper provides proofs for the propositions.
Critical weaknesses
  1. In the method, the paper does not present a background to on-policy distillation or how current/existing methods use counterfactuals to improve it. The use of counterfactuals in distillation is not entirely new, e.g., [1,2]. The paper misses these critical references. It would be better if the paper can provide this information in the beginning and discuss how the proposed method builds over other approaches or what is the novelty over existing works. Without this background and contrast, it is harder to follow and understand the method or its significance.

[1] Zeming Chen, Qiyue Gao, Antoine Bosselut, Ashish Sabharwal, and Kyle Richardson. Disco: Distilling counterfactuals with large language models. ACL 2023.

[2] Faisal Hamman, Pasan Dissanayake, Yanjun Fu, and Sanghamitra Dutta. Few-shot knowledge distillation of LLMs with counterfactual explanations. NeurIPS 2025.

  1. The presented results seem to be based on a single fixed set of 140 questions from LoCoMo. The evaluation could be repeated with different training sample selections to reduce potential data selection bias. The mean and variance of the results should then be reported to assess the robustness of the method to variations in the training samples.

  2. Limited evaluations: The paper uses only one decoder-only LLM for the evaluations (though it has been done with various sizes of the same LLM family). It would be interesting to see if the same method generalizes to different model families.

Questions for authors
  1. What does a “cell” mean in the paper? I noticed that the authors are referring to the word “cell” in many places, but I didn’t understand what that actually is. For instance, the paper also says, “let C assign tokens to cells”. Can you explain the difference between a token position and a cell?

  2. Perform experiments by selecting different train sets with different seeds? Report the standard deviations as well.

  3. Include experiments using at least one other LLM model family

4 comments
AuthorsOpenReview ↗

Response to Reviewer doKL: Llama-3 Generalization (+4.18 pp), 5 Data Resplits, and Background

Thank you for recognizing our gains, ablations, and analysis. We added 5 random data resplits (10 models) and 5-seed Llama-3 experiments, where MCC-OPD beats Full-context OPD by +4.18 pp (5/5 runs positive).

1. Background on OPD and counterfactual distillation.

Unlike off-policy distillation on static teacher outputs, OPD rolls out student answers and applies next-token teacher supervision at each student-generated prefix. Rather than perturbing text to synthesize examples, MCC-OPD queries full-memory (QFQ_F) and evidence-removed (QCQ_C) teachers at the same student prefix, isolating how evidence shifts next-token probabilities without requiring unanswerable inputs or flipped final answers. Using the equal-strength sign-reversed shift around QFQ_F, our mirror KL budget bounds contrastive displacement without answer-correctness rewards. We will add this background.

2. Relationship to DISCO and CoD.

We will cite and contrast both: DISCO generates text perturbations and teacher-filters counterfactual training examples, while CoD uses minimally edited prediction-flipping inputs for few-shot boundary transfer. MCC-OPD instead contrasts paired distributions (QFQ_F, QCQ_C) at identical on-policy student prefixes and dynamically bounds token-level KL shift via mirror calibration.

3. "Cell" versus generation position.

A position is a generation step; a cell groups vocabulary tokens at that step. At prefix "Dana lives in", selecting "Paris" and "Oslo" forms three cells, {"Paris"}, {"Oslo"}, and {all other tokens}, where (0.6, 0.3, 0.1) gives probability 0.1 to the tail group. Student and teachers share this partition at the current step and rebuild it next step: union top-K tokens form singleton cells plus one tail cell. We will add this example.

4. Training-sample selection (5 random 140/77 resplits).

Our original 140/77 split follows the public split in App. B across all methods (5-seed original split: 43.35±0.74%). To isolate split variance, we drew 5 new random 140/77 splits from the 217-item development pool (no LoCoMo test overlap, fixed training seed, 10 models):

Judge scores are percentages. Entries are mean±sample SD across five splits; macro averages four benchmarks equally.

BenchmarkUncalibrated ControlMCC-OPD
LoCoMo62.30 ± 0.6361.81 ± 1.03
REALTALK25.74 ± 0.4425.27 ± 0.00
MuSiQue37.74 ± 0.9438.15 ± 0.69
LongMemEval47.37 ± 0.9947.49 ± 0.76
Macro Judge (%)43.29 ± 0.4643.18 ± 0.48

MCC-OPD stays tightly within 42.59-43.92% (43.18±0.48%), matching the original split. Against the uncalibrated counterfactual control, it leads in 3/5 splits and on MuSiQue and LongMemEval, with a mean paired difference of −0.11 pp.

5. Cross-family generalization (Llama-3).

Training Llama-3.1-8B-Instruct → Llama-3.2-3B-Instruct across 5 seeds on the 140 questions (α=1.5\alpha=1.5; gains before rounding) yields:

Benchmark entries are seed means; macro reports mean±sample SD across five seeds. Gains are percentage points.

BenchmarkFull-context OPDMCC-OPDGain
LoCoMo67.6371.85+4.21
REALTALK28.4633.63+5.16
MuSiQue46.3748.68+2.31
LongMemEval49.3054.35+5.05
Macro Judge (%)47.94 ± 0.5452.13 ± 0.30+4.18

MCC-OPD gains +4.18 pp macro Judge (5/5 runs positive), improves every benchmark, and nearly halves seed SD (0.30 vs. 0.54).

With these clarifications, checks, and Llama-3 gains, we respectfully invite you to consider raising your score.

AuthorsOpenReview ↗

Follow-up: Complete Lexical and Exact-Match Metrics on Llama-3

Complementing the Llama Judge results in our first response, evaluating the same saved test answers for Llama-3.1-8B-Instruct → Llama-3.2-3B-Instruct across all remaining metrics shows consistent gains on every macro criterion:

Macro Metric5-Seed Gain vs. Full OPD (pp)
Judge accuracy+4.18
F1+3.96
Exact Match+2.29
BLEU-1+2.81

Each macro metric weights LoCoMo, REALTALK, MuSiQue, and LongMemEval equally within each run before averaging over all five training seeds (computed before rounding). Across all four benchmarks and four metrics, complete MCC-OPD improves 15/16 seed-averaged scores over Full-context OPD (only MuSiQue Exact Match changes by −0.89 points), confirming that its cross-family benefits transfer to token overlap, exact matching, and BLEU-1 alongside LLM-judged semantic correctness.

AuthorsOpenReview ↗

Follow-up before the discussion deadline

Dear Reviewer,

Thank you again for your time and constructive feedback. We would like to follow up on our rebuttal and supplementary response, where we have provided additional experiments, analyses, and clarifications addressing your concerns.

We would greatly appreciate your thoughts on these updates and whether any questions remain. If the additional evidence resolves your main concerns, we would be grateful if you would consider raising your score. We are happy to provide any further clarification that would help your assessment.

Thank you for your continued consideration.

Best regards,
The Authors

Reviewer_doKLOpenReview ↗

Rebuttal Acknowledgement

I read the rebuttal. While the responses address my concerns, and I appreciate the new experiments, I believe more substantial experimental evaluation and comparison with baselines is needed for this paper to be fully ready. I maintain my score.

Reviewer_maosOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The authors study on-policy distillation (OPD) for answering questions from supplied memory.

A frozen teacher scores each student prefix twice: once with full memory and once with the supporting evidence removed. The two teacher predictions are then combined into a contrastive target. The strength of this correction is set at each position by a "mirror" KL budget. The full correction is kept if it moves the target no farther than the opposite move would; otherwise, it is shrunk by bisection.

The student is trained on 140 questions and tested on four question-answering benchmarks. The authors report that the method outperforms existing methods.

Critical strengths
  • The comparison is broad and includes a good number of baselines and benchmarks: For example, the method is compared with two standard on-policy distillation variants, counterfactual token selection, two spurious-signal-aware token selectors, reward-extrapolated distillation, on-policy self-distillation, reinforcement learning with correctness rewards, supervised fine-tuning on teacher answers, and an outcome-based calibration. The evaluation uses four benchmarks covering conversational memory.

  • Some useful theoretical results support the method. For example, Proposition 1 shows that the KL from the full-context teacher is convex and nondecreasing in the correction strength, which justifies the bisection search. Proposition 2 and Lemma 2 state what the outcome-based calibration estimates and when it holds.

  • The experiments are well structured. For example, Figure 4 uses a length-matched control and shows that sensitivity to evidence does not mean the target favors correct answers.

Critical weaknesses

Pls see the following similarities with prior work. The method mainly combines known pieces:

In my opinion, choosing the strength of an exponential tilt under a KL budget by a 1-D search is a standard step, as in V-MPO (Song et al., 2019). Proposition 1 is also a known fact: KL to an exponential tilt is a log-moment-generating function. The new part is using the mirror move as the budget, and this choice is not justified.

  • Please also see the recent works, VCSD (Liang et al., 2026) and VAD (Zhang et al., 2026), which use the same kind of evidence-contrast target in on-policy distillation. This is not counted as a weakness, but the paper should discuss it.

  • Eq. (5): The budget only shrinks a correction when it mainly boosts low-probability tokens. A large but symmetric correction is never shrunk.

  • Section 4.4 shows that the uncalibrated target promotes wrong answers, but this test is not repeated with the calibrated target. So there is no direct evidence that calibration fixes the problem.

  • The gain from calibration is not robust. It depends on α\alpha and on the number of rollouts N.

  • Why α=1.5\alpha = 1.5?

  • All results come from single runs.

  • The method needs annotated supporting evidence, which the main baselines do not use. So reward-free still requires extra labels.

  • All four test sets were corrected by the authors. The scores are not directly comparable to published results.

Questions for authors

Please see Weaknesses.

3 comments
AuthorsOpenReview ↗

Response to Reviewer maos: AdaCAD & Fixed-KL Controls, 5-Seed Runs, and Target Diagnostics

Thank you for your constructive review. Under the submitted split and protocol, we added a 5-seed AdaCAD-style baseline, a matched single-run fixed-KL control, 5-seed repeats verified independently by MiMo v2.6 Pro, and the calibrated-target diagnostic.

1. Contribution and adaptive-contrast comparison.

CAD supplies the contrastive form, and exponential tilting with convex scalar optimization, as in V-MPO, solves the KL projection. However, AdaCAD and CoCoA adjust contrast heuristically via context/no-context JSD or logit interpolation without bounding KL from the teacher, whereas V-MPO requires an external KL threshold. MCC-OPD derives a prefix-specific KL budget internally from the sign-reversed counterfactual shift, needing no external KL cap. We will cite AdaCAD, CoCoA, and V-MPO, and expand our comparison with VCSD and VAD regarding target construction, reference distribution, and budget rules.

Adapting AdaCAD’s base-2 JSD rule to our partition on Qwen3-8B→1.7B, we selected s=2∈{1,1.5,2}s=2 \in \{1, 1.5, 2\} by 77-item validation F1 and ran 5 seeds. MCC (α=1.5\alpha=1.5) beats AdaCAD-style on macro Judge (43.35±0.74 vs. 42.97±0.55), leading on 3/4 benchmarks (AdaCAD-style leads on REALTALK) and gaining +0.53/+0.47/+0.52 macro F1/EM/BLEU-1 under a shared supervision, training, and evaluation setup.

2. Mirror budget rationale and fixed-KL control.

Equal-magnitude positive and negative log shifts around QFQ_F yield different KL displacements after normalization. The sign-reversed shift at the same prefix sets an internal reference (recovering QCQ_C at α=1\alpha=1), and Proposition 1 retains the largest feasible forward step without an absolute KL threshold.

For centered shift δ\delta, contraction (c<1c \lt 1) holds iff EQF[sinh⁡(δ)]>0\mathbb{E}_{Q_F}[\sinh(\delta)] \gt 0. Boosting rare tokens under QFQ_F can cause this asymmetry, though probability alone does not dictate contraction. Symmetric centered shifts produce equal forward and mirror KL (c=1), so the mirror rule preserves the full shift rather than imposing an absolute cap. In a matched single-run comparison at α=1.5\alpha=1.5 against a validation-tuned fixed-KL budget (selected from three training-derived candidates), MCC gains +1.84/+1.92/+2.63/+1.21 Judge points on LoCoMo/REALTALK/MuSiQue/LongMemEval (+1.90 mean), winning 15/16 metrics.

3. Target diagnostic and answer correctness.

Repeating the Sec. 4.4 frozen-trajectory diagnostic with calibration (α=1.5\alpha=1.5) cuts mean KL from QFQ_F by ~56% (0.530 → 0.234 nats); at 0.01 nats/answer-token, question-averaged incorrect-answer promotion is 22.80% → 23.87% (no reduction on frozen trajectories, matching Fig. 4’s distinction between evidence sensitivity and correctness). MiMo v2.6 Pro rejudged all 80 evaluations from 20 training runs: MCC/FC × two α\alpha values × 5 seeds, each evaluated on four benchmarks. Using fixed rubrics, hidden method labels, and 2 votes plus a third on disagreement, MiMo confirms positive macro MCC−FC gains of +0.097/+0.151 points, matching DeepSeek’s +0.095/+0.160.

4. Robustness to α\alpha, N, and training seeds.

In the paper, Fig. 6 swept five α\alpha values (α=1.5\alpha=1.5 peaked in 3-benchmark Judge); Table 2 and App. C.1/Table 4 isolated calibration at α=1.5\alpha=1.5, N=4 (all 4 Judge scores and all 4 MuSiQue metrics improved); and App. C.4/Table 8 tested the complete method at N∈{4,8}N \in \{4, 8\} (N=4 best at α=1.5\alpha=1.5, N=8 still above Full OPD in three-benchmark mean Judge accuracy). Extending matched MCC vs. FC to 5 seeds (equal benchmark weights per run, mean±sample SD) preserves positive Judge gains at both strengths:

α\alphaMCC JudgeUncalibrated FC JudgePaired Gain (pp)
1.543.35 ± 0.7443.26 ± 0.37+0.095
243.46 ± 0.2443.31 ± 0.46+0.160

MCC leads in 3/5 runs per α\alpha. At α=2\alpha=2, calibration nearly halves seed SD (0.24 vs. 0.46) and improves 3/4 benchmarks (−0.12/+0.49/+0.10/+0.16; FC is +0.14/+0.21/+0.18 higher on F1/EM/BLEU-1). At α=1.5\alpha=1.5, MuSiQue improves in all 5/5 runs (+1.18 pp), while LongMemEval averages −1.28.

5. Supervision and revised benchmarks.

Supporting-evidence spans are context-level supervision, while "reward-free" means no answer-level reward. Full OPD uses no spans, whereas MCC, FC, Fixed-KL, and AdaCAD-style share identical evidence labels, isolating calibration. All methods share identical revised inputs and scoring rules; original/revised data, rationales, and scripts are ready for release, and we will distinguish revised-benchmark scores from historical unrevised results.

We hope these controls, runs, and blind re-evaluations address your concerns and merit an updated assessment.

AuthorsOpenReview ↗

Follow-up: Validation-Selected Global Contrast and Unchanged-Item Rankings

We share two additional analyses showing that mirror calibration outperforms validation-selected global contrast strength and preserves its #1 standing on strictly unchanged test items.

  1. Mirror calibration vs. validation-selected global contrast. Retrospectively applying the same rule to both methods—final-checkpoint F1 on the 77 validation questions over α∈{1,1.5,2}\alpha \in \{1, 1.5, 2\} from one training run per candidate—selects MCC at α=2\alpha=2 and uncalibrated FC at α=1.5\alpha=1.5. Evaluated across all five training seeds (equal benchmark weights per run, mean ± sample SD), MCC outperforms validation-selected FC by +0.208 pp with lower seed variance:
MethodSelected α\alphaMacro Judge (%)
Uncalibrated FC1.543.257 ± 0.365
MCC-OPD243.465 ± 0.241

Benchmark differences are +0.18 (LoCoMo), +0.91 (REALTALK), +0.90 (MuSiQue), and −1.16 (LongMemEval). Because constant-c scaling induces effective strength cαc\alpha, FC at α=1.5\alpha=1.5 also implements constant contraction c=0.75 relative to nominal α=2\alpha=2. Combined with our first response, three controls cleanly separate alternative hypotheses: validation-selected FC tests global strength tuning, AdaCAD-style tests an alternative position-wise JSD rule on the shared OPD partition, and single-run fixed-KL tests a constant vs. mirror-derived prefix budget.

  1. Rankings and gains on unchanged test items. Holding the submitted main-table checkpoints fixed across all 11 student methods under DeepSeek Judge (distinct from the five-seed runs above), we recomputed scores on items unchanged across all recorded fields and on a stricter subset further excluding item-specific Judge guidance:
Evaluation SetItemsMCC−Full OPD (pp)Judge #1
Full test set4,591+2.3184/4
Unchanged4,391+2.3164/4
Strict unchanged4,264+2.5284/4

Unchanged also excludes the 40 MuSiQue items with decomposition-only edits, while Strict unchanged additionally removes item-specific Judge guidance. MCC ranks #1 on 12/16 benchmark-metric pairs on the unchanged set and 11/16 on the strict subset, confirming that both its #1 Judge ranking on every benchmark and its macro gain over Full OPD hold on unrevised examples.

AuthorsOpenReview ↗

Follow-up before the discussion deadline

Dear Reviewer,

Thank you again for your time and constructive feedback. We would like to follow up on our rebuttal and supplementary response, where we have provided additional experiments, analyses, and clarifications addressing your concerns.

We would greatly appreciate your thoughts on these updates and whether any questions remain. If the additional evidence resolves your main concerns, we would be grateful if you would consider raising your score. We are happy to provide any further clarification that would help your assessment.

Thank you for your continued consideration.

Best regards,
The Authors

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.