- Rating
- 1
- Confidence
- 3
- Informativeness
- 1
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
The paper proposes VaCrit, a framework for post-hoc self-correction in mathematical reasoning. It represents a textual critique as a latent diagnosis connecting an initial response to a corrected response. Training first develops a reference-free critic through error-localization supervision and refinement-utility learning. It then jointly optimizes a reference-conditioned critique posterior and a refinement model, while using the frozen critic as a prior and a distillation teacher to preserve diagnostic capability. At inference, only the final model is retained; the same checkpoint generates the initial response, diagnoses it, and produces a revision. Experiments with Qwen3-4B and Qwen3-8B on seven mathematical benchmarks report improvements in final accuracy and a favorable balance between error repair and preservation of initially correct answers.
Critical strengths
-
The problem is relevant, and the evaluation considers both benefits and harms of revision. Improving self-correction requires repairing incorrect responses while avoiding damage to correct ones. Reporting correction gains, error-repair rates, and correctness-preservation rates provides a more informative evaluation than final accuracy alone.
-
The complete system produces encouraging empirical improvements. Table 1 reports macro-average final accuracies of 49.10% and 50.48% for the 4B and 8B models, exceeding the strongest same-backbone baseline, SCoRe, by 2.33 and 1.91 percentage points. A single correction round improves accuracy on all seven benchmarks for both model sizes. The reported average preservation rates of initially correct responses are 97.28% and 96.64%, respectively.
-
The paper provides useful implementation and feedback analyses. The algorithm, prompts, feedback extraction, and training settings make the intended procedure reasonably concrete. Table 3 also provides an informative finding: retaining the repair suggestion without the explicit error index preserves almost the full gain, whereas retaining the index alone provides essentially no net improvement. This helps identify which feedback content the trained refiner actually uses.
Critical weaknesses
-
The conceptual contribution appears limited, and the incremental methodological advance is not sufficiently established. The paper's own related-work discussion describes feedback-guided correction and critics trained using downstream refinement outcomes. VaCrit combines these ideas with reference-conditioned feedback generation, variational regularization, and diagnostic distillation. The resulting training procedure is concrete, but the manuscript does not clearly establish what additional capability requires this particular formulation. Representing an intermediate critique as a latent variable and deriving a variational objective does not, by itself, demonstrate a substantial advance over existing critique-refinement training. The central question is whether the proposed objective provides a meaningful benefit over simpler ways of training the same diagnosis-and-correction pipeline; the current evidence does not resolve this question.
-
The ablations do not isolate the components that constitute the main claimed contribution. Table 3 studies critic initialization and inference-time feedback fields. It does not isolate the reference-conditioned posterior, posterior-prior regularization, diagnostic self-distillation, or the advantage of jointly updating the posterior and refiner. Thus, the results support the effectiveness of the complete pipeline, but do not establish whether its gains require the proposed variational machinery or could be obtained with a simpler reference-assisted SFT or critique-refinement RL procedure. The error-index ablation further suggests that the explicit localization field adds little beyond the suggestion, although this does not imply that localization training is unnecessary.
-
The connection between joint training and the diagnostic behavior actually deployed at inference remains insufficiently demonstrated. During joint training, correction outcomes directly optimize the reference-conditioned posterior , whereas the deployed model learns its reference-free diagnostic behavior through distillation from the frozen prior . The deployed diagnosis policy is not directly rewarded according to the usefulness of its own sampled critiques. Parameter sharing may still improve its diagnostic behavior, but this needs evidence. In particular, the ProcessBench results in Table 5 evaluate the prior before and after meta-utility training, rather than the final unified model after joint optimization. They therefore do not directly establish diagnostic improvement or retention through the joint-training stage.
-
The relationship between the stated ELBO and the implemented objective needs clarification. Equation (5) contains for the deployed diagnosis distribution, while Equation (9) replaces this with and a separate distillation penalty . These terms are not interchangeable. In the stated special case , Equation (9) is naturally an ELBO for the mixture using the frozen prior, , rather than the deployed mixture, . The remark that Equation (9) “recovers the standard ELBO” should specify which marginal is bounded and explain its connection to the objective in Equation (1). The additional distillation penalty alone does not establish that the resulting objective bounds the deployed model's success probability.
-
The computational cost of the improvements is not adequately characterized. Appendix D.2 specifies data alignment and decoding limits, but the paper does not provide an accuracy-versus-generated-tokens or comparable total-budget evaluation. VaCrit adds a diagnostic generation with thinking enabled, while initial generation and refinement use thinking-disabled modes. Training also involves multiple sampled critiques and refinements. Without reporting actual training and inference costs, it is difficult to assess how much benefit the proposed training scheme provides relative to simpler methods with a comparable computation budget.
Questions for authors
-
Incremental contribution: What specific benefit does the proposed variational objective provide over a critique-refinement pipeline trained with filtered supervised trajectories or direct outcome-based RL? Can this benefit be demonstrated while controlling for backbone, training problems, access to reference solutions and localization labels, and computation budget?
-
Core component ablations: What happens when reference conditioning is removed from the posterior, posterior-prior regularization is removed, or diagnostic distillation is removed? A direct critique-refinement RL baseline would also help establish the need for the variational formulation.
-
Diagnostic capability at deployment: Can the authors evaluate the final reference-free model on ProcessBench and on its own generated responses, and compare it with the frozen critic? How large is the correction-performance gap when the refiner receives critiques from the posterior, the frozen prior, or the final model under their respective available inputs?
-
Meaning of the variational bound: After replacing the diagnosis prior in Equation (5) by the frozen critic in Equation (9), what probability does the claimed ELBO bound? What assumptions or additional derivation connect it to the deployed model's correction probability?
-
Efficiency: What are the measured training costs and average generated tokens per problem, broken down by generation, diagnosis, and refinement? Do the gains persist under a matched total inference budget?
7 comments
Response to Reviewer CYj3 - Part 2
[W2/Q2] The ablations do not isolate the components that constitute the main claimed contribution
Thank you for requesting more targeted ablations. We address the requested critique-refinement RL comparison in W1/Q1 through CURE and Joint correctness-RL, with the latter directly optimizing expected correctness under the same supervision and regularization. Here, we complement these comparisons with four training ablations: (i) w/o reference conditioning removes the reference solution from the posterior’s input while retaining posterior learning; (ii) Frozen posterior freezes the posterior at the common initialization while continuing to train the unified model; (iii) Sequential training first trains the posterior with the unified model fixed, then trains the unified model with the learned posterior fixed; and (iv) w/o diagnostic distillation sets while retaining the remaining objectives.
All variants in the table use Qwen3-4B, the same critic initialization, and the same training data. Results are unweighted averages across seven benchmarks, with each final model generating its own initial responses, reference-free critiques, and refinements.
| Method | Initial | Final | Gain | CS | CL |
|---|---|---|---|---|---|
| VaCrit (full) | 44.80 | 49.10 | +4.30 | 13.19 | 97.28 |
| w/o reference conditioning | 44.08 | 47.71 | +3.63 | 12.66 | 96.90 |
| Frozen posterior | 44.76 | 48.26 | +3.50 | 11.42 | 97.05 |
| Sequential training | 44.49 | 47.65 | +3.16 | 11.06 | 96.34 |
| w/o diagnostic distillation | 46.70 | 50.47 | +3.77 | 13.93 | 95.80 |
Reference conditioning and joint adaptation contribute beyond critic initialization. Removing reference conditioning weakens correction, consistent with references providing additional evidence for identifying errors and repair directions. Freezing the posterior reduces correction gain despite nearly identical initial accuracy, and sequential training also underperforms joint updates. Joint updates allow critique generation and refinement to adapt to one another, which can help keep feedback useful as the refiner changes.
Diagnostic distillation improves correctness preservation and net correction gain. Removing it yields higher final accuracy alongside stronger initial responses, but a smaller improvement through correction. Retaining distillation anchors the unified model’s diagnostic behavior to the learned critic during joint training, which can help limit harmful revisions of initially correct answers.
Response to Reviewer CYj3 - Part 3
[W3/Q3] The connection between joint training and the diagnostic behavior actually deployed at inference remains insufficiently demonstrated
Can the authors evaluate the final reference-free model on ProcessBench and on its own generated responses, and compare it with the frozen critic?
Thank you for highlighting the need for direct evidence on the deployed model. We evaluate the final unified model after joint training, examining diagnostic retention on ProcessBench and the correction utility of its own critiques. All evaluations below use Qwen3-4B.
First, we assess diagnostic retention on ProcessBench by comparing the final model with the frozen prior obtained after meta-utility training. Both receive no reference solutions and follow identical evaluation settings within each output format. Entries report F1 scores; Avg. is the unweighted mean across the four subsets.
| Format | Model | GSM8K | MATH | Olympiad | OmniMATH | Avg. |
|---|---|---|---|---|---|---|
| XML | Frozen prior | 84.89 | 84.23 | 79.58 | 74.45 | 80.79 |
| XML | Final model | 84.05 | 84.75 | 80.20 | 73.52 | 80.63 |
| Markdown | Frozen prior | 85.99 | 82.74 | 78.12 | 73.14 | 80.00 |
| Markdown | Final model | 85.69 | 83.95 | 76.85 | 73.47 | 79.99 |
As shown in the table above, the final model largely retains the frozen prior’s average diagnostic performance in both formats. This is consistent with initializing the unified model from the learned critic and maintaining diagnostic distillation during joint training, which penalizes departures from the critic’s diagnostic behavior as the shared model learns refinement.
How large is the correction-performance gap when the refiner receives critiques from the posterior, the frozen prior, or the final model under their respective available inputs?
To complement this diagnostic evaluation, we test whether the final model’s reference-free critiques provide useful feedback for correcting its own generated responses. We fix the initial responses generated by the final model, the final refiner, decoding settings, and generation budgets, changing only the critique source. The frozen prior and final model receive identical reference-free inputs. The posterior additionally receives the reference solution , which is unavailable at deployment.
Initial accuracy is fixed at 86.40% on MATH-500 and 54.15% on OlympiadBench. Entries report final accuracy, with correction gain in percentage points in parentheses; bold identifies the better reference-free result.
| Critique source | Reference solution | MATH-500 | OlympiadBench |
|---|---|---|---|
| Frozen prior | No | 89.00 (+2.60) | 58.46 (+4.31) |
| Final model | No | 90.00 (+3.60) | 59.05 (+4.90) |
| Posterior | Yes | 90.60 (+4.20) | 68.25 (+14.10) |
As shown in the table above, the final model’s own critiques outperform frozen-prior feedback under identical reference-free conditions. With initial responses and the refiner held fixed, this comparison directly assesses the usefulness of the supplied feedback to the deployed refiner. The frozen prior was optimized with a frozen base-model refiner, whereas the unified model’s shared parameters continue to adapt during joint training, potentially making its own critiques better suited to the final refiner. With access to reference solutions unavailable at deployment, posterior feedback exceeds self-feedback by 0.60 and 9.20 percentage points on MATH-500 and OlympiadBench, respectively.
Response to Reviewer CYj3 - Part 1
[W1/Q1] The conceptual contribution appears limited, and the incremental methodological advance is not sufficiently established
Thank you for asking us to clarify the methodological benefit. VaCrit is designed to learn reference-informed critiques that adapt to the current refiner, while retaining reference-free diagnosis in the deployed model. Our reference-conditioned posterior learns which critiques induce successful refinements as the refiner changes, while the learned reference-free critic regularizes the posterior and anchors the unified model’s diagnostic behavior. The intended benefit is more effective error repair while preserving already-correct responses.
We evaluate this benefit through an existing critique-guided RL baseline and a controlled objective comparison. (i) CURE, included in our main experiments, provides a comparison with an established critique-guided RL approach that jointly trains critique generation and guided solving using downstream outcomes. (ii) Joint correctness-RL replaces the expected log-success objective Eqϕ[logsθ] with expected correctness Eqϕ[sθ] in both posterior and unified-model updates, where sθ denotes the probability of a correct refinement given a critique. This control retains reference conditioning, joint updates, and both KL terms. In this way, the control implements joint outcome-based RL under the same supervision and regularization, allowing us to assess whether the variational log-success objective provides an additional benefit. VaCrit and Joint correctness-RL share the same training problems, initialization, access to reference solutions and localization supervision, optimization settings, and rollout counts (K=M=3), with β1=β2=0.1.
All three methods use Qwen3-4B and the same training data. Results are unweighted averages across seven benchmarks, with each model generating its own initial responses and performing correction without reference solutions.
VaCrit’s stronger correctness preservation than CURE allows successful repairs to translate into a positive net correction gain. This is consistent with diagnostic distillation anchoring the unified model’s critique behavior to the learned critic during joint training, which can help limit harmful revisions of correct answers as the refiner adapts.
Against Joint correctness-RL, VaCrit improves error repair at comparable correctness preservation. For a given critique, log-success assigns greater value to the same increase in correction probability when that probability is low, encouraging improvement on less successful cases. This provides a rationale consistent with the stronger error repair observed here. The complementary roles of reference conditioning, posterior adaptation, joint updates, and diagnostic distillation are examined in W2/Q2.