- Rating
- 2
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper studies multi-constraint instruction following and identifies partial satisfaction as a primary failure mode. While models maintain high per-constraint satisfaction, their all-satisfaction rate drops substantially as the number of constraints increases. The paper proposes Constraint-Decoupled On-Policy Distillation (CD-OPD), which decomposes an instruction into single-constraint prompts, estimates the token-level effect of each constraint against an unconstrained prompt, and combines these signals using soft min-pooling. The resulting correction is added to a full-instruction teacher distribution and distilled into the student on its own trajectories. Experiments on three Qwen3 teacher–student pairs and four instruction-following benchmarks show consistent improvements over standard OPD and reduced performance decay as the number of constraints increases.
Critical strengths
- CD-OPD offers a clear and well-motivated approach to multi-constraint instruction following. By retaining constraint-level information through decomposition and min-pooling, it directly addresses the partial-satisfaction problem that scalar RLVR and standard OPD struggle with.
- The experiments provide useful insights beyond the main performance gains. In particular, the comparisons show the different behaviors of code-based and LLM-based rewards, how standard OPD inherits the teacher’s compositional decay, and how CD-OPD changes performance as the number of constraints increases. The ablations and efficiency analysis further help explain where the gains come from.
Critical weaknesses
-
The presentation of the core method contains several inconsistencies that make it difficult to verify exactly what was implemented. The main text defines the contrastive signal using raw logit differences and describes zero-mean, unit-variance normalization, while Appendix E uses log-probability differences and uncentered RMS normalization. The claim that min-pooling “guarantees” strict conjunction is also not formally established, and the reported teacher forward passes appear inconsistent with the views required by the formulation. These issues suggest that the paper has not been sufficiently proofread and weaken confidence in the technical claims and efficiency analysis.
-
Although the experiments show consistent improvements, they are not broad enough to support the paper’s general claims. All three teacher–student settings use models from the Qwen3 family and share the same training dataset. The evaluation also does not establish whether the method remains effective or computationally practical when the number of constraints grows beyond the tested range. It is therefore unclear how well the findings generalize across model families, training data, and more complex multi-constraint settings.
Questions for authors
-
I may be missing something here. The main text uses raw logit differences with zero-mean normalization, while Appendix E uses log-probability differences with uncentered RMS normalization. Since the latter is not shift-invariant, why does the partition-function offset have no effect? Which formulation was used for the reported results?
-
The paper claims that min-pooling “guarantees” strict conjunction, but it only operates on next-token distributions. Does this guarantee extend to sequence-level constraints? If not, the claim should be stated more cautiously.
-
I may have misunderstood the computation count, but the method appears to require teacher views, while the paper reports . Is one distribution reused or omitted, and are all teacher computations included in the reported training time?
-
All teacher–student settings use Qwen3 models, and constraint counts are tested only within a limited range. Is there evidence that CD-OPD generalizes to other model families or scales to more constraints? Otherwise, the generality claims should be narrowed.
Clear answers to these questions, particularly the first three, would materially affect my assessment. I would consider updating my score if these concerns are satisfactorily resolved.