- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper proposes PAVIF, a two-stage reinforcement learning framework for improving instruction following in large language models. In the first stage, the authors remove instruction constraints to generate responses with varying levels of compliance, then use teacher scores to train the policy’s verification ability. In the second stage, the same policy generates and evaluates responses, combining self-verification scores with programmatic rewards for GRPO training. Evaluations cover three Qwen models and six instruction-following benchmarks, alongside analyses of verification ability, general capabilities, and training efficiency. PAVIF improves over the corresponding base models overall, with the clearest gains on the 4B model, and reduces second-stage training time per step relative to a separate verifier of the same size.
Critical strengths
- The method is clear and practically useful. Training verification ability within the policy and reusing that policy to provide rewards is a workable training approach. The improvements across several benchmarks on the 4B model, together with lower per-step training time than a separate same-sized verifier, make the approach worth considering.
- The evaluation is broad and reports limitations. The experiments cover six instruction-following benchmarks, three model sizes, two model-based judges, and ablations of the training stages and reward components. The authors also evaluate general capabilities and explicitly report that generation optimization does not further improve verification accuracy. These results help establish the scope of the method.
Critical weaknesses
- The contribution over prior work remains unclear. Constraint-removal data construction is closely related to MuSC [1], while learning to evaluate responses and using self-generated feedback has precedents in Self-Rewarding Language Models [2]. PAVIF offers a specific combination of these ideas, but the ablations do not clearly establish the added value of verification pretraining and dynamic parameter sharing.
- The reliability and consistency of the benefits need stronger support. Most verification gains from Stage 1 disappear after Stage 2, yet the reliability of online self-scores is not directly examined. Improvements over VerIF are also small on the 1.7B and 8B models, with no repeated-run results to establish their stability.
[1] Huang, H., Liu, J., He, Y., Li, S., Xu, B., Zhu, C., Yang, M., & Zhao, T. (2025). MuSC: Improving complex instruction following with multi-granularity self-contrastive training. In Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) (pp. 10667–10686). [2] Yuan, W., Pang, R. Y., Cho, K., Li, X., Sukhbaatar, S., Xu, J., & Weston, J. E. (2024). Self-rewarding language models. In Proceedings of the 41st International Conference on Machine Learning, PMLR, 235, 57905–57923.
Questions for authors
- Please clarify PAVIF’s main methodological contribution relative to MuSC and Self-Rewarding Language Models. Are there existing comparisons showing that GRPO-based verification training provides additional benefits over verification SFT on the same data?
- Is the frozen verifier in Table 2 initialized from the base model or the Stage 1 checkpoint? Are the generation policy’s initialization, training data, and budget otherwise identical to those of full PAVIF? This distinction is important for interpreting the contribution of dynamic self-verification.
- Could existing checkpoints or training logs be used to examine agreement between Stage 2 self-scores and independent judgments—for example, the proportion of responses receiving full self-scores despite violating constraints, and how this changes during training? Results on soft constraints would be particularly informative.
- Are repeated training runs or other uncertainty analyses available to assess whether the smaller improvements over VerIF are consistent?
4 comments
W2 & Q3: Self-verification reliability and reward inflation during Stage 2
W2: The reliability and consistency of the benefits need stronger support. Most verification gains from Stage 1 disappear after Stage 2, yet the reliability of online self-scores is not directly examined. Improvements over VerIF are also small on the 1.7B and 8B models, with no repeated-run results to establish their stability.
Q3: Could existing checkpoints or training logs be used to examine agreement between Stage 2 self-scores and independent judgments—for example, the proportion of responses receiving full self-scores despite violating constraints, and how this changes during training? Results on soft constraints would be particularly informative.
Answer: Thank you for suggesting a direct assessment of scoring reliability. We examine this from two complementary perspectives.
(1) Verification ability on human-annotated data. We evaluated verification accuracy on a validation set of 300 responses annotated by humans with three-level scores (0, 0.5, and 1). As shown in Table B, verification accuracy was largely maintained after Stage 2 training.
(2) Agreement and over-scoring during training. On soft-only examples sampled from the Stage 2 training dataset, we compared each method's training verifier scores with judgments from the independent judge gpt-oss-120B. Although the frozen-reward variant showed higher agreement at some early checkpoints, PAVIF consistently achieved higher agreement and a lower over-scoring rate across the later checkpoints (Table B2). This supports better scoring reliability in later training, without establishing the absence of reward hacking or generalization to held-out data.
Table B: Verifier agreement and over-scoring during Stage 2.
\hline \textbf{Checkpoint} & \textbf{Agreement with human labels, n (\\%)} & \textbf{Overrate ratio, n (\\%)} \\\\ \hline \text{Stage 1 (step 0)} & \text{139 (46.3\\%)} & \text{95 (31.7\\%)} \\\\ \text{Stage 2, step 50} & \text{136 (45.3\\%)} & \text{100 (33.3\\%)} \\\\ \text{Stage 2, step 100} & \text{136 (45.3\\%)} & \text{108 (36.0\\%)} \\\\ \text{Stage 2, step 150} & \text{142 (47.3\\%)} & \text{104 (34.7\\%)} \\\\ \text{Stage 2, step 200} & \text{129 (43.0\\%)} & \text{112 (37.3\\%)} \\\\ \text{Stage 2, step 250} & \text{137 (45.7\\%)} & \text{104 (34.7\\%)} \\\\ \text{Stage 2, step 300} & \text{132 (44.0\\%)} & \text{111 (37.0\\%)} \\\\ \text{Stage 2, step 350} & \text{140 (46.7\\%)} & \text{109 (36.3\\%)} \\\\ \text{Stage 2, step 400} & \text{143 (47.7\\%)} & \text{105 (35.0\\%)} \\\\ \text{Stage 2, step 450} & \text{144 (48.0\\%)} & \text{109 (36.3\\%)} \\\\ \text{Stage 2, step 500} & \text{139 (46.3\\%)} & \text{108 (36.0\\%)} \\\\ \text{PAVIF (Ours), step 550 (final)} & \text{138 (46.0\\%)} & \text{100 (33.3\\%)} \\\\ \hline \end{array}$$ **Table B2: Agreement and over-scoring relative to gpt-oss-120B on soft-only training examples (all values in %).** $$\begin{array}{lcccc} \hline \textbf{Checkpoint} & \textbf{PAVIF agree.} & \textbf{PAVIF over-score} & \textbf{Frozen agree.} & \textbf{Frozen over-score} \\\\ \hline \text{Stage 2, step 50} & 73.6 & 22.0 & 73.3 & 22.6 \\\\ \text{Stage 2, step 100} & 74.5 & 21.4 & 78.5 & 19.4 \\\\ \text{Stage 2, step 150} & 75.1 & 21.9 & 78.0 & 17.6 \\\\ \text{Stage 2, step 200} & 75.6 & 21.4 & 76.9 & 17.9 \\\\ \text{Stage 2, step 250} & 73.2 & 23.7 & 74.3 & 22.6 \\\\ \text{Stage 2, step 300} & 74.0 & 23.0 & 71.3 & 23.6 \\\\ \text{Stage 2, step 350} & 72.4 & 23.6 & 69.2 & 25.7 \\\\ \text{Stage 2, step 400} & 73.7 & 22.9 & 71.0 & 25.6 \\\\ \text{Stage 2, step 450} & 74.7 & 22.0 & 71.0 & 24.8 \\\\ \text{Stage 2, step 500} & 76.3 & 20.3 & 68.9 & 26.4 \\\\ \text{PAVIF (Ours), step 550 (final)} & 75.4 & 20.8 & 72.1 & 22.1 \\\\ \hline \end{array}$$Q4: Consistency of performance gains over VerIF across repeated evaluations (1/2)
Q4: Are repeated training runs or other uncertainty analyses available to assess whether the smaller improvements over VerIF are consistent?
Answer: We thank the reviewer for highlighting the need to quantify variability in the comparison with VerIF. At each model size, we evaluated one checkpoint per method at temperature 0.7 using three decoding seeds (42, 233, and 1024). PAVIF achieves a higher six-benchmark average than VerIF in eight of the nine evaluations.
Table C: Instruction-following Performance (seed 42).
\hline \smash{\lower0.6em{\textbf{Model}}} & \kern3.3651em\rlap{\textbf{IFEval}}\kern-3.3651em & & & & \kern2.1957em\rlap{\textbf{Multi-IF}}\kern-2.1957em & & & \kern2.9191em\rlap{\textbf{IFBench}}\kern-2.9191em & & & & \kern-1.6030em\rlap{\textbf{FollowBench}}\kern1.6030em & & \kern0.7774em\rlap{\textbf{CFBench}}\kern-0.7774em & & & \textbf{InfoBench} & \smash{\lower0.6em{\textbf{Avg.}}} \\\\ & \textbf{Pr.S} & \textbf{Pr.L} & \textbf{Ins.S} & \textbf{Ins.L} & \textbf{Turn 1} & \textbf{Turn 2} & \textbf{Turn 3} & \textbf{Pr.S} & \textbf{Pr.L} & \textbf{Ins.S} & \textbf{Ins.L} & \textbf{HSR} & \textbf{SSR} & \textbf{CSR} & \textbf{ISR} & \textbf{PSR} & \textbf{Overall} & \\\\ \hline \rlap{\textit{Qwen3-1.7B}} & & & & & & & & & & & & & & & & & & \\\\ \text{Base} & 68.4 & 71.5 & 76.6 & 79.7 & 63.5 & 40.8 & 22.1 & 19.0 & 24.0 & 20.9 & 26.5 & 44.9 & 68.4 & 58 & 24 & 31 & 78.4 & 51.9 \\\\ \text{+ VerIF} & 73.2 & 76.2 & 80.8 & 82.9 & 66.2 & 42.7 & 25.9 & 18.3 & 23.0 & 20.3 & 25.6 & 50.3 & 74.2 & 59 & 25 & 33 & 79.0 & 54.2 \\\\ \text{+ PAVIF (Ours)} & 73.6 & 78.6 & 80.9 & 84.9 & 66.3 & 43.5 & 26.2 & 17.0 & 22.7 & 21.2 & 26.2 & 48.5 & 72.4 & 56 & 23 & 30 & 78.7 & 53.7 \\\\ \hline \rlap{\textit{Qwen3-4B-Instruct-2507}} & & & & & & & & & & & & & & & & & & \\\\ \text{Base} & 83.2 & 87.1 & 88.4 & 91.1 & 78.1 & 60.3 & 43.7 & 31.7 & 34.3 & 33.7 & 37.5 & 67.7 & 84.9 & 71 & 43 & 50 & 84.5 & 66.3 \\\\ \text{+ VerIF} & 87.8 & 90.0 & 91.5 & 93.0 & 82.0 & 63.0 & 46.6 & 31.7 & 34.7 & 34.6 & 38.4 & 68.9 & 85.7 & 71 & 42 & 49 & 86.4 & 67.8 \\\\ \text{+ PAVIF (Ours)} & 86.9 & 90.9 & 91.0 & 93.8 & 81.5 & 63.5 & 47.2 & 33.0 & 37.3 & 35.5 & 39.8 & 68.1 & 85.7 & 73 & 44 & 50 & 85.9 & 68.3 \\\\ \hline \rlap{\textit{Qwen3-8B}} & & & & & & & & & & & & & & & & & & \\\\ \text{Base} & 83.9 & 86.3 & 89.1 & 90.6 & 80.4 & 63.3 & 47.4 & 23.7 & 26.7 & 27.0 & 32.0 & 61.9 & 83.6 & 69 & 37 & 45 & 84.6 & 64.4 \\\\ \text{+ VerIF} & 85.4 & 87.6 & 89.9 & 91.5 & 80.0 & 61.6 & 46.4 & 23.0 & 29.3 & 27.0 & 33.4 & 65.3 & 84.3 & 70 & 38 & 46 & 83.9 & 64.9 \\\\ \text{+ PAVIF (Ours)} & 84.8 & 87.8 & 89.3 & 91.4 & 82.2 & 63.7 & 47.5 & 27.0 & 31.7 & 29.7 & 35.5 & 64.0 & 83.8 & 69 & 36 & 45 & 84.4 & 65.3 \\\\ \hline \end{array}$$Q4: Consistency of performance gains over VerIF across repeated evaluations (2/2)
**Q4 (continued).
Table D: Instruction-following Performance (seed 233).
\hline \smash{\lower0.6em{\textbf{Model}}} & \kern3.3651em\rlap{\textbf{IFEval}}\kern-3.3651em & & & & \kern2.1957em\rlap{\textbf{Multi-IF}}\kern-2.1957em & & & \kern2.9191em\rlap{\textbf{IFBench}}\kern-2.9191em & & & & \kern-1.6030em\rlap{\textbf{FollowBench}}\kern1.6030em & & \kern0.7774em\rlap{\textbf{CFBench}}\kern-0.7774em & & & \textbf{InfoBench} & \smash{\lower0.6em{\textbf{Avg.}}} \\\\ & \textbf{Pr.S} & \textbf{Pr.L} & \textbf{Ins.S} & \textbf{Ins.L} & \textbf{Turn 1} & \textbf{Turn 2} & \textbf{Turn 3} & \textbf{Pr.S} & \textbf{Pr.L} & \textbf{Ins.S} & \textbf{Ins.L} & \textbf{HSR} & \textbf{SSR} & \textbf{CSR} & \textbf{ISR} & \textbf{PSR} & \textbf{Overall} & \\\\ \hline \rlap{\textit{Qwen3-1.7B}} & & & & & & & & & & & & & & & & & & \\\\ \text{Base} & 69.7 & 72.3 & 77.7 & 80.1 & 64.1 & 41.0 & 22.5 & 19.3 & 22.0 & 20.6 & 25.0 & 48.3 & 70.7 & 58 & 24 & 31 & 76.9 & 52.2 \\\\ \text{+ VerIF} & 71.0 & 75.2 & 79.5 & 82.9 & 66.7 & 44.1 & 25.3 & 16.3 & 20.3 & 18.6 & 24.1 & 53.0 & 74.5 & 60 & 27 & 34 & 78.6 & 54.1 \\\\ \text{+ PAVIF (Ours)} & 74.3 & 78.4 & 81.5 & 84.8 & 65.8 & 42.5 & 26.0 & 18.7 & 25.3 & 22.4 & 28.2 & 50.0 & 72.9 & 57 & 25 & 32 & 78.3 & 54.3 \\\\ \hline \rlap{\textit{Qwen3-4B-Instruct-2507}} & & & & & & & & & & & & & & & & & & \\\\ \text{Base} & 84.3 & 88.4 & 89.2 & 91.8 & 78.1 & 59.6 & 44.7 & 30.3 & 34.0 & 33.7 & 38.4 & 66.2 & 84.2 & 72 & 44 & 50 & 84.6 & 66.4 \\\\ \text{+ VerIF} & 86.1 & 88.5 & 90.5 & 92.1 & 82.3 & 63.3 & 47.5 & 31.7 & 34.7 & 34.3 & 38.1 & 66.8 & 85.3 & 72 & 43 & 50 & 87.2 & 67.8 \\\\ \text{+ PAVIF (Ours)} & 86.1 & 90.8 & 90.8 & 93.8 & 81.4 & 63.4 & 47.8 & 31.7 & 36.3 & 34.6 & 39.2 & 70.4 & 87.5 & 73 & 44 & 51 & 86.2 & 68.5 \\\\ \hline \rlap{\textit{Qwen3-8B}} & & & & & & & & & & & & & & & & & & \\\\ \text{Base} & 84.5 & 86.7 & 89.3 & 90.9 & 80.5 & 63.4 & 47.4 & 24.7 & 30.0 & 28.5 & 34.0 & 64.3 & 84.2 & 69 & 38 & 46 & 84.4 & 65.1 \\\\ \text{+ VerIF} & 85.2 & 87.4 & 89.9 & 91.5 & 79.1 & 62.0 & 46.4 & 23.7 & 29.0 & 27.3 & 32.8 & 63.4 & 84.4 & 69 & 38 & 46 & 84.9 & 64.8 \\\\ \text{+ PAVIF (Ours)} & 85.8 & 88.5 & 90.3 & 92.2 & 82.2 & 63.7 & 48.5 & 26.0 & 31.3 & 29.4 & 35.2 & 60.9 & 82.0 & 70 & 38 & 46 & 84.7 & 65.3 \\\\ \hline \end{array}$$ **Table E: Instruction-following Performance (seed 1024).** $$\begin{array}{lcccccccccccccccccc} \hline \smash{\lower0.6em{\textbf{Model}}} & \kern3.3651em\rlap{\textbf{IFEval}}\kern-3.3651em & & & & \kern2.1957em\rlap{\textbf{Multi-IF}}\kern-2.1957em & & & \kern2.9191em\rlap{\textbf{IFBench}}\kern-2.9191em & & & & \kern-1.6030em\rlap{\textbf{FollowBench}}\kern1.6030em & & \kern0.7774em\rlap{\textbf{CFBench}}\kern-0.7774em & & & \textbf{InfoBench} & \smash{\lower0.6em{\textbf{Avg.}}} \\\\ & \textbf{Pr.S} & \textbf{Pr.L} & \textbf{Ins.S} & \textbf{Ins.L} & \textbf{Turn 1} & \textbf{Turn 2} & \textbf{Turn 3} & \textbf{Pr.S} & \textbf{Pr.L} & \textbf{Ins.S} & \textbf{Ins.L} & \textbf{HSR} & \textbf{SSR} & \textbf{CSR} & \textbf{ISR} & \textbf{PSR} & \textbf{Overall} & \\\\ \hline \rlap{\textit{Qwen3-1.7B}} & & & & & & & & & & & & & & & & & & \\\\ \text{Base} & 68.9 & 72.5 & 77.0 & 80.0 & 63.5 & 40.3 & 21.8 & 18.7 & 24.0 & 21.8 & 28.2 & 47.7 & 70.6 & 57 & 24 & 30 & 78.7 & 52.4 \\\\ \text{+ VerIF} & 71.5 & 75.0 & 78.9 & 82.1 & 66.3 & 43.2 & 25.5 & 19.7 & 24.3 & 23.3 & 27.9 & 50.3 & 72.4 & 58 & 26 & 34 & 79.0 & 54.2 \\\\ \text{+ PAVIF (Ours)} & 75.4 & 79.9 & 81.7 & 85.1 & 65.7 & 43.4 & 26.1 & 21.0 & 25.0 & 23.0 & 28.8 & 48.1 & 71.4 & 57 & 24 & 30 & 78.9 & 54.3 \\\\ \hline \rlap{\textit{Qwen3-4B-Instruct-2507}} & & & & & & & & & & & & & & & & & & \\\\ \text{Base} & 83.5 & 87.6 & 88.7 & 91.5 & 78.4 & 60.0 & 44.3 & 31.0 & 33.0 & 33.7 & 36.3 & 65.5 & 84.7 & 71 & 42 & 49 & 86.2 & 66.2 \\\\ \text{+ VerIF} & 86.3 & 88.4 & 90.4 & 91.7 & 82.1 & 62.7 & 46.6 & 31.3 & 35.0 & 33.7 & 37.8 & 66.4 & 84.3 & 72 & 42 & 50 & 88.3 & 67.6 \\\\ \text{+ PAVIF (Ours)} & 86.5 & 89.6 & 90.9 & 92.9 & 81.6 & 62.7 & 46.4 & 32.7 & 38.7 & 34.9 & 41.3 & 69.2 & 86.7 & 74 & 42 & 51 & 85.6 & 68.3 \\\\ \hline \rlap{\textit{Qwen3-8B}} & & & & & & & & & & & & & & & & & & \\\\ \text{Base} & 83.5 & 85.8 & 88.5 & 90.0 & 80.0 & 62.7 & 47.4 & 25.3 & 29.3 & 27.6 & 32.0 & 63.0 & 82.8 & 69 & 37 & 44 & 83.7 & 64.3 \\\\ \text{+ VerIF} & 84.5 & 86.5 & 89.1 & 90.6 & 79.8 & 61.8 & 46.7 & 24.3 & 28.7 & 27.3 & 32.6 & 65.5 & 85.1 & 69 & 38 & 47 & 83.6 & 64.8 \\\\ \text{+ PAVIF (Ours)} & 83.9 & 87.1 & 89.1 & 91.2 & 82.3 & 63.8 & 47.9 & 25.3 & 30.0 & 28.5 & 33.7 & 64.7 & 84.8 & 70 & 39 & 47 & 84.8 & 65.6 \\\\ \hline \end{array}$$
W1 & Q1–Q2: Methodological distinctions and ablations of verification training and dynamic self-verification
W1: The contribution over prior work remains unclear. Constraint-removal data construction is closely related to MuSC [1], while learning to evaluate responses and using self-generated feedback has precedents in Self-Rewarding Language Models [2]. PAVIF offers a specific combination of these ideas, but the ablations do not clearly establish the added value of verification pretraining and dynamic parameter sharing.
Q1: Please clarify PAVIF’s main methodological contribution relative to MuSC and Self-Rewarding Language Models. Are there existing comparisons showing that GRPO-based verification training provides additional benefits over verification SFT on the same data?
Answer: MuSC [1] and Self-Rewarding Language Models [2] both use preference optimization, while PAVIF trains the policy through reinforcement learning using its own verification feedback. MuSC uses constraint removal to construct preference pairs, treating the response to the full instruction as chosen and the response to the reduced instruction as rejected. However, a response may still satisfy an omitted constraint. We therefore assess each response against the full instruction. We generate responses under three levels of constraint coverage (none, partial, and full) to encourage a more balanced distribution across the three verification scores (0, 0.5, and 1).
In Self-Rewarding Language Models [2], the model scores its own responses to construct preference pairs for the next round of DPO. In PAVIF, the same policy generates and verifies responses during reinforcement learning. Under this dynamic self-verification mechanism, the policy’s own verification scores serve as a reward for instruction following. In our ablation study, dynamic self-verification yielded a higher average instruction-following score than using a frozen verifier initialized from the same Stage 1 checkpoint (69.05 vs. 67.93, Table 2).
Table A compares GRPO (PAVIF) with verification SFT (PAVIF+) on Qwen3-4B-Instruct-2507. Both models used the same Stage 1 examples and Stage 2 procedure. PAVIF averaged 69.3 across the six benchmarks, compared with 65.0 for PAVIF+. On InfoBench, PAVIF scored 87.8% against 85.1% for PAVIF+. We will include this comparison in the appendix of the revised manuscript. Owing to potential variation in model judgments across deployments, we reran all evaluations, using gpt-oss-120B as the judge for FollowBench, CFBench, and InfoBench.
Table A: GRPO vs. SFT for verification training.
\hline \smash{\lower0.6em{\textbf{Model}}} & \kern-0.0785em\rlap{\textbf{IFEval}}\kern0.0785em & & \kern2.1957em\rlap{\textbf{Multi-IF}}\kern-2.1957em & & & \kern-0.5245em\rlap{\textbf{IFBench}}\kern0.5245em & & \kern-1.6030em\rlap{\textbf{FollowBench}}\kern1.6030em & & \textbf{CFBench} & \textbf{InfoBench} & \smash{\lower0.6em{\textbf{Avg.}}} \\\\ & \textbf{Pr.S} & \textbf{Pr.L} & \textbf{Turn 1} & \textbf{Turn 2} & \textbf{Turn 3} & \textbf{Pr.S} & \textbf{Pr.L} & \textbf{HSR} & \textbf{SSR} & \textbf{ISR} & \textbf{Overall} & \\\\ \hline \rlap{\textit{Qwen3-4B-Instruct-2507}} & & & & & & & & & & & & \\\\ \text{Base} & 83.5 & 87.6 & 82.1 & 72.1 & 62.9 & 28.3 & 31.0 & 62.6 & 74.6 & 45 & 86.4 & 64.6 \\\\ \text{PAVIF+} & 86.7 & 89.8 & 86.3 & 73.1 & 59.6 & 28.3 & 31.3 & 65.7 & 76.4 & 43 & 85.1 & 65.0 \\\\ \text{PAVIF (Ours)} & 88.4 & 91.7 & 84.3 & 74.7 & 65.3 & 31.7 & 35.3 & 72.9 & 86.1 & 50 & 87.8 & 69.3 \\\\ \hline \end{array}$$ **Q2: Is the frozen verifier in Table 2 initialized from the base model or the Stage 1 checkpoint? Are the generation policy’s initialization, training data, and budget otherwise identical to those of full PAVIF? This distinction is important for interpreting the contribution of dynamic self-verification.** **Answer:** Yes. We initialized both the policy and the verifier from the same Stage 1 checkpoint used in PAVIF. We kept the verifier frozen throughout Stage 2 and held all other training settings fixed, including the training data and number of policy updates. We have clarified this setup in Section 5.4 of the revised manuscript. [1] Huang, H., Liu, J., He, Y., Li, S., Xu, B., Zhu, C., Yang, M., & Zhao, T. (2025). MuSC: Improving complex instruction following with multi-granularity self-contrastive training. In Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) (pp. 10667–10686). [2] Yuan, W., Pang, R. Y., Cho, K., Li, X., Sukhbaatar, S., Xu, J., & Weston, J. E. (2024). Self-rewarding language models. In Proceedings of the 41st International Conference on Machine Learning, PMLR, 235, 57905–57923.