- Rating
- 1
- Confidence
- 3
- Informativeness
- 1
- Presentation
- 1
- Soundness
- 2
Read review
Paper summary
This paper proposed GAD-RL to address a problem where VLM models always unconsciously correct the original wrong words to plausible right words
Critical strengths
- idea is straight-forward and the motivation is reasonable
- experiment design is good
Critical weaknesses
- If teacher model needs to take a glance of GT, why not just using GT to build token guidance (no teacher model is needed)
- no code; I highly recommend authors provide a small demo that at least can match some numbers described in the paper; I'll run it
- The “Teacher errors” values in Table 5 are exactly the same in the “Incompatible” column of Table 9; I think authors need to explain this
- Section 5.2said the fixed-weight OPD also uses SA-FKL; it gets a score of 0.5549. In Table 4, the “w/o RAA” entry (i.e., gating + SA-FKL only) achieved a score of 0.5414, which is lower than that of the fixed-weight method; Can authors explain this?
- The SNR dropped from 12.57 to 0.114, mainly because erroneous tokens have almost disappeared, not because the teacher has gotten worse. This is the data on which the core motivation is based, and it has been exaggerated
Questions for authors
- If teacher model needs to take a glance of GT, why not just using GT to build token guidance (no teacher model is needed)
- no code; I highly recommend authors provide a small demo that at least can match some numbers described in the paper; I'll run it
- The “Teacher errors” values in Table 5 are exactly the same in the “Incompatible” column of Table 9; I think authors need to explain this
- Section 5.2said the fixed-weight OPD also uses SA-FKL; it gets a score of 0.5549. In Table 4, the “w/o RAA” entry (i.e., gating + SA-FKL only) achieved a score of 0.5414, which is lower than that of the fixed-weight method; Can authors explain this?
- The SNR dropped from 12.57 to 0.114, mainly because erroneous tokens have almost disappeared, not because the teacher has gotten worse. This is the data on which the core motivation is based, and it has been exaggerated
- “Harmful signals” may be a false impression caused by tokenization drift. If this explanation is true, the correct adjustment would be to align the segmentation of teachers and students, rather than reducing the number of teacher tokens
Reply 5:
We thank the reviewer for highlighting the role of changes in error proportions. However, the teacher model remains frozen throughout training. Therefore, our motivation is not based on the assumption that the teacher’s own capability deteriorates, but rather on understanding how the effectiveness of its supervision changes as the student’s generation trajectories evolve. We agree that the overall B/H ratio depends not only on the proportions of correct and incorrect tokens in the student’s outputs, but also on the teacher’s conditional supervision behavior at correct and incorrect positions. To disentangle these factors, we conducted a two-factor reweighting analysis. Specifically, when replacing the proportions of correct and incorrect tokens in the Base model with those observed at training step 300, while keeping the within-category supervision-signal rates of the Base model unchanged, the B/H ratio decreases from 12.57 to 1.656, rather than the actually observed value of 0.114. This reweighting shows that the changing error proportion alone does not reproduce the observed B/H ratio; the correctness-conditioned directional signal rates also change as the student evolves.
Reply 6:
We thank the reviewer for raising this concern. Both the student and teacher models used in this analysis are based on Qwen3.5-2B. We have verified that all response token IDs involved in the evaluation decode to exactly the same content under both models’ tokenizers. During teacher scoring, we directly concatenate the original response token IDs generated by the student to the tokenized privileged prompt, without first decoding the response into text and then re-tokenizing it. Therefore, both models score the same actual generated token IDs under identical response token prefixes. The changes in token counts discussed in Section 5.1 refer to differences in the outputs generated by the student across training checkpoints, rather than inconsistencies in tokenization between the teacher and student when scoring the same response. GAD-RL does not modify the tokenization scheme or reduce the number of tokens required to represent a given response. Instead, it achieves adaptive distillation control by regulating whether teacher supervision is applied at student-generated token positions and how strongly that supervision is applied. Our compatibility analysis in Table 9 explicitly allows alternative tokenizations: a teacher Top-1 candidate is classified as GT-compatible if its decoded content matches a prefix of the GT continuation. Thus, a different tokenization alone is not counted as a teacher content error.