- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 1
- Soundness
- 2
Read review
Paper summary
The paper studies bias propagation in weak-to-strong generalization (W2SG) rather than capability recovery alone. It constructs biased and clean Qwen2.5-7B teachers, trains matched Qwen2.5-32B students on their pseudo-labels, and evaluates three sycophancy-related behaviors such as factual errors under incorrect assertions, persona-conditioned opinion agreement, and answer changes under pressure. It introduces BAF and nBAF, as well as a factual teacher student pathway decomposition.
Critical strengths
Important safety question: The paper shifts W2SG evaluation from aggregate task capability toward whether structured undesirable behaviors in weak supervision affect the stronger student.
Careful reporting: The authors explicitly separate absolute rates, signed teacher-student gaps, BAF, and nBAF, and correctly note that BAF alone can be misleading.
Useful factual audit: The exact count decomposition distinguishes same-error, different-error, teacher-only, and student-only cases rather than relying only on aggregate rates.
Critical weaknesses
- The headline conclusions rely on one training run per condition.
All main Qwen results come from a single training run, and the reported bootstrap intervals capture evaluation item uncertainty rather than initialization, data order, or optimization variability. This is especially important for the opinion result, where the claimed sign reversal is considerably smaller than the factual and pressure effects.
The additional LLaMA experiment does not constitute a successful replication, since the factual teacher manipulation reverses direction and the other teacher contrasts are weak. A multi seed study and at least one successful second model family replication would substantially strengthen the conclusions.
- Positioning with respect to literature must be strengthened.
The paper’s W2SG discussion is somewhat narrow given recent work on the reliability and trustworthiness of weak supervision. For example, "Trust Functions" (https://arxiv.org/abs/2606.01000) studies learning when weak labels should be trusted and filtering unreliable supervision, and I am curious whether their method can be extended for trusting the teacher in terms of desirable behavior in addition to capability. As another example, "Generalizing Trust" (https://arxiv.org/abs/2501.00418) studies whether desirable properties such as fairness, robustness, and privacy transfer from weak to strong models, and I think this directly overlaps with this work. Moreover, "How to Mitigate Overfitting in Weak-to-Strong Generalization?" studies how weak-label errors can induce overfitting and explores supervision filtering, which also seems relevant.
Inclusion and discussion of these works would sharpen the positioning.
- BAF is somewhat misleading.
BAF is the ratio between the two student-condition rates, not a direct measure of teacher-to-student amplification. The paper itself acknowledges this. For example, pressure has BAF = 2.84 despite the student biased - clean gap being smaller than the teacher gap (nBAF = 0.79).
Signed gaps or nBAF seem more directly aligned with the paper’s propagation question and could arguably be the primary quantities, with BAF treated as secondary.
Questions for authors
I am open to increasing my score conditioned on satisfactory addressing of my concerns above. Below are related questions:
- Do the factual, opinion, and pressure contrasts remain stable across multiple independent training seeds?
- How would trust-based weak-label selection, such as Trust Functions, affect the biased supervision used here?
- Would the conclusions change if the main quantity were the signed teacher-to-student gap change rather than BAF?