- Rating
- 2
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper proposes Audit-First VAPO, which uses confidence and agreement signals to decide whether to accept a grader’s feedback, ask another grader, or skip a training update. It chooses decision thresholds on separate labelled examples to limit incorrect feedback, though how the main experiments produce the input scores is unclear. On GSM8K, it reports 74.1% answer accuracy while accepting 41% of proposed updates, about 7% of which use incorrect feedback.
Critical strengths
- Checking whether feedback is trustworthy before deciding how strongly to train on it is a useful idea. Making an update smaller does not fix feedback that rewards the wrong answer.
- The comparisons help test whether the method chooses better feedback. Random selection uses a similar number and size of updates, while comparisons with other second-grader strategies help assess whether selectively asking for another opinion is worth the cost.
Critical weaknesses
-
It is unclear how the main experiments calculate the scores used to accept or reject feedback. The appendix gives a formula for an older simulated setup, but does not clearly explain how the actual graders produce confidence and other signals. Specify those inputs and how they are combined so the method can be reproduced.
-
The claimed error guarantee needs explaining. If SVAMP’s 200 calibration examples are the samples used in the formula, no updates should pass certification at the stated 8% target, yet the paper reports accepting 40%. Show the sample counts and calculations, and explain why a guarantee calculated on calibration examples also applies to later training updates.
Questions for authors
See weaknesses