- Rating
- 2
- Confidence
- 2
- Informativeness
- 1
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
The paper asks how difficult it is to measure changes in true performance after optimizing a proxy score. It identifies two obstacles: old data may rarely contain the outputs that become common after optimization, and monitoring features may miss changes relevant to the target. It formalizes these obstacles, derives sample and representation requirements under specified assumptions, and studies them using reweighted language-model outputs and GRPO checkpoints.
Critical strengths
-
The paper distinguishes limited access to new labels from limitations of the monitoring features. This helps clarify two different reasons why an audit can fail.
-
The experiments include controlled distribution shifts and actual GRPO checkpoints. The authors also report several limitations and negative results, including substantial differences between the controlled setting and trained policies.
Critical weaknesses
-
The likelihood-ratio measurements are inconsistent. New-policy and baseline samples use different tokenization and scoring procedures. This weakens the quantitative overlap conclusions in Figure 4.
-
The main auditing experiments are disconnected from actual trained policies. They evaluate an estimator on artificially reweighted distributions, while the GRPO results show substantial departures from this distribution family. The estimator is not directly evaluated on those checkpoints.
-
E1 does not separate estimator limitations from intrinsic difficulty. It tests one binning estimator without comparisons against standard importance-sampling alternatives. Its empirical sample requirements therefore do not establish that other estimators would struggle similarly, even though the paper provides separate lower bounds under specific assumptions.
-
The monitor experiments do not directly establish failure to estimate correctness. They primarily measure reconstruction of the distribution shift, which is a different objective. The stronger linear certificates also use weights that are not renormalized on the evaluated subset, limiting their interpretation as hidden changes in target performance.
-
The practical importance of the restricted-label setting is insufficiently demonstrated. The mathematical benchmarks allow direct correctness checks. They provide a controlled test, but do not show when obtaining new labels becomes the main obstacle in a realistic application.
-
The added value over existing overlap and importance-sampling results needs clearer justification. The paper establishes additional results under specific assumptions, but the experiments do not clearly demonstrate what these results explain beyond established coverage difficulties.
Questions for authors
Please see weaknesses above.