acceptodds
Under review as a conference paper at ICLR 2027

Curse of Corpus: Do Models Reason Beyond Counting And Composition?

Abstract

Process reward models are often evaluated by pooling all labeled steps into one AUROC. We study this practice under the absorbing-label convention, which labels the first incorrect step and every later step as incorrect. Pooled AUROC then mixes two nuisances: comparisons across heterogeneous sources and an association between position and incorrectness. Both are measurable exactly, because a rank statistic is a sum over positive-negative pairs: on our held-out set only 0.59% of the 371,765,485 pairs compare steps of one source at one index, and 93.84% cross sources. A scorer that reads no text and knows only the corpus therefore reaches pooled absorbing AUROC 0.9140, against 0.9151 for a fine-tuned 31B verifier. Across five trained verifiers, held-out raw pooled AUROC exceeds mean-macro AUROC by 0.0963-0.1244. Across seven corpora, a text-blind relative-position baseline obtains absorbing-label pooled AUROC from 0.6370 to 0.9768; imposing absorption raises it by as much as 0.3124 without changing text or scores. We evaluate first-error detection with , which excludes post-error suffixes and compares errors with correct steps at the same index. Absolute index alone therefore receives exactly 0.5, mean-macro and weighted-macro variants also exclude cross-source comparisons, and adding length strata returns relative position to 0.5. The estimands then disagree about direction and not only magnitude: averaged over the first and last five checkpoints of a matched 2,300-step run, every ranking estimand rises with an interval excluding zero, pooled AUROC by 0.0708 and weighted-macro by 0.0363, while accuracy at naming the step that first broke falls by 0.0335, interval . In a forced-verdict diagnostic, pooled raw AUROC places Qwen 0.0847 above Gemma whereas source-aware first-error intervals include zero; conversely weighted-macro favors Muse over GLM by 0.0597 with interval while pooled raw AUROC is inconclusive.

Primary area:
foundation or frontier models, including LLMs
Keywords:
composition reasoning, process reward models, incremental prediction, fine tuning, domain verifiers, cross domain reasoning

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_4pwfOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper studies the practice of process reward model evaluation through pooled AUROC. It argues that the pooled AUROC can be high even if verifier never reads the reasoning. dataset source leakage and position leakage are enough to get a high score. To address this, the authors propose A_hazard which only looks at step up to the first error and only compares steps from the same source at the same index, then validated this method on 5 models, 5 fine-tuned verifiers and 3 public PRMs.

Critical strengths
  1. The pair decomposition and text-blind baseline results in Eq.7 and Table 15 provide a simple, cost-effective sanity check to audit position and source shortcuts for PRMs.
  2. The estimand ablation study in table 3 and table 14 is solid. The ablation applies each restriction separately and each shortcut is attributed to one restriction that removes it.
Critical weaknesses
  1. The paper lacks more direct and convincing evidence that A_hazard is a better evaluation metric than pooled AUROC, from the results shown in section 5.4 and table 17, the advantage of A_hazard over pooled raw AUROC is marginal.
  2. The core critique of multi-source pooled AUROC relies on a non-standard setup, it would be better if there's a more comprehensive comparison with more published PRM evaluation baselines, the only citation for this setup is Xia et al. (2025)
Questions for authors
  1. Could you give one concrete scenario where A_hazard leads to a better choice than pooled AUROC, supported by downstream outcomes such as checkpoint selection, or best-of-N? Neither section 5.4 nor Table 17 shows it.
  2. Could you provide more evidence of published papers that pool heterogeneous sources into unstratified AUROC? If this practice is rare, the contribution of the main finding in this paper drops significantly.
Reviewer_mDinOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
1
Soundness
1
Read review
Paper summary

The paper argues that pooled step-level AUROC, widely used to evaluate process reward models (PRMs), is dominated by two shortcuts on multi-source evaluation sets: source identity and step position. The authors decompose pooled AUROC by pair type and show that on their held-out set, 93.84% of positive–negative step pairs are cross-source and only 0.59% are same-source same-index. They construct text-blind controls (absolute index, relative position, source prior) and report that a source-prior control reaches pooled absorbing AUROC 0.9228, above a fine-tuned 31B Gemma verifier at 0.9151. To address this, they propose A_{\mathrm{hazard}}, a first-error risk-set estimand that compares event vs. non-event steps only within the same absolute index, with mean-macro and weighted-macro source-aware variants and a length-stratified extension. They report that rankings change under AhazardA_{\mathrm{hazard}} (e.g., Qwen–Gemma gap disappears; Muse–GLM gap appears), and that on a matched Gemma run, pooled AUROC rises by 0.1074 while first-error localization drops by 0.0755. They contribute an exact pair-type decomposition of pooled AUROC, a controlled measurement of position leakage, the AhazardA_{\mathrm{hazard}} estimator family with invariance guarantees, and paired evidence that a ranking estimand and a first-error readout can reverse a conclusion.

Critical strengths
  1. The diagnostic framing is consequential. The pair-type decomposition makes precise what “pooled AUROC is contaminated by source and position” means in accounting terms. The text-blind source-prior control outranking a fine-tuned verifier is a compelling demonstration that the metric can reward corpus composition rather than verification ability. This directly affects how the community should report PRM evaluations.

  2. AhazardA_{\mathrm{hazard}} has construction-level invariance, not just empirical improvement. The fact that position-only and source+index-only scorers receive exactly 0.5 is an identity, not a tuning result. This makes the estimator conceptually clean and easy to audit, even if the empirical claims around it need stronger support.

Critical weaknesses
  1. Reproducibility is effectively zero from the submitted material. The paper references an ablation_metric_design.py script and extensive appendix tables, but the provided submission does not include a complete appendix, code, data, or checkpoint links. Key empirical claims—the source-prior control’s 0.9228, Gemma’s 0.9151, the +0.1074 pooled AUROC gain and −0.0755 localization loss—cannot be independently checked. For a paper whose central contribution is a measurement critique, the inability to reproduce the measurements is a critical weakness, not a presentation issue.

  2. The aggregation problem is acknowledged but unresolved. The paper shows that global, mean-macro, and weighted-macro AhazardA_{\mathrm{hazard}} can name three different best models, and that dropping a single source can flip the mean-macro winner. The recommendation to “pre-declare the target and report sensitivity” is honest but does not give practitioners a decisive protocol. Combined with the limited ablation coverage (only Gemma and gpt-oss retain per-step scores), the estimator family is less actionable than the paper’s framing suggests.

Questions for authors
  1. Where are the code, data, and checkpoint artifacts? Specifically, is ablation_metric_design.py available, and does it reproduce Tables 14–17 from the released scores? If the artifacts are not included in the submission, can the authors provide a link or a self-contained reproduction package?

  2. The paper reports that a source-prior control reaches 0.9228 on held-out absorbing AUROC while Gemma reaches 0.9151. Can the authors clarify the exact fitting and application procedure for the deployable source prior, including how source-level first-error rates are estimated on validation and whether any smoothing or clipping is applied?

  3. For the matched Gemma run, the pooled AUROC gain is +0.1074 and the first-error localization loss is −0.0755. Can the authors provide the per-checkpoint trajectory (not just endpoints) and the exact localization readout definition in the main text, so that readers can assess whether the sign reversal is robust to the choice of localization metric?

  4. The paper states that mean-macro AhazardA_{\mathrm{hazard}} is undefined when a source has no comparable first-error pair. How often does this occur across the five trained models, and how are the reported intervals constructed when a bootstrap draw leaves a source without a comparable pair?

Reviewer_Li3HOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
3
Read review
Paper summary

The paper investigates how evaluation choices can exaggerate the apparent ability of process reward models. It decomposes pooled AUROC to show how comparisons across datasets and step positions can reward shortcuts. It uses text-blind baselines and controlled label changes to isolate these effects. It then proposes a first-error metric that compares correct steps and first errors at matching positions within the same dataset, with an additional variant controlling solution length. Its main contribution is diagnosing evaluation bias and reducing specific shortcuts.

Critical strengths
  1. The text-blind baselines are convincing.
  2. The label intervention is well controlled.
  3. The proposed metric has clear mathematical guarantees.
Critical weaknesses
  1. The proposed source-aware metric can still reward shortcuts based on how close a step is to the end of a solution.
  2. The training comparisons have limited experimental control. Each model family has only one training run, and most use different training budgets. Some detailed analyses, like Tables 3, 16, and 17, cover only two models.
  3. The paper does not demonstrate improvements in practical applications. It remains unclear whether selecting models with the proposed metric improves reasoning search, step repair, or final-answer accuracy.
Questions for authors
  1. How sensitive is the metric to how reasoning is divided into steps?
  2. Does training with absorbing labels actually make the model worse at finding the first error?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.