acceptodds
Under review as a conference paper at ICLR 2027

What Best-of- Hides: Auditing Multimodal Process Reward Models

Abstract

A process reward model (PRM) scores the intermediate steps of a reasoning trace, and multimodal PRMs are judged almost entirely by one number: how much Best-of-N (BoN) selection improves over the baseline. That number turns out not to identify what it is meant to measure. Whenever a verifier’s scores vary little within a problem, PRM-weighted majority just returns the plain majority answer and PRM-min returns a uniformly random completion, so the reported delta tracks score dispersion, not ranking quality—a pattern we confirm by recomputing within-problem ranking on 9 scored cells of a released artifact, our own earlier submission among them: 7 scorers over 4 benchmarks, every cell in [0.4795, 0.5425], and 2 of the published cells outright scoring failures whose completions all tie. The obvious fix, within- problem step AUC, fails for a different reason: label one chain per problem, as this literature does, and a step-index baseline alone reaches 0.6694 against the verifier’s own 0.7903 [0.5759, 0.9180], because the within-problem contrast is quietly a positional one. Repairing the deployed three-valued verdict readout, which puts 4346 held-out steps on just 2 distinct values, recovers +0.2581 once training has fixed the objective, but only −0.0605 before it has. The one level that escapes both problems is the trajectory, where a problem’s completions are exchangeable and neither objection applies—and there, the repaired verifier ranks at 0.5069–0.5749, no better than a reranker that just counts words. This is, in the end, a negative-results paper: what follows is not a verifier that works, but an account of what it takes to find out whether one does.

Primary area:
foundation or frontier models, including LLMs
Keywords:
process reward models, multimodal reasoning, vision-language models, best-of-N, reinforcement learning with verifiable rewards, GRPO, step-level verification, reward calibration

Reviews

Among:
Shape:
1234this paper, Top 15%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.00 over 3 reviews, above 85% of the others.
Reviewer_BSDAOpenReview ↗
Rating
3
Confidence
1
Informativeness
3
Presentation
3
Soundness
3
Read review
Paper summary

This paper is a careful negative-results audit of how multimodal process reward models are evaluated, rather than a proposal of a new verifier. Its central claim is that the field’s headline instrument—the aggregate Best-of-N improvement over maj@N or pass@1—is non-identifiable: when within-problem verifier scores have low dispersion, PRM-weighted majority collapses to plain majority vote and PRM-min collapses to uniform selection, so the reported delta tracks score dispersion rather than ranking quality. The authors prove a simple degeneracy lemma, reproduce nine released scored cells with within-problem AUC, tie rate, and parse rate, and show that several published gains are arithmetically indistinguishable from majority vote or parser fallback. They then trace one mechanism to a three-valued argmax readout, show that a continuous verdict-probability readout helps only after the training objective is fixed, and demonstrate that the obvious replacement metric: step-level within-problem AUC. The only admissible level they identify is trajectory-level ranking among exchangeable completions, where their repaired verifier remains near chance and is matched by a length reranker, while an external VisualPRM-8B clears chance on the same cache; the result is therefore scoped as evidence about this checkpoint, protocol, and measurement stack, not as a refutation of multimodal process supervision in general.

Critical strengths
  • The paper identifies an important and under-reported failure mode in PRM evaluation: aggregate BoN deltas can be dominated by score dispersion, majority vote, parser fallback, and the mixed-problem denominator rather than by ranking quality.
  • The analysis is unusually forensic and self-critical: it audits the authors’ own released artifact, recomputes AUC/tie/parse diagnostics from logged scores, decomposes headline gains, and reports negative or inconvenient cells rather than silently dropping them.
  • The measurement chain is methodologically strong: a formal degeneracy lemma motivates within-problem metrics; nuisance baselines test position and length confounds on the same pairs; trajectory-level AUC is justified as the admissible exchangeable comparison; and alternative explanations for the null are measured rather than argued away.
  • Reproducibility practices are exemplary in intent: machine-generated numbers, macro provenance, problem-grouped splits, bootstrap at the problem level, release manifests, and explicit statements of which pipeline stages did or did not run.
Critical weaknesses

The evidentiary base remains narrow: one main generator/cache family, one trained in-house checkpoint at one seed, one external verifier control, no human audit of MC step labels, and an unmeasured merged-checkpoint AUC limit how far the negative conclusion can travel.

Questions for authors

See weakness

4 comments

Thank you for a careful summary and review. You are right that a base of one main generator, one in-house checkpoint at one seed, one external verifier, no human audit of the labels and an unmeasured merged checkpoint would limit how far a negative conclusion can travel. Below is the evidence on each of your five points and where it sits in the PDF. Conventions for both parts: unless stated, numbers are on MathVista, MathVision and MathVerse, on mixed problems (at least one correct and one wrong completion), under softmin aggregation of step scores (beta = 4), with 95% bootstrap intervals over problems (over solutions on VisualProcessBench); AUC is within-problem AUC, random = pass@1, top-1 = picking the top-scored completion, wmaj = PRM-weighted majority, * = interval excludes 0. The three instruments are AUC, the top-1 gain over random and the weighted-majority gain over maj@16 (Sec. 1); an instrument registers a contrast when its paired interval excludes 0. Section, table and appendix numbers refer to the PDF on OpenReview unless marked as the submission's.

A correction first, on our verifier (the in-house checkpoint): its trajectory numbers in the submission (Tables 1 and 10, and our verifier's column of Table 11) came from the served path described in App. C, which did not deliver the image. Scored through the corrected path (image sent, native template, verdict read by token id; App. C), its within-problem AUC on the original cache of Qwen2.5-VL-7B completions is 0.545 to 0.661 (App. G, Table 3), mostly on problems that contributed a training chain; on held-out MathVista and MathVision problems it is 0.575 [0.526, 0.623] and 0.542 [0.491, 0.592] (App. D, Table 2; the MathVerse held-out row is not a clean test). Even through the served path, on the full caches of 64 completions, its scores beat the length baseline on MathVista, MathVision and MathVerse (App. C), so the submission's statement that its ranking was matched or beaten by a length reranker does not hold. The comparisons of instruments below are measured on released PRMs.

1. One main generator, one cache family. You are right that Qwen2.5-VL-7B is the main generator: our verifier, the zero-shot judge and the original cache use only its completions, and so do most trajectory comparisons (Sec. 8, Scope). A second generator, InternVL3-8B, gives 16 completions per problem, and its cache, like the fresh Qwen2.5-VL-7B cache, is scored by all five released PRMs (Sec. 2, App. H). For VisualPRM-8B with its released code (mixed problems in brackets; AUC and wmaj - maj@16 with intervals in Tables 5 and 6, shares and signs in App. H):

completionsbenchmark (mixed)AUCmaj@16 - randomwmaj - maj@16top-1 - maj@16
Qwen2.5-VL-7BMathVista (485)0.624+0.146*+0.015*-0.090*
MathVision (1122)0.597+0.070*+0.016*-0.002
MathVerse (2157)0.558+0.050*+0.014*-0.106*
InternVL3-8BMathVista (484)0.711+0.147*+0.012-0.027
MathVision (1206)0.579+0.081*-0.002-0.017
MathVerse (1845)0.627+0.089*+0.013*-0.006

With either generator, majority vote supplies 78% to 102% of VisualPRM-8B's weighted-majority gain over random (above 100%, the weighted vote fell below majority vote), and its top choice never significantly beats maj@16, under any of six aggregators (App. H). Per generator, AUC, top-1 and the weighted vote register 9, 4 and 2 of 12 image contrasts (Qwen2.5-VL-7B) and 10, 6 and 1 of 10 (InternVL3-8B; Table 8). The generator changes how much a PRM needs the image: a blank canvas costs VisualPRM-8B 0.006 to 0.041 AUC on Qwen2.5-VL-7B completions and 0.012 to 0.137 on InternVL3-8B ones, so the image check must run on the completions being reranked (Secs. 5, 8).

2. One in-house checkpoint at one seed. Beyond our checkpoint, two released versions of one PRM show the gap between the instruments directly: on Qwen2.5-VL-7B MathVista completions VisualPRM-8B-v1_1 ranks above VisualPRM-8B (AUC +0.071 [+0.040, +0.104]) and selects better (top-1 +0.080 [+0.029, +0.130]), yet their weighted votes differ by +0.006 [-0.012, +0.023]; elsewhere the two rank alike or v1_1 lower (Sec. 6; intervals in App. H). Our verifier has one seed: a rerun of our recipe with the same seed reproduces its held-out within-problem step AUC (0.651 vs 0.653 on the 3367 soft pairs, the within-problem pairs of steps whose MC success counts differ; 0.774 [0.595, 0.894] vs 0.790 on the 124 hard pairs of a positive and a negative step; Sec. 4, App. D), which checks reproducibility, not seed variance.

3. One external verifier. Five released PRMs from three groups, 7B to 72B (VisualPRM-8B and -v1_1; Qwen-VL-PRM-7B, with a readout we reconstructed from its released training data and could not check against the authors' code, App. C; text-only Qwen2.5-Math-PRM-7B and -72B), a zero-shot Qwen2.5-VL-7B judge and degenerate scorers (a constant, a length reranker); within each cache all scorers see identical completions (PDF Table 1; App. H). Over 22 image contrasts on the three multimodal PRMs and 60 comparisons between released PRMs, AUC registers 19 and 46, top-1 10 and 40, and the weighted vote 3 and 25 (Secs. 5, 6). At the best threshold on the "+" probability, chosen on the benchmark, our harness reaches VisualPRM-8B's published VisualProcessBench macro-F1 (62.5 against 62.0; 55.4 at threshold 0.5; App. C).

4. No human audit of MC step labels. We did not relabel our corpus by hand; we applied our Wilson labelling rule (App. E) to 700 VisualProcessBench solutions that carry human labels (App. I). It calls 49.8% of human-correct steps wrong with our corpus recipe (question blank); with the question shown, 20.7% (Qwen2.5-VL-7B) and 15.9% (InternVL3-8B), and 35-40% on the hardest third of problems. This fits Prop. 4 (Sec. 7): an MC label records whether this generator can finish from a step, not whether the step is valid. The position check (Sec. 4) runs on both label sources, which tilt in opposite directions:

scorerhuman labels, within solution (19,054 pairs)human, pooled (step AUC)our MC labels, soft pairs (3,367)rising (2,082)falling (1,285)
position only0.872 (earlier higher)0.6260.618 (later higher)10
VisualPRM-8B0.6960.6950.4320.2230.770
VisualPRM-8B-v1_10.6110.642---
Qwen2.5-Math-PRM-7B0.5860.6460.5190.4960.555
Qwen2.5-Math-PRM-72B0.5430.646---
Qwen-VL-PRM-7B (our readout)0.3190.538---
our verifier--0.6530.7440.506

Human labels: VisualProcessBench, where 87% of within-solution pairs have the correct step first; within solutions every released PRM is below position (each paired interval excludes 0), while pooled across solutions four of the five score above it. MC labels: our held-out steps, one chain per problem, soft pairs as in point 2, split by whether the value rises or falls along the chain (Sec. 4, Apps. D and H).

5. Unmeasured merged-checkpoint AUC. Our verifier's trajectory AUCs (Part 1) come from its merged form, the LoRA adapter merged into the bf16 base model (Sec. 2). Its held-out AUC as a merged checkpoint cannot be measured: the original weights were not retained, and every number for it comes from stored scores. On the same-seed rerun, merging moves held-out within-problem step AUC from 0.774 to 0.758, inside the unmerged adapter's interval, pooled step AUC from 0.868 to 0.867 and soft-pair AUC from 0.651 to 0.654 (App. D).

How far the conclusion travels: that a Best-of-N gain does not identify what a verifier does is measured on five released PRMs, two generators and two label sources (Secs. 3 to 6), and the Scope paragraph (Sec. 8) states what it does not cover.

Thanks again. Your review named one weakness with five specific gaps -- we provided an answer for each with measurements above. If those address your concern, would appreciate if you consider raising your score. We are glad to answer anything further before the discussion closes

Thank you again for recognizing the paper’s methodological contribution and soundness. Your main reservation concerned the breadth of the evidence. We have substantially expanded it to five released PRMs from three groups, two generators, and comparisons against human labels on 700 VisualProcessBench solutions. We also measured the effect of checkpoint merging on a reproduced training run, while explicitly retaining the limitation concerning seed variation.

These additions provide evidence that the central evaluation findings extend beyond our original checkpoint and cache. Given your positive assessment of the contribution, would this broader validation support a stronger acceptance recommendation? We would appreciate knowing whether any specific concern still limits your assessment.

Reviewer_cdm1OpenReview ↗
Rating
3(+2)
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper analyses the evaluation of multimodal process reward models (PRMs). It shows how gains under Best-of-N setting can hide weak verifier contributions and studies whether within-problem score ranking metrics provide a more informative evaluation of PRMs.

Critical strengths
  • The paper identifies an important problem in the evaluation of PRMs, where improvements may come mainly from majority voting rather than the verifier's capabilities.

  • The controls are useful, testing step-level ranking against position and length baselines. These experiments highlight why aggregate accuracy alone may be insufficient to assess a PRM.

Critical weaknesses
  • The majority of the negative results seem specific to the faulty training and serving pipeline rather than multimodal PRMs. Appendix C explains several implementation bugs in the authors' PRM setup. When evaluating an existing PRM, VisualPRM-8B achieves above-chance trajectory ranking and beats the length baseline across all four benchmarks, showing that the observed trajectory-level failure does not generalise to other multimodal PRMs.

  • The paper is poorly written and very difficult to follow. It is formatted like a debugging report of an earlier submission rather than a self-contained paper, and several planned validations were left incomplete.

Questions for authors

Please refer to the Weaknesses section above.

5 comments

Thank you for the review. Both strengths you list, majority vote's share of the BoN gain and the position controls, hold on released PRMs (items 1 and 4 below). Conventions for both parts: unless stated, numbers are on MathVista, MathVision and MathVerse (MMMU-Pro, with 58 and 91 mixed problems at N = 16, is in App. H), on mixed problems (at least one correct and one wrong completion), under softmin aggregation of step scores (beta = 4), with 95% bootstrap intervals over problems (over solutions on VisualProcessBench); AUC is within-problem AUC, random = pass@1, top-1 = picking the top-scored completion, wmaj = PRM-weighted majority, * = interval excludes 0, and a measure registers a contrast when its paired interval excludes 0. Section, figure, table and appendix numbers refer to the PDF on OpenReview unless marked as the submission's.

W1. The negative results seem specific to our pipeline, not to multimodal PRMs. Our checkpoint first. Its corpus has an empty question field, so it was trained, and is scored, without the question, and its Monte Carlo (MC) continuations were generated without it and graded on the continuation alone (Sec. 2, App. E). A correction: its trajectory numbers in the submission (Tables 1 and 10, and our verifier's column of Table 11) came from the served path described in App. C, which did not deliver the image. Scored through the corrected path (image sent, native template, verdict read by token id; App. C), its within-problem AUC on the original cache of Qwen2.5-VL-7B completions is 0.545 to 0.661 (App. G, Table 3), mostly on problems that contributed a training chain; on held-out MathVista and MathVision problems it is 0.575 [0.526, 0.623] and 0.542 [0.491, 0.592] (App. D, Table 2; the MathVerse held-out row is not a clean test). Even through the served path, on the full caches of 64 completions, its scores beat the length baseline on MathVista, MathVision and MathVerse (App. C), so the submission's statement that its ranking was matched or beaten by a length reranker does not hold. In the main text, our checkpoint or its corpus enters the results in the MC-label half of Sec. 4, the step-level image probe of Sec. 5 and Sec. 7 (readout; Fig. 5b); items 1 to 4 below use only released PRMs and public human labels.

On VisualPRM-8B your reading holds with its released code: on a fresh cache of 16 Qwen2.5-VL-7B completions per problem its AUC is 0.624, 0.597 and 0.558, against 0.510, 0.510 and 0.506 for length (PDF Table 1). All five released PRMs we score rank above chance on these three benchmarks for both generators (App. H): VisualPRM-8B and -v1_1, Qwen-VL-PRM-7B (readout reconstructed from its released training data, not checked against the authors' code; App. C), and the text-only Qwen2.5-Math-PRM-7B and -72B. The submission's central claim, that a Best-of-N gain does not identify what a verifier does, holds for these verifiers, which do rank:

1. Majority vote supplies most of VisualPRM-8B's gain, and its top choice does not beat majority vote (released code, 16 completions per problem, mixed problems in brackets; AUC and wmaj - maj@16 with intervals in Tables 5 and 6, shares and signs in App. H):

completionsbenchmark (mixed)AUCmaj@16 - randomwmaj - maj@16top-1 - maj@16
Qwen2.5-VL-7BMathVista (485)0.624+0.146*+0.015*-0.090*
MathVision (1122)0.597+0.070*+0.016*-0.002
MathVerse (2157)0.558+0.050*+0.014*-0.106*
InternVL3-8BMathVista (484)0.711+0.147*+0.012-0.027
MathVision (1206)0.579+0.081*-0.002-0.017
MathVerse (1845)0.627+0.089*+0.013*-0.006

Majority vote's share of the weighted-majority gain over random is 78% to 102% (above 100%, the weighted vote fell below majority vote). Over six aggregators (its own mean included), four benchmarks and both generators, its top choice is more accurate than maj@16 in 4 of 48 cells, never significantly (App. H). On MathVista, Sec. 3's value as beta -> inf (-0.088; Fig. 2 right, App. H) differs from top-1 - maj@16 by 1/485, from one problem whose top score is shared across answers (App. A).

2. On Qwen2.5-VL-7B MathVerse completions, a rank-preserving rescaling of VisualPRM-8B's scores (weights e^(beta s)) turns a significant gain over maj@16 into a significant loss: +0.014 [+0.007, +0.022] with the standard weights, -0.065 [-0.083, -0.046] at beta = 16 and -0.106 [-0.126, -0.085] as beta -> inf (Prop. 1; Sec. 3, Fig. 2 right).

3. On whether within-problem ranking is more informative (your summary): over 22 image contrasts on the three multimodal PRMs (a blank canvas or another problem's image, identical completions) and 60 comparisons between released PRMs, AUC registers 19 and 46, top-1 10 and 40, and the weighted vote 3 and 25 (Secs. 5, 6). AUC is lower without the real image in all 22, so our harness delivers the image; at the best threshold on the "+" probability, chosen on the benchmark, it reaches VisualPRM-8B's published VisualProcessBench macro-F1 (62.5 against 62.0; 55.4 at threshold 0.5; App. C).

4. Within human-labelled VisualProcessBench solutions, where the correct step comes first in 87% of pairs, a position-only scorer (0.872) outranks all five released PRMs (0.319 to 0.696; Sec. 4); pooled across solutions (step AUC), four of the five score above it (0.642 to 0.695 against 0.626; App. H).

W2. Hard to follow, formatted like a debugging report, validations left incomplete. Agreed that the submission was hard to follow. In the PDF the argument sits as follows: the question and the three instruments (Sec. 1); three checks, each scoring a verifier next to a degraded counterpart on identical completions (Secs. 3 to 5: degenerate verifier, position, image); the within-problem test across verifiers (Sec. 6); four propositions that explain or flag the failures (Sec. 7); and a reporting protocol built on within-problem AUC and top-1 gain over random (Sec. 8). App. C states which scoring path each number uses.

On the validations the submission left open, what was run and what it shows:

  • External PRMs. The five released PRMs of W1, from three groups and 7B to 72B, scoring identical completions within each cache (Sec. 2, Table 1, App. H); items 1 to 4 are their results.
  • A second generator. InternVL3-8B, 16 completions per problem, scored by all five (Sec. 2, Table 6). With it, as with Qwen2.5-VL-7B, majority vote supplies most of VisualPRM-8B's gain and its top choice never significantly beats maj@16 (item 1). AUC, top-1 and the weighted vote register 10, 6 and 1 of its 10 image contrasts, and 9, 4 and 2 of the 12 on Qwen2.5-VL-7B completions (Table 8).
  • Audit of the step labels against human labels. Not of our corpus: we applied the MC labelling rule to VisualProcessBench solutions that carry human labels (700 solutions; App. I). With our corpus recipe (question blank) it calls 49.8% of human-correct steps wrong; with the question shown, 20.7% (Qwen2.5-VL-7B) and 15.9% (InternVL3-8B), and 35-40% on the hardest third of problems. Sec. 4 runs the position check on both label sources (item 4).
  • Merged-checkpoint cost. The trajectory caches were scored with the LoRA adapter merged into the bf16 base model (Sec. 2). On a same-seed rerun of our recipe, merging moves held-out within-problem step AUC (124 hard pairs) from 0.774 to 0.758, inside the adapter's interval [0.595, 0.894], and AUC on the 3367 soft pairs from 0.651 to 0.654, against 0.653 [0.626, 0.681] for the original checkpoint (Sec. 4, App. D).
  • Perception versus reasoning. We did not attempt this attribution; what we measured is whether the verifiers use the image (the 22 contrasts of item 3; Sec. 5).

Thanks Again. If these answers resolve your two concerns, with the major rewriting improvements, would appreciate it if you could please consider raising your score. If anything still looks unconvincing, tell us what, and we will answer before the discussion closes.

Reviewer_N3k8OpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper studies whether Best-of-N (BoN) accuracy is a reliable way to evaluate multimodal process reward models (PRMs). The authors show that BoN improvements can largely come from majority voting rather than the PRM itself. They propose evaluating how well a PRM ranks correct and incorrect solutions to the same problem. Their experiments reveal a large gap between step-level performance and full-solution ranking, and identify several issues in reward scoring and data labeling.

Critical strengths

The paper addresses an important problem in PRM evaluation and shows why commonly reported BoN improvements can be misleading.

The analysis is thorough, with useful theoretical observations, simple baselines, and experiments that identify several hidden issues in existing evaluation pipelines.

The paper provides practical insights for building and evaluating PRMs, especially the importance of continuous reward scores and within-problem comparisons.

Critical weaknesses

The theoretical results mainly describe cases where PRM scores have little variation. They do not fully establish that BoN improvements are generally unreliable indicators of PRM quality.

Several implementation issues, including images not being passed to the verifier, weaken the experimental conclusions. Fixing these issues substantially improves ranking performance on some benchmarks.

The experiments focus mostly on one trained PRM, and the step-level evaluation has a small effective sample size. It remains unclear how well the findings generalize to stronger PRMs or different training setups.

Questions for authors

The paper shows that BoN accuracy does not necessarily reflect ranking quality. However, a PRM can also achieve high pairwise AUC while consistently failing to select the best solution. Why should within-problem AUC be the preferred metric, and how well does it predict actual improvements in solution selection?

The labels are based on whether the generator can successfully complete a solution from a given step. However, a correct step can have a low continuation success rate simply because the remaining problem is difficult. How can we distinguish learning to verify reasoning from learning to predict the generator's continuation success? Would this distinction become more important when evaluating a different generator?

4 comments

Thank you for a careful and fair review. Conventions for all three parts: unless stated, trajectory-level numbers are on MathVista, MathVision and MathVerse, on mixed problems (at least one correct and one wrong completion), under softmin aggregation of step scores (beta = 4), with 95% bootstrap intervals over problems (over solutions on VisualProcessBench); AUC is within-problem AUC, random = pass@1, top-1 = picking the top-scored completion, and the weighted vote = PRM-weighted majority. Section, figure, table and appendix numbers refer to the PDF on OpenReview unless marked as the submission's.

W1. The theory covers low-variation scores and does not show that BoN gains are generally unreliable. Agreed on both counts. Lemma 1 is only a sufficient condition for that regime and does not carry a general claim on its own, though the regime is common (its condition alone holds on 27 to 60% of VisualPRM-8B's mixed problems; App. H). Two results need no dispersion assumption. Prop. 1 (Sec. 7): AUC and top-1 ignore any strictly increasing transform of the scores, while weighted majority with weights e^(beta s) runs from maj@N (beta = 0) to the top-scored completion's cluster (beta -> inf), so for a fixed ranking its gain over maj@N ranges over a set containing 0 and, absent ties at the top score, top-1's gain over maj@N. For VisualPRM-8B (released code), one ranking of Qwen2.5-VL-7B MathVerse completions (N = 16) gives a gain over maj@16 of +0.014 [+0.007, +0.022] with the standard weights (w = s), -0.065 [-0.083, -0.046] at beta = 16 and -0.106 [-0.126, -0.085] as beta -> inf; Qwen2.5-Math-PRM-7B moves the other way, +0.029 to +0.071 (Sec. 3, Fig. 2 right, App. H). Where scores spread, the vote does move: Qwen2.5-Math-PRM-72B, whose scores spread most, gains up to +0.053 [+0.036, +0.072] over maj@16 (MathVision), two thirds of what its top choice gains over maj@16 (+0.079; Sec. 3). Prop. 2 (Sec. 7) is an identity: an all-problem gain is the mixed fraction (set by generator, benchmark and N) times the mixed-problem gain, so on MMMU-Pro (81 of 1730 problems mixed at N = 64) no reranker can move the aggregate by more than 0.047 (Sec. 3).

What we claim is therefore specific to PRM-weighted majority: its gain over pass@1 includes what majority vote earns without a verifier (more than three quarters of VisualPRM-8B's at N = 16, with either generator; App. H), and its gain over maj@N depends on score scale. Between verifiers, the standard weights compress large differences in both directions: Qwen2.5-Math-PRM-7B's top choice is 0.176 [0.154, 0.200] more accurate than VisualPRM-8B's on Qwen2.5-VL-7B MathVerse completions and 0.114 [0.066, 0.163] less accurate on InternVL3-8B MathVista completions, yet their weighted-vote gains differ by only 0.014 and 0.021 (Tables 5, 6). Top-1 gain over random is a BoN number too, and one of the two we recommend (Q1).

W2. Implementation issues, including images not reaching the verifier, weaken the conclusions. You are right about our checkpoint, and a correction is due. Its trajectory numbers in the submission (Tables 1 and 10, and our verifier's column of Table 11) came from the served path described in App. C, which did not deliver the image. Scored through the corrected path, its within-problem AUC on the original cache of Qwen2.5-VL-7B completions is 0.545 to 0.661 (App. G, Table 3), mostly on problems that contributed a training chain; on held-out MathVista and MathVision problems it is 0.575 [0.526, 0.623] and 0.542 [0.491, 0.592] (App. D, Table 2; the MathVerse held-out row is not a clean test). Even through the served path, on the full caches of 64 completions, its scores beat the length baseline on MathVista, MathVision and MathVerse (App. C), so the submission's statement that its ranking was matched or beaten by a length reranker does not hold.

The corrected path, like training, leaves the question field empty, so our checkpoint never sees the question (Sec. 2, App. C). As you say, the path matters on some benchmarks: a replica of the served path that shares the corrected readout, and so differs in image, template and question field together, is 0.046 [0.023, 0.069] AUC below the corrected path on MathVista and 0.106 below on MathVerse, but not below it on MathVision (App. C, Fig. 6). At N = 16 (64 on MMMU-Pro) the weighted vote barely sees the difference: paired intervals exclude 0 in 12 of 32 benchmark-aggregator cells for AUC, 7 for top-1, 3 for the weighted vote.

The checks themselves run on released PRMs, and there the image reaches the verifier: on the three multimodal ones, a blank canvas or another problem's image lowers AUC in all 22 settings, significantly in 19, against 10 for top-1 and 3 for the weighted vote (Sec. 5, Table 8).

W3. One trained PRM and a small step-level sample; stronger PRMs or other training setups. Agreed that one trained PRM cannot settle this. The checks run on five released PRMs from three groups, 7B to 72B: VisualPRM-8B, its successor VisualPRM-8B-v1_1 and Qwen-VL-PRM-7B (multimodal), and the text-only Qwen2.5-Math-PRM-7B and -72B; on 16 completions per problem from two generators, Qwen2.5-VL-7B and InternVL3-8B; and on VisualProcessBench human step labels (Sec. 2, App. H). At the best threshold on the "+" probability, chosen on the benchmark, our harness reaches VisualPRM-8B's published VisualProcessBench macro-F1 (62.5 against 62.0; 55.4 at threshold 0.5; App. C). For Qwen-VL-PRM-7B we reconstructed the readout from its released training data and could not check it against the authors' code (App. C). Larger and newer PRMs: on Qwen2.5-VL-7B MathVista completions, Qwen2.5-Math-PRM-72B and VisualPRM-8B-v1_1 rank better than Qwen2.5-Math-PRM-7B and VisualPRM-8B, respectively (AUC +0.053 [+0.032, +0.073], +0.071 [+0.040, +0.104]; Tables 5, 7; elsewhere v1_1 ranks alike or lower), yet within human-labelled solutions they reach 0.543 and 0.611 against 0.872 for ranking earlier steps higher (step AUC pooled across solutions: 0.646 and 0.642 against 0.626; all in App. H, and 0.872 also in Sec. 4). Training data may leave signatures too: with our readout, Qwen-VL-PRM-7B's scores rise along a solution, which puts it below chance within human-labelled solutions (0.319), consistent with the shard of its training data we inspected, where all 3860 chains with an error end at that step, so late steps are almost always labelled correct.

Step-level sample: using every within-problem pair of held-out steps whose Monte Carlo (MC) success counts differ (soft pairs) gives 3367 pairs from 446 problems, against 124 hard pairs (a positive and a negative step) from 39 problems (Sec. 4). Our verifier then beats candidate length clearly (+0.143 [+0.109, +0.177]) and step index narrowly (+0.035 [+0.002, +0.071]), and the same pairs put VisualPRM-8B at 0.432, below chance, because its scores fall along a chain while these labels rise. On human labels, VisualProcessBench gives 19,054 within-solution pairs (table in Q2).

Q1. High AUC can coexist with failing to select. Why prefer AUC, and does it predict selection? You are right. With k correct and m wrong completions, a confident wrong completion on top costs at most k of the km pairs, so for any AUC up to 1 - 1/m some ordering puts a wrong completion on top (App. A). We also measured a milder case: on Qwen2.5-VL-7B MathVerse completions VisualPRM-8B's AUC is above chance (0.558 [0.548, 0.569]) while its top choice is worse than random (-0.055 [-0.071, -0.040]; Table 1, intervals in Table 5); on InternVL3-8B completions its MathVerse top-1 gain is +0.084 (Sec. 6). So we recommend AUC together with top-1 gain over random on mixed problems, not AUC alone (Sec. 8).

Both are rank statistics, so neither moves with score scale (Prop. 1). AUC leads the pair for two reasons. It separates ranking from base rate: its chance level is 0.5 for any share of correct completions, while near chance the top-1 gain is about 4(k/N)(m/N)(A - 1/2), with A the AUC (Prop. 3). And, with continuous scores, pooling every correct-wrong pair (top-1 reads one completion per problem) gives it more power: its paired interval excludes 0 in 19 of 22 image contrasts and 46 of 60 between-PRM comparisons, against 10 and 40 for top-1 and 3 and 25 for the weighted vote (Secs. 5, 6).

On prediction: over the 330 cells of Fig. 2 (left; six aggregators), AUC's rank correlation is 0.89 with the top-1 gain and 0.72 with the weighted-vote gain (Sec. 6; descriptive, cells share problems). Prop. 3 gives a closed form: under Lehmann alternatives (Lehmann, 1953), correct scores follow F0^theta and wrong ones F0, so AUC = theta/(1+theta) and P(top-1 correct) = kA / (kA + m(1-A)). With each cell's AUC and each problem's own k and m, the prediction tracks the observed top-1 gain at r = 0.82 over the 84 cells of Fig. 5a (four aggregators; mean absolute error 0.029; values below in App. H):

scorer, Qwen2.5-VL-7B completions (AUC)predicted top-1 gainobserved
zero-shot Qwen2.5-VL-7B judge, MathVista (0.568)+0.044+0.044
VisualPRM-8B, MathVista (0.624)+0.081+0.056
VisualPRM-8B, MathVerse (0.558)+0.037-0.055
Qwen2.5-Math-PRM-7B, MathVerse (0.618)+0.080+0.121

On the same MathVista completions Qwen2.5-Math-PRM-72B has VisualPRM-8B's AUC (0.624), and so the same prediction (+0.081), but twice its top-1 gain (+0.112; Sec. 7); the deviation tells them apart. A shortfall flags confident wrong completions at the top. We use the model as a diagnostic, not a law.

Q2. Verifying reasoning vs predicting the generator's continuation success; a different generator. Agreed, and Prop. 4 (Sec. 7) states it: an MC label estimates the generator's value function V(prefix), its chance of finishing correctly from there. Along the generator's own chains V is a martingale from the problem's pass rate to the chain's outcome, so it tracks difficulty across problems and progress within a chain; within-problem metrics remove the first but not the second.

(i) Direction. A scorer that locates errors should beat chance on falling pairs, where the MC value drops along the chain. On our held-out MC soft pairs (W3; one chain per problem, continuations generated without the question, App. E), step index scores 1 on rising pairs and 0 on falling ones, so its 0.618 is just the share of rising pairs. Our verifier has no such prior: on the 3027 pairs with equal MC values it ranks the later step higher 0.505 [0.464, 0.550] of the time. Against that rate, comparing pairs the same number of steps apart, a rise makes it favour the later step more often by 0.237 [0.183, 0.290], a fall the earlier step by only 0.014 [-0.041, +0.066] (rising pairs 0.744 [0.694, 0.788], falling 0.506 [0.453, 0.559]; App. D). It registers progress and ignores drops, so on these labels there is essentially no evidence that it locates where a chain goes wrong. The same holds with success counts at least 4, 8 or 12 of 16 apart: falling pairs 0.547, 0.555 and 0.500, rising 0.803, 0.802 and 0.886 (App. D).

(ii) The difficulty effect you describe, on human labels (VisualProcessBench: 700 solutions, 6593 steps; M = 16 continuations per prefix, k of them correct; problem thirds by the generator's empty-prefix pass rate; App. I):

MC labels fromAUC of k/M (within solution)human-correct steps with k = 0same, by third: hard / mid / easy
Qwen2.5-VL-7B, our corpus recipe (question blank, prefix as a closed turn)0.633 (0.558)49.8%-
Qwen2.5-VL-7B, question shown, prefix continued0.743 (0.609)20.7%40.0 / 11.9 / 6.2%
InternVL3-8B, question shown, prefix continued0.762 (0.614)15.9%35.3 / 7.8 / 4.9%

Your example is common on hard problems: with the question shown, the rule calls 35-40% of human-correct steps wrong on the hardest third, for either generator.

(iii) A different generator. On the same 6593 steps the two generators' verdicts under our Wilson rule (positive, negative or abstain; App. E) differ on 27% (opposite on 2.6% of the steps both label), and k = 0 status changes on 13% of human-correct steps (App. I). So yes: the label measures whether this generator can finish, and a different generator changes the target.

(iv) Telling the two apart in practice: compare a verifier with a position-only scorer on the same pairs, under human as well as MC labels, which tilt in opposite directions (Sec. 4, Apps. D and H):

scorerhuman labels, within solution (19,054 pairs)human, pooled (step AUC)our MC labels, soft pairs (3,367)rising (2,082)falling (1,285)
position only0.872 (earlier higher)0.6260.618 (later higher)10
VisualPRM-8B0.6960.6950.4320.2230.770
VisualPRM-8B-v1_10.6110.642---
Qwen2.5-Math-PRM-7B0.5860.6460.5190.4960.555
Qwen2.5-Math-PRM-72B0.5430.646---
Qwen-VL-PRM-7B (our readout)0.3190.538---
our verifier--0.6530.7440.506

On VisualProcessBench 87% of within-solution pairs have the correct step first; within solutions every released PRM is below position (each paired interval excludes 0), while pooled across solutions four of the five score above it. VisualPRM-8B, which its authors trained on MC-labelled data, ranks human labels above chance but falls below chance on our MC pairs, where its scores fall along a chain while the labels rise. Sec. 8 therefore asks for a positional baseline, the rise and fall split, and more than one labelled chain per problem where possible. Setlur et al. (2025) and Zhang et al. (2025) made the conceptual point first.

Thank you for the detailed rebuttal. The additional experiments have addressed most of my concerns. I appreciate the authors' transparency regarding the implementation issues and their efforts to correct them. Overall, I find the central idea interesting and the additional evidence convincing. I will increase my score to 3.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.