acceptodds
Under review as a conference paper at ICLR 2027

Tracing Error Origins and Propagation: Dependency-Grounded Process Supervision for Mathematics and Physics

Abstract

Mathematical and physical reasoning is central to AI for Science, and process reward models help improve it by evaluating intermediate steps. Derivations in these domains reuse quantities, equations, and assumptions, creating identifiable dependencies between the information a step uses and the results it produces. These dependencies provide a basis for distinguishing newly introduced mistakes from inherited errors. We propose dependency-grounded process supervision, which extracts these input–output relationships and checks whether each error arises locally or is inherited through the information used. The resulting supervision teaches a model to judge step correctness, distinguish new mistakes from inherited consequences, and classify newly introduced mistakes. This labeling preserves valid uses of information from partially incorrect steps and accounts for steps that both inherit an error and introduce another. Experiments in mathematics and physics show improved error detection compared with models trained to judge step correctness, with gains persisting on longer mathematical solutions. Controlled comparisons show that learning to distinguish new and propagated errors improves diagnosis beyond rules based on correctness and step order, with similar detection performance. The learned scores also support answer selection and training-data selection. These findings suggest that reasoning dependencies can guide the design of more informative feedback about how errors arise and propagate in scientific problem solving.

Primary area:
foundation or frontier models, including LLMs
Keywords:
process supervision, process reward model, dependency-grounded supervision, mathematical reasoning, physics reasoning, large language models

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_XynpOpenReview ↗
Rating
2
Confidence
3
Informativeness
3
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes dependency-grounded process supervision for mathematical and physical reasoning. SaT-3L-SM proposes candidate text spans, which Qwen3.8-27B groups into reasoning steps while extracting inputs, assumptions, outputs, and dependencies. A subsequent attribution stage distinguishes newly introduced mistakes from inherited errors, including mixed cases. These annotations train a Qwen3-4B process reward model to predict correctness, error role, and intrinsic error type. Evaluations cover PRM800K, an annotated physics subset, and ProcessBench. In the controlled shared-label study, full-trace supervision improves detection over first-error masking. The hierarchy achieves similar Error AP to full-trace binary supervision while improving role macro-F1 over a correctness-based sequential rule. Additional analyses examine reasoning length and error density, alongside downstream answer voting and SFT data selection, with both improvements and regressions across settings. The PRM800K and physics evaluations use reference labels generated by the proposed annotation pipeline.

Critical strengths
  1. The dependency-based distinction between consumed invalid content and locally introduced mistakes is well motivated and more expressive than assigning propagation solely by step order. The thin-film example clearly illustrates correct use of an unaffected output, inherited invalidity, and a mixed error.

  2. The experimental analysis is unusually transparent. The paper includes shared-label controls, length and error-density slices, role confusion matrices, explicit negative findings, class counts, training configurations, score mappings, and bootstrap procedures. It also honestly reports the sparse-error degradation and GSM8K localization loss.

  3. The shared-label comparison provides quantitative support for improved error-role diagnosis. The learned role head raises macro-F1 from 66.24% to 70.80% over the correctness-based sequential rule (+4.56 points, 95% CI [3.37, 5.72]), while the hierarchical and full-trace binary models have similar detection AP (Table 2). This supports the value of explicit role supervision beyond correctness scores and step order.

Critical weaknesses
  1. The practical value of the type head remains insufficiently demonstrated. In Table 2(a), adding type supervision to correctness+role leaves Error AP essentially unchanged (86.59% versus 86.58%) and increases role macro-F1 by only 0.08 percentage points. Although the head recognizes some error categories, its macro-F1 over the six types with test support is 26.67%. On this test set, F1 is reported as 0.00% for assumption/condition and answer mismatch errors, with 56 and 13 reference examples, respectively (Table 14). These scores are conditional on reference new errors with known types. The downstream studies neither use type predictions directly nor isolate the contribution of type supervision. The authors acknowledge that type prediction needs improvement and leave type-guided correction to future work, but the current results provide limited evidence for the practical benefit of this component.

  2. The downstream voting results do not establish a consistent advantage over the three-class PRM. Figure 4(a) and Table 19 show lower accuracy for the hierarchical model at N = 2–5 and small gains at N = 6–8, while the prose in Section 5.4 emphasizes N = 8. A more balanced discussion across budgets would be helpful. Although these results support the utility of correctness scores for voting, a matched downstream ablation is needed to isolate the contribution of role and type supervision.

  3. The organization of the results section makes it difficult to connect individual findings with their experimental settings. For example, the analyses of reasoning length and first-error position in Table 4 are discussed in separate sections, with other comparisons in between. Details on checkpoint selection, scoring rules, and related analyses are distributed across Appendices C–E, requiring repeated cross-referencing to interpret the results. Grouping related analyses and providing a consolidated summary of the main experimental settings would improve the coherence and readability of the presentation.

Questions for authors
  1. What fraction of traces or steps were rejected as unverifiable, malformed, or otherwise ineligible? Please provide a human audit of dependency extraction, role attribution, and intrinsic types, including agreement rates and common failure modes.

  2. Could the authors report how frequently mixed errors and correct uses of unaffected outputs occur in the evaluation sets, and how the model performs on these cases? This would help connect the aggregate results to the distinctions illustrated in Figure 2.

  3. Appendix E.3 leaves open whether correctness-only decoding would recover the GSM8K loss. Could the authors compare the hierarchical model's current correctness–role decoder with decoding based only on its error probability, using the same checkpoint, inputs, and threshold of 0.5? Reporting localization accuracy, correct-trace acceptance, and F1 on all four ProcessBench subsets would help isolate the decoder's contribution.

  4. The threshold is tuned on the validation set (Appendix E.4), but the rule still treats only the first predicted error as new. Could the authors also test a rule that uses the error probabilities and how they change across steps, and allows later errors to be labeled as new? It would be useful to see whether the role head still has an advantage over this less restrictive baseline.

5 comments
AuthorsOpenReview ↗

Response to Reviewer Xynp (1/3)

Thank you for the detailed review and for asking us to clarify our evaluation protocol and what the results support.

Comment 1 · Type diagnosis, voting, and experimental settings

W1. What the type head adds. We present type prediction as an auxiliary diagnostic target in the revision. Adding role supervision to correctness + type improves conditional type macro-F1 on 153 independently typed steps by +2.96 [0.59, 5.48] and +3.85 [0.58, 6.78] points across two seeds. Metrics are conditioned on reference local-error steps with known types. The practical benefit of using predicted types for repair remains untested.

W2. Voting at every budget. Both PRMs beat majority voting on the same 1,319 GSM8K questions. The hierarchy trails the three-class PRM at N=2–5 and leads slightly at N=6–8, with all seven paired intervals including zero. In the matched Table 2 replay, differences from full-trace binary range from −0.38 to +0.08 points, with all intervals including zero. The revision limits the voting claim to correctness-score utility and reports the full budget breakdown in the appendix.

W3. Evaluation settings. We grouped trace length and first-error position in RQ3 and added a single selection and decoding table. These PRM800K slices use the existing three-class and hierarchical models at scores ≥0.5, with length cutoffs of ≤6, 7, and ≥8 steps and position cutoffs at 1/3 and 2/3 on erroneous traces. The existing hierarchy was selected by validation balanced accuracy. The shared-label and explicit-IO models were selected by validation Error AP. ProcessBench uses the same application hierarchy for both decoders at a strict >0.5 threshold.

AuthorsOpenReview ↗

Response to Reviewer Xynp (2/3)

Comment 2 · Data screening and independent review

Q1. Screening and audit. These exclusion rates use the source-checked pools. Evaluation conclusions apply to the retained, fully evaluable traces.

PoolExcludedRecorded exclusions
PRM800K train/validation, before splitting31,327/73,797 (42.45%)29,493 hard rejections, 1,674 held for review, 160 without usable attribution
PRM800K test1,282/2,659 (48.21%)19 unverifiable, 1,183 other hard rejections, 75 held for review, five without usable attribution
Physics32/667 (4.80%)Eight hard rejections, ten held for review, 13 held after cessation review, one without usable attribution

Of the five PRM800K test traces without usable attribution, two failed output validation and three timed out.

In the independent audit of 100 PRM800K questions, the two annotators agreed on 97.08% of four-way roles (κ = 0.954, 616 jointly determinate steps). On 617 final determinate steps, the automatic reference agreed with humans on 84.28% of correctness decisions and 75.85% of four-way roles. Of the 61 human-identified mixed errors, the automatic reference labeled 3 as mixed, 7 as pure new, 34 as propagated, and 17 as correct. Under the training mapping, mixed and pure new both receive the new-error role, so 10/61 receive a new-error label. Human type agreement was 133/153 (86.93%) when both annotators found a local error. On the final 153 human-typed steps, the automatic reference supplied a type for 69 and matched 41 of those. One recurring mismatch was assigning derivation/logic to method/formula errors.

In a stratified blind audit of 80 physics steps, two reviewers agreed on 71/75 jointly determinate four-way roles (94.67%, κ = 0.947). After adjudication, automatic labels agreed with humans on 61/76 correctness decisions (80.26%) and 51/76 four-way roles (67.11%). The sets of source steps matched exactly on 61/78 steps with determinate source references. The cases in Q2 illustrate why matching the source step alone can miss an error in the extracted content or its causal attribution.

Q2. Mixed errors and unaffected outputs. The automatic PRM800K reference marks 295/9,073 steps (3.25%) as mixed, across 208/1,377 traces. The hierarchy flags 283 as errors and predicts new for 99. The independent mathematics audit identifies 61/617 determinate steps as mixed. On these steps, direct and explicit-IO training give 68.85% and 71.31% recall as new (+2.46 points [−0.94, 5.93]). In a targeted audit with premise links and local judgments fixed, false propagation predictions on unaffected uses are 3/3 for the step-level rule and 2/3 for the output-level rule. Misses on genuine propagation are 0/11 and 1/11. We restrict the mathematics rule comparison to the targeted audit, whose cases were selected for the comparison. An evaluation-wide frequency would require a representative sample.

In physics, the human audit identified 13 mixed errors. The automatic reference labeled six as mixed, six as pure propagation, and one as correct. The hierarchy flags 12/13 as errors and predicts new for 3/13. The audit found four correct reuses of unaffected outputs, three in the screen-positive stratum and one in the screen-negative stratum. This gives an exploratory inverse-sampling-weighted estimate of 3.52% in the physics evaluation set. Treating unresolved judgments as all absent or all present gives 3.50–4.00%. This range captures sensitivity to unresolved adjudications alone. Each evaluated PRM accepts 2/4 of these steps.

In U058, the extractor bundled a valid oscillation claim with the incorrect claim that the object never stops at the endpoints. The target used the oscillation claim alone, but the automatic label marked it as propagated. In U118, the extractor correctly isolated the conclusion that 8 m/s² was unattainable, which the target reused. The automatic attribution still marked that valid conclusion as the source of propagation.

AuthorsOpenReview ↗

Response to Reviewer Xynp (3/3)

Comment 3 · Same-model decoding and a stronger score-only rule

Q3. Same-model ProcessBench decoding. Both decoders use the same hierarchical checkpoint and inputs and flag the first step with a score strictly above 0.5. Correctness-only uses the error probability. Joint decoding weights it by the probability of new conditional on the two error roles. Joint decoding increases correct-trace acceptance on all four subsets. All four localization intervals include zero. Each cell gives correctness-only → joint (%) and the paired difference in points with a 95% interval. We resampled question clusters within each subset 2,000 times.

SubsetExact first-error localizationFully correct trace acceptanceF1
GSM8K55.56 → 54.11 (−1.45 [−3.85, 0.49])91.19 → 91.71 (+0.52 [0.00, 1.90])69.05 → 68.06 (−0.99 [−2.94, 0.55])
MATH58.92 → 57.91 (−1.01 [−2.73, 0.67])75.86 → 77.83 (+1.97 [0.74, 3.36])66.33 → 66.41 (+0.08 [−1.12, 1.28])
OlympiadBench54.16 → 54.46 (+0.30 [−0.80, 1.49])53.69 → 59.00 (+5.31 [3.07, 7.74])53.92 → 56.64 (+2.72 [1.43, 4.13])
Omni-MATH54.55 → 54.55 (0.00 [−1.33, 1.34])55.19 → 58.09 (+2.90 [0.90, 5.14])54.86 → 56.26 (+1.40 [0.21, 2.70])
Unweighted mean55.80 → 55.26 (−0.54 [−1.40, 0.28])68.98 → 71.66 (+2.68 [1.79, 3.62])61.04 → 61.84 (+0.80 [0.11, 1.53])

Q4. Score-only rule for later new errors. The rule predicts new at the start of a flagged segment or when the error score jumps by at least γ. We jointly select the flagging threshold and γ by maximizing three-way role macro-F1 on 13,930 validation steps for each seed. The jump grid runs from 0.05 to 0.50 in increments of 0.05. The flagging grid runs from 0 to 1 in increments of 0.01 and includes the original sequential threshold. Both seeds select γ=0.35, with flagging thresholds of 0.44 and 0.39. We freeze them before test scoring. The role head's gains over this rule on the human reference are positive in both seeds, with both intervals including zero. Values below are role macro-F1 (%), with paired 95% intervals in points.

Reference, seedSequentialScore-onlyRole headScore-only − sequentialHead − score-only
Automatic, 4266.2466.4570.80+0.21 [−0.57, 0.98]+4.35 [3.34, 5.33]
Automatic, 4365.5066.2070.87+0.70 [0.03, 1.39]+4.68 [3.75, 5.61]
Human, 4261.5164.1366.95+2.62 [−0.19, 5.54]+2.82 [−0.66, 6.10]
Human, 4362.6163.3065.23+0.69 [−2.07, 3.35]+1.92 [−1.77, 5.54]
Reviewer_xe1iOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper addresses a core limitation of Process Reward Models (PRMs): standard binary step supervision conflates new intrinsic errors with propagated errors (steps executing valid reasoning over faulty premises inherited from earlier steps). The proposed pipeline:

Segments reasoning traces using SaT-3L-SM Extracts input premises, assumptions, and atomic outputs for each step via Qwen3.8-27B Annotates each step with contextual correctness, an error role (correct, new error, propagated error, cessation), and an intrinsic error type (8-way classification) conditioned on dependency support.

A Qwen3-4B model equipped with three linear heads is trained with a masked, multi-task objective (L=LC+0.5LR+0.5LT). The approach is evaluated across re-annotated PRM800K, a physics subset of PRMBench Preview, ProcessBench, and downstream best-of-N voting and SFT data selection. The authors report that the hierarchical model gains +10.55 Error AP on PRM800K and +8.40 on physics over baselines, alongside modest improvements on ProcessBench (58.61 to 61.84 F1). The conceptual formalization of dependency-grounded error attribution and the handling of mixed errors are well reasoned. However, the evaluation framework suffers from circularity: training and test labels originate from the same unvalidated LLM pipeline without human ground truth. Furthermore, controlled experiments demonstrate that the headline detection improvements stem from training-label distribution shifts rather than the hierarchical architecture. The role and type heads exhibit low recall and F1, remain untested across multiple training seeds, and are not utilized in downstream tasks. Positioning against uncited foundational prior art (PARC, ICML 2025) and providing rigorous human validation are necessary before this work meets the ICLR publication threshold.

Critical strengths

The framework establishes a clear, rigorous taxonomy that separates contextual correctness from error role and intrinsic error type. The explicit handling of "mixed" steps (steps that inherit a flawed premise but simultaneously introduce an intrinsic calculation/reasoning mistake) is a valuable formalization. The controlled study (Table 2) fixes initialization, labels, splits, and training budgets (16,712 updates) across five loss configurations. The authors report transparently that full hierarchy and BFE-style binary training tie on error detection (+0.05 AP, 95% CI [-0.31,0.43]), demonstrating strong scientific candor. The evaluation uses 2,000 paired cluster-bootstrap iterations across 410 problem clusters, providing informative slices across trace length, error density, prefix versus suffix, and first-error location. Recomputed metrics—including AP deltas, macro-F1 values, class-loss weights, confusion matrix derivations (Fig. 3), and ProcessBench F1 averages—are verified to be mathematically consistent throughout the manuscript.

Critical weaknesses

Unvalidated LLM "Ground Truth" and Closed Evaluation Loop: All step-correctness, role, and error-type test labels on PRM800K and PRMBench-Physics are generated by the same Qwen3.8-27B pipeline used to build the training set. The paper presents zero human validation, zero inter-annotator agreement metrics (e.g., Cohen's ), and no comparison against PRM800K’s original human step annotations. Evaluating a model against synthetic labels generated by its own supervisor pipeline measures distillation agreement rather than true verification accuracy.

Confounded Headline Detection Gains: The +10.55 AP gain on PRM800K (Table 1; abstract L23–25) is primarily an artifact of training data and label-distribution alignment, not hierarchical supervision:

The baseline models were trained on original PRM800K ratings, whereas the hierarchical model was fine-tuned on 267k LLM-annotated steps where 82% of suffix steps are marked incorrect. Under strictly controlled shared-label conditions (Table 2), the hierarchical model matches simple full-trace binary supervision (+0.05 AP, [-0.31,0.43]). The claim that hierarchical supervision improves error detection over step-correctness models is not supported by the controlled data. Absence of Competitive External PRMs and Sub-SOTA ProcessBench Results: The evaluation omits standard open-weight PRM baselines: Qwen2.5-Math-PRM-7B (ProcessBench avg 73.5), PathFinder-PRM-7B (69.5), and official BFE checkpoints. On ProcessBench (the only human-labeled test set), the model achieves 61.84 F1, well below existing 7B baselines. Furthermore, GSM8K localization drops by 7.76 F1, and the overall gain is heavily driven by conservative acceptance of fully correct traces rather than refined error localization.

Weak Diagnostic Baseline and Low Type-F1

The role-F1 gain (66.24 to 70.80) is benchmarked against a naive heuristic ("first error is new, all subsequent errors are propagated"). This rule cannot represent independent later errors by construction, inflating the apparent advantage of the learned head. The model achieves only 60.3% recall on new errors, confusing them with correct steps and propagated errors roughly 20% of the time each (Fig. 3). Intrinsic type prediction is very weak (macro-F1 of 26.67; 0.00 F1 on assumption and answer-mismatch errors). Experiments use a single training seed (seed 42); bootstrap CIs reflect test sampling noise rather than training stability.

Missing Critical Prior Work: PARC: The paper fails to cite PARC (Mukherjee et al., "Premise-Augmented Reasoning Chains Improve Error Identification in Math Reasoning with LLMs", ICML 2025), which explicitly links steps to premise inputs and formalizes the exact distinction between "native" (new) errors and "accumulation" (propagated) errors. The submission must position its contribution relative to PARC, BFE, and PathFinder-PRM. Downstream Applications Do Not Leverage Role or Type Heads: Neither best-of-N voting nor SFT data selection utilizes the learned role or type heads; both rely solely on scalar correctness probabilities. On GSM8K voting, the hierarchical model does not outperform a simpler three-class PRM. On SFT selection, comparisons are made only against random subsets, not against three-class PRM filtering, leaving the practical utility of the multi-head architecture unverified.

Contamination and Validation Leakage: The validation set for the application hierarchy shares 1,683 normalized problems (affecting 2,027 of 2,099 validation traces) with the training split (L864–866), compromising early-stopping and checkpoint selection. Additionally, the exact source composition of the 267k training steps is never detailed.

Minor Issues Formal Inconsistencies in Eq. 5 and Eq. 6: Equation 5 leaves the role undefined for the case ci=I,νi=0,Ai=∅c_i = I, \nu_i = 0, A_i = \emptyset. Equation 6 introduces the cessation label, which is missing from Eq. 5's initial role set and later merged into correct. Implicit Loss Attenuation (Eq. 9): Because the denominator counts masked non-error tokens, the effective weight on the type loss is attenuated by the sparsity of typed new errors (≈16%\approx 16\%), explaining the poor type-F1 performance. Model Naming Ambiguity: The text conflates the "application hierarchy" (Table 1, selected by balanced accuracy on an overlapping split) with the "full hierarchy" (Table 2, selected by validation AP on a disjoint split). Untabulated Textual Claims: Claims regarding late-group false-positive reductions (25.96% to 15.19%, L372) and long-minus-short error recall gains (+4.82, L296) do not appear in any table.

Questions for authors

Can you provide a human-annotated validation study on a stratified sample of 200–300 steps comparing human agreement () against the Qwen3.8-27B annotator for correctness, role (new vs. propagated vs. mixed), and error type? If the Table 1 baseline models are trained on the same 267k dependency-annotated steps using correctness-only loss (matching the BFE-style binary setup in Table 2), does the +10.55 AP gap disappear? How does an explicit, non-learned dependency rule perform on role prediction (e.g., classifying an error as propagated if and only if its extracted premise set Di references a previously flagged step)? How do established open-weight PRMs (such as Qwen2.5-Math-PRM-7B and PathFinder-PRM-7B) perform when evaluated on your test splits and on ProcessBench under your exact evaluation harness? Across 3 random training seeds, what is the mean and variance of the role macro-F1 and type macro-F1 scores?

6 comments
AuthorsOpenReview ↗

Response to Reviewer xe1i (1/3)

Comment 1 · Human validation and the detection gap

Thank you for the detailed review. We address the independent reference, the source of the detection gains, and the stronger role baseline first.

W1/Q1. Independent reference. Two annotators independently assessed 625 steps from 100 randomly sampled PRM800K questions with existing labels and predictions hidden. Four-way role agreement before adjudication was 97.08% (κ = 0.954) on 616 jointly determinate steps. On the 617 final determinate steps, the automatic reference agreed with humans on 84.28% of correctness decisions and 75.85% of four-way roles, identifying 3/61 mixed errors as mixed. When both annotators found a local error, they agreed on its type in 133/153 cases (86.93%). Among the 153 final human-typed steps, the automatic reference supplied a type for 69 and matched 41 of those. Of the 9,073 evaluation steps, 1,349 aligned one-to-one with rated steps in the original PRM800K. Correctness agreement was 84.36% with neutral counted as non-error and 84.67% with neutral excluded.

We also conducted a blinded audit of 80 physics steps from 66 questions, with two independent reviewers per step and a third reviewer for 16 cases. Before adjudication, four-way role agreement was 71/75 on jointly determinate steps (94.67%, κ = 0.947). After adjudication, automatic labels agreed with human correctness on 61/76 steps (80.26%) and four-way role on 51/76 (67.11%). Of 13 human-identified mixed errors, the automatic reference marked six as mixed, six as pure propagation, and one as correct. These rates describe the audited sample.

On the independent mathematics reference, Error AP averaged over the two matched IO seeds is 88.13% with direct annotation and 87.97% with explicit IO (difference −0.16 points, 95% CI [−0.73, 0.37]). We observe no clear Error AP improvement from explicit IO in this audit. The same comparison shows gains in propagation recall and conditional type prediction.

W2/Q2. What explains the Table 1 AP gap. The existing models in Table 1 have different training histories. In Table 2, we continued training its three-class checkpoint with full-trace correctness supervision on 267,361 steps. On the same 9,073 test steps, the resulting binary model reaches 86.53% and 86.35% Error AP across two seeds, compared with the existing hierarchy's 86.33%. Adding role and type under matched training changes AP by +0.05 [−0.31, 0.43] and −0.07 [−0.45, 0.29] points. The full-trace binary advantage over first-error masking remains +12.03 points [10.78, 13.32] after active-target normalization. We revised the abstract and RQ1/RQ2 to distinguish these findings. Full-trace correctness supervision improves detection in the controlled comparison. Role supervision improves error-origin diagnosis at similar AP.

AuthorsOpenReview ↗

Response to Reviewer xe1i (2/3)

Comment 2 · External PRMs and the direct-premise baseline

W3/Q4. External PRMs. On the same 3,400 ProcessBench traces at threshold 0.5, Qwen2.5-Math-PRM-7B scores 72.99% mean F1 and our 4B model scores 61.84%. The largest localization gap is on GSM8K (71.01% vs. 54.11%). On OlympiadBench and Omni-MATH the larger gaps are in fully correct trace acceptance (26.25 and 24.90 points). On our automatically annotated mathematics and physics references, the hierarchy reaches 86.33% and 84.23% Error AP, versus 74.02% and 70.23% for Qwen on the same steps using its native scoring adapter. For context, PathFinder-PRM-7B reports 69.5% F1 under its published protocol. Public checkpoint availability for the original BFE model remains unverified. Table 2's BFE-style row is our full-trace binary control on shared labels. PathFinder results

W4a/Q3. Direct-premise rule. We kept the binary scores, validation thresholds, and flagged steps fixed. The new rule labels a flagged step as propagated when a premise extracted before attribution links it directly to an earlier flagged step. Otherwise it predicts new. The rule improves over the sequential rule on the automatic reference in both seeds. The learned role head improves further. On the human reference, its gain over the premise rule has a positive interval in seed 42 and an interval including zero in seed 43. Values are role macro-F1 (%), with paired 95% intervals for gains in points.

Reference, seedSequentialPremise ruleRole headPremise − sequentialHead − premise
Automatic, 4266.2467.5070.80+1.25 [0.68, 1.80]+3.31 [2.28, 4.41]
Automatic, 4365.5066.9570.87+1.45 [0.91, 2.04]+3.92 [3.02, 4.83]
Human, 4261.5163.3166.95+1.80 [−0.30, 3.94]+3.64 [0.37, 6.78]
Human, 4362.6164.5365.23+1.91 [−0.02, 3.88]+0.70 [−2.68, 3.92]

Comment 3 · Diagnostic results, prior work, and downstream use

W4b. New errors and intrinsic types. Type prediction remains an auxiliary diagnostic target. Adding role supervision to correctness + type improves conditional type macro-F1 on human-typed steps by +2.96 [0.59, 5.48] and +3.85 [0.58, 6.78] points across the two seeds. These results condition on reference local errors. We narrowed the paper's typing claim to this setting.

W4c/Q5. Training seeds. We completed two of the three requested seeds for all four full-trace conditions. Full-hierarchy role macro-F1 was 70.80% and 70.87%, and conditional type macro-F1 was 26.67% and 27.44%. On the human reference, the role head gains over the sequential rule were +5.44 [1.99, 8.64] and +2.61 [−0.89, 5.77] points. The paired intervals reflect test-question uncertainty within each of the two runs.

W5. Relation to prior work. PARC links premise steps and identifies native and accumulation errors. A step can use a valid output from a partially incorrect premise, as in Figure 2, step 3. Our annotation follows the output actually used. For mixed steps, it retains the inherited cause alongside the new mistake. BFE uses propagation and cessation to construct full-trace binary labels. PathFinder-PRM conditions its reward on mathematical and consistency error predictions. Our PRM separately predicts correctness, error origin, and the type of a newly introduced error. Across two seeds in the matched annotation comparison, the explicit-IO condition has a +3.20-point difference in human-reference propagation recall [0.00, 6.52] and a +1.45-point difference in conditional type macro-F1 [0.35, 2.93]. PARC · BFE · PathFinder-PRM

W6. Downstream scores. Voting and SFT selection assess correctness-score utility. Matched correctness-based voting shows no consistent gain from adding role or type supervision. The SFT selection logs identify the three-class Qwen3-4B PRM as the selector, and we corrected its attribution in the revision. With the verifier, inputs, and threshold fixed on ProcessBench, adding the role score raises mean F1 from 61.04% to 61.84% (+0.80 points [0.11, 1.53]), mainly by accepting more correct traces. Localization changes are inconclusive.

AuthorsOpenReview ↗

Response to Reviewer xe1i (3/3)

Comment 4 · Data splits and model selection

W7a. Validation overlap. The application split shared 1,683 normalized questions, covering 2,027/2,099 validation traces. This compromised the original validation-based selection. We repeated selection after removing shared questions. The original 7,000-step checkpoint still ranked first among three candidates on the remaining 70 questions (balanced accuracy 85.95%, versus 85.77% and 85.38%). We added this limited sensitivity check to the revision. Table 2 uses a separate question- and trajectory-grouped split.

W7b and M3. Source, splits, and result mapping. Both setups use the same screened, dependency-annotated PRM800K Phase 2 pool. Screening required complete evaluations and assessable steps, and export checked consistency of correctness and role. The application hierarchy used a trajectory-ID hash split of 42,470 trajectories (267,233 training and 14,058 validation steps) and supplies Table 1's hierarchical results, the ProcessBench and PRM800K slice analyses, and the hierarchical voting results. Table 2 pooled these partitions and split groups linked by normalized question text, available question IDs, or trajectory IDs, assigning about 5% of groups to validation. The test-overlap check removed no rows at this stage. This setup uses 267,361 training and 13,930 validation steps, and selects models by validation Error AP. Both setups use the same 9,073-step PRM800K test set.

Comment 5 · Equations and reported statistics

M1. Equations 5 and 6. We revised the text around the equations to specify the domain and training mapping. Equation 5 applies to validated incorrect steps. A local mistake makes the role new, and propagation requires a dependency-supported prior error. An incorrect annotation with neither fails validation. Equation 6 records recovery at a contextually correct step, which maps to correct for correctness and role training with type loss masked.

M2. Type loss. Equation 9 divides by all target steps in the batch. Eligible type labels enter the numerator on 42,782 of 267,361 training steps (16.0%). Relative to averaging over eligible type targets alone, this scales the type term by the eligible fraction in each batch. Our type F1 results characterize the batch-normalized objective. Averaging over eligible targets is an untested variant.

M4. Late first errors and trace length. Both calculations compare the Table 1 three-class and application hierarchy at a threshold of 0.5. When the first error occurs after two-thirds of the trace, false positives fall from 176/678 (25.96%) to 103/678 (15.19%) among reference-correct steps. Error recall rises from 78.97% to 80.72% on traces of ≤6 steps and from 75.45% to 82.02% on traces of ≥8 steps. The long-minus-short gain is +4.82 points (paired 95% CI [0.88, 8.80]). We added the false-positive counts alongside the length and position analysis in RQ3.

Reviewer_qqhsOpenReview ↗
Rating
2(+1)
Confidence
2(−1)
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

Authors identify errors in LLMs solving Math/Physics problems. Further more, they train a PRM with the identified weakness.

Method include 2 parts. (1) use an LLM to cut answer traces into pieces, extract each step's input, output, assumption and relying configurations, and extract errors there. (2) train a PRM with qwen3-4B, identifying (a) correctness, (b) error type, (c) error character (e.g. new mistake/propergated mistake).

Authors show training with all error signals throughout the phase would lead to better PRM performance. The PRM can also be used as reward model in down-stream to boost BoN performance.

Critical strengths

(1) The authors conduct valid experiments to train and obtain PRM based on the error identification process. This idea makes sense and looks promising to lift LLM's performance, especially for those that cannot do Math/Physics very well.

(2) The experiments are valid and show improvement of the proposed method on PRM performance.

Critical weaknesses

(1) The idea to use hierachical process (reward/correctness/etc.) to supervise/grade LLM answers are not new. For example https://icml.cc/virtual/2025/poster/46462.

(2) Experiments show improvement on PRMs, but not show improvements on LLMs trained with process rewards. For example, can this better fine-grained process reward bring better learning performance during the grokking phase of LLMs?

(3) The dataset and problem researched (Physics & Math) seems a bit old given current situations where LLMs' Math/Physics problem solving capability has become human expert level. Whether the testbed of this proposed method, the models and dataset researched here, can generalize to the problem of frontier Math & Physics capabilities, is questionable.

Questions for authors

Please see weakness.

3 comments
AuthorsOpenReview ↗

Response to Reviewer qqhs

Thank you for pointing us to PARC. We have added it to the revised manuscript's Related Work.

W1. Relation to PARC. We attribute errors through the specific outputs consumed by each step. A step can reuse a valid output from a partially incorrect source, inherit an error through an invalid output that affects its conclusion, or both inherit an error and introduce a new one. In mixed cases, the annotation retains the inherited cause and assigns the intrinsic type to the new mistake. These judgments determine the correctness, error-role, and intrinsic-type targets for PRM training.

PARC links premise steps and distinguishes native from accumulation errors. Its accumulation rule checks the error label of a cited step. In one audited trace, a source step proposes an invalid ceiling rule involving 4.5, and a later step computes 4.5² = 20.25. With premise links and local judgments fixed, PARC's accumulation rule flags the calculation. Our annotation accepts it because the calculation uses the value 4.5 independently of the invalid rule. PARC, Algorithm 1

We tested the contribution of explicit inputs and outputs against direct annotation with the same annotator, text, review procedure, and PRM training setup. On independently reviewed PRM800K questions, recall on 125 pure propagation steps rose from 62.00% to 65.20% (+3.20 points, paired 95% CI [0.00, 6.52]). Conditional type macro-F1 rose from 20.20% to 21.65% (+1.45 points [0.35, 2.93]). Both gains were positive in both seeds. Overall role macro-F1 changed by +0.92 points [−0.38, 2.35].

W2. Training the generator with process rewards. Generator RL and grokking remain untested in this study. We limit the claims in the revision to the uses we evaluated. Voting and SFT data selection use correctness scores. The SFT records identify the selector as the three-class Qwen3-4B PRM, which we have corrected in the revision. In a separate test that uses the role predictions directly, adding the role score to the same verifier raises mean ProcessBench F1 from 61.04% to 61.84% (+0.80 points, paired 95% CI [0.11, 1.53]), primarily through higher acceptance of fully correct traces.

W3. Scope of the mathematics and physics evaluations. Adding the role score to the same verifier changes ProcessBench F1 by +2.72 points [1.43, 4.13] on OlympiadBench, +1.40 [0.21, 2.70] on Omni-MATH, and −0.99 [−2.94, 0.55] on GSM8K. On the automatically annotated physics subset (635 traces from 226 problems), the hierarchy scores 84.23% Error AP versus 75.83% for the existing three-class PRM. The models have different training histories. We limit our conclusions to the tasks and trace-generation settings evaluated here.

I'd like to thank the authors for their supplementary rebuttal.

I'd like to increase score to 2: weak reject. Still overall I am still concerned about novelty, and whether this method proposed could generalize very well on frontier problems (though it's indeed not feasible for authors to validate this).

AuthorsOpenReview ↗

Thank you for the thoughtful review and constructive discussion

Thank you for reconsidering our paper and for the thoughtful review and follow-up. Your comments have helped us clarify our contribution and identify important questions for future work.

We initially focused on mathematics and physics because their derivations often explicitly reuse quantities, assumptions, and intermediate results, making input–output dependencies particularly clear for studying error propagation. These fields also have active research communities and well-established benchmarks that support systematic evaluation.

We appreciate your continued concerns about novelty and generalization, which highlight important directions for our future research. We recognize the limits of our current evaluation and will investigate whether and how our approach can generalize to other disciplines and more challenging frontier problems. Thank you again for the time and expertise you have contributed to this discussion.

Comments on the paper

1 comment

Dear Reviewers,

A friendly reminder to check the authors’ responses and the other reviewers’ comments. It would be helpful to hear your thoughts on the differing assessments and whether they affect your evaluation.

Thank you for your time and effort!

AC

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.