acceptodds
Under review as a conference paper at ICLR 2027

A Strong MoE Is All You Need for Process Supervision

Abstract

Process reward models (PRMs) provide fine-grained supervision for multi-step reasoning, yet obtaining reliable process labels remains a central bottleneck. Existing approaches typically rely on human annotation, rollout-based estimation, or auxiliary verification. In this work, we explore a different source of process supervision natively exposed by Mixture-of-Experts (MoE) models: expert routing. We show that simple routing statistics provide an intrinsic process-confidence signal that is competitive with conventional output-logit confidence. Building on this observation, we construct routing confidence by combining overall routing mass, head routing mass, and within-step routing variation, and introduce RoCo-Rethink, which identifies the least confident reasoning step and selectively regenerates the subsequent trajectory. We further introduce RoCo-PRM, which uses a strong MoE to automatically annotate reasoning trajectories with routing-derived step scores and distills them into a standalone PRM. Across multiple MoE architectures and diverse reasoning tasks, both RoCo-Rethink and RoCo-PRM consistently outperform strong baselines, while RoCo-PRM transfers effectively across both model architectures and tasks. Our findings suggest that expert routing is more than a computational dispatch mechanism: it provides an architecture-native source of process supervision for reasoning.

Primary area:
foundation or frontier models, including LLMs
Keywords:
Mixture-of-Experts, Process Supervision, Process Reward Models, Test-Time Reasoning, Expert Routing

Reviews

Among:
Shape:
1234this paper, Top 15%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.00 over 3 reviews, above 85% of the others.
Reviewer_PYoMOpenReview ↗
Rating
3
Confidence
3
Informativeness
3
Presentation
3
Soundness
2
Read review
Paper summary

This paper investigates whether native expert-routing signals in Mixture-of-Experts models can serve as step-level process supervision. The authors construct a confidence score from three statistics: the overall routing probability mass assigned to selected experts, the routing mass of the initial tokens in each reasoning step, and the variation of routing signals inside one step. The authors proposed 2 methods: RoCo-Rethink identifies the reasoning step with the lowest routing confidence and regenerates the remaining trajectory. RoCo-PRM uses a strong MoE teacher to produce confidence labels and distills them into a standalone process reward model, allowing it to guide both MoE and dense target models. These methods are evaluated using three MoE models, two dense target models, and six benchmark.

Critical strengths
  1. The proposed method proposed a novel process supervision method, which leverages the architectural internal signals into reasoning supervision.
  2. The writting is clear and easy to follow.
  3. The experiments are thorough.
Critical weaknesses
  1. The authors do not discuss the potential computational and storage costs introduced by the proposed method.
  2. The head-routing feature uses only the first three tokens of a step, which might be only connection words and contain limited information.
  3. The proposed method may be sensitive to step segmentation. Including more ablation of reasoning traces might be helpful.
  4. The reported Spearman correlations between routing statistics and external verifier scores are generally low, with many results around 0.05–0.25.
Questions for authors

Please refer to the Weaknesses.

3 comments
AuthorsOpenReview ↗

Response to Reviewer PYoM — Part 1: Computational overhead and head-routing features (W1–W2)

We appreciate your questions about the practical cost of routing-based revision and the information contained in its head-token feature. We measured scoring overhead separately from generation and tested alternative head-token windows.

W1. Computational and storage overhead

Routing replay and step scoring account for about 2.1% of RoCo-Rethink’s measured incremental revision latency.

  1. Latency measurements.

    We evaluated all 1,319 GSM8K test problems with OLMoE. All methods started from shared initial trajectories, and the measured latency includes scoring and revision.

    MethodScoring replay (s/problem)Total revision latency (s/problem)
    Random RethinkNo scoring replay10.37
    LECO0.21715.17
    RoCo-Rethink0.21010.21

    RoCo’s routing replay and scoring took 0.210 s per problem, or approximately 2.1% of its total revision latency. Relative to Random Rethink, the total latency difference was −0.16 s per problem (95% CI [−0.83, 0.53]).

    These measurements include the scoring cost introduced by RoCo. Total latency also depends on the suffix regenerated and the number of rounds executed.

  2. Storage requirements.

    During replay, we aggregate router statistics into step-level scores and retain those scores for revision-point selection. The procedure requires no persistent storage of full router probabilities.


W2. Do the first three tokens provide a useful routing signal?

The three-token head feature improved revision success in a controlled ablation.

  1. Ablation protocol.

    We evaluated 100 initially incorrect OLMoE trajectories from GSM8K. All variants used the same candidate steps and eight cached revisions per candidate. We varied the number of head tokens and tested skipping explicit step numbers and bullet markers.

  2. Results.

    Head-routing variantEstimated revision success
    Head feature removed4.50% (36/800)
    First token5.00%
    First three tokens5.88% (47/800)
    First five tokens4.63%
    Three tokens after skipping explicit numbering and bullet markers5.88% (47/800)

    Adding the original three-token feature increased success by 1.375 percentage points (paired 95% CI [0.50, 2.50]). The three-token window had the highest observed success rate among the tested lengths.

  3. Interpretation.

    The head feature is computed from routing behavior at the beginning of a step. Its usefulness can therefore be tested even when the initial words contain little semantic content. Removing the feature reduced revision success, while skipping explicit numbering and bullet markers retained the same aggregate success rate.

    These results support the empirical usefulness of the three-token routing feature. The prefix-skipping experiment specifically tests explicit formatting markers.

AuthorsOpenReview ↗

Response to Reviewer PYoM — Part 2: Segmentation sensitivity and routing informativeness (W3–W4)

W3. How sensitive is RoCo to step segmentation?

Sentence-level refinement changed few selected revision starts and retained the same aggregate revision success.

  1. A refinement that changes step boundaries.

    On 100 initially incorrect OLMoE–GSM8K trajectories, we preserved the original newline boundaries and split steps further at sentence boundaries.

    • 56 of 100 trajectories gained additional boundaries.
    • The refinement added 79 boundaries.
    • The total step count increased from 706 to 785.
  2. Comparing actual revision starts.

    We compared selected starts by their absolute token offsets in the original generation. This identifies where regeneration actually begins even when step indices change under the new segmentation.

    Among the 56 trajectories whose segmentation changed, the selected start moved in 3 cases (5.4%).

  3. Revision success.

    With eight revisions evaluated per selected start, both segmentations yielded 47 successful revisions out of 800 (5.88%).

    The selected revision start was therefore retained in most affected trajectories, and aggregate revision success was unchanged under this sentence-level refinement.


W4. How informative are routing statistics given their modest correlations with external verifiers?

Routing features jointly add predictive information beyond step characteristics and logit entropy.

  1. What the additional analysis tests.

    We agree that Table 3 shows modest rank correlations between individual routing statistics and verifier scores. We tested whether the three features jointly contribute information after controlling for step token length, relative position, formatting indicators, and logit entropy.

    We repeated grouped cross-validation across the same 18 model–task–verifier settings as Table 4. The control and augmented predictors used identical samples and folds.

  2. Controlled prediction results.

    Candidate stepsControl AUROCControl + routing AUROCSettings improved
    All steps0.60790.631714/18
    Non-answer steps0.59390.618515/18

    For the non-answer analysis, we excluded explicit answer steps while retaining the verifier labels computed on the original trajectories.

    Adding routing features improved macro-average AUROC by 0.0238 with all steps and 0.0246 after excluding answer steps. This supports their incremental predictive value given the specified controls.

  3. Direct revision-point evaluation.

    We also evaluated 100 initially incorrect OLMoE–GSM8K trajectories, restricting all selectors to the same non-answer candidates and using eight cached revisions per candidate.

    SelectorEstimated revision success
    Longest step6.75%
    Last non-answer step2.50%
    Uniform random non-answer step5.00%
    RoCo9.13%

    RoCo’s advantage persisted against length-based and fixed-position selection rules. This complements the verifier-based analysis by testing how often revision from the selected step corrects the final answer.

AuthorsOpenReview ↗

Gentle Reminder to Revisit Our Rebuttal and Evaluation

Dear Reviewer PYoM,

Thank you again for your positive assessment and support of our work. We sincerely appreciate your recognition of the method’s novelty, clear presentation, and thorough experiments.

Your questions helped us strengthen the evaluation with measurements of computational overhead, ablations of the head-routing feature, step-segmentation analysis, and additional controls for routing informativeness.

We would be grateful if you could revisit our responses and consider whether the additional evidence warrants an updated assessment. We hope these results address your concerns and would welcome any remaining questions.

Thank you very much for your time and continued support.

Best regards,
The Authors

Reviewer_pGeMOpenReview ↗
Rating
3
Confidence
1
Informativeness
3
Presentation
2
Soundness
3
Read review
Paper summary

The paper proposes using native Mixture-of-Experts (MoE) routing as an intrinsic step-level process-supervision signal. It combines overall routing mass, head routing mass, and within-step routing variation into a routing-confidence score, where higher overall/head mass and lower variation indicate higher confidence. Building on this, it introduces RoCo-Rethink, which selects the lowest-confidence reasoning step and regenerates the suffix at test time, and RoCo-PRM, which uses a strong MoE teacher to annotate reasoning steps with routing-derived scores and distills them into a standalone PRM. Experiments cover multiple MoE and dense models across six reasoning benchmarks, reporting consistent gains over test-time and PRM baselines.

Critical strengths
  • Novel and simple idea. Routing is produced during ordinary MoE inference; RoCo-Rethink requires no extra rollouts or auxiliary verifier (Sec. 3.2, Sec. 4.1).
  • Reasonably controlled comparisons. RoCo-Rethink is compared with LECO and random rethink under the same selective-regeneration protocol; RoCo-PRM shares backbone/teacher with MC-PRM and CPMI-PRM (Sec. 5.1, Appendix F).
  • Broad and consistent empirical results. Table 1 reports best performance in all 18 RoCo-Rethink settings and best/tied-best in all 30 RoCo-PRM settings, with transfer to dense models.
  • Useful analyses. Component ablation, revision-oracle analysis, teacher/backbone ablation, and leave-one-task-out transfer support the practical value of the signal (Fig. 3, Tables 9–10, Sec. 6.3, Fig. 6).
Critical weaknesses
  • Raw routing–reliability evidence is modest. Table 3 shows small Spearman correlations between routing statistics and verifier step quality, sometimes near zero; Table 4 shows routing predictors beat logit predictors in only 13/18 settings. Downstream gains are stronger than this raw correlation evidence.
  • Trajectory-level normalization complicates interpretation. The routing-confidence score is min-max normalized over the whole trajectory (Appendix A.3), so the target for a prefix depends on later steps, while the PRM input sees only the prefix; test-time scores are again normalized within the trajectory (Appendix E.3). This makes the signal closer to a within-trajectory ranking/confidence score than a calibrated process reward.
  • “PRM/process supervision” framing is somewhat strong. RoCo-PRM is trained by regression to routing confidence (Eq. 8), not to step correctness or a reward. It is effective for revision-point selection, but the paper does not establish that it is a process reward suitable for RL or absolute verification.
Questions for authors
  • Since the confidence target uses trajectory-level normalization but the PRM only sees the prefix, how stable is training when the target depends on future steps? Can you report a prefix-only or non-trajectory-normalized variant?
  • Could routing confidence largely proxy for step position, length, or formatting? Can you add position/length/logit-entropy controls?
  • Are token/compute budgets strictly matched between RoCo-Rethink, LECO, and random rethink across all settings?
3 comments
AuthorsOpenReview ↗

Response to Reviewer pGeM — Part 1: Routing beyond confounds and prefix-only supervision (W1/Q2, W2/Q1)

We appreciate your questions about routing’s predictive value and the interpretation of its scores. We added controls for step characteristics and logit entropy, and tested targets computed from the prefix alone.

W1/Q2. Does routing add predictive information beyond position, length, formatting, and logit entropy?

Routing features add predictive information after jointly controlling for these factors.

  1. Controlled prediction experiment.

    We repeated grouped cross-validation across all 18 model–task–verifier settings in Table 4, using identical samples and folds.

    • The control predictor used step token length, relative position, formatting indicators, and logit entropy.
    • The augmented predictor additionally used routing features (MM, HH, and DD).
  2. Results with and without answer steps.

    Candidate stepsControl AUROCControl + routing AUROCImprovementSettings improved
    All steps0.60790.6317+0.023814/18
    Non-answer steps0.59390.6185+0.024615/18

    For the non-answer analysis, we excluded explicit answer steps while retaining the verifier labels computed on the original trajectories.

    The improvement persists after excluding answer steps. Table 3 measures individual routing statistics’ correlations; this experiment establishes their joint predictive contribution beyond the specified controls.

  3. Direct revision-point comparison.

    On 100 initially incorrect OLMoE–GSM8K trajectories, all selectors used the same non-answer candidates within each trajectory, with eight cached revisions per candidate.

    RoCo achieved 9.13% estimated revision success, compared with 6.75% for longest-step selection, 2.50% for last-non-answer selection, and 5.00% for uniform random non-answer selection.

    This directly measures whether revision corrects the final answer. RoCo’s advantage persisted against selection rules based on length or a fixed late position.


W2/Q1. Can RoCo-PRM learn from targets computed from the prefix alone?

Removing dependence on later steps preserved first-error localization and improved macro-average accuracy at both checkpoints.

  1. The dependency in the original target.

    Full-trajectory normalization can assign different targets to the same prefix depending on later steps. The target expresses confidence relative to the complete trajectory, while the PRM sees only the problem and current prefix.

  2. Matched fixed-normalization control.

    We trained a matched control using fixed training-set statistics in both normalization stages (Appendices A.1 and A.3). Each target can then be computed from the prefix alone.

    We compared both variants at steps 250 and 500. For each variant and checkpoint, we calibrated a separate threshold on GSM8K and froze it for the other three ProcessBench subsets.

  3. Exact first-error localization results.

    DatasetStep 250 OriginalStep 250 FixedStep 500 OriginalStep 500 Fixed
    MATH41.75%41.25%44.61%44.95%
    OlympiadBench29.65%38.28%32.53%39.49%
    Omni-MATH29.78%39.00%31.23%40.45%
    Macro-average33.73%39.51%36.12%41.63%

    Both variants improved from step 250 to step 500. Fixed normalization achieved 5.78 and 5.51 percentage points higher macro-average accuracy, respectively.

    These results favor fixed normalization for prefix-only first-error localization and show that routing-derived targets computed without later steps can support this task.

  4. Inference-time normalization.

    The min–max transformation in Appendix E.3 preserves within-trajectory score ordering and therefore the revision point selected by taking the lowest-scoring step.

Fixed normalization is a promising refinement. We will evaluate it through full training and end-to-end revision experiments to determine whether the localization gains translate into improved reasoning performance, and update the manuscript’s training recipe and results accordingly.

AuthorsOpenReview ↗

Response to Reviewer pGeM — Part 2: Evidence for process supervision and realized inference costs (W3, Q3)

W3. What does the process-supervision evidence support?

Independent error annotations show that the learned routing-derived scores support first-error localization.

  1. Training objective and intended use.

    RoCo-PRM regresses routing-derived confidence scores from the problem and reasoning prefix. In the revision pipeline, these scores rank candidate steps within a trajectory, and the lowest-scoring step determines where regeneration begins.

    We agree that the training objective and downstream evaluation should be distinguished when interpreting the learned scores.

  2. Independent validation on ProcessBench.

    We evaluated the original RoCo-PRM checkpoint using human-annotated first-error locations. We calibrated a threshold on GSM8K and froze it for the three other subsets.

    DatasetExact first-error accuracyUniform random stepProcessBench F1
    MATH49.2%18.1%45.2%
    OlympiadBench41.7%13.6%40.0%
    Omni-MATH40.9%13.8%40.6%

    Exact first-error accuracy is measured on erroneous trajectories. A prediction of “no error” counts as a miss.

    RoCo-PRM exceeded uniform random step selection by 27.1–31.1 percentage points across the three subsets. This evaluates the learned step scores directly against independent annotations, outside the revision pipeline.

  3. Scope of the conclusion.

    These results support using routing-derived supervision to train a predictor for revision-point selection and first-error localization. Calibrating the scores as process rewards and assessing their use in RL remain open questions.


Q3. Are token and compute budgets strictly matched across all settings?

The methods share revision rules and budget limits; realized resource use varies with the selected revision point and early stopping.

  1. Shared protocol and realized costs.

    The revision methods in Table 1 use a common revision protocol. The selected revision point determines how much of the trajectory is regenerated, while convergence-based stopping determines how many rounds are executed. We therefore measured generated tokens, executed rounds, and latency explicitly in a supplementary comparison.

  2. Supplementary comparison under shared limits.

    We evaluated all three methods on the full set of 1,319 GSM8K test problems with OLMoE. They used:

    • Identical initial trajectories, with 49.96% accuracy.
    • Greedy revision decoding.
    • The same cap of four rounds and 512 new tokens per round.
    • The same convergence-based stopping and answer-aggregation rules.
    MethodFinal accuracyNew tokens/problemRevision rounds/problemRevision latency (s/problem)
    Random Rethink50.27%57.91.0910.37
    LECO51.33%84.21.1315.17
    RoCo-Rethink52.54%55.71.0710.21

    Latency starts from the shared initial trajectory and includes scoring and revision.

  3. Paired comparisons and interpretation.

    Relative to LECO, RoCo-Rethink achieved:

    • +1.21 percentage points in accuracy (paired 95% CI [0.40, 2.06]).
    • −28.43 generated tokens per problem (paired 95% CI [−32.55, −24.32]).
    • Lower measured mean revision latency: 10.21 s versus 15.17 s.

    Relative to Random Rethink, RoCo-Rethink achieved:

    • +2.27 percentage points in accuracy (paired 95% CI [1.62, 2.98]).
    • A generated-token difference of −2.14 per problem (paired 95% CI [−5.90, 1.75]).

    In this OLMoE–GSM8K comparison, RoCo-Rethink achieved higher accuracy under the shared limits, with fewer generated tokens than LECO and similar token use to Random Rethink.

AuthorsOpenReview ↗

Gentle Reminder to Revisit Our Rebuttal and Evaluation

Dear Reviewer pGeM,

Thank you again for your positive assessment and support of our work. We greatly appreciate your thoughtful questions, which motivated additional controls, a prefix-only supervision experiment, and independent error-localization evaluation.

We hope our detailed responses provide a clearer basis for assessing the method and its empirical conclusions. We would be very grateful if you could revisit the rebuttal and consider updating your assessment and confidence level, where appropriate.

Your continued support is greatly appreciated. Please let us know if any remaining questions would help you reach a more confident assessment.

Thank you very much for your time and consideration.

Best regards,
The Authors

Reviewer_hzsfOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper uses MoE routing statistics to score reasoning steps. RoCo-Rethink revises the lowest-confidence step, while RoCo-PRM distills routing scores from a stronger MoE into a separate process reward model. Both improve downstream accuracy over the tested baselines, by about 1–2 points across several small MoE and dense models on six benchmarks.

Critical strengths
  • The routing-based signal requires no extra verification rollouts for labeling.
  • The paper tests several model families and tasks, and includes component and revision-oracle analyses.
Critical weaknesses
  • What does routing confidence measure? Its correlations with reference PRM scores are modest. The score may also favor longer, non-terminal steps because it uses within-step variation and head-token statistics. Random-step revision is not a strong enough control to rule this out. Length- and position-matched baselines are missing.
  • Limited validation as a PRM. RoCo-PRM is mainly tested as a step selector in the authors' revision pipeline. Direct step-error detection or best-of-N evaluation would better support the process-supervision claim. Some reported values also need clarification, particularly Table 15's OLMoE “before” scores. Also, Vanilla is described as greedy decoding, yet it has nonzero variance across seeds in Table 1.
Questions for authors
  1. How does RoCo compare with selecting the longest step, a fixed non-answer step, or a random non-answer step? How strongly is routing confidence correlated with step length?
  2. Can RoCo-PRM be evaluated on held-out step-error labels or best-of-N reranking?
  3. Please clarify the Table 15 scores and report comparable inference budgets across the revision baselines.
4 comments
AuthorsOpenReview ↗

Response to Reviewer hzsf — Part 1: Step length and position bias, and independent PRM validation (W1/Q1, W2/Q2)

Thank you for highlighting the need to distinguish routing-based revision gains from simple step-selection effects and to evaluate RoCo-PRM independently of our revision pipeline. We address these concerns below with controlled revision-point comparisons and human-annotated first-error evaluation.

W1/Q1. Does RoCo favor longer or non-terminal steps?

RoCo’s revision advantage persists when all selectors are restricted to the same non-answer candidates.

  1. Controlled comparison.

    We evaluated 100 initially incorrect OLMoE trajectories from GSM8K. We excluded all 200 explicit answer steps, leaving 506 non-answer candidates across the trajectories. All selectors chose from this same candidate set within their respective trajectories, using eight cached revisions per candidate.

    We compared RoCo with three selectors:

    • Longest step, directly testing whether choosing longer candidates explains the gains.
    • Last non-answer step, testing a fixed late position.
    • Uniform random non-answer step, measuring revision success under random selection from the same candidate set.
  2. Revision success.

    SelectorEstimated revision success
    Longest step6.75%
    Last non-answer step2.50%
    Uniform random non-answer step5.00%
    RoCo9.13%

    RoCo improved estimated revision success by:

    • 2.38 percentage points over longest-step selection (paired 95% CI [0.97, 4.22]).
    • 4.13 percentage points over uniform random selection (paired 95% CI [1.50, 6.95]).
  3. Association with step length.

    The mean within-trajectory Spearman correlation between routing confidence and step length was:

    • −0.6794 when answer steps were included.
    • +0.0357 among non-answer steps.

    The average rank correlation was close to zero within the non-answer candidate set. Together with the revision comparisons, this provides evidence that RoCo’s gains extend beyond selecting the longest step or a fixed late step on these trajectories.


W2/Q2. Can RoCo-PRM identify step errors on held-out annotations?

RoCo-PRM identifies independently annotated first errors on ProcessBench, outside the revision pipeline.

  1. Independent evaluation protocol.

    We evaluated the existing RoCo-PRM checkpoint on ProcessBench using human-annotated first-error locations. We calibrated the decision threshold on GSM8K (400 trajectories) and applied it unchanged to MATH, OlympiadBench, and Omni-MATH (1,000 trajectories each).

    Exact first-error accuracy and Within-1 are evaluated on erroneous trajectories using thresholded predictions. Predicting “no error” on an erroneous trajectory counts as a miss for both metrics.

  2. Results with the frozen threshold.

    DatasetExact first-error accuracyUniform random stepWithin-1ProcessBench F1
    MATH49.2%18.1%61.2%45.2%
    OlympiadBench41.7%13.6%58.0%40.0%
    Omni-MATH40.9%13.8%52.8%40.6%

    RoCo-PRM exceeded uniform random step selection by 27.1–31.1 percentage points in exact first-error accuracy across the three subsets.

  3. What this evaluation establishes.

    The evaluation compares the learned step scores with human error annotations directly. It therefore provides evidence that routing-derived supervision supports first-error localization beyond its use for selecting revision points.

AuthorsOpenReview ↗

Response to Reviewer hzsf — Part 2: Table 15 corrections, decoding clarification, and inference budgets (W2, Q3)

W2/Q3. What are the correct OLMoE “before” scores in Table 15?

We corrected the baseline entries and all associated gains. Leave-one-task-out improvements remain positive in all 12 settings.

  1. Source of the discrepancy.

    The OLMoE “before” entries in Table 15 came from the MC-PRM row of Table 1. The intended comparison is against Vanilla decoding. The corrected entries are:

    TaskOriginal “before”Correct Vanilla “before”RoCo-PRM “after”Corrected gain
    MATH50010.29.411.8+2.4 pp
    GSM8K50.648.752.0+3.3 pp
    MMLU49.246.849.6+2.8 pp
    HumanEval29.929.331.5+2.2 pp

    We also verified the remaining “before” entries and all “after” entries.

  2. Updated results and manuscript.

    Across all three target models and four tasks, the corrected mean gain is 2.45 percentage points, with improvements in all 12 model–task settings.

    We updated Table 15, Figure 6, Section 6.4, and Appendix G.2 in the revised manuscript.


W2. Why does Vanilla have nonzero standard deviation in Table 1?

The Vanilla baseline uses sampling, while RoCo-Rethink generates its initial response greedily.

The original description left these two decoding settings ambiguous:

  • Vanilla generates a single response with temperature 0.3 and top-p 0.95. Its nonzero standard deviations reflect variation across decoding seeds.
  • RoCo-Rethink generates its initial response greedily and samples revised suffixes with temperature 0.3 and top-p 0.95.

We clarified the settings in Section 5.1 and Appendix C.1.


Q3. How do inference budgets compare across revision methods?

Under shared revision limits, RoCo-Rethink achieved higher accuracy than LECO with fewer generated tokens and lower measured revision latency.

  1. Supplementary comparison under shared limits.

    We compared all three revision methods on the full set of 1,319 GSM8K test problems with OLMoE. They shared:

    • Identical initial trajectories, with 49.96% accuracy.
    • Greedy revision decoding.
    • A cap of four revision rounds and 512 new tokens per round.
    • The same convergence-based stopping and majority-vote rules.

    This supplementary experiment measures resource use under a common protocol. Realized costs vary with the selected revision point and the number of rounds before stopping.

  2. Accuracy and realized resource use.

    MethodFinal accuracyNew tokens/problemRevision rounds/problemRevision latency (s/problem)
    Random Rethink50.27%57.91.0910.37
    LECO51.33%84.21.1315.17
    RoCo-Rethink52.54%55.71.0710.21

    Latency starts from the shared initial trajectory and includes scoring and revision.

  3. Paired comparisons.

    Relative to LECO, RoCo-Rethink achieved:

    • +1.21 percentage points in accuracy (paired 95% CI [0.40, 2.06]).
    • −28.43 generated tokens per problem (paired 95% CI [−32.55, −24.32]).

    Relative to Random Rethink, RoCo-Rethink achieved:

    • +2.27 percentage points in accuracy (paired 95% CI [1.62, 2.98]).
    • −2.14 generated tokens per problem (paired 95% CI [−5.90, 1.75]).

    These measurements show an accuracy advantage under the shared limits, with lower generated-token use than LECO and similar generated-token use to Random Rethink.

AuthorsOpenReview ↗

Gentle Reminder to Revisit Our Rebuttal and Evaluation

Dear Reviewer hzsf,

Thank you again for your time and for recognizing the value of routing-based supervision and our broad experimental evaluation. We sincerely appreciate your suggestions, which helped us strengthen the evidence and improve the manuscript.

We have provided detailed responses to all your concerns, including the requested non-answer selection baselines, independent ProcessBench evaluation, inference-cost measurements, and manuscript corrections.

We would be very grateful if you could revisit our responses and reconsider your rating in light of this additional evidence. Your reassessment would mean a great deal to us. If any concerns remain, please let us know so that we can address them.

Thank you very much for your time and consideration.

Best regards,
The Authors

AuthorsOpenReview ↗

Thank You for Reassessing Our Paper

Dear Reviewer hzsf,

Thank you very much for revisiting our paper and updating your rating. We sincerely appreciate your time and support.

Your questions motivated stronger selection controls, independent PRM evaluation, and clearer reporting. We are grateful for your contribution to improving the work and remain available to clarify any remaining questions.

Best regards,
The Authors

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.