acceptodds
Under review as a conference paper at ICLR 2027

Between Tokens and Trajectories: Event-Level Test-Time Optimization for LLM Reasoning

Abstract

Test-time scaling has emerged as an effective way to improve the reasoning ability of large language models. Recent work has begun to complement sampling and search with instance-level test-time optimization, using reward signals to refine a reasoning trajectory. However, representative approaches differ in optimization granularity: some revise one token at a time, while others optimize token decisions independently. As a result, it is difficult for them to use a reward computed over the full response to coordinate updates across dependent reasoning decisions. We instead treat a short span of discrete token decisions that are autoregressively dependent as a single reasoning event, allowing one reward signal to guide their coordinated revision. Based on this formulation, we introduce Adaptive Event Policy Gradient (AEPG). At its core is a temporary event policy, adapted with reward-guided policy gradients to coordinate the revision of the entire event. During this process, each revised token is fed back into the frozen language model, so subsequent decisions condition on the updated context. Across three instruction-tuned LLMs and five mathematical reasoning benchmarks, AEPG achieves the highest average accuracy among the evaluated test-time methods on all three backbones. Ablation studies further show that multi-token event revision is an important contributor to these gains, highlighting event-level optimization as a promising direction for more effective test-time reasoning.

Primary area:
foundation or frontier models, including LLMs
Keywords:
test-time optimization, LLM reasoning

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_zYpUOpenReview ↗
Rating
2
Confidence
3
Informativeness
1
Presentation
1
Soundness
2
Read review
Paper summary

This paper proposes Adaptive Event Policy Gradient (AEPG), a test-time method that revises short spans of dependent tokens within a reasoning trajectory. It uses reward-guided policy gradients to adapt a lightweight controller while keeping the language model and reward model frozen. Experiments on three models and five mathematical reasoning benchmarks show improvements in average accuracy, supported by ablations and analyses of search coverage and inference efficiency.

Critical strengths
  1. Revising several dependent tokens together is a reasonable idea for correcting reasoning errors. The lightweight controller allows this adaptation while keeping the language model frozen.
Critical weaknesses
  1. The novelty appears incremental. The method combines existing reward-guided search and policy-gradient techniques, and its added value over PRM-guided search is not clearly established through discussion or direct comparisons.

  2. The computational budgets are not fully matched. AEPG uses additional probe rollouts beyond the final responses, making it difficult to separate the benefits of the method from those of additional search computation.

  3. The presentation should be clearer. Figure 1 contains too much text and lacks a clear visual focus, making it difficult to identify the main contribution and understand how the method differs from existing approaches.

Questions for authors
  1. What evidence distinguishes the proposed event-level adaptation from conventional reward-guided local search, particularly PRM-guided search?

  2. Is there evidence that the selected events correspond to coherent reasoning decisions, beyond showing that local prefix changes can improve subsequent generation?

2 comments
AuthorsOpenReview ↗

Response to Reviewer zYpU(Response1)

> Response to Question 1: Distinguishing event-level adaptation from PRM-guided search

There are several aspects to this question.

First, our problem is instance-level test-time optimization, where the key question is how to use a reward to revise an already generated CoT. We introduce this setting in the third paragraph of the Introduction. Intuitively, we focus on this direction because we believe it has greater potential and a higher ceiling than conventional generate-and-search procedures, and this is also partly supported by the comparison results reported in the instance-level optimization work we cite.

Second, regarding the distinction from PRM-guided search that you are most concerned about, we believe the most direct evidence comes from our ablation study. If we do not update ZZ, that is, if we do not jointly revise the already generated content, the procedure becomes closer to the reward-guided local search you mention: it still uses a reward to screen local prefixes, but it does not perform a joint policy update over multiple dependent decisions within an existing event. Empirically, our method performs better. We should note that Z=0Z=0 is not strictly equivalent to PRM-guided search: the former still uses an outcome reward for local prefix selection, whereas the latter is a search guided by dense step-level rewards. Nevertheless, Z=0Z=0 is the closest controlled setting to the “select but do not jointly revise” paradigm, and it therefore isolates the gain from joint revision.

In addition, our event is not a strict PRM step. A PRM step usually corresponds to a relatively complete semantic step boundary, whereas an AEPG event is a short sequence of token decisions determined by a pivot and an adaptive stopping rule, and it does not necessarily align with semantic step boundaries. Figure 4 in the paper is a concrete example: the revised event covers a short local span of decisions around the symmetry analysis, rather than a complete PRM step. Thus, AEPG is not simply renaming a PRM step; it jointly revises a shorter, autoregressively dependent span of token decisions.

Finally, there is also a factual distinction: PRM-guided search typically requires an additional process reward model and scores or prunes each step during search, whereas AEPG does not introduce a PRM and does not perform step-level beam search during generation. Instead, it jointly revises an already generated event.

AuthorsOpenReview ↗

Response to Reviewer zYpU(Response2)

> Response to Question 2: Evidence that selected events correspond to coherent reasoning decisions

On this question, our evidence comes mainly from the event selection mechanism, and not merely from showing that a local prefix change can improve the subsequent generation.

Intuitively, whether a position is worth using as the starting point of an event depends on whether it can have a sufficiently large effect on the subsequent generation. We consider two factors. First, the current position must admit alternative next tokens: if a different token is generated, the subsequent tokens are likely to change as well, and this factor can be measured by entropy. Second, the larger the semantic divergence among the candidate tokens, the more likely they are to lead to completely different subsequent reasoning. For example, if the candidates are “so” and “and”, the subsequent reasoning may still follow the same decision; but if the candidates are “however” and “so”, they are likely to lead to opposite reasoning directions. This factor can be formalized as the semantic dispersion of the candidate tokens. Therefore, at the beginning of the procedure, we use entropy * semantic dispersion of candidates as the criterion to select the starting point of an event, and then use an adaptive stopping rule to determine where the event ends.

This is directly supported by the results and analysis in our ablation study: compared with entropy-only, dispersion-only, first-uncertain, and random pivot selection, the joint criterion achieves the best pooled accuracy. Hence, our evidence is not only that local prefix changes can improve the subsequent generation, but also that the event selection mechanism itself tends to locate decision points that have a large influence on subsequent reasoning and whose candidate directions differ substantially, and the ablation results support the effectiveness of this selection criterion.

Reviewer_SjZoOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes Adaptive Event Policy Gradient (AEPG), a test-time optimization method for LLM reasoning. AEPG identifies pivot positions in an initial reasoning trajectory and optimizes short spans of autoregressively dependent tokens through a lightweight controller. A frozen reward model evaluates short probe rollouts, providing feedback for policy-gradient updates and guiding the selection of promising prefixes. These prefixes are subsequently extended into complete responses, with the final output selected by the same reward model. Experiments on three LLMs and five mathematical reasoning benchmarks show improvements over several test-time reasoning baselines.

Critical strengths

S1. Diverse baseline comparisons.

The paper compares AEPG with a diverse set of test-time reasoning methods, including response-level sampling and selection (BoN), answer aggregation (SC), planning-based search (RAP), latent-space optimization (LatentSeek and LTPO), and gradient-based logit refinement (∇-Reasoner). This provides relatively broad empirical coverage of existing test-time computation paradigms.

S2. Useful empirical and theoretical analyses.

Beyond final accuracy, the paper analyzes oracle search coverage and reward-based selection accuracy (Section 4.3), highlighting the gap between discovering correct solutions and successfully selecting them. Appendix D also derives the event-policy gradient under stated assumptions and characterizes the controller's single-state expressiveness, while explicitly distinguishing the practical surrogate update from the ideal objective. These analyses help clarify both the method and its limitations.

Critical weaknesses

W1. Limited methodological novelty and insufficient justification for event-level optimization.

The main components of AEPG are closely related to existing approaches. The pivot selection criterion is adopted from ROSE (Zhao et al., 2026), and the reward models are pretrained Skywork-Reward-V2 models. The main methodological contribution therefore lies in constructing multi-token reasoning events and adapting a lightweight controller through policy gradients.

However, the motivation for the specific event construction remains insufficiently justified. Although the paper defines events as variable-length token spans with heuristic stopping criteria, it does not adequately establish why these boundaries are preferable to simpler pivot-based resampling or alternative span constructions. The existing L=1L=1 ablation supports multi-token optimization but does not isolate the benefits of the proposed adaptive event boundaries.

Moreover, each controller is updated at most once (Appendix B.3), and the additional gains from controller adaptation are relatively modest in some settings. For example, removing the controller update reduces the average AIME accuracy from 14.44% to 12.22% on Qwen2.5-3B (Table 2). Although larger gains are observed on Qwen3-8B, the paper does not adequately establish whether controller adaptation provides sufficient benefits over simpler local search strategies under comparable computational budgets.

W2. Insufficient experimental rigor and unclear fairness of comparisons.

The evaluation does not sufficiently establish that the reported improvements are robust or achieved under comparable inference budgets.

First, while BoN and SC each generate eight complete responses, AEPG additionally performs up to 128 short probe rollouts before generating up to eight complete continuations. As acknowledged in Appendix A.4, matching the number of full responses does not imply matching total inference computation. Although Sections 4.4–4.5 analyze full-response budget scaling and inference latency, they do not establish the relative accuracy of these methods under matched total computation. It is therefore difficult to isolate the benefits of the proposed optimization mechanism from those of additional exploration.

Second, the evaluation protocol lacks important details, particularly the sampling hyperparameters, such as temperature and top-p, used by the baselines. The number of independent evaluation runs and random seeds used for the main results are also not clearly reported in the paper.

Finally, the reported gains are relatively small in several settings, especially on the AIME benchmarks, which contain only 30 problems each. For example, on Qwen2.5-7B, AEPG solves only three additional problems across AIME24/25/26 compared with ∇-Reasoner. Without repeated evaluations or uncertainty estimates, it is unclear whether these improvements are robust to sampling variability.

W3. Insufficient analysis of reward-model reliability and sensitivity.

AEPG relies heavily on reward models to guide controller updates, select promising prefixes, and rank final responses. However, the general reliability of these reward signals remains insufficiently established. In Appendix C.3, the reported reward-ranking consistency of 73.58%–82.22% is measured only on selected probe pairs with large reward differences (>3), and most pairs show no difference in final-answer correctness. While this provides some evidence for the usefulness of short-probe rewards, it does not fully establish their reliability across typical sampled probes.

Moreover, although two Skywork-Reward-V2 models are used for different backbone families, the paper does not conduct a controlled analysis of how reward-model choice affects optimization and final performance. Given the central role of reward signals in AEPG, further analysis of reward-model sensitivity and suitability for evaluating truncated probes is needed.

W4. Insufficient clarity of key methodological details.

Several important methodological details are deferred to the appendices, including the adaptive event stopping rule (Appendix B.1), the practical controller objective and update-skipping criterion (Appendix B.2), and the detailed controller parameterization (Appendix B.3). Moreover, although Algorithm 1 outlines prefix selection, it does not fully specify how the candidate pool from pre-update and post-update probes is maintained and ranked for full continuation. These details are essential to understanding the proposed optimization and search procedure. A more self-contained description of these core mechanisms in the main text would improve clarity.

Questions for authors

Q1. Necessity of event-level optimization.

Beyond the existing Z=0Z=0 ablation, could the authors compare AEPG against a simpler pivot-based resampling baseline that directly samples full continuations from the same selected pivots, without event-level controller optimization or short-probe screening? How do these approaches compare under matched total inference computation? This would help establish the incremental value of the proposed mechanism over standard pivot-based search.

Q2. Event construction.

What motivates the uncertainty–dispersion-based adaptive event boundaries? Although the paper includes L=1L=1 ablations and uses several maximum event-length settings, could the authors directly compare adaptive stopping against fixed-length events without uncertainty-based stopping, under otherwise comparable settings?

Q3. Evaluation protocol and reliability.

Could the authors clarify the sampling configurations of the baselines, including temperature and top-p, as well as the random seeds and number of independent runs for the main experiments? Are the reported accuracies from single runs or averaged over multiple runs? Given the small AIME test sets, results across multiple random seeds with confidence intervals would be important.

Q4. Compute-matched evaluation.

Beyond the full-commit budget scaling and latency analyses already provided, could the authors report accuracy-versus-compute comparisons under matched total generated tokens or end-to-end GPU time, accounting for probe generation and reward-model evaluation? This would help isolate the benefits of controller adaptation from those of additional sampling.

Q5. Reward-model sensitivity.

Could the authors compare different reward models under the same backbone and inference settings? In particular, how does reward-model choice affect short-probe ranking reliability, controller optimization, and final-answer accuracy? Such an ablation would help clarify which reward models are most suitable for AEPG and whether its improvements generalize beyond the current Skywork-Reward-V2 models.

5 comments
AuthorsOpenReview ↗

Response to Reviewer SjZo(Response1)

> Response to Q1: Necessity of event-level optimization over pivot-based resampling

In response to your suggestion, we added a simpler pivot-based resampling baseline to more directly test the incremental value of the event-level mechanism. This baseline reuses AEPG's initial response and selected pivots, and directly samples eight complete continuations from these positions, without event-level controller optimization and without short-probe screening (since the reward model has limited ability to recognize short events, we use model self-evaluation to obtain a relative preference when event scoring is needed). Complete responses are then screened by the Skywork reward model, and the answer is selected from a candidate pool that includes the initial response. For ease of comparison, the table below lists full AEPG, the existing Z=0Z=0 ablation, and the new pivot-based resampling baseline.

Table 1. Comparison with pivot-based resampling on Qwen2.5-3B-Instruct (seeds 42/44/46, mean ± std).

MethodAIME24AIME25AIME26Pooled AIME (90)
AEPG14.44 ± 1.9215.56 ± 1.9214.44 ± 1.9214.81 ± 0.64
Z=0Z=08.89 ± 1.9210.00 ± 3.3311.11 ± 1.9210.00 ± 1.92
Pivot-based resampling10.00 ± 0.008.89 ± 5.098.89 ± 1.929.26 ± 2.31

The results show that AEPG outperforms both Z=0Z=0 and the new pivot-based resampling baseline on all three AIME datasets and on pooled AIME.

AuthorsOpenReview ↗

Response to Reviewer SjZo(Response2)

> Response to Q2: Motivation and validation of adaptive event boundaries

On this question, we would like to clarify the motivation behind the uncertainty–dispersion-based criterion. Whether an event is worth selecting depends on whether it can have a sufficiently large effect on the subsequent generation. We consider two factors. First, the current position must admit alternative next tokens: if a different token is generated, the subsequent tokens are likely to change as well, and this factor can be measured by entropy. Second, the larger the semantic divergence among candidate tokens, the more likely they are to lead to completely different subsequent reasoning. For example, if the candidates are “so” and “and”, the subsequent reasoning may still follow the same decision; but if the candidates are “however” and “so”, they are likely to lead to opposite reasoning directions. This factor can be formalized as the semantic dispersion of the candidate tokens. We therefore use entropy * semantic dispersion of candidates as the criterion. When the score of subsequent positions remains below a threshold, it indicates that extending the event further would have little influence on subsequent reasoning; stopping at this point avoids wasting the revision budget on decisions that are already determined.

Second, regarding the fixed-length comparison you mentioned: from the design intuition of the stopping rule, fixed-length events have an inherent issue. Intuitively, if the score of a subsequent token is already below the threshold, meaning that revising the current token would have little effect on subsequent reasoning, then there is little reason to revise it; we should instead adjust the preceding content to steer the generation toward a different token, rather than directly revising this already determined token. A fixed-length event cannot distinguish this case: it either continues extending over tokens with very high certainty, wasting the revision budget, or truncates too early at positions where revision is still possible. Therefore, adaptive stopping and fixed-length events are not choices on the same equivalent axis. Likewise, our ablation with L=1L=1 considers only the first token when subsequent tokens may still change, and this ablation is necessary because it directly shows the importance of the event—i.e., more than a single token—for steering subsequent autoregressive generation.

AuthorsOpenReview ↗

Response to Reviewer SjZo(Response3)

> Response to Q3: Evaluation protocol, sampling configurations, and multi-seed reliability

For the sampling configurations: BoN, SC, and RAP all use temperature=0.8; BoN and SC use top-p=1.0, while RAP uses top-p=0.95. For ∇\nabla-Reasoner, in the AIME experiments, the rollout stage uses temperature=0.6 and top-p=0.95, and the resampling stage uses temperature=0.5 and top-p=0.95. Greedy, LatentSeek, and LTPO use greedy decoding for answer generation. AEPG also uses greedy decoding for the initial response and full continuations, while sampling within an event according to the controller policy.

Our experiments use seed 42, and the accuracies reported in the paper come from single runs. Given the small size of the AIME test sets, we agree that multiple random seeds are indeed necessary. We therefore prioritize additional multi-seed evaluation on AIME24, AIME25, and AIME26, using seeds 42, 44, and 46, and include AEPG, BoN, SC, and ∇\nabla-Reasoner. The table below reports the per-run accuracy, mean accuracy, and 95% confidence intervals.

Table 2. Multi-seed evaluation on AIME24/25/26 (mean ± std).

ModelMethodAIME24AIME25AIME26Pooled AIME (90)
Qwen2.5-3BBoN5.56 ± 1.938.89 ± 1.9310.00 ± 3.338.15 ± 1.70
Qwen2.5-3BSC10.00 ± 0.006.67 ± 3.335.56 ± 1.937.41 ± 0.64
Qwen2.5-3B∇\nabla-Reasoner10.00 ± 0.008.89 ± 1.937.78 ± 1.938.89 ± 1.11
Qwen2.5-3BAEPG14.44 ± 1.9315.56 ± 1.9314.44 ± 1.9314.81 ± 0.64
Qwen2.5-7BBoN13.33 ± 3.3315.56 ± 1.9314.44 ± 3.8514.44 ± 2.94
Qwen2.5-7BSC15.56 ± 1.9311.11 ± 1.9314.44 ± 3.8513.70 ± 1.28
Qwen2.5-7B∇\nabla-Reasoner22.22 ± 1.9318.89 ± 1.9317.78 ± 1.9319.63 ± 0.64
Qwen2.5-7BAEPG24.44 ± 1.9324.44 ± 1.9320.00 ± 3.3322.96 ± 0.64
Reviewer_U9eROpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
1
Soundness
2
Read review
Paper summary

This paper proposes AEPG, a test-time optimization method that revises short spans of reasoning tokens while keeping the language model frozen. It selects revision positions in an initial response, uses rewards from short probe rollouts to update a temporary controller over several token decisions, and generates complete answers from promising revised prefixes. Experiments on three instruction-tuned models and five math benchmarks show higher average accuracy than the evaluated baselines. Ablations examine the contributions of controller updates and multi-token revision, alongside analyses of search coverage and inference efficiency.

Critical strengths

The paper studies an interesting problem: how to use reward feedback to improve reasoning during inference without updating the base model. The experiments cover several models and benchmarks, with ablations examining controller updates and revision length.

Critical weaknesses
  1. I found parts of the method difficult to follow, especially the motivation for event-level optimization. Changing a token and regenerating autoregressively already allows later tokens to condition on that change. It would help to explain more concretely what additional coordination AEPG provides and which limitation of existing methods it addresses.

  2. I would like to better understand the contribution beyond a simpler search procedure: branching from the same pivots, sampling short probes without controller updates, selecting promising prefixes by reward, and generating complete answers. The Z=0Z=0 ablation is helpful, but could the authors clarify whether it uses the same number of probes and reward evaluations? This would help separate the benefit of controller optimization from additional sampling and selection.

  3. The computational comparison could be more informative. AEPG uses up to 128 short probes and their reward evaluations before generating up to eight complete continuations, while BoN generates eight complete answers. A comparison under matched total computation, allowing BoN to sample more answers within that budget, would help assess the practical benefit of the method.

  4. More information about result stability would be helpful. Each AIME benchmark contains only 30 problems, so some improvements correspond to one or two additional correct answers. Could the authors clarify how many independent runs were performed and report variation across random seeds?

Questions for authors

See Weakness.

2 comments
AuthorsOpenReview ↗

Response to Reviewer U9eR(Response1)

> Response to Weakness 1: Motivation and clarity of event-level optimization

We thank the reviewer for pointing this out.

Why an event? Intuitively, a useful revision needs to satisfy two things: the revised prefix itself should be good, and the revision should be substantial enough to change the direction of subsequent reasoning, rather than just producing a local wording change. Existing single-token methods optimize or accept one next-token decision at a time and then let the frozen language model continue. They give fine-grained control, but their update unit is narrow: local wording may change while the key reasoning span stays the same, so the continuation still follows the original wrong logic. A single-token change is also easier to be pushed in the wrong direction by one or two reward-model misjudgments. LatentSeek optimizes token-wise latents and decodes them independently before autoregressive continuation, while ∇\nabla-Reasoner refines and commits the next-token decision one step at a time; neither directly coordinates a span of interdependent reasoning decisions under one reward. That is why I optimize a short span jointly, and why I call it an event.

What happens after we define an event? Prior instance-level test-time optimization methods do introduce updatable quantities, such as token-wise latents or logits, and optimize them with policy-gradient-style updates. However, these quantities are typically tied to individual token positions. As a result, existing methods can mainly revise the current token's logits or latent representation, and making several positions change together is difficult. For event-level optimization, I attach a temporary policy parameter ZZ. ZZ is not tied to a specific token position; it is shared across multiple token positions within the event. Because ZZ is shared, the token decisions inside the event can be updated jointly under the same reward signal, instead of each position being changed separately.

Take the square-table example in Figure 4. The initial response makes a mistake in the symmetry analysis and reduces the square-table problem to a circular-table-style computation, 7!=50407! = 5040. The key issue is not a single wrong token; rather, a short reasoning span—symmetry analysis, fixing a reference, and the counting rule—needs to change together. AEPG selects this position as a pivot and jointly revises this reasoning event, so the continuation moves to 8!/4=100808!/4 = 10080. In contrast, methods that only revise local tokens may change the surrounding wording but keep the same erroneous computation.

AuthorsOpenReview ↗

Response to Reviewer U9eR(Response2)

> Response to Weaknesses 2–4: Concerns about Fairness

We understand that your main concerns are twofold: (i) AEPG uses multiple probes to assist prefix selection, which increases token generation; and (ii) the multi-seed stability of the results.

1) Fairness of using multiple probes

Why multiple probes? In test-time computation, the reward model is not perfectly reliable. Scoring only the event itself can be noisy, so we introduce a small number of short probe continuations to help evaluate the reward of the revised event. These probes are short-prefix rollouts, not additional full responses; the final full-commit budget remains at most eight complete continuations. For clarity, the Z=0Z=0 ablation uses exactly the same number of probes, reward evaluations, and full-commit budget as full AEPG; the only difference is that the controller ZZ is not updated.

Motivated by your concern, we ran a more controlled fairness check by removing the short probes. To keep reward scoring reasonably accurate without probes, we use a carefully prompted model-based self-reward to score the event, so that event revision can still be evaluated without probe rollouts. We ran this variant with three seeds, focusing on the representative Qwen2.5-7B model and the AIME benchmarks due to time and compute constraints.

Table 1. Probe-free fairness check on AIME24/25/26 (mean ± std).

SelectionAIME24AIME25AIME26Pooled AIME (90)
AEPG24.44 ± 1.9324.44 ± 1.9320.00 ± 3.3322.96 ± 0.64
AEPG(no probes)24.44 ± 3.8523.33 ± 3.3421.00 ± 1.7322.96 ± 1.70

The results show that even without the extra probe generation, AEPG still retains a clear advantage.

2) Multi-seed experiments

The main paper reports results with a single seed. Considering time and compute constraints, we chose AIME—the benchmark most sensitive to random variation due to its small size—and compared against three strong baselines: BoN, SC, and ∇\nabla-Reasoner. Results are shown below.

Table 2. Multi-seed evaluation on AIME24/25/26 (mean ± std).

ModelMethodAIME24AIME25AIME26Pooled AIME (90)
Qwen2.5-3BBoN5.56 ± 1.938.89 ± 1.9310.00 ± 3.338.15 ± 1.70
Qwen2.5-3BSC10.00 ± 0.006.67 ± 3.335.56 ± 1.937.41 ± 0.64
Qwen2.5-3B∇\nabla-Reasoner10.00 ± 0.008.89 ± 1.937.78 ± 1.938.89 ± 1.11
Qwen2.5-3BAEPG14.44 ± 1.9315.56 ± 1.9314.44 ± 1.9314.81 ± 0.64
Qwen2.5-7BBoN13.33 ± 3.3315.56 ± 1.9314.44 ± 3.8514.44 ± 2.94
Qwen2.5-7BSC15.56 ± 1.9311.11 ± 1.9314.44 ± 3.8513.70 ± 1.28
Qwen2.5-7B∇\nabla-Reasoner22.22 ± 1.9318.89 ± 1.9317.78 ± 1.9319.63 ± 0.64
Qwen2.5-7BAEPG24.44 ± 1.9324.44 ± 1.9320.00 ± 3.3322.96 ± 0.64

Although the main experiments use a single-seed protocol, we hope these additional results alleviate your concern about stability.

Comments on the paper

2 comments
AuthorsOpenReview ↗

Summary of Additional Experiments and Clarifications

We believe that the main concerns raised by the reviewers center on three aspects: methodological novelty and the necessity of event-level optimization, computational fairness with respect to probe-based reward evaluation, and result stability on the small AIME benchmarks. To address these concerns, we substantially expanded our empirical evaluation during the rebuttal period.

Specifically, we (1) further clarify the distinction between AEPG and existing reward-guided local search / PRM-guided search, and add a pivot-based resampling baseline to directly test the incremental value of the event-level mechanism over simpler pivot-based search; (2) explain that the short probes are short-prefix rollouts and are not counted toward the final full-commit budget, and add probe-free fairness experiments as well as reward-model sensitivity experiments, showing that the core advantage of AEPG does not rely on the extra probe computation; and (3) add multi-seed experiments on AIME24/25/26 for AEPG, BoN, SC, and ∇\nabla-Reasoner, reporting mean, standard deviation, and pooled AIME results to provide a more stable comparison.

These additional experiments directly address the main concerns regarding novelty, computational fairness, and stability. Across all newly added evaluations, AEPG remains competitive under the same experimental settings. We believe these results substantially address the issues raised by the reviewers. We hope the reviewers and the Area Chair will evaluate the paper in light of both the original manuscript and the newly provided evidence; all newly added experiments and analyses will be incorporated into the final version of the paper.

Hi Reviewers,

The authors have provided responses. Please check if they have addressed your concerns.

Thanks, AC

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.