acceptodds
Under review as a conference paper at ICLR 2027

Bending the Thought: Curvature-Guided Reinforcement Learning for LLM Reasoning

Abstract

Reinforcement learning with verifiable rewards (RLVR) improves large language model reasoning using only final-answer feedback. This raises a token-level credit assignment problem: only a small subset of local computations may account for the final result, while the reward fails to indicate the locations of such computations. Existing token-level credit allocation researches, such as token entropy or update magnitude, are mostly defined on the policy side and mainly capture uncertainty or distributional change in the probability space, while neglecting the model’s internal computational processes. Inspired by the cognitive neural-manifold, we view reasoning as a trajectory in the hidden representation space. Under a smooth-trajectory idealization, we derive an upper bound on the second-order variation of a smooth scalar readout that includes a curvature-dependent term. This motivates us to treat curvature as a candidate signal for token-level update credit assignment. Building on this analysis, we introduce ARC (Adaptive Reweighting by Curvature), a curvature-guided RLVR framework that converts response-normalized curvature scores into bounded, detached weights on the token-level policy surrogate. Empirical analysis reveals a correlation between high-curvature positions and reasoning markers, alongside layer-dependent differences in entropy-based token selections. Evaluations on six mathematical reasoning benchmarks show improved performance over the baseline for both Qwen3-8B-Base and Qwen3-14B-Base, including gains of 5.98 and 8.27 percentage points on AIME 2024, respectively.

Primary area:
reinforcement learning
Keywords:
reinforcement learning, credit assignment, neural manifold

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_m9QhOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes a token-level reweighting scheme called ARC for improving large language model reasoning through reinforcement learning with verifiable rewards. The scheme treats hidden states as trajectories and, based on local curvature, assigns higher weights to tokens near points where the trajectory direction changes; this design is motivated by the concept of semantic change boundaries. The authors further examine the method’s alignment with inference tokens and its complementarity with token entropy, and report performance improvements on six mathematics benchmarks using Qwen3-8B and Qwen3-14B. While the idea is compelling and the reported improvements are encouraging, the current evidence is insufficient to clearly explain why curvature is effective, nor does it fully demonstrate the reliability of these gains.

Critical strengths

(1) This paper explores a useful representation-level signal for token-level credit attribution. The proposed soft weights are bounded and normalized within each reply, making it relatively straightforward to adapt the model to the DAPO objective.

(2) The experiments were quite comprehensive, covering two Qwen3 model sizes and six mathematical reasoning benchmarks. The authors also set up entropy-based baselines, conducted ablation experiments by shuffling weights, and performed additional diagnostic experiments on Llama.

Critical weaknesses

Major concerns

(1) The theoretical result does not truly establish the proposed connection to credit allocation. Proposition 1 merely provides an upper bound on the second-order variation of a certain smooth readout function, but high curvature does not necessarily imply a meaningful change in reasoning or a greater contribution to the final reward. The geometric argument here serves, at most, to provide motivation for the method; it does not prove that it identifies important tokens.

(2) These experiments are not sufficient to distinguish curvature from simpler hidden-state signals. Although the paper compares entropy-based weights with shuffled weights, it does not examine alternative metrics such as step size, rotation angle or second-order differences. Without these comparisons, it is difficult to determine whether the performance improvement is actually attributable to curvature itself.

(3) Table 1 does not present results for different independent training seeds. Sampling 32 answers per question helps to estimate the evaluation accuracy, but fails to reflect differences between multiple reinforcement learning runs. This is particularly crucial given that the improvements achieved by Qwen3-8B on several benchmarks are already modest. Consequently, the existing results still do not demonstrate whether the reported improvements are stable.

Minor concerns

(1) The paper needs to clarify the specific source of the hidden states used in Eq.(4) to (5), for example, whether they are derived from the residual flow output or the normalised output of a particular module. Different choices may affect the curvature score and reproducibility.

(2) Section 4.5 notes that the recovery length increases rapidly when strong curvature selection is employed, but does not provide statistical data on the length or training curves. If a small-scale quantitative comparison were included, this design trade-off could be better assessed.

Questions for authors

(1) How were the curvature layers and the weight parameter γ = 0.6 selected? Are the main results robust to other layers and weighting strengths?

(2) Could the authors discuss whether other geometric measures of hidden-state changes, such as step length or turning angle, might also be useful for token reweighting?

(3) The curvature-sensitive layer differs between Qwen3 and Llama. Could the authors discuss what might explain this difference?

3 comments

We thank the reviewer for recognizing our method and its performance improvements. We address your questions below.

  1. Theoretical Result (Weakness 1)

Proposition 1 provides a geometric motivation for curvature. Curvature thus has a clear mathematical connection to rapid changes induced by the trajectory. The definition of the score follows from this connection independently of the marker-word correlations. The upper bound also has a clear limitation. High curvature allows for larger changes, but does not guarantee that they occur. This explains why curvature can identify candidate turning points without directly equating them with contributions to correctness. We further test the usefulness of ARC in controlled training experiments.

We use two levels of evidence. Analyses of reasoning markers, position, and overlap with entropy characterize the scoring signal. Tables 1 and 3 directly test its usefulness when incorporated into reinforcement learning updates. In particular, Entropy-Soft uses exactly the same soft-weighting mechanism and changes only the scoring signal. ARC scores higher in 11 of the 12 reported model--benchmark combinations. This comparison directly addresses the explanation that the gains come entirely from soft weighting.

In addition, the diagnostic of curvature-only positions examines the extent of overlap with entropy, and the performance drop after shuffling the weights tests the role of matching scores to token positions. These controls support the practical value of representation geometry.

  1. Other Metrics (Weakness 2, Question 2)

Our method derives a theoretical upper bound on semantic change based on curvature. The derivation determines our choice of score. We discuss the two proposed metrics from this theoretical perspective.

\textbf{Step length cannot distinguish straight motion from turning.} For example, two trajectory segments can have exactly the same adjacent step lengths, with one continuing in a straight line and the other turning through 90∘90^\circ. Step length assigns them the same score, whereas ARC gives near-zero curvature to the straight segment and positive curvature to the turn.

\textbf{Turning angle captures directional change but lacks an arc-length scale.} The curvature estimate in Eq. (5) is \begin{equation*} \widehat{\kappa}_t \approx 2\sin(\theta_t/2)/\bar{\ell}_t, \end{equation*} where θt\theta_t is the angle between adjacent displacements and ℓˉt\bar{\ell}_t is the mean step length. If two positions have the same turning angle but one completes the turn over half the distance of the other, the turning angle alone assigns them the same score, while ARC's curvature estimates differ by a factor of two.

Thus, step length omits the direction change, and turning angle omits the distance over which that change occurs. ARC retains both pieces of information and directly corresponds to the geometric quantity that bounds the trajectory-bending term in Proposition 1.

  1. Evaluation with Multiple Seeds and Ablations (Weakness 3, Question 1)

We clarify that representation geometry differs across layers. Our experiments show that, at particular layers, curvature represents semantic transitions better than entropy and random selection. The reasoning process itself determines the relevant layers, since only the geometry at certain layers captures this signal. We then use this optimal signal to improve the model's subsequent behavior.

Table compares the two-run mean and the γ=1\gamma=1 results with the original DAPO and ARC results from Table 1.

SettingAIME24AIME25AMC23MATH500MinervaOlympiad
DAPO (Table 1)29.6522.1877.3287.4035.0949.76
ARC (γ=0.6\gamma=0.6, Table 1)35.6325.6378.2888.4036.6350.75
ARC (γ=1\gamma=1)32.1324.5677.6488.1035.5950.27
ARC (mean over two seeds)35.6625.6178.0988.4336.8750.74
  1. Choice of Hidden States (Weakness 4)

We compute curvature from the residual stream after the intermediate decoder block. Equation (4) normalizes the displacement vectors between adjacent hidden states to construct unit tangents.

  1. Changes in Response Length during Training (Weakness 5)

Thank you for raising this point. We will include the changes in response length during training in the appendix of the revised manuscript.

  1. Differences in Curvature Sensitivity across Model Families (Question 3)

One possible explanation is that the two models differ in architecture and post-training objectives, causing checks, corrections, or transitions to conclusions during reasoning to appear as more pronounced turns in the hidden-state trajectory at different depths. Moreover, the two models generate their own responses. Differences in tokenization and marker usage may also affect the alignment measured within a ±3\pm 3-token window. The current results show that curvature is associated with reasoning markers in the intermediate layers of both models, but the peak layer needs to be identified separately for each model.

If our response has addressed your concerns, we would appreciate it if you could reconsider your overall assessment of the paper.

Reviewer_E2f5OpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes ARC, a curvature-guided reinforcement learning method for improving token-level credit assignment in LLM reasoning. The key idea is to view the model's hidden representations as a trajectory and use local curvature to identify potentially important reasoning transitions. The authors provide a theoretical analysis relating trajectory curvature to semantic variation and incorporate curvature-based weights into the DAPO training objective. Experiments on Qwen3-8B and Qwen3-14B across 6 mathematical reasoning benchmarks show improvements over standard DAPO and entropy-based reweighting methods.The paper also includes analyses of curvature alignment with reasoning markers and its complementarity with token entropy.

Critical strengths

The paper explores an interesting direction for improving RLVR credit assignment by leveraging internal hidden-state geometry rather than relying solely on output-level statistics such as token entropy.

The proposed ARC method is relatively simple and can be integrated into existing RLVR training objectives without requiring additional process-level reward annotations.

Critical weaknesses

1. Limited novelty relative to closely related work.

I find the novelty of this paper somewhat limited. Several recent works have already investigated the geometry of LLM hidden-state trajectories, including curvature. For example, The Geometry of Reasoning (ICLR 2026) studies reasoning through representation-space trajectories using geometric quantities such as velocity and curvature, while Representational Curvature Modulates Behavioral Uncertainty (ICML 2026) directly investigates the relationship between hidden-state curvature and token-level uncertainty. Furthermore, methods such as SHEAR (2026) and FlowTracer (ICML 2026) have already explored using internal model representations or information flow for fine-grained RL credit assignment. These highly relevant works are not adequately discussed or compared against. The main methodological contribution of ARC appears to be replacing existing token-importance signals with curvature scores and applying a relatively standard soft-reweighting mechanism to DAPO. I am not convinced that this constitutes a substantial methodological advancement.

2. Insufficient theoretical and empirical justification for curvature as a reasoning-importance signal.

The central assumption that high-curvature positions correspond to important reasoning transitions is not sufficiently established. The theoretical result in Proposition 1 is essentially an upper bound on the second-order variation of a smooth semantic readout, derived using standard differential-geometric arguments. Importantly, this bound does not imply that high curvature necessarily corresponds to large semantic changes, let alone greater contribution to the final reasoning outcome. Empirically, the evidence mainly consists of correlations with surface-level reasoning markers (e.g., "therefore" and "however") and response positions. These may reflect linguistic or structural transitions rather than genuinely important reasoning decisions. Stronger validation, such as counterfactual interventions, token perturbations, or annotations of critical reasoning steps, would be needed to establish the proposed interpretation.

3. Missing key ablations to isolate the contribution of curvature.

The paper does not sufficiently establish whether the performance improvements specifically come from the proposed curvature measure. Curvature combines changes in hidden-state direction with local step lengths, while the diagnostic results also show a strong preference for tokens in the middle of reasoning trajectories. Therefore, simpler alternatives such as turning-angle-only, step-length-only, or position-based reweighting could potentially explain part of the observed gains. Although the authors include entropy-based and shuffled-weight controls, these do not isolate the individual factors underlying curvature. Moreover, the method relies on selecting a specific Transformer layer (Layer 21 for 8B and Layer 24 for 14B) and a fixed reweighting strength (γ = 0.6), without systematic ablations of these choices. These experiments are important for establishing both the necessity and robustness of the proposed mechanism.

4. Insufficient experimental robustness, reproducibility, and efficiency analysis.

While ARC improves performance across the reported mathematical benchmarks, several improvements are relatively modest, particularly for the 8B model. The paper does not report multiple training seeds, confidence intervals for the main performance gains, or statistical significance against the strongest entropy-based baseline. In addition,important RLVR training details, such as training steps, learning rate, batch size, and checkpoint selection, are insufficiently specified, making reproduction difficult. The computational overhead of extracting and processing intermediate hidden states is also not quantified in terms of gpu memory, training throughput, or wall-clock time. Finally, although diagnostic experiments are conducted on Llama, actual RLVR improvements are demonstrated only on qwen3 models and mathematical reasoning tasks, leaving the broader generalizability of ARC unclear.

Questions for authors

Novelty: Could the authors clarify the key differences between ARC and closely related works such as The Geometry of Reasoning, Representational Curvature Modulates Behavioral Uncertainty, SHEAR, and FlowTracer? What is the main conceptual advancement beyond applying curvature-based token reweighting to an existing RLVR objective?

Theoretical and semantic justification: How does Proposition 1 justify assigning greater RL credit to high-curvature tokens, given that it only establishes an upper bound on semantic variation? Could the authors provide stronger empirical evidence that high-curvature positions correspond to genuinely important reasoning decisions rather than linguistic transitions?

Ablation studies: Could the authors compare curvature-based weighting against turning-angle-only, step-length-only, and position-based weighting? How sensitive is ARC to the selected Transformer layer and the reweighting coefficient γ?

Robustness and efficiency: Could the authors report multi-seed training results, complete training configurations, and the additional GPU memory and training-time overhead of curvature computation? Additionally, does ARC provide similar benefits on other model families or non-mathematical reasoning tasks?

3 comments

We thank the reviewer for recognizing our method and its performance improvements. We address your questions below.

  1. Novelty (Weakness 1, Question 1)

These four studies differ from ARC in what they analyze and how they define their scores. The Geometry of Reasoning pools multiple tokens within each reasoning step into a single point and uses step-level Menger curvature to compare whether different derivations share the same logical structure. King also analyze token-level trajectories, but average recent turning angles to predict next-token entropy. The two studies examine geometric similarity across derivations and predictive uncertainty, respectively.

SHEAR computes the Wasserstein distance between span-level hidden-state distributions from correct and incorrect responses to the same question. FlowTracer computes how much flow passes through a token in the attention graph from the question to the generated answer. Our method derives a curvature-based theoretical upper bound on semantic change and validates it experimentally.

If the reviewer believes that prior work already covers ARC's allocation mechanism, we would appreciate clarification as to which scoring definition or allocation rule specifically overlaps.

  1. Evidence for Curvature (Weakness 2, Question 2)

Proposition 1 provides a geometric motivation for curvature. Curvature thus has a clear mathematical connection to rapid changes induced by the trajectory. The definition of the score follows from this connection independently of the marker-word correlations. The upper bound also has a clear limitation. High curvature allows for larger changes, but does not guarantee that they occur. This explains why curvature can identify candidate turning points without directly equating them with contributions to correctness. We further test the usefulness of ARC in controlled training experiments.

We use two levels of evidence. Analyses of reasoning markers, position, and overlap with entropy characterize the scoring signal. Tables 1 and 3 directly test its usefulness when incorporated into reinforcement learning updates. In particular, Entropy-Soft uses exactly the same soft-weighting mechanism and changes only the scoring signal. ARC scores higher in 11 of the 12 reported model--benchmark combinations. This comparison directly addresses the explanation that the gains come entirely from soft weighting.

In addition, the diagnostic of curvature-only positions examines the extent of overlap with entropy, and the performance drop after shuffling the weights tests the role of matching scores to token positions. These controls support the practical value of representation geometry.

  1. Ablation Studies (Weakness 3, Question 3)

First, I consider the proposed comparisons unrelated to our method. We derive a theoretical upper bound on semantic change based on curvature, and this derivation motivates our choice of score. The proposed methods based on step length or turning angle do not follow from our theoretical argument.

\textbf{Step length cannot distinguish straight motion from turning.} For example, two trajectory segments can have exactly the same adjacent step lengths, with one continuing in a straight line and the other turning through 90∘90^\circ. Step length assigns them the same score, whereas ARC gives near-zero curvature to the straight segment and positive curvature to the turn.

\textbf{Turning angle captures directional change but lacks an arc-length scale.} The curvature estimate in Eq. (5) is \begin{equation*} \widehat{\kappa}_t \approx 2\sin(\theta_t/2)/\bar{\ell}_t, \end{equation*} where θt\theta_t is the angle between adjacent displacements and ℓˉt\bar{\ell}_t is the mean step length. If two positions have the same turning angle but one completes the turn over half the distance of the other, the turning angle alone assigns them the same score, while ARC's curvature estimates differ by a factor of two.

Thus, step length omits the direction change, and turning angle omits the distance over which that change occurs. ARC retains both pieces of information and directly corresponds to the geometric quantity that bounds the trajectory-bending term in Proposition 1.

We also clarify that representation geometry differs across layers. Our experiments show that, at particular layers, curvature represents semantic transitions better than entropy and random selection. The reasoning process itself determines the relevant layers, since only the geometry at certain layers captures this signal. We then use this optimal signal to improve the model's subsequent behavior.

Table compares the two-run mean and the γ=1\gamma=1 results with the original DAPO and ARC results from Table 1.

SettingAIME24AIME25AMC23MATH500MinervaOlympiad
DAPO (Table 1)29.6522.1877.3287.4035.0949.76
ARC (γ=0.6\gamma=0.6, Table 1)35.6325.6378.2888.4036.6350.75
ARC (γ=1\gamma=1)32.1324.5677.6488.1035.5950.27
ARC (mean over two seeds)35.6625.6178.0988.4336.8750.74
  1. Training details (Weakness 4, Question 4)

Thank you for asking about the training details. Apart from the curvature layer and the weighting strength, all other parameters are the same as in DAPO. We will also document the training details in the appendix to facilitate reproduction. Regarding the training results, we would like to clarify that the evidence in the current Table 1 does not depend on the 0.16-percentage-point gain on 8B Olympiad. ARC outperforms DAPO on all six benchmarks for both models and outperforms Entropy-Soft in 11 out of 12 settings. On AIME24, the gains over Entropy-Soft are 4.07 and 4.01 percentage points for 8B and 14B, respectively. The 14B model shows positive gains on all six benchmarks.

If our response has addressed your concerns, we would appreciate it if you could reconsider your overall assessment of the paper.

Reviewer_JB4yOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes ARC (Adaptive Reweighting by Curvature), a method for allocating token-level policy updates in reinforcement learning with verifiable rewards. ARC treats hidden states at a fixed transformer layer as a trajectory across response positions and estimates its local curvature from changes in normalized tangent directions. Response-normalized curvature scores are mapped to bounded, detached weights with unit mean, which reweight the DAPO policy surrogate while retaining the original outcome-based supervision.

The paper provides a geometric bound motivating curvature as a candidate signal, diagnostic analyses of its relationship with reasoning markers and entropy, and reinforcement learning experiments on Qwen3-8B-Base and Qwen3-14B-Base. ARC improves over DAPO on six mathematical benchmarks, including gains of 5.98 and 8.27 percentage points on AIME 2024.

Critical strengths
  1. An interesting representation-based signal for policy-update allocation: The paper explores hidden-state trajectory geometry as an alternative to output-level uncertainty. Computing local curvature provides a concrete, inexpensive-to-define signal that can be incorporated into an existing RLVR objective without additional process annotations or a learned process reward model.
  2. A simple and reasonably controlled training modification: Bounded soft weights, response-level mean normalization, and stop-gradient make the intervention easy to understand. The method retains updates for all response tokens and preserves the verifier-derived advantage. The reported failure of hard curvature selection also provides useful motivation for the softer formulation.
  3. Consistent improvements over DAPO and informative comparison methods: ARC improves over DAPO across both model sizes and all six benchmarks. The Entropy-Soft baseline is particularly useful because it retains the same weighting scheme while replacing the scoring signal. ARC also outperforms this baseline on most benchmarks, suggesting that its benefit is not explained solely by introducing soft reweighting.
  4. Extensive diagnostics and a meaningful shuffled-weight control: The paper examines layer dependence, response positions, marker proximity, entropy overlap, and sensitivity to diagnostic thresholds. Repeating the diagnostics on Llama provides evidence that the observed associations are not unique to Qwen. The shuffled-weight experiment preserves the weight distribution while disrupting its correspondence with tokens, supporting the importance of this correspondence.
Critical weaknesses
  1. The main training results lack sufficient reproducibility and uncertainty reporting: The paper specifies the curvature layers and weighting strength but does not adequately document the complete training protocol, including learning rate, training duration, batch and rollout configuration, and generation limits. Results are averaged over 32 sampled responses per problem, which does not establish robustness across independent training runs. This distinction matters for smaller improvements, such as the 0.16-point advantage over Entropy-Soft on 8B Olympiad. Training memory and runtime overheads are also not quantified.
  2. Response length and computational budget are insufficiently controlled: The hard-curvature variant reportedly causes rapid response-length growth, indicating that allocation choices can substantially change generation behavior. However, the paper does not provide corresponding length statistics or learning curves for ARC and its main baselines. Consequently, it is difficult to determine how much of the accuracy improvement reflects better update allocation versus changes in generated reasoning length or effective training compute. Evaluation under matched generation budgets would strengthen the empirical conclusion.
  3. The evidence supports a useful heuristic more strongly than the proposed reasoning interpretation: The theoretical result bounds semantic second-order variation from above by a curvature-dependent expression. High curvature therefore permits, but does not imply, substantial semantic change. Likewise, proximity to words such as “therefore,” “assume,” or “answer” does not establish that a token is consequential for correctness. The paper acknowledges this limitation, but stronger claims about reasoning transitions and improved credit assignment should remain qualified. Controls based on response position or simpler hidden-state statistics would help determine what curvature specifically contributes.
  4. Layer selection and training robustness are not sufficiently investigated: The diagnostics show that curvature is informative only at particular layers and can perform worse than random selections in early layers. Nevertheless, the training experiments use fixed layers 21 and 24 without a systematic study of how these choices affect downstream performance. Sensitivity to the training coefficient γ, and stability of the diagnostic signal as the policy changes, are also unexplored. The appendix’s selection-ratio and marker-window sweeps address diagnostic choices rather than these training dependencies.
  5. Some diagnostic evidence requires more cautious interpretation: At the strongest Qwen layer, marker alignment is approximately 15% for curvature versus 13.7% for entropy, while entropy performs better at many other layers. Moreover, the Layer-21 boxplot in Figure 7 does not clearly show the claimed distance-to-marker advantage over shuffled or random selections. This should be reconciled with the accompanying description. The Llama experiments establish cross-model diagnostic associations, but do not demonstrate cross-family improvements from ARC training.
Questions for authors
  1. Can you provide the complete training and evaluation configuration and results across independent training seeds? Are the smaller gains over Entropy-Soft reproducible?
  2. How do response length, total generated training tokens, and evaluation compute compare across DAPO, Entropy-Soft, and ARC? Do the gains persist under matched generation budgets?
  3. How were layers 21 and 24 selected? How sensitive are downstream results to layer choice and γ, and does curvature remain informative throughout reinforcement learning?
  4. How does ARC compare with similarly normalized weights based on token position, hidden-state step length, or turning angle without arc-length normalization? These controls could clarify the contribution of the specific curvature formulation.
  5. Can you clarify the Layer-21 distance distributions in Figure 7? Also, is Cur-Shuffled’s consistent degradation below DAPO reproducible across seeds, and do training curves help explain it?
3 comments

We thank the reviewer for recognizing our method and its performance improvements. We address your questions below.

1、Training Details and Multiple Seeds (Weakness 1, Question 1)

Thank you for asking about the training details. Apart from the curvature layer and the weighting strength, all other parameters are the same as in DAPO. We will also document the training details in the appendix to facilitate reproduction. Regarding the training results, we would like to clarify that the evidence in the current Table 1 does not depend on the 0.16-percentage-point gain on 8B Olympiad. ARC outperforms DAPO on all six benchmarks for both models and outperforms Entropy-Soft in 11 out of 12 settings. On AIME24, the gains over Entropy-Soft are 4.07 and 4.01 percentage points for 8B and 14B, respectively. The 14B model shows positive gains on all six benchmarks.

Table compares the two-run mean and the γ=1\gamma=1 results with the original DAPO and ARC results from Table 1.

SettingAIME24AIME25AMC23MATH500MinervaOlympiad
DAPO (Table 1)29.6522.1877.3287.4035.0949.76
ARC (γ=0.6\gamma=0.6, Table 1)35.6325.6378.2888.4036.6350.75
ARC (γ=1\gamma=1)32.1324.5677.6488.1035.5950.27
ARC (mean over two seeds)35.6625.6178.0988.4336.8750.74

2、Response Length during Training (Weakness 2, Question 2)

Thank you for raising this point. We will include the changes in response length during training in the appendix of the revised manuscript.

3、Evidence for Curvature (Weakness 3, Question 4)

Proposition 1 provides a geometric motivation for curvature. Curvature thus has a clear mathematical connection to rapid changes induced by the trajectory. The definition of the score follows from this connection independently of the marker-word correlations. The upper bound also has a clear limitation. High curvature allows for larger changes, but does not guarantee that they occur. This explains why curvature can identify candidate turning points without directly equating them with contributions to correctness. We further test the usefulness of ARC in controlled training experiments.

We use two levels of evidence. Analyses of reasoning markers, position, and overlap with entropy characterize the scoring signal. Tables 1 and 3 directly test its usefulness when incorporated into reinforcement learning updates. In particular, Entropy-Soft uses exactly the same soft-weighting mechanism and changes only the scoring signal. ARC scores higher in 11 of the 12 reported model--benchmark combinations. This comparison directly addresses the explanation that the gains come entirely from soft weighting.

In addition, the diagnostic of curvature-only positions examines the extent of overlap with entropy, and the performance drop after shuffling the weights tests the role of matching scores to token positions. These controls support the practical value of representation geometry.

Finally, I consider the proposed comparisons unrelated to our method. We derive a theoretical upper bound on semantic change based on curvature, and this derivation motivates our choice of score. The proposed methods based on step length or turning angle do not follow from our theoretical argument.

\textbf{Step length cannot distinguish straight motion from turning.} For example, two trajectory segments can have exactly the same adjacent step lengths, with one continuing in a straight line and the other turning through 90∘90^\circ. Step length assigns them the same score, whereas ARC gives near-zero curvature to the straight segment and positive curvature to the turn.

\textbf{Turning angle captures directional change but lacks an arc-length scale.} The curvature estimate in Eq. (5) is \begin{equation*} \widehat{\kappa}_t \approx 2\sin(\theta_t/2)/\bar{\ell}_t, \end{equation*} where θt\theta_t is the angle between adjacent displacements and ℓˉt\bar{\ell}_t is the mean step length. If two positions have the same turning angle but one completes the turn over half the distance of the other, the turning angle alone assigns them the same score, while ARC's curvature estimates differ by a factor of two.

Thus, step length omits the direction change, and turning angle omits the distance over which that change occurs. ARC retains both pieces of information and directly corresponds to the geometric quantity that bounds the trajectory-bending term in Proposition 1.

4、Layer choice (Weakness 4, Question 3)

We clarify that representation geometry differs across layers. Our experiments show that, at particular layers, curvature represents semantic transitions better than entropy and random selection. The reasoning process itself determines the relevant layers, since only the geometry at certain layers captures this signal. We then use this optimal signal to improve the model's subsequent behavior.

5、Figure Correction (Weakness 5, Question 5)

We clarify that the panels for Layers 1 and 21 were swapped in Fig. 7. We will correct the figure in the revised manuscript. For cross-family validation, our experiments on the Llama family also found that higher-curvature positions correspond to more semantic transitions.

If our response has addressed your concerns, we would appreciate it if you could reconsider your overall assessment of the paper.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.