acceptodds
Under review as a conference paper at ICLR 2027

GradCuit: Credit-Assigned Gradient Flow Enables Robust and Interpretable Test-Time Latent Reasoning

Abstract

Optimization-based latent reasoning improves large language model outputs by optimizing instance-specific continuous states at test time while keeping model parameters frozen. Existing methods, however, typically connect these states to the reasoning trajectory through decoded tokens, making sequence-level credit assignment indirect and obscuring how latent updates shape subsequent reasoning. We introduce GradCuit(gradient through circuit), which inserts optimizable latent states at a selected Transformer layer between the hidden representations of the prompt and the generated continuation. Causal self-attention provides every continuation-token log-probability with a differentiable path to every preceding latent state through the remaining Transformer blocks, enabling reward-weighted gradients from the entire continuation to be assigned directly to the latents. Across five instruction-tuned backbones, three reasoning benchmarks, and two answer formats, GradCuit achieves an average accuracy of 64.5%, outperforming chain-of-thought prompting by 6.6 percentage points and the strongest competing method by 2.4 points. GradCuit also demonstrates greater robustness: across seven learning-rate settings, it consistently outperforms LatentSeek while reducing the standard deviation of accuracy from 1.53 to 0.82, and even its random-walk variant remains competitive with LatentSeek. For interpretability, token-level gradient attribution reveals that latent influence concentrates on reasoning-connector tokens, while layer analysis identifies early-to-middle Transformer layers as the most effective optimization space. These results demonstrate effective test-time latent reasoning with improved robustness and interpretability through direct credit-assigned gradient flow.

Primary area:
unsupervised, self-supervised, semi-supervised, and supervised representation learning
Keywords:
LLM Reasoning, Latent Reasoning, Test-Time Optimization, Credit Assignment

Reviews

Among:
Shape:
1234this paper, Top 15%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.00 over 4 reviews, above 85% of the others.
Reviewer_FazTOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes GradCuit, a test-time latent reasoning method. It optimizes a continuous prefix in an intermediate Transformer layer within the frozen LLM. It uses self-rewards and gradients of continuation-token to update the prefix at test time. Across five backbones, three benchmarks, and two answer formats, it reports 64.5% average accuracy: 6.6pts above CoT and 2.4pts above self-consistency. Ablations examine reward directions, layer placement, and token-level sensitivity.

Critical strengths
  1. Broad empirical gains. GradCuit improves over CoT in all reported settings and exceeds self-consistency on average. The breadth across model families and answer formats supports its practical value.
  2. The full method exceeds the fixed-prefix variant by 4.6pts on avg in the boxed answer format. This supports a contribution from latent updates beyond an additional reasoning prompt.
Critical weaknesses
  1. The credit-assignment and interpretability claims exceed the evidence. Eq. 7 assumes trajectories sampled from the policy, while implementation uses greedy trajectories, its update is therefore a biased estimator for that expectation. The random-direction ablation changes the optimizer and update magnitude, so it doesn't isolate direction quality (a cleaner control should compute the actual Adam update as the full GradCuit, and replace its direction with a random one while retaining its magnitude). Also, large gradient norms on connector tokens show local sensitivity, but don't necessarily establish that those tokens causally explain improved reasoning.
  2. It reports fewer generation rounds but no wall-clock latency provided. GradCuit requires sequential updates with self-verification and backward passes (these can't be parallelized), while self-consistency can sample candidates in parallel. Fewer rounds alone don't necessarily establish an efficiency advantage.
Questions for authors

See weaknesses.

2 comments

Thanks for the additional experiments. That addresses most of my concerns. The connector experiment is useful, though it doesn't show these tokens are uniquely important. The timing results support better throughput, but latency versus self-consistency remains unclear. Overall, I will raise my score to 3.

Thanks for adding this comparison. Please make this trade-off clear in the paper and distinguish throughput from per-request latency. I will keep my score at 3.

Reviewer_H5NSOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes GRADCUIT, a test-time latent reasoning method for improving model reasoning. Specifically, the method first appends a fixed textual prefix to the prompt, takes the hidden states of this prefix at a middle Transformer layer, and optimizes a trainable offset on these hidden states while keeping the model frozen. The continuation tokens attend to the prefix states through the remaining layers, so the gradient of log-probability on every continuation token can reach every latent state. The authors present this as direct credit assignment in latent space instead of decoding into an output token like LatentSeek, a directed baseline. At each step, the model generates a greedy continuation, and the same backbone verifies the extracted final answer with a reward of 0 (accept) or −1 (reject). The latents are updated with a policy-gradient-style step until the verifier accepts or ten updates are reached. The paper provides evaluation across five instruction-tuned backbones, three benchmarks, and two answer formats; GRADCUIT shows competitive performance compared with all the baselines.

Critical strengths

For me, I like the motivation of this work, which directly optimizes latent states inside an intermediate Transformer layer without decoding into text, which avoids the token-decoding bottleneck. The paper is well written and easy to understand. The experimental design is cover wide variety of models and tasks.

Critical weaknesses

According to section 2.2, line 182, after each generation, the same model checks the final answer. If it accepts, the reward is 0, and optimization stops immediately. If it rejects, the reward is −1, and the latents are updated. So every update that actually happens uses R = −1. As a result, the latent update does not use the reward for credit assignment as claimed, and for me, the gain mostly comes from perturbing and regenerating until the self-verifier accepts.

The second thing is that this method improves upon test-time latent-optimization methods; however, one of the important baselines in this field (LTPO [1]) is missing. It is important to see a comparison with prior work on optimizing input-level latents at test time (LTPO) as a baseline.

Finally, the method seems to work well at some selected layers. However, the method optimizes latent states at only one Transformer layer, although the layer-sensitivity study shows that different depths have different effects. I have a question: why restrict the optimization to a single layer? Are we able to extend it to more than one layer by jointly optimizing latent states at multiple layers, and does this provide additional gains or simply increase the optimization cost and instability?

Some minor things are about the paper presentation, such as moving Table 2 to the appendix or presenting it in a single paragraph to save space. [1]. Ye, Wengao, Yan Liang, and Lianlei Shan. "Thinking on the fly: Test-time reasoning enhancement via latent thought policy optimization." International Conference on Learning Representations. Vol. 2026. 2026.

Questions for authors

The paper reports that GradCuit uses fewer optimization rounds; however, for me, fewer rounds do not mean it is more computationally efficient because the method requires a backward pass through the Transformer followed by another full generation. It would be useful to show the efficiency of the method by reporting wall-clock time, FLOPs, or GPU cost under a matched compute budget.

Reviewer_UVUpOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

GradCuit performs test-time adaptation by optimizing a short prefix of hidden states at an intermediate Transformer layer while keeping model parameters frozen. The model generates a complete response, evaluates its final answer using the same backbone as a self-verifier, and updates the latent prefix using continuation-token log-probability gradients when the answer is rejected. Across five backbones, three benchmarks, and two answer formats, the method improves average accuracy over the evaluated baselines. Further analyses examine prefix insertion, random updates, learning-rate sensitivity, and token-level gradient attribution.

Critical strengths
  1. The method provides a concrete way to adapt instance-specific latent states using gradients from the full continuation, without updating model weights or accessing ground-truth answers. The appendix clearly describes the optimization and shared self-verification protocol.

  2. The evaluation is reasonably broad, and the fixed-prefix ablation supports a benefit beyond the added reasoning cue. The full method improves over the fixed-prefix variant in all 15 Boxed settings.

Critical weaknesses
  1. The central credit-assignment and robustness claims are not adequately isolated. The random-update control changes both the optimizer and update magnitude: it uses a unit-normalized Gaussian direction without Adam or norm matching. The reported advantage therefore cannot be attributed specifically to informative gradient directions. Likewise, larger token-gradient norms establish sensitivity, not a causal explanation of improved reasoning.

  2. The incremental value under comparable inference costs remains unclear. Average generation rounds omit backward computation and verifier costs, and no matched-cost comparison with adaptive sampling is provided. LTPO is also a particularly relevant missing baseline, since it already optimizes continuous input states at test time. Its characterization as relying on decoded-token interfaces in the introduction should be corrected.

Questions for authors
  1. Additional evidence: Does the advantage over random updates remain when optimizer treatment and realized update norms are matched? Please include repeated random runs and report the actual latent displacement under the stated numerical precision.

  2. Additional evidence: How does GradCuit compare with adaptive verifier-based sampling under matched wall-clock or total-compute budgets, including verification and forward/backward computation?

  3. Additional evidence and clarification: Could you compare with LTPO and clarify the distinction from existing continuous-prefix optimization? LTPO optimizes continuous inputs without textual decoding during optimization, which appears inconsistent with the introduction’s characterization.

  4. Clarification and additional evidence: What exact scalar is differentiated in Figure 4, particularly for accepted trajectories whose reward is zero? Do connector-token effects persist after controlling for token probability and position, and under interventions on their gradient contributions?

1 comment
AuthorsOpenReview ↗

Response to Reviewer UVUp (6/6)

Question 4. What scalar is differentiated in Figure 4? Do reasoning connectors still have the strongest gradients after controlling for token probability or position, or intervening on their gradient contributions?

Scalar differentiated in Figure 4 and trajectories included in the analysis. Thank you for asking about the calculation in Figure 4. Section 3.4 of the original paper describes the calculation in words. We take the trajectory generated at the last step at which a prefix update is actually performed, compute the gradient for each continuation token separately with respect to all prefix states being optimized, take its L2 norm as the gradient strength of that token, and then average by token category and across evaluation examples.

Specifically, let cc be the input question, xtx_t the tt-th continuation token in the current trajectory, and z(l)z^{(l)} all prefix states being optimized at layer ll. According to Equation (7) in Section 2.2, each token's scalar contribution to the update objective is its log-probability multiplied by the reward for that trajectory. The reward is treated as a constant when differentiating:

st=R(x,c)log⁡π(xt∣x<t,z(l),c)s_t = R(x,c)\log \pi(x_t \mid x_{<t}, z^{(l)}, c)

As stated in Section 2.2 of the original paper, R∈{−1,0}R\in\{-1,0\}. When the self-verifier judges the final answer to be correct, R=0R=0, optimization stops immediately, and the accepted trajectory is excluded from the analysis in Figure 4. For trajectories included in the analysis, R=−1R=-1, so the gradient strength of the tt-th token is

= \Vert \nabla_{z^{(l)}} \log \pi(x_t \mid x_{<t}, z^{(l)}, c) \Vert_2$$ Here, **gradients are backpropagated from every continuation token**, and the L2 norm of each token's gradient is computed separately over all prefix positions being optimized and all their hidden dimensions. We will add the above equations after the existing description in Section 3.4 of the revised paper and state explicitly how accepted trajectories are handled. **Gradient strength of reasoning connectors after different gradient interventions.** Following your suggestion, we separately restricted the tokens contributing to the update by their generation probability and by their position in the sequence, and directly adjusted the gradient contributions of reasoning connectors to the update. In each experiment, we performed one prefix update after the intervention, then generated a new reasoning trajectory and measured the gradient strength of each token category in the **next round** using the same procedure as in Figure 4. **Table R2-7. Gradient strength and rank of reasoning connectors in the next reasoning trajectory after a single-step gradient intervention** | Update condition in the current round | Connector gradient strength | Second-strongest category and its gradient strength | 95% bootstrap interval for the lead | Connector rank | | ------------------------------------------------------------ | --------------------------: | --------------------------------------------------- | ----------------------------------- | -------------: | | Backpropagate gradients only from tokens with generation probability $p>0.5$ | 0.1231 | Answer Marker (0.0231) | [0.0635, 0.1375] | 1/5 | | Backpropagate gradients only from tokens in the last 20% of the trajectory | 0.1756 | Answer Marker (0.0686) | [0.0502, 0.1610] | 1/5 | | Connector gradient weight $\alpha=0.5$ | 0.1512 | Answer Marker (0.0356) | [0.0830, 0.1465] | 1/5 | | Connector gradient weight $\alpha=1$ (standard one-step update) | 0.1645 | Answer Marker (0.0514) | [0.0689, 0.1479] | 1/5 | | Connector gradient weight $\alpha=2$ | 0.1708 | Answer Marker (0.0369) | [0.0907, 0.1614] | 1/5 | In all five settings, **reasoning connectors rank first in gradient strength among the five token categories**. After one prefix update, connectors still have the highest average gradient strength in the new trajectory whether we filter tokens by probability or position, or decrease or increase connector gradient contributions. The above interventions measure the local sensitivity of reasoning connectors in the next trajectory. Regarding the role of connector gradients in producing the final answer, **Table R2-2** above further shows that when only the gradients from reasoning connectors are backpropagated, GradCuit still retains **92.1%, 80.3%, and 89.4%** of the full method's gains over CoT on GPQA-Diamond, GSM8K, and MATH-500, respectively.
Reviewer_fVezOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

This paper addresses a central question: whether a model can correct its own errors at test time by adjusting its internal states while keeping its weights fixed. In practice, GradCuit first appends a fixed reasoning prefix to the prompt and takes its hidden representations at an intermediate Transformer layer as the variables to optimize. The model then generates a response and checks its final answer using the same backbone. If the answer is rejected, GradCuit computes gradients from the log-probabilities of the generated tokens, updates the prefix representations, and generates a new response. This process repeats until the answer is accepted or the update budget is exhausted. Since subsequent tokens can attend to the optimized representations, gradients from the entire continuation can contribute to their updates. Experiments across five backbones and three reasoning benchmarks show higher average accuracy than CoT and the evaluated enhanced-reasoning baselines.

Critical strengths

The idea is simple and easy to follow. The method can work on existing models without updating their weights. Only a small set of internal states is optimized for each problem.

The experiments cover five models, three benchmarks, and two answer formats. GradCuit gives better average accuracy than the compared baselines, with a 2.4-point gain over the strongest baseline.

The ablation study is useful. It shows that the improvement is not only from adding the fixed reasoning prefix. Updating the latent states gives further improvement.

Critical weaknesses

The method is close to optimizing a soft prefix at test time. The paper explains its difference from LatentSeek, but the additional contribution over existing continuous-prefix methods could be more clear.

The compute comparison is not sufficient. Table 6 mainly counts generation rounds. GradCuit also needs backward computation and answer verification. Fewer rounds do not necessarily mean lower total cost.

The interpretation of the gradient analysis seems too strong. Larger gradients on reasoning connectors show that these tokens are more sensitive to the latent states. This does not show that the method identifies or corrects the wrong reasoning steps.

The random-update comparison changes more than the update direction. The random version does not use Adam, and its update size is not matched to GradCuit. Therefore, this comparison does not fully isolate the benefit of the gradient direction.

Questions for authors

How does GradCuit compare with TTT-style or soft-prompt optimization methods that update a fast state at test time? Could the same answer feedback be used to update a small adaptation module, so we can check whether optimizing prefix states is a better choice?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.