- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 1
Read review
Paper summary
This paper proposes Failure Logit Extrapolation (FLEx), a training-free decoding method that combines predictions from a clean context and a context containing previous failed attempts for test-time scaling. Experiments across eight models and six benchmarks show improvements over independent sampling and sequential revision. The paper also shows that trajectories collected with FLEx can improve downstream performance when used for supervised fine-tuning.
Critical strengths
- The experimental evaluation is extensive, covering eight models across multiple scales and families, as well as six benchmarks spanning mathematics, coding, and instruction following.
- FLEx is a simple, training-free decoding method that reuses the model's own failed attempts and does not require an auxiliary model.
Critical weaknesses
- My main concern is the overall problem setting. FLEx assumes that a reliable verifier is available to determine whether each attempt is correct, but I am not convinced that this is a realistic assumption for practical test-time scaling. For example, the math experiments use reference answers to identify failures, whereas these answers are usually unavailable when solving new problems.
- The motivation is not fully convincing. Around line 75, the paper states: “Our experiments expose a fundamental tension between these primitives. Independent sampling produces more diverse solutions, but cannot use prior failures.” How is this conclusion supported, and how does it motivate FLEx? It seems heuristic to me.
- Some of the design choices and analyses appear heuristic:
- How do the authors choose the extrapolation direction? When should they use the default direction, extrapolating away from , and when should they use the reverse direction?
- Where are the results of the and sweeps reported? Appendix A.5 describes the selection protocol, but only the selected configurations are shown.
- If FLEx is applied to a new task, do the hyperparameters and extrapolation direction need to be tuned again? How expensive is this tuning process?
- Are the prompt distributions in Table 2 matched? If not, could this make the comparison unfair?
Questions for authors
See the Weaknesses.
4 comments
Response to Reviewer H84i [2/2]
[W3]: Extrapolation direction, hyperparameter sensitivity, tuning cost, and prompt matching
We thank the reviewer for raising these points. Here we address them:
-
Extrapolation direction. We extrapolate away from the branch that is less performant, and in the vast majority of cases, the branch with failure-filled context. The reason we have used the reverse direction for a specific model family x evaluation benchmark is when informative execution feedback makes sequential revision stronger than independent sampling. We further verify this explanation in Table 7, where replacing execution feedback with binary correctness largely removes this advantage.
-
Hyperparameter sweeps. Appendix A.5 describes our hyperparameter selection protocol, and Table 6 reports the selected configurations. We agree that showing the hyperparameter sensitivity would be useful. We now report the results of a sweep over λ and α for Qwen3 on mathematics benchmarks, as shown below.
| λ / α | 1e-4 | 1e-3 | 1e-2 | mean |
|---|---|---|---|---|
| 0.1 | 35.17 | 34.36 | 33.58 | 34.37 |
| 0.2 | 36.06 | 35.35 | 33.33 | 34.91 |
| 0.4 | 35.23 | 35.60 | 36.73 | 35.85 |
| 0.8 | 37.39 | 38.89 | 36.04 | 37.44 |
| 1.0 | 38.24 | 36.24 | 37.39 | 37.29 |
| mean | 36.42 | 36.09 | 35.41 |
In this combination, lambda is the dominant factor. The row mean rises from 34.37 at lambda=0.1 to 37.44 at lambda=0.8. It then flattens at lambda=1.0 (37.29), as now we additionally evaluate. Alpha matters less. Values of 1e-4 and 1e-3 are close (36.42 vs 36.09). A value of 1e-2 is clearly worse (35.41). A loose mask admits low-probability tokens, which the extrapolation then amplifies.
-
Generalization and tuning cost. We sweep once per family x domain, not per benchmark or per model size. A selected hyperparameter transfers to any size within the same family and different benchmarks of the same domain (e.g., math). A practitioner adopting FLEx on a new domain only needs to run this once on a fraction of data.
-
Prompt distribution. We clarify that both methods collect trajectories from the exact same 20K prompts under the same sampling budget. While the distributions of retained correct trajectories may differ, this is an intended consequence of the sampling methods. In particular, FLEx can discover correct trajectories for prompts that independent sampling fails to solve. This is precisely one of the benefits of improved solution coverage. Our comparison evaluates the downstream effectiveness of the resulting trajectory collection pipelines. It does not aim to control away the improvement in solution coverage.
Dear Reviewer H84i,
As the author-reviewer discussion period is approaching its end tomorrow, we wanted to follow up in case you have had a chance to consider our response, particularly our additional verifier-free majority@k experiments, hyperparameter sensitivity analysis, and clarifications regarding the motivation and experimental design.
We sincerely hope these additional results and explanations have helped address your concerns. We would be very grateful for any updated thoughts or remaining questions you might have, and would welcome the opportunity to provide further clarification before the discussion closes.
Thank you again for your time and thoughtful consideration!
Best,
The Authors
Dear Reviewer H84i,
As the author-reviewer discussion period comes to a close, we wanted to reach out one final time.
We have made a substantial effort to address your concerns through additional verifier-free evaluations, hyperparameter sensitivity analyses, and clarifications of the motivation and experimental design. We sincerely hope these additional results and explanations have helped address your reservations.
We would be very grateful if you could share any updated thoughts on our responses, particularly whether your main concerns have been addressed or if any remain unresolved. We remain available to respond promptly over the next few hours and would sincerely appreciate the opportunity for further discussion.
Thank you again for your time and thoughtful consideration.
Best,
The Authors
Response to Reviewer H84i [1/2]
We thank the reviewer for acknowledging our extensive evaluation across models and benchmarks, as well as the simplicity of our training-free approach. We address the concerns and questions below.
[W1]: Reliance on a verifier
We thank the reviewer for this concern. We clarify that FLEx is a sampling primitive designed for verifier-guided search, a well-established setting in prior literature. For example, AlphaCode [1], FunSearch [2] and AlphaEvolve [3] all rely on automated evaluators to guide test-time search.
Our contribution is to improve how trajectories are sampled, rather than how they are verified. While our math experiments use reference answers as verifiers, this directly corresponds to practical post-training settings such as rejection sampling and RL with verifiable rewards, where such feedback is available. We will expand the related work discussion and clarify the verifier-guided setting and our contribution in the revised manuscript.
We also invite the reviewer to check out our response to Reviewer TB9N regarding the performance impact of using majority@k instead of oracle answers as this metric highlights an interesting property of FLEx when interacting naively with majority voting verification method. In short, FLEx improves majority@k by +0.2 to +2.1pp in the settings we tested, though the net effect is a balance between widened coverage and weakened vote conversion.
[1] Competition-Level Code Generation with AlphaCode
[2] Mathematical discoveries from program search with large language models
[3] AlphaEvolve: A Gemini-powered coding agent for designing advanced algorithms
[W2]: Convincingness of motivation
We thank the reviewer for this question. We clarify that independent sampling cannot leverage previous failures by construction, as each attempt is sampled independently from the same original context without access to previous attempts.
The remaining observation on sequential revision is empirically grounded. As already discussed in the Introduction (L51), multiple prior works [1, 2, 3] have identified that sequential TTS often suffers from repeated errors and reduced exploration. This motivated us to investigate the underlying reasons more closely. Through preliminary experiments, we observed that sequential revision produces significantly less diverse answers than independent sampling, as reported in Figure 3. This observation led us to explore how previous failures could be used without sacrificing diversity, ultimately motivating FLEx. We will clarify this progression in the revised manuscript.
[1] Contextual Drag: How Errors in the Context Affect LLM Reasoning
[2] Understanding Performance Gap Between Parallel and Sequential Sampling in Large Reasoning Models
[3] Old Habits Die Hard: How Conversational History Geometrically Traps LLMs