acceptodds
Under review as a conference paper at ICLR 2027

Test-time Learning from Reasoning Failures

Abstract

When an attempt is rejected during deployment, large language models (LLMs) commonly retry through resampling or textual feedback for self-correction, while keeping parameters fixed. Yet a rejected attempt itself provides a negative learning signal: the model can update its parameters to reduce the attempt's likelihood, without adding it to the context. However, simply unlearning a rejected attempt before retrying does not guarantee better overall performance. To this end, we introduce Negative Space, a bilevel optimization framework that trains LLMs to adapt from rejected attempts at test time. Specifically, during training, the lower level simulates test-time adaptation from failure: it applies a single unlearning step to rejected responses between two successive attempts. The upper level learns to make this adaptation beneficial for overall performance through first-order policy optimization that maximizes rewards across initial and retry attempts. At test time, a rejected attempt triggers the same unlearning step; the model then retries the original query and discards the update. Across code generation and mathematical reasoning, Negative Space improves average accuracy over the strongest evaluated retry baseline by 4.8 points on two coding benchmarks and 3.7 points on five mathematics benchmarks. Further analysis shows that, although trained with fewer retries, it retains its advantage when given more at test time.

Primary area:
foundation or frontier models, including LLMs
Keywords:
test-time adaptation, learning from failure, meta-learning, low-rank adaptation, reinforcement learning with verifiable rewards, large language models, reasoning models

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_RrNaOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper proposes Negative Space, a bi-level optimization method that trains LLMs to adapt from unsuccessful attempts at test time. The method introduces a LoRA adapter whose shared matrix A is jointly trained with the base model parameters through reinforcement learning, while its query-specific matrix B is updated at test time to reduce the likelihood of rejected responses. Experiments on code generation and mathematical reasoning show that the proposed method achieves higher accuracy than RL without a retry loop, RL with independent retry, and RL with self-correction. On coding tasks, the method also improves Pass@k and repair rates, but requires more inference time and computation.

Critical strengths
  • The proposed method shows consistent performance gains across coding and mathematical reasoning, improving average accuracy over the strongest baseline by 4.8% and 3.7%, respectively. Higher deployment pass@k and repair rate on coding tasks provide further evidence that the adaptation improves subsequent attempts.
Critical weaknesses
  • The main contribution of this paper is using failed responses for test-time unlearning and learning a shared LoRA matrix that makes these updates improve subsequent attempts. However, LoRA-based meta-learning has already been explored in PERK [1] and Meta-LoRA [2]. In particular, Meta-LoRA learns shared LoRA components and adapts the remaining components to individual subjects at test time. Outer-loop RL for optimizing test-time adaptation has been proposed in SEAL [3] and MASS [4]. The unlearning objective itself is also standard. Therefore, the conceptual novelty limited, with the main contribution being the combination of these existing ideas for learning from failed attempts at test time.
  • The experiments use only Qwen3-4B and Qwen3-8B in non-thinking mode. It is therefore unclear whether the reported gains extend to models using extended reasoning. This is particularly relevant because suppressing an entire failed response using the unlearning objective may penalize useful intermediate reasoning steps.

[1] Zeming Chen, Angelika Romanou, Gail Weiss, Antoine Bosselut, PERK: Long-Context Reasoning as Test-Time Learning, ICLR 2026
[2] Bo Cheng, Xu Wang, Jinda Liu, Yi Chang, Yuan Wu, MeTA-LoRA: Data-Efficient Multi-Task Fine-Tuning for Large Language Models, arXiv 2025
[3] Adam Zweiger, Jyothish Pari, Han Guo, Ekin Akyürek, Yoon Kim, Pulkit Agrawal, Self-Adapting Language Models, NeurIPS 2025
[4] Zeyneb N. Kaya, Nick Rui, Test-Time Meta-Adaptation with Self-Synthesis, arXiv 2026

Questions for authors
  • Could you please evaluate the method with thinking models? It would be important for establishing the applicability of the method to reasoning models.
  • Could you please compare the method with baselines under the same total compute budget, allowing the baselines to use additional samples or retries?
5 comments
AuthorsOpenReview ↗

Response by Authors

W1. Clarifying the Conceptual Novelty

We thank the reviewer for highlighting these related works. We agree that LoRA-based meta-learning, outer-loop optimization, and unlearning have established precedents. Our contribution is not a new meta-learning operator, but a framework that integrates failure-driven test-time adaptation into LLM post-training. Specifically, we train models to correct their own reasoning failures at test time: after a rejection, the model temporarily updates its parameters to suppress the failed attempt, retries the same question, and discards the update.

Distinction from prior work. While existing methods primarily adapt to additional information or training examples, Negative Space learns to correct the model's own failed attempts:

Table R1: Comparison of Negative Space with related test-time adaptation methods.

MethodAdaptation objectiveInner-loop signal
PERKEncode long-context informationContext likelihood
MeTA-LoRATransfer knowledge across tasksTask-specific training data
SEALIncorporate new knowledge or tasksSelf-generated training data
MASSImprove reasoning on a given problemSelf-synthesized examples
Negative SpaceCorrect a rejected attempt on the same questionUnlearning rejected responses

Two properties make this framework non-trivial:

  1. The inner loop unlearns rather than learns. Unlike the cited approaches, which adapt by fitting additional information, our framework uses rejected attempts as negative learning signals, without positive targets or additional training examples at deployment.

  2. Learning to make unlearning effective. Directly suppressing a failed response may also suppress useful reasoning steps and degrade subsequent performance. Our outer loop learns to make such potentially harmful updates beneficial. On LiveCodeBench (Table 3a), norm-matched random-A\mathbf{A} unlearning achieves 35.1%, below no adaptation (38.9%), whereas learned-A\mathbf{A} adaptation reaches 42.4%.

Importantly, the asymmetric LoRA factorization provides a computationally efficient implementation of this framework, rather than constituting its conceptual novelty. Our contribution is the post-training framework that learns to turn rejected reasoning attempts into effective test-time corrections.


W2 and Q1. Generalization to Thinking Models

We thank the reviewer for this suggestion. We extend our evaluation to Qwen3-4B in thinking mode to examine whether Negative Space remains effective with extended reasoning.

Table R2: Results on Qwen3-4B in thinking mode (accuracy, %).

MethodCCLCBAVG
Qwen3-4B (thinking)35.449.142.3
+ independent retry40.852.046.4
+ retry with feedback43.756.750.2
RL w/ retry feedback57.363.760.5
Negative Space62.167.364.7

Negative Space outperforms the strongest retry baseline by 4.2 percentage points on average, demonstrating that failure-driven adaptation remains effective with extended reasoning, despite the potential suppression of useful intermediate steps.


Q2. Comparison under the Same Total Compute Budget

We thank the reviewer for this suggestion. We compare Negative Space with RL with independent retry, the strongest coding baseline, under matched compute on LiveCodeBench, allowing the baseline additional retries. At K=1K=1, Negative Space uses 1.86× the FLOPs and 1.26× the wall-clock time of independent retry per returned response (Table 3b). We pair each retry budget of Negative Space with the smallest independent-retry budget that uses at least as much compute:

Table R3: Compute-matched comparison on LiveCodeBench (accuracy, %).

Budget matched byNegative SpaceIndependent retry (≥ same budget)
FLOPsK=1 (3.7×): 41.2K=3 (4.0×): 40.8
FLOPsK=2 (6.4×): 43.9K=6 (7.0×): 43.2
Wall-clockK=1 (2.5×): 41.2K=2 (3.0×): 39.1
Wall-clockK=2 (4.0×): 43.9K=4 (5.0×): 42.7
Wall-clockK=3 (5.6×): 45.6K=5 (6.0×): 43.2

Costs are per returned response, relative to a single attempt. Independent retry saturates at 43.2% for K≥5K \ge 5, below Negative Space at every K≥2K \ge 2. Thus, even with at least as much compute, independent retry does not reach the accuracy of Negative Space: the margin is small at the lowest FLOPs budget and grows at larger budgets, where additional resampling no longer helps. Under wall-clock time, which reflects deployment latency, Negative Space leads at every budget.


Thank you once again for your insightful comments, which have strengthened our work. We hope that our responses address your concerns, and we welcome any further feedback you may have.

Thank you for the additional experiments. I believe the experiment on Qwen3-4B in thinking mode is crucial. Therefore, I have raised my score from 2 to 3.

Thank you for your constructive feedback and for raising your score. Your comments have helped us improve the paper, and we appreciate your time and effort.

Reviewer_w9hqOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

In this paper, the authors propose a novel test-time learning method that incorporates negative feedback from each retry into the B matrix of LoRA, while leveraging the A matrix as a steering mechanism to efficiently integrate the parameterized feedback into subsequent generations. By introducing a reflection-information loss function, the method trains the steering matrix A while initializing the reflection matrix B to zero for each new query. To evaluate its effectiveness, the authors conduct experiments on mathematical reasoning and coding benchmarks using two LLMs from the Qwen family. Through additional analyses of parameter configurations and ablation studies, the paper demonstrates the method's generalization capabilities and validates the contributions of its key design components.

Critical strengths
  1. The idea of parameterizing experiences and reflections from previous failed attempts using LoRA, while assigning distinct roles to the A and B matrices, is valuable. With the suppression loss-guided update, the parameterized reflections can provide more direct and informative signals to guide subsequent generations compared to conventional text-based reflection feedback.

  2. The authors provide a detailed description of the proposed method, along with clear explanations of the motivations behind its key design choices, making the paper well-structured and easy to follow.

  3. Comprehensive experiments are conducted using two LLMs across 7 datasets spanning both mathematical reasoning and coding tasks. Further analyses of hyperparameter settings and ablation studies demonstrate the effectiveness of the proposed method and validate the contributions of its individual components.

Critical weaknesses
  1. Some claims in the paper may not be sufficiently precise and require further clarification. For example, although the paper primarily emphasizes learning from failures through unlearning, the loss function in Eq. (6) also incorporates rewards from successful first attempts as supervision signals to enhance the model's underlying capabilities. Therefore, it would be valuable to further disentangle the contributions of these two supervision signals. For instance, the authors could conduct an ablation study by freezing the backbone model's parameters and updating only the LoRA factors A and B to examine the resulting performance changes.

  2. Several important details regarding the experimental setup require further clarification. First, for baselines such as GRPO, the paper only mentions task-level rewards without specifying how these rewards are defined or calculated. It is also unclear whether the baseline models are trained using full-parameter fine-tuning or LoRA-based parameter-efficient fine-tuning (PEFT). Second, the motivations behind certain experimental choices are not adequately explained. For example, why is GRPO not used for mathematical reasoning tasks, while TTRL, which does not require reference answers, is selected instead? Given that the proposed method uses ground-truth labels as supervision signals in Eq. (6), this comparison may introduce fairness concerns. Finally, the authors should provide detailed statistics for each dataset, including the training, validation, and test splits, to improve experimental transparency and reproducibility.

  3. Although the authors state that they conduct independent experiments with two random seeds and report the average performance, the variance across runs and statistical significance of the improvements should also be reported. This is particularly important given the inherent randomness in both generation and optimization processes, especially for mathematical reasoning tasks where the performance differences between the proposed method and the strongest baselines are relatively small.

  4. Some experimental results, particularly the performance of certain baselines, warrant further investigation. For nearly half of the evaluated tasks, the retry-with-feedback baseline performs worse than independent retries. This observation is somewhat unexpected, especially considering that the models have already undergone reinforcement learning with a retry-based training procedure. Such results may indicate suboptimal experimental configurations, ineffective feedback utilization, or potential overfitting during training. Further analysis is needed to explain these performance patterns and ensure that the baseline comparisons are reliable.

  5. The experiments are conducted using only two relatively small LLMs from the same model family. Evaluating the proposed method across different model families and including medium- or large-scale LLMs would provide stronger evidence of its generalizability and scalability. This is particularly important because the proposed method also involves fine-tuning the backbone model's parameters. Investigating how its effectiveness and computational efficiency change with increasing model size would help establish its practical applicability to larger and more capable LLMs.

Questions for authors

Please refer to the Weaknesses section for my main questions.

In addition, I believe the parameterized reflection matrix B may offer further advantages in terms of information compression and computational efficiency. As the number of retries (K) increases, B accumulates historical feedback through iterative updates, potentially providing a more compact representation of past failures compared to text-based reflections, which require increasingly longer contexts and may introduce additional computational overhead. Therefore, I am curious whether the proposed method would demonstrate greater advantages as K increases. Although the authors present a case study on one coding dataset, it would be valuable to investigate whether this advantage generalizes across different tasks and datasets, particularly in terms of not only performance improvements but also computational efficiency.

4 comments
AuthorsOpenReview ↗

Response by Authors [1/2]

W1. Disentangling Backbone Learning from Failure-Driven Adaptation

We thank the reviewer for this valuable suggestion. We clarify that training uses rewards from both initial attempts and retries, whereas test-time adaptation relies solely on rejected responses. In Eq. (10), the first-attempt term updates only θ\theta, while the retry term updates both θ\theta and A\mathbf{A}.

To disentangle these contributions, we present two existing results and one new ablation:

  1. Backbone training is matched with the baseline (Table 1 in the manuscript). RL with independent retry uses the same two-term reward objective for backbone training. Nevertheless, Negative Space achieves 4.8 percentage points higher average accuracy on coding benchmarks, showing that the gains cannot be explained by first-attempt reward supervision alone.

  2. Test-time adaptation provides additional gains (Table 3a in the manuscript). Disabling the test-time update on the same trained checkpoint reduces LiveCodeBench accuracy from 42.4% to 38.9%, demonstrating a 3.5-point gain from failure-driven adaptation beyond the trained backbone.

  3. Frozen-backbone ablation (new). Following the suggestion, we freeze the backbone θ\theta and train only A\mathbf{A} using adapted-retry rewards, while B\mathbf{B} is updated from rejected responses. Compared with independent retry on the same frozen backbone, this improves accuracy by 4.3 and 4.0 points on CodeContests and LiveCodeBench, respectively, demonstrating that learning to adapt from failures remains effective without backbone fine-tuning.

Table R1: Frozen-backbone ablation (accuracy, %).

BackboneRetry methodCodeContestsLiveCodeBench
Qwen3-4B (frozen)Independent retry14.129.2
Qwen3-4B (frozen)Negative Space (A\mathbf{A} only)18.433.2

W2. Clarifications on Experimental Setup and Baselines

We thank the reviewer for these points. We clarify the experimental details as follows:

Rewards. Training uses binary correctness rewards r(x,y)∈{0,1}r(x,y)\in\{0,1\}: for code, r=1r=1 if the response passes the training-time correctness tests; for mathematics, r=1r=1 if the final answer matches the reference answer.

Baseline parameters. All baselines use full-parameter fine-tuning.

GRPO vs. TTRL. We apologize for the confusion caused by the TTRL label in Table 2. The reported baseline was standard GRPO, trained on the same training data using reference-answer rewards. All mathematical methods use the same reward supervision; majority voting is used only for retry decisions.

Dataset statistics. Table R2 lists the dataset statistics.

Table R2: Dataset statistics.

DomainSplitDataset# Problems
CodeTrainCodeContests (filtered)7,054
CodeTestCodeContests test (filtered)103
CodeTestLiveCodeBench (v3–v5, Aug–Dec 2024)171
MathTrainMATH, levels 3–58,520
MathTestMATH-500 / AIME 2024 / AMC 2023 / OlympiadBench / HMMT Feb 2025500 / 30 / 40 / 675 / 30

W3. Variance

We thank the reviewer for this suggestion. To better quantify variability, we conduct repeated full deployments for Negative Space and the strongest mathematics baseline (RL with retry feedback): 32 runs on AIME 2024, AMC 2023, and HMMT 2025, and 4 runs on MATH-500 and OlympiadBench. Each entry reports the mean ± std across runs, averaged over the two training seeds.

Table R3: Accuracy (%) over repeated deployments on mathematics (mean ± std).

MethodAIME 2024AMC 2023OlympiadBenchMATH-500HMMT 2025AVG
RL w/ retry feedback36.9 ± 4.675.4 ± 5.158.3 ± 1.387.7 ± 0.311.2 ± 3.353.9
Negative Space37.1 ± 3.678.4 ± 4.160.6 ± 1.189.9 ± 0.815.5 ± 2.956.3

Negative Space achieves higher mean accuracy than the strongest baseline on all five benchmarks. A paired bootstrap over problems, stratified by benchmark, yields a 95% confidence interval of [0.7, 4.1] percentage points, confirming a statistically significant improvement.

AuthorsOpenReview ↗

Response by Authors [2/2]

W4. Retry with Feedback vs. Independent Retry

We thank the reviewer for this careful observation. This pattern is consistent with prior findings that self-correction often fails to beat resampling [1, 2] and that models anchor to their own prior outputs [3, 4]. Our analysis points to the same anchoring effect.

Mechanism (Figure 4). On LiveCodeBench, feedback-conditioned retries stay much closer to the failed program than independent retries (tree edit distance 0.067 vs. 0.216), are shorter (length ratio 0.51 vs. 1.03), and repair fewer failures (7.1% vs. 13.5%): conditioning on the failure leads to local patches rather than new solutions.

Baseline reliability. The baseline follows the Self-Debugging prompts for code [5] and RLEF-style training [6], with the same data, reward, and retry budget as independent retry. In code, the pattern holds with and without retry-loop training, so it is not a training artifact. Retry-loop training still improves this baseline on most OOD benchmarks in both domains, and in mathematics it surpasses independent retry (56.1% vs. 53.7%).

References [1] Large Language Models Cannot Self-Correct Reasoning Yet. ICLR 2024. [2] Is Self-Repair a Silver Bullet for Code Generation? ICLR 2024. [3] Try Again, Don't Look Back: Blind Resampling Outperforms Self-Repair in Small Code Models. arXiv 2026. [4] From Inertia to Objectivity: Improving Deep Research Agents with Noise Isolation. EMNLP 2026. [5] Teaching Large Language Models to Self-Debug. ICLR 2024. [6] RLEF: Grounding Code LLMs in Execution Feedback with Reinforcement Learning. ICML 2025.


W5. Generalizability and Scalability

We thank the reviewer for this suggestion. To further evaluate scalability and generalizability, we extend our experiments to Qwen3-14B and Qwen3-4B in thinking mode.

Table R4: Results on Qwen3-14B (accuracy, %).

MethodCCLCBAVG
Qwen3-14B17.533.925.7
+ independent retry18.435.727.1
+ retry with feedback21.437.429.4
RL w/ retry feedback31.046.838.9
Negative Space35.049.142.1

Table R5: Results on Qwen3-4B in thinking mode (accuracy, %).

MethodCCLCBAVG
Qwen3-4B (thinking)35.449.142.3
+ independent retry40.852.046.4
+ retry with feedback43.756.750.2
RL w/ retry feedback57.363.760.5
Negative Space62.167.364.7

On Qwen3-14B and Qwen3-4B in thinking mode, Negative Space achieves the best average accuracy, demonstrating promising scalability to larger models.


Q1. Test-Time Scaling and Computational Efficiency

We thank the reviewer for this insightful suggestion. As shown in Eq. (9), B\mathbf{B} accumulates information from past failures in a fixed-size representation, without retaining failed responses in the context. To examine whether the advantages observed on LiveCodeBench (Figure 3a) generalize to mathematics, we extend the evaluation to AMC 2023 with K∈{1,2,4,6}K \in \{1,2,4,6\}.

Table R6: Test-time scaling on AMC 2023: deployed accuracy (%) and total inference cost (TFLOPs).

K=1K=1K=2K=2K=4K=4K=6K=6
Retry with feedback, accuracy75.077.580.080.0
Negative Space, accuracy77.582.585.085.0
Retry with feedback, TFLOPs89799211231284
Negative Space, TFLOPs972104011211192

Accuracy and efficiency. Negative Space's advantage increases from 2.5 points at K=1K=1 to 5.0 points at K≥2K \ge 2. Its FLOPs grow by only 23% from K=1K=1 to K=6K=6, compared with 43% for retry with feedback. Notably, at K=4K=4, Negative Space achieves 5.0 points higher accuracy at comparable cost; at K=6K=6, it maintains this advantage with 7.2% fewer FLOPs.

Together with Figure 3a, these results demonstrate favorable test-time scaling across coding and mathematical reasoning, in both accuracy and computational efficiency.


Thank you once again for your insightful comments, which have strengthened our work. We hope that our responses address your concerns, and we welcome any further feedback you may have.

Thanks for the detailed responses. Most of my concerns have been resolved, and I have improved my score accordingly.

Thank you for your positive feedback and for updating your rating. We appreciate your time and thoughtful comments, which have helped us improve the paper.

Reviewer_cMPTOpenReview ↗
Rating
2
Confidence
3
Informativeness
1
Presentation
2
Soundness
1
Read review
Paper summary

The paper looked into whether an LLM can improve following attempts by learning from rejected responses at test time. The proposed method is Negative space and unlike previous approaches such as independent resampling or textual self-correction, Negative space utilize the failed responses to update model parameters and retries the original query without adding the failure to the prompt

Critical strengths

Negative space can convert rejected responses into parameter updates without needing an overhead of detailed textual feedback or failed responses in the subsequent prompts, which could be potentially useful for systems where responses are restricted to acceptance or rejection

Critical weaknesses

The mathematics verifier used 16 sampled solutions to obtain a majority answer, but the paper does not seem to compare against directly returning that answer using the same sample pool. If the consensus remains the same, it feels like any accepted retry will match the majority answer already available to the system. The paper could benefit from establish what adaptation adds beyond majority voting. Also, it was bit unclear whether voting samples are reused or regenerated after adaptation.

The paper's motivation is that naive unlearning approaches can lead to diminishing performance; however, there was no direct test of a tuned unlearning baseline on a conventional GRPO or independent-retry checkpoint. The random-A ablation study continues to employ the base model trained utilizing Negative Space, thereby failing to demonstrate outcomes on a conventional checkpoint. The existing empirical evidence appears to prove the efficacy of the integrated system more convincingly than the usefulness of its acquired geometric properties

Questions for authors

How does Negative Space compare with directly returning the majority answer from the same 16-sample pool? Clarification on whether the pool is reused or regenerated after adaptation is appreciated as well as reporting total inference cost, including voting samples verification, and parameter updates.

What performance improvements are achieved by implementing a calibrated unlearning modification to a baseline GRPO or independent-retry model checkpoint? Additionally, how do the results compare when employing a static random LoRA coefficient or a comprehensive parameter adjustment? Regarding the current random-A ablation study, was the resulting update scale aligned with the learned-A scenario? Furthermore, was the learning rate optimized using a separate validation dataset?

Figure 3b appears to show Negative Space's deployment pass@2 below 30%, but Table 1 reports 31.1% accuracy for a single deployment run. Figure 3a's K=1 values are also different. Clarification on what the 2 seeds vary and potentially uncertainty estimates would be helpful

5 comments
AuthorsOpenReview ↗

Response by Authors [1/2]

W1 and Q1. Comparison with Direct Majority Voting

We thank the reviewer for this question. We additionally compare Negative Space against directly returning the majority answer from the same 16-sample pool. These are two distinct deployment paradigms: direct majority voting selects the consensus answer from 16 candidates, whereas retry-based methods use this consensus as an acceptance signal for at most two attempts (K=1K=1). The voting pool is generated once and reused after adaptation.

Table R1: Comparison with majority voting on mathematical benchmarks (average accuracy, %).

SystemDeployed (K=1K=1)Maj@16Gap
Qwen3-8B + independent retry50.756.45.7
Qwen3-8B + retry with feedback48.856.47.5
GRPO + independent retry54.460.46.0
GRPO + retry with feedback52.460.48.0
RL w/ independent retry53.761.57.8
RL w/ retry feedback56.162.26.1
Negative Space59.864.04.2

Maj@16 achieves higher accuracy than all retry-based methods. Nevertheless, among the evaluated retry systems, Negative Space achieves the smallest gap to its corresponding majority-vote baseline (4.2 vs. 5.7–8.0 percentage points).

Inference cost. Direct Maj@16 requires 16 generations. Negative Space additionally requires an initial attempt, a retry for rejected queries, and one unlearning update. Including all generation, verification, and adaptation costs, Negative Space requires 972 TFLOPs and 13.2 seconds per problem, compared with 891 TFLOPs and 10.1 seconds for Maj@16.

Scope. Our mathematics experiments evaluate failure-driven adaptation within a shared retry protocol, rather than claiming superiority over direct majority voting. In code generation, the acceptance check uses public tests, which do not directly provide a candidate solution.


W2 and Q2. Calibrated Unlearning and Learned Adaptation Geometry

We thank the reviewer for this suggestion. We conduct additional experiments to evaluate calibrated test-time unlearning on conventional checkpoints and clarify the role of learned adaptation geometry.

Calibrated unlearning on conventional checkpoints. We apply random low-rank and full-parameter unlearning to three checkpoints, tuning the step size over η∈{0.1,0.3,0.8}\eta \in \{0.1, 0.3, 0.8\} on an independent validation set. The best value is η=0.1\eta=0.1; smaller values produce negligible suppression from a single update.

Table R2: Changes in deployed accuracy (percentage points) after calibrated unlearning, relative to the corresponding no-adaptation baseline.

CheckpointUnlearning methodCodeContestsLiveCodeBench
Qwen3-4BRandom LoRA−1.0−1.2
Qwen3-4BFull-parameter−1.0−1.8
GRPORandom LoRA0.0−2.3
GRPOFull-parameter−1.9+0.6
RL w/ independent retryRandom LoRA−1.9−1.8
RL w/ independent retryFull-parameter−2.9−2.3

These results show that direct unlearning on conventional checkpoints generally degrades retry performance, with only one modest improvement (+0.6 points).

Learned adaptation geometry (Table 3a). The random-A\mathbf{A} control is norm-matched to the learned-A\mathbf{A} update to ensure comparable update magnitudes. Random-A\mathbf{A} unlearning achieves 35.1%, below no adaptation (38.9%), whereas learned-A\mathbf{A} adaptation reaches 42.4%.

Frozen-backbone ablation (new). To further isolate the contribution of learned adaptation, we freeze the Qwen3-4B backbone and train only A\mathbf{A} using adapted-retry rewards.

Table R3: Frozen-backbone ablation (deployed accuracy, %).

BackboneRetry methodCodeContestsLiveCodeBench
Qwen3-4B (frozen)Independent retry14.129.2
Qwen3-4B (frozen)Negative Space (A\mathbf{A} only)18.433.2

On the same backbone, direct random-LoRA unlearning reduces accuracy by 1.0 and 1.2 points on CodeContests and LiveCodeBench, whereas learning A\mathbf{A} improves accuracy by 4.3 and 4.0 points, respectively, without backbone fine-tuning.

Together, these results show that effective failure-driven adaptation requires more than simply applying unlearning updates: learning the adaptation geometry is crucial in our evaluated settings.

AuthorsOpenReview ↗

Response by Authors [2/2]

Q3. Clarification of Evaluation Variability

We thank the reviewer for pointing out these discrepancies. All evaluations use identical sampling settings. Table 1 averages two seeds, whereas Figure 3 uses independently sampled evaluation runs (Figure 3b uses one different seed), explaining the numerical differences. We report the per-seed results underlying Table R4 below.

Table R4: Per-seed results (accuracy, %), reported as seed 1 / seed 2 (average).

MethodCodeContests (103)LiveCodeBench (171)AVG
No RL training
Qwen3-4B9.7 / 11.7 (10.7)24.0 / 25.1 (24.6)16.8 / 18.4 (17.6)
+ independent retry14.6 / 13.6 (14.1)30.4 / 28.1 (29.2)22.5 / 20.8 (21.7)
+ retry with feedback12.6 / 15.5 (14.1)25.7 / 26.9 (26.3)19.2 / 21.2 (20.2)
RL training without a retry loop
GRPO12.6 / 17.5 (15.0)32.7 / 33.3 (33.0)22.7 / 25.4 (24.0)
+ independent retry20.4 / 28.2 (24.3)39.2 / 37.4 (38.3)29.8 / 32.8 (31.3)
+ retry with feedback19.4 / 22.3 (20.9)33.9 / 35.1 (34.5)26.7 / 28.7 (27.7)
RL training with a retry loop
Independent retry22.3 / 29.1 (25.7)38.6 / 38.0 (38.3)30.5 / 33.6 (32.0)
Retry with feedback20.4 / 23.3 (21.8)36.8 / 34.5 (35.7)28.6 / 28.9 (28.8)
Negative Space30.1 / 32.0 (31.1)40.9 / 43.9 (42.4)35.5 / 37.9 (36.8)

We additionally conduct 32 independent evaluation runs on CodeContests to quantify sampling variability.

Table R5: Accuracy across repeated evaluation runs on CodeContests (mean ± std, %).

MethodAccuracy (%)
Qwen3-4B (first attempt only)11.5 ± 2.3
Negative Space30.4 ± 3.0

These results confirm noticeable sampling variability on CodeContests (103 problems). Nevertheless, Negative Space consistently outperforms all evaluated baselines on both coding benchmarks.


Thank you once again for your insightful comments, which have strengthened our work. We hope that our responses address your concerns, and we welcome any further feedback you may have.

AuthorsOpenReview ↗

Additional Follow-up on W1 and Q3

We thank the reviewer again for the careful feedback. We provide two additional updates below.

  1. Recovery beyond majority voting (W1). We further analyze whether retry-based methods can recover when majority voting fails. These are particularly challenging cases where the majority answer from 16 sampled solutions is incorrect. We report the averaged recovery rates over mathematics benchmarks, with only one retry (K=1K=1), as shown in Table R6. Negative Space achieves the highest recovery rate, improving over the strongest retry-trained baseline by 9.0 percentage points.

Table R6. Recovery rates when majority voting fails.

MethodWrong-majority recovery rate (%)
RL w/ independent retry3.6
RL w/ retry feedback12.0
Negative Space21.0
  1. Updated Figure 3 (Q3). We have redrawn Figure 3 using the same two training seeds and evaluation protocol as Table 1.

We have updated the manuscript accordingly, with changes highlighted in blue for easy reference.


With the discussion period nearing its end, we would appreciate any feedback on whether these additional results and analyses help alleviate your concerns.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.