acceptodds
Under review as a conference paper at ICLR 2027

From Past Attempts to Better Decisions: Generalizing Verified Experience through Parametric Memory

Abstract

A language model's first attempt on a difficult reasoning problem is often wrong. Correct answers may emerge only after repeated sampling or revision. Yet the failed and successful attempts are then discarded, so the next problem starts from scratch and the same mistakes can recur. We ask which lessons from this trial-and-error history are useful, how to store them in model parameters, and whether they generalize to unseen problems. We introduce BVEd, a parametric experience memory for frozen solvers. Starting from repeated training attempts, BVEd removes final predictions and copied answer text, then uses a teacher to distill successes and failures into candidate lessons about useful reasoning steps, missed constraints, and recurring mistakes. Source-side screens measure whether lessons increase support for the known answer, limit answer steering without the problem, and detect copied answer cues; at most one training target is selected from each admitted source. The selected lessons train a helper to transform a new, label-free problem into fresh problem-specific guidance, which the unchanged solver uses to produce its answer. Across four reasoning benchmarks, BVEd yields absolute accuracy gains of –. Together, these results suggest that trial-and-error histories can supply useful supervision: BVEd extracts experience from past attempts, consolidates it into model parameters, and generates guidance for previously unseen problems.

Primary area:
transfer learning, meta learning, and lifelong learning
Keywords:
Reasoning, Experience Distillation, Parametric Memory, Self-Improvement, Deductive Reasoning

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_3daXOpenReview ↗
Rating
1
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper introduces BVEd, a parametric experience-memory framework that transfers information from past reasoning attempts to unseen problems. The method collects successful and failed attempts from a frozen solver, uses a teacher model to extract candidate lessons, and applies source-side screening and benchmark-specific selection strategies to construct training targets. These lessons are distilled into a helper model that generates problem-specific guidance for new questions without retrieving past trajectories or modifying the solver. Experiments on four deductive-reasoning benchmarks, including FOLIO, FLD×2, ProofWriter, and RuleTaker, report accuracy improvements of 3.09–14.87 percentage points over direct answering.

Critical strengths

S.1. Well-motivated research problem. The paper addresses an important question of how information accumulated through repeated reasoning attempts can be reused on unseen problems. The distinction between solving an individual problem and consolidating past experience into reusable guidance provides a meaningful motivation for the study.

S.2. Positive empirical results across multiple benchmarks. BVEd achieves positive accuracy gains across all four reasoning benchmarks under the evaluated protocols, with relatively small variation across helper training runs. The use of paired evaluations and additional uncertainty analysis on FOLIO strengthens the empirical presentation of these results.

S.3. Informative analysis beyond aggregate accuracy. The paper examines how guidance changes individual predictions through repair–harm analysis and evaluates its effects across different recipient models. These analyses provide a more detailed characterization of the guidance's effects on individual decisions and its variation across recipient models.

Critical weaknesses

W.1: Insufficient ablations of experience extraction and screening.

The paper does not isolate the contributions of success/failure histories or individual screening components (ABG, blind-JSD, and lexical filtering). Existing controls demonstrate the value of problem-specific guidance but not the necessity of the proposed experience-learning pipeline. Comparisons with success-only histories, history-free guidance distillation, and unfiltered lessons under matched training conditions are needed.

W.2: Insufficient comparison with standard reasoning baselines.

The main experiments compare answer-only Qwen3-8B with BVEd, which introduces an additional trained 4B helper. However, short-reasoning and zero-shot CoT baselines using the same Qwen3-8B are missing. Appendix E provides limited reasoning controls on smaller solvers. Comparisons under controlled reasoning lengths and end-to-end inference costs are needed to establish benefits beyond additional inference computation.

W.3: Insufficient analysis of answer-equivalent test-time guidance.

Although BVEd screens training lessons, Appendices C and G.1 show that test-time guidance can still contain answer-equivalent conclusions after postprocessing. The paper does not quantify how frequently this occurs or how much it contributes to accuracy gains. Without such analysis, the role of reusable reasoning guidance versus direct answer provision by the helper remains unclear.

Questions for authors

Q1. How much do experience histories and individual screening components contribute compared with success-only, history-free, and unfiltered guidance under matched training conditions?

Q2. Does BVEd outperform pretrained-helper, short-reasoning, zero-shot CoT, and training-free methods (e.g., Self-Consistency and Self-Refine) under comparable inference costs on the main Qwen3-8B benchmarks?

Q3. How frequently does test-time guidance contain answer-equivalent conclusions after postprocessing, and how much of BVEd's accuracy gain remains when such conclusions are removed?

Minor Comment: Presentation and Visualization Quality The overview figure (Figure 1) is overly crowded, with small text that reduces readability. Figure 3 on page 8 also contains excessive whitespace, thin bars, and small labels. A more compact layout and clearer visual design would improve the overall presentation.

1 comment

Thank you for your detailed response and for carrying out the extra experiments. I appreciate the authors' efforts to address my concerns, and I have no further questions at this stage. In my opinion, it was unnecessary to repeatedly emphasize the 95% confidence intervals, as they were not central to my main concerns. I still feel that the paper's methodology and empirical evidence could be strengthened further. As shown in Table R2, the proposed screening strategy offers no clear advantage over random selection. Therefore, I will maintain my original rating.

Reviewer_tVmBOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper converts verified attempt histories into parametric memory. A frozen Qwen3-8B solver samples four attempts per labeled training problem; a frozen teacher sees the problem, the sanitized attempts, and outcome categories but not the answer options or gold label, and writes four candidate lessons. Three screens then admit at most one lesson per source: answer-belief gain against the gold answer under the solver's own length-normalized likelihood, a blind Jensen-Shannon divergence computed with the problem context hidden, and a lexical screen against answer strings and verdict words. A Qwen3-4B helper is supervised on problem-to-lesson pairs and at inference emits guidance that is prepended to a frozen solver's prompt. Paired gains over direct answering are 14.87 on FOLIO, 10.60 on FLD x2, 7.61 on ProofWriter, and 3.09 on RuleTaker, with length-matched generic text and shuffled-guidance controls, a five-recipient audit, and premise-cluster bootstrap intervals on FOLIO.

Critical strengths

The evaluation design is more careful than most papers in this area. The paired repair and harm decomposition with permutation-matched controls is the right way to show that guidance is doing something content-specific rather than acting as a generic prompt, and the premise-context cluster bootstrap on FOLIO is a genuinely appropriate response to the small sample. The authors are also unusually candid in Appendix C, where they document cases in which the generated test-time guidance does reveal the answer, and they state plainly that the blind-bias screening limits answer steering without proving non-leakage.

Critical weaknesses

No literature method is evaluated anywhere in the paper. Every comparison is against the same solver with no prefix, a generic instruction, a permuted guidance, or the unadapted helper. The central thesis is that experience is better internalized into parameters than retrieved, and that thesis is untested without a nearest-neighbor retrieval baseline over the training lessons. Equally, the direct baseline is a single greedy answer-only decode while the method itself is built on four sampled attempts, so self-consistency at zero additional cost to the authors is a glaring omission. A plain teacher-generated chain-of-thought prefix is also missing, and since the helper's guidance is an injected rationalization, that arm is the most plausible alternative explanation for the FOLIO gain. Second, the screening machinery has no ablation. Nothing compares screened selection to arbitrary or unscreened lesson selection, so there is no evidence that the three screens are necessary or that the ABG, blind-JSD, and lexical components contribute individually. This compounds a hyperparameter-selection concern: Equation 5 selects the configuration that maximizes the margin against the generic and shuffled controls rather than accuracy over direct alone, which optimises the very contrast Table 3 reports. The headline numbers are also protocol-sensitive in a way the paper does not fully reconcile: ProofWriter shows a gain of 7.61 in Table 2 and 1.7 in the Figure 3 audit on the same 10,158-question panel with the same recipient, and FOLIO shows 14.54 against 4.8. The FOLIO cluster intervals, roughly 8.8 to 21.5, indicate that the largest reported effect is also the least precisely estimated.

Questions for authors

What happens against a retrieval baseline that returns the nearest training lesson for each test problem? Against a self-consistency baseline using the four attempts the pipeline already generates? Against a plain teacher-generated chain-of-thought prefix? Without an ablation over the screening, how can we tell that the ABG and blind-JSD filters do anything rather than that any teacher-generated lesson would have worked? The screening is applied to training candidates, but at test time there is no gold label so only the lexical postprocessor runs, and Appendix C shows semantic disclosure slipping through. Is there any way to quantify test-time leakage? Please reconcile the Table 2 and Figure 3 results, and report bootstrap intervals on all four benchmarks rather than FOLIO alone. Would distillation of success-derived lessons alone recover most of the gain, which would undercut the trial-and-error framing? Finally, the seeds vary neither the pretrained initialization nor candidate construction, so the reported standard deviations understate the true uncertainty.

Reviewer_G156OpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper proposes BVEd, a method that turns a model’s past attempts into guidance for new problems. A teacher extracts lessons from the solver’s past attempts, the lessons are screened for usefulness and answer leakage, and a helper model is trained on the selected lessons. At inference, the helper generates guidance from the new problem alone to improve the performance of a frozen solver.

Critical strengths
  1. The presentation is clear and the problem is well motivated.

  2. Experiments are rather thorough, and there are consistent gains across benchmarks.

Critical weaknesses
  1. The improvement might come from the helper solving the problem and giving too many hints rather than providing genuine past experience. Even though final answers are removed before the teacher extracts lessons during training, the leftover intermediate reasoning steps and those that failed to get extracted can still give the answer away, training the helper to be stronger and give the answers directly. Additionally, at test time, the safeguard is only a word filter, which may not be robust (some experiments can be conducted to show, in both cases, how many data answers are actually filtered out). The baseline setup is also unfairly weak: the baseline solver isn't allowed to reason at all, while the guided solver gets extra reasoning from a second model.

  2. The experimental evaluation feels incomplete. The untrained-helper control is only tested on one dataset (FOLIO) with smaller models. Additionally, what happens if you just fine-tune the solver or helper on correct attempts, or if the helper simply tries to solve the problem directly? Finally, other methods like ExpeL or Buffer of Thoughts should also be added to the experiments.

Questions for authors
  1. Could the authors report how many candidate lessons each screen (ABG, blind JSD, lexical) rejects and how often the selected lessons include the final answer?

  2. What happens if the helper is trained on a random candidate lesson without any screening?

  3. What's the reason for using answer-only decoding?

  4. Could the authors evaluate stronger solver models? The only solver larger than the main Qwen3-8B, Qwen2.5-14B, shows a drop on FOLIO (−2.02%, Table 4), which suggests the guidance may be harmful as the solver gets stronger.

  5. What's the compute overhead at inference time?

Comments on the paper

1 comment
Area_Chair_GPKCOpenReview ↗

Reminder: Please Follow Up on the Author Rebuttal

Dear Reviewers,

As we are approaching the end of the discussion period, I would like to remind you to review the authors' rebuttal and assess whether your concerns have been adequately addressed.

If you have any remaining concerns, please raise them as soon as possible to allow the authors sufficient time to respond. If your concerns have been resolved, please leave a brief acknowledgment to let the authors know that you have read and considered their responses. Thank you for your time and efforts in the review process.

Best regards, Area Chair

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.