- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper investigates whether LLMs can independently rediscover foundational algorithms after suppressing their prior knowledge of these algorithms. The authors use targeted unlearning followed by algorithm recovery tasks with different levels of hints and verifier feedback. Experiments across 20 algorithms and three models show that many algorithms cannot be recovered without hints, while more detailed guidance substantially improves recovery. The authors also find that suppressing algorithm names during decoding reduces performance, suggesting a reliance on memorized knowledge.
Critical strengths
- I think this paper studies a very important research question. Distinguishing genuine algorithmic discovery from the reuse of memorized knowledge is an important problem for understanding the scientific discovery capabilities of LLMs. The proposed experimental setting is interesting and potentially useful.
- the evaluation is systematic. The paper evaluates multiple models and foundational algorithms, with different hint levels and verifier settings. These experiments provide useful observations about algorithm recovery and the role of external feedback.
Critical weaknesses
- My main concern is that unlearning does not necessarily isolate algorithmic innovation ability. The unlearning procedure explicitly trains models not to disclose target knowledge, which may also affect their willingness or ability to reason about related problems. Preserving general benchmark performance does not establish that the algorithm-specific reasoning capabilities remain intact. Consequently, failures after unlearning do not necessarily indicate an inability to independently rediscover the algorithms. This significantly weakens the paper's central conclusion about foundational innovation.
- The algorithm-name suppression experiment has substantial confounding factors. The token masks include common tokens, such as individual letters, that may be important for unrelated reasoning. The paper also reports performance degradation on general reasoning benchmarks under these masks. Therefore, the observed recovery degradation cannot be confidently attributed to reliance on memorized algorithm names rather than disruption of the generation process.
Questions for authors
- How can the authors distinguish the effects of suppressing memorized knowledge from impairing the reasoning capabilities needed to rediscover the algorithms? Are there additional controls that could establish preservation of algorithm-specific reasoning abilities after unlearning?
- Can the authors provide stronger controls for algorithm-name suppression, such as masking matched sets of unrelated tokens? This would help determine whether the performance drop is specific to algorithm-name retrieval rather than a general consequence of token restrictions.
- How should the recovery results be interpreted as evidence of independent invention rather than knowledge retrieval or implementation ability? In particular, what evidence supports treating recovery after unlearning as an approximate upper bound on foundational innovation capability?
4 comments
A gentle reminder
Dear Reviewer 1vY4,
As the discussion period is drawing to a close, we would greatly appreciate it if you could take a look at our response. In particular, we have evaluated the unlearned models on non-target algorithms to provide a more direct assessment of their general algorithmic problem-solving ability, and designed a new replacement-based algorithm-name suppression experiment that bans no tokens, which is intuitively a much cleaner and lighter intervention than the original token masking. Based on your comments, we have also revised the relevant statements in the paper. We hope these updates could help address the concerns about capability preservation and the confounding factors in algorithm-name suppression. We would be grateful to hear whether our response has been helpful, and we are happy to provide any further clarification.
Thank you for your time and consideration.
Best regards,
Authors
Thank you for your reply. I think the response "We agree that a suppression-based design differs from a clean pretraining-time exclusion of target algorithms" implies that the evaluation protocol cannot provide meaningful conclusions. With unlearning, the measured performance is neither an upper bound nor a lower bound, because unlearning is known for suppressing the mention of the "unlearned" knowledge while still storing them in the weights. For example, the model may suppress the algorithm even if it's able to "rediscover" it; on the other hand, the model may just pretend to "rediscover" it since the knowledge itself is not removed in its weights.
Thank you for the follow-up. We understand the concern is whether our results remain informative even though they do not serve as a strict upper or lower bound. Rather than only measuring an absolute capability, our framework provides extensive controlled comparisons across algorithms, hint levels, and interventions, which yield consistent, reliable, and informative patterns:
- Easier targets remain recoverable. Under the identical unlearning pipeline, the unlearned models still recover 11 of the 12 easier targets without hints, showing that unlearning does not prevent them from producing the target algorithms.
- Off-target ability is preserved. On non-target algorithms, the unlearned models achieve recovery rates close to those of the original models (-4.9 to +1.4 pp).
- Hints restore recovery. With level-2 hints, the unlearned models reach 100% coverage on Qwen3-4B-Thinking and Gemma-4-12B.
- Training-free name suppression shows a consistent pattern. Without changing any weights, name suppression still yields a consistent difficulty-dependent decline.
We therefore believe our findings are meaningful: together, these results consistently reveal that even for foundational classical textbook algorithms, the tested models rely heavily on recall for the harder ones, and our framework offers a systematic way to examine whether LLMs can recover foundational algorithms after direct recall of the relevant knowledge is suppressed.
We sincerely thank you for your time and engagement, and we hope our response could be helpful.
We sincerely thank you for the insightful feedback. We have uploaded a revised version of the paper, with all additions highlighted in green. We will give our point-wise responses below.
W1 & Q1 & Q3: "My main concern is that unlearning does not necessarily isolate algorithmic innovation ability ... does not establish that the algorithm-specific reasoning capabilities remain intact."
Thank you for raising this important question. We agree that a suppression-based design differs from a clean pretraining-time exclusion of target algorithms and cannot completely rule out interference to the rediscovery task itself. Within these bounds, we present several lines of evidence to narrow the uncertainty and support the reliability of our findings and conclusions.
To test algorithm-specific reasoning more directly than general benchmarks, we add a new evaluation of each model's 9 checkpoints (unlearned on the never-recovered targets) across all 20 algorithms under the no-hint recovery protocol with 16 trials per pair (will extend to 128 in the final revision). For each checkpoint, we separate recovery rate (RR) on its own unlearned algorithm (on-target) from the other 19 algorithms (off-target):
Across all three models, on-target RR drops sharply to 0.0% (-22.5 to -75.3 pp), whereas off-target RR remains close to the original baseline (-4.9 to +1.4 pp)—confirming that unlearning preserves general algorithmic problem solving.
Beyond this new control, three findings in our paper also provide evidence that algorithm-specific reasoning and task engagement remain substantially preserved:
We have revised the Abstract, §3.2, §5.1, §7, and §8 accordingly to add these results. Specifically, we have removed overclaims such as "approximate upper bound" and stated our conclusions more precisely to reflect the scope and limitations of our setting.
W2 & Q2: "The algorithm-name suppression experiment has substantial confounding factors ... stronger controls for algorithm-name suppression"
Thank you for highlighting this confound. We agree that static token-ID masking bans shared letter and symbol tokens globally, which may impair general reasoning performance.
To address the concern about token masking, we design a cleaner algorithm-name suppression logic: whenever a target algorithm name appears during generation, we replace it with
"the algorithm"and resume generation from the edited prefix:We evaluate this intervention across all 20 algorithms on Qwen3-4B-Thinking and Gemma-4-12B with 32 trials per pair (will extend to 128 in the final revision):
With single-letter and mathematical tokens fully available, recovery still drops across both models (-17.6 and -35.8 pp), showing the same direction as token masking. We have added this replacement experiment to §5.2 and Appendix B.
We thank the reviewer again and hope the above explanation could help address the concerns.