acceptodds
Under review as a conference paper at ICLR 2027

Natively Unlearnable Large Language Models

Abstract

Unlearning aims to remove the influence of specific training data sources, but this has proved challenging because the contributions of different sources are entangled within the model. Isolating source contributions to disjoint parameters makes removal easier, though it obstructs joint learning across sources. We propose NULLs (Natively Unlearnable LLMs), a model class that satisfies the two opposing goals of isolating source-specific contributions and learning jointly across sources, by training a set of shared backbone neurons alongside a pool of sparsely activated sinks. During training, information specific to a source naturally concentrates in its sinks while information shared across sources accumulates in the backbone. A source is then unlearned at deployment by disabling its corresponding sinks, with no gradient updates and no access to the retained data. We show that NULLs scales to Wikipedia's  6M articles, isolating each as an independent source. Unlearning a single article removes knowledge specific to it while preserving facts shared with semantically related articles, closely matching retraining from scratch. We note that unlearning with NULLs is also robust: in a case study of unlearning the Harry Potter books, NULLs resists both adversarial extraction and relearning that reverses post-hoc unlearning. Finally, NULLs preserves general language capabilities, matching a standard transformer on downstream benchmarks. Together, these results suggest that source-level unlearning need not be an afterthought. It can be built natively into LLM training while retaining the benefits of shared representation learning.

Primary area:
foundation or frontier models, including LLMs
Keywords:
Unlearning, Modularity, Memorization

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_taDsOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper proposes NULLs, an LLM architecture in which MLP neurons are partitioned into a shared backbone and a sink pool where each training source is assigned a deterministic sparse mask over the sink pool, so source-specific knowledge concentrates in sinks while shared knowledge accumulates in the backbone. Unlearning a source then requires only disabling/zeroing its mask at inference (no gradient updates or retained-data access). Experiments train 1B models on Wikipedia with ~6M articles, showing Sink-Off closely matches gold-standard retraining, resists relearning attacks that reverse NPO in <10 steps, and preserves downstream benchmark performance vs. a standard transformer (Table 1).

Critical strengths
  • Performance matches the retraining gold standard at unprecedented source granularity. NULLs scales to ~6M independently controllable sources without linearly scaling parameters (via combinatorial masks), and Sink-Off tracks retraining's performance (Fig. 4), while post-hoc baselines (NPO & GA) degrade shared and target knowledge in tandem (Fig. 3). This shows that training-time design can resolve the entanglement problem that post-hoc unlearning structurally cannot.
  • Robust removal where prior methods fail. NULLs with permanent removal matches retraining against fine-tuning relearning attacks (Fig. 7b), directly addressing the documented failure mode of most post-hoc unlearning methods.
Critical weaknesses
  • The core mechanism is inherited wholesale from Ghosal et al. (2025). The training is repurposed without modification. The delta is a granularity shift (sequences to sources). Whether this constitutes sufficient conceptual novelty is questionable. The paper also provides no mechanism guaranteeing shared/sink role separation; it is only validated in experiments. In general, this paper reads like an application of Ghosal et al. (2025).

  • Models are just 1B parameters. It is unclear whether this scales to larger models, which have much larger capacity for storing and memorizing source-specific knowledge. This is a critical limitation for real-world deployment.

Questions for authors
  • Figure 1 seems misleading as sinks actually overlap by using the combinatorial mask.

  • How does the training ensure that such a separation effectively achieve the knowledge disentanglement goal? The model could very well ignore these memorization sinks, right? Do you design any mechanism to avoid such a case?

  • The combinatorial mask leaves it questionable about the claim that the knowledge is actually disentangled. After permanent removal of source A, what is the change in Sink-On truth ratio for semantically adjacent sources B? In particular, for source pairs with maximal mask overlap, how much does B's article-specific recall degrade?

1 comment

Figure 1

Figure 1 seems misleading as sinks actually overlap by using the combinatorial mask.

Thank you for pointing this out. Figure 1 is a schematic, and we draw each source's sink as a separate block for visual simplicity. As described in the caption and Section 3.2, each sink is implemented as a sparse mask over a shared sink pool, so masks of different sources partially overlap. The overlap is small by design: with 100 of 8,000 sinks per source, two masks share 1.25% of their sinks in expectation, and Section 4.3 (Figure 8) shows that unlearning stays close to retraining across all overlap ratios we test. We will add a sentence to the caption making this explicit.

Scaling

Models are just 1B parameters. It is unclear whether this scales to larger models, which have much larger capacity for storing and memorizing source-specific knowledge. This is a critical limitation for real-world deployment.

On scale

Thank you for raising this. We agree that we test only 1B-parameter models, as stated in Section 5. Our results do, however, separate the two ways scale could affect NULLs.

  • More capacity to memorize. In NULLs, source-specific knowledge is held by the sinks, and Figure 8b scales their capacity directly: increasing the active sinks per source raises memorization (Sink-On truth ratio from ≈1 at 50 neurons to ≈5.8 at 250) with no consistent change under Sink-Off. Added capacity increases how much the model memorizes, not whether it can be removed.
  • Leakage into the backbone. Isolation is governed by interference, which the overlap ratio controls (Figure 8a). Each sink is active for about 1.3% of sources, while the backbone is active for all of them. This asymmetry is a design parameter independent of model size, and the sink pool can grow with the model to keep it fixed.

Disentanglement of Knowledge

The combinatorial mask leaves it questionable about the claim that the knowledge is actually disentangled. After permanent removal of source A, what is the change in Sink-On truth ratio for semantically adjacent sources B? In particular, for source pairs with maximal mask overlap, how much does B's article-specific recall degrade?

As the combinatorial mask is assigned psuedo-randomly from the source identifier, so removal of a paricular source mask is not expected to have any specific effect of semantically related articles. This is in contrast to what is observed in gradient-based unlearning methods. In the facts we tested, removing a single source permanently matched preserved decreased the Truth Ratio on neighboring facts by 0.05 (reflecting a degradation of around 1%, still well above he backbone-only model). Our Harry Potter experiments also showed a similar trend, with loss on other C4 clusters increasing by just 0.6% when that cluster was deleted.

As we note in our response to other reviewers, permanent deletion with multiple sources can cause sink-specific information to be lost. On the other hand, our routing based deletion allows any number of sources to be deleted without harming other sources.

Reviewer_auF8OpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes Natively Unlearnable LLMs (NULLs), a pretraining-time framework for source-level unlearning. Building on memorization sinks, NULLs partitions MLP neurons into a shared backbone and a pool of sparsely activated, source-specific sinks, aiming to isolate source-specific knowledge while preserving cross-source learning. Unlearning is achieved by disabling the corresponding sinks without additional training. Experiments on 1B-parameter models trained on Wikipedia and a Harry Potter/C4 mixture demonstrate forgetting performance comparable to retraining, preservation of shared knowledge and general capabilities, and robustness against adversarial extraction and relearning attacks.

Critical strengths
  • This paper focuses on a practically important problem by incorporating unlearning capabilities directly into the pre-training stage.

  • The fine-grained evaluations about source-level unlearning effectiveness, knowledge preservation, and behavioral similarity to a retrained reference model are impressive.

  • The framework of NULLs is straightforward and easy to follow.

Critical weaknesses
  • My main concern is the methodological novelty of this work. To the best of my knowledge, the general idea of isolating specific subsets of training data within dedicated model components has already been explored in prior studies. In particular, NULLs appears to build substantially upon Memorization Sinks (Ghosal et al, 2025) and shares conceptual similarities with FlexOLMo (Shi et al., 2025).

  • The experimental evaluation is primarily conducted on 1B-scale models under article-level and topic-level unlearning settings, using Wikipedia and Harry Potter benchmarks. I am not fully convinced that they are sufficient to establish the robustness and generalizability of NULLs. In particular, it remains unclear whether NULLs can be directly extended to more fine-grained unlearning scenarios, such as entity-level or sample-level unlearning (e.g. TOFU/MUSE/WMDP benchmarks). Given the polysemantic nature of LLM neurons, fine-grained knowledge may not be cleanly isolated within designated sink components.

  • Another important concern is that the source isolation mechanism underlying NULLs is not rigorously established. As I understand it, the approach relies on the hypothesis that source-specific information will be preferentially stored in dedicated sink components, while its influence on the shared backbone will gradually diminish during training. However, this hypothesis does not necessarily imply that source-specific knowledge is exclusively stored in these components or that removing the corresponding sinks eliminates the influence of the associated training data. The shared backbone may still encode information derived from the target sources, even if such information is no longer readily accessible under the evaluated inference conditions.

  • The experimental comparisons are largely limited to NPO, GA, and the retrained model. They do not adequately position NULLs relative to methods that similarly exploit component-level isolation or training-time mechanisms.

Questions for authors

Q1: What would be the performance difference between NULLs and a straightforward source-aware extension of Memorization Sinks (Ghosal et al, 2025)? Specifically, if we replace sequence-level identifiers with source-level identifiers and introduce a corresponding source-aware routing mechanism into Memorization Sinks, how much additional benefit would the remaining components of NULLs provide?

Q2: Could the authors clarify the fundamental methodological differences between NULLs and closely related approaches, including Memorization Sinks (Ghosal et al, 2025), FlexOLMo (Shi et al., 2025), and Example-tied Dropout (Maini et al., 2023)?

Q3: Since NULLs refers to naively unlearnable LLMs, how can it be extended to entity-level or example-level unlearning, e.g., via TOFU/MUSE/WMDP benchmarks?

Q4: Could the authors provide additional experimental results on larger models or different model architectures?

Q5: The evaluation appears to rely on ground-truth routing, i.e., oracle routing. Is the source identity required for this routing realistically available in practical unlearning scenarios, particularly when the provenance of queried knowledge is ambiguous or multiple sources contribute to the same knowledge?

2 comments

Thank you for your careful review of our paper. We were encouraged to hear that you found that:

  • The problem is practically important.
  • The fine-grained evaluations are impressive
  • The framework is easy to follow.

Novelty and Contribution

Q1: What would be the performance difference between NULLs and a straightforward source-aware extension of Memorization Sinks (Ghosal et al, 2025)? Specifically, if we replace sequence-level identifiers with source-level identifiers and introduce a corresponding source-aware routing mechanism into Memorization Sinks, how much additional benefit would the remaining components of NULLs provide?

Q2: Could the authors clarify the fundamental methodological differences between NULLs and closely related approaches, including Memorization Sinks (Ghosal et al, 2025), FlexOLMo (Shi et al., 2025), and Example-tied Dropout (Maini et al., 2023)?

1. Novelty relative to Ghosal et al. (2025)

Our main novelty in this work is demonstrating that the mechanism proposed by Ghosal et al. can be scaled to enable selective removal of individual pre-training documents at scale. Our findings demonstrate novel findings beyond what was shown in Ghosal et al:

  • NULLs Matches Retraining Gold Standard. Facts stated only in the article are removed unless other sources make them inferable; those are kept, at the retrained model's rate (Fig. 4). NPO and gradient ascent, even given the target, degrade shared and specific knowledge at similar rates (Fig. 3c).
  • Individual documents stay separable at corpus scale. Each sink neuron serves ~75K articles on average, yet excluding one article's mask removes its unique facts, even with the most similar article's sink active instead (Fig. 4). Expert- and routing-based isolation (Gururangan et al., 2021; Shi et al., 2025; Cloud et al., 2024; Shilov et al., 2025) needs dedicated parameters per source, so it isolates domains or a designated forget set. NULLs makes each of ~6M documents separately removable from a fixed pool, while still learning across them.
  • Sinks store knowledge, not just strings. They hold facts, tested with cloze questions rather than verbatim continuation (Fig. 4, 5b), and the model needs them: with all sinks off, article-specific facts vanish (answer probability 0.08 → 0.005, Fig. 9). NULLs therefore keeps sinks live, routes queries to them (App. A.2), and removes one only on request.
  • Removal is robust. Under GCG extraction and relearning attacks, the model with its Harry Potter sink zeroed matches one never trained on Harry Potter, while NPO is undone within 10 fine-tuning steps (Fig. 7).

1. Novelty relative to FlexOlmo

NULLs differs from FlexOlmo in two fundamental ways.

Granularity of removal. FlexOlmo isolates each data source in its own expert, so its unit of removal is a whole dataset: seven in their experiments. NULLs makes each of Wikipedia's ~6M articles separately removable, nearly six orders of magnitude more units. The gap is built into the designs: a FlexOlmo source costs a full expert, so per-document removal would need millions of MLPs, each trained on one document. To implement a takedown of one news article, FlexOlmo would have to drop or retrain its entire news expert. On the other hand, NULLs removes just that article while closely matching retraining (Fig. 4).

Shared representation learning. FlexOlmo trains each expert separately against a frozen backbone, so no closed dataset updates any shared parameter. Knowledge common to several sources must be learned separately in each expert, and no source benefits from another's data. On the other hand, NULLs trains all sources jointly, so knowledge supported by several sources is learned once, in the backbone (Fig. 9).

Extending Beyond Source-Level Unlearning

Since NULLs refers to naively unlearnable LLMs, how can it be extended to entity-level or example-level unlearning, e.g., via TOFU/MUSE/WMDP benchmarks?

Thank you for this question. Because unlearnability is built in during training, the unit that can be removed is whatever is declared a source at training time, and NULLs is agnostic to that choice: a source can be a document, an entity, a data owner or a topic cluster. Our case studies already span two granularities: ~6M individual Wikipedia articles (each essentially one entity's page) and a topic cluster (the Harry Potter books). The benchmarks the reviewer mentions correspond to different choices of source:

  • TOFU (entity-level). Each fictitious author's facts appear only in that author's QA pairs. With one source per author, forgetting an author amounts to disabling its sinks, and TOFU's retain model is exactly the retraining reference NULLs is designed to match. Example-level unlearning follows by making each author's information its own source.
  • MUSE (document-level). MUSE-Books unlearns the Harry Potter series, which is our §4.2 setting; MUSE-News corresponds to one source per article, as in our Wikipedia study. Each removal is gradient-free and needs no retained data.
  • WMDP (domain/capability-level). WMDP targets a capability rather than a fixed corpus, so the natural reference is pretraining data filtering. In NULLs this is the coarsest granularity: a single hazard-domain source assigned during pretraining (e.g., by a classifier) and removed before deployment, as with our Harry Potter cluster and in the spirit of gradient routing.

Larger models / other architectures

We agree that further increasing model scale is a valuable direction for future wok, and our Limitations (Sec. 5) note that we only test 1B-parameter models. Every configuration needs its own retrain-from-scratch baseline, so with our compute we chose to scale the number of sources rather than model size. The paper does show NULLs scaling along that axis: it handles ~6M independently removable sources over ~32B tokens (Sec. 4.1) and still matches a standard transformer on benchmarks and knowledge capacity (Table 1, Fig. 8c). Giving each source more sink neurons also raises knowledge capacity without hurting unlearning (Fig. 8b).

We would expect larger models to help rather than hurt, since wider MLPs leave room for larger sink pools, which helped in Fig. 8. The underlying training dynamic also held at both 360M and 1.7B in Ghosal et al. (2025).

We think other architectures are a promising next step. NULLs only adds an elementwise mask on MLP activations, so it and is broadly applicable to any model with MLP blocks, including mixture-of-experts models. We will add this to the future work discussion.

Routing

Q5: The evaluation appears to rely on ground-truth routing, i.e., oracle routing. Is the source identity required for this routing realistically available in practical unlearning scenarios, particularly when the provenance of queried knowledge is ambiguous or multiple sources contribute to the same knowledge?

In our work, we focus on the setting where the unlearning target is pre-defined source and believe that it corresponds to real unlearning scenarios. For example, removing a document that contains an individual's private information or a copyright takedown request. Thus, the ability to remove information in NULLs does not depend on any specific routing choices.

The main role of routing in NULLs is to decide which sink to activate for a particular query. We provide some analysis of a straightforward routing strategy in our Appendix A.2, and show that even a basic embedding based router selects the source article (out of 6 million Wikipedia articles) for 73.1% of the Cloze prompts (85.9% top-5, 89.1% top-10). However, we believe that designing better routing strategies or co-training a learnable router can be a valuable avenue for future work.

NULLs automatically handles cases where facts may appear in many articles. In Appendix A.6., we show that such facts are stored in the backbone neurons and thus don't depend on routing. Thus, only access to article-specific facts depends on routing, and in that case, routing is unambiguous.

Reviewer_7cw1OpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper builds unlearning into pretraining. Each data source (e.g., each Wikipedia article) gets its own small set of "sink" neurons, plus a shared backbone that every source uses. To delete a source, its sink neurons are turned off. The paper tests this on about 6M Wikipedia articles and on Harry Potter books.

Critical strengths
  1. The idea is interesting and simple: just one mask in the MLP, with no extra parameters per source.
  2. The paper checks that facts which can be learned from other articles are kept after deletion (Fig. 4), which is the right thing to test.
Critical weaknesses
  1. The experiments are thin. There is only one 1B model, each setting is trained once, the general-ability test uses four benchmarks where scores are near random, and the only baselines are GA and NPO without a retain loss.
  2. Deletion at scale is not really tested. On Wikipedia, sinks are only turned off through routing, which anyone with the weights can undo. With 6M articles, each sink neuron is shared by about 75k articles, so actually zeroing one article's sinks also touches many others.
Questions for authors
  1. What happens when you zero the sink weights for 10, 100 or 1000 articles in the 6M model? How much do the other articles that share those neurons lose?
  2. The model sees each article for 7 epochs. In real pretraining most documents are seen once. Does the source-specific knowledge still collect in the sinks in that case?
  3. The real router is right only 73% of the time, and a wrong pick loses most article-specific knowledge (Fig. 9). How much does the deployed model actually know about articles that were not deleted?
  4. How was the retrained model behind "closely matches retraining" built?
2 comments

Permanent Deletion of Multiple Sources

What happens when you zero the sink weights for 10, 100 or 1000 articles in the 6M model? How much do the other articles that share those neurons lose?

Deleting a large number of sources would not impact the general capabilities of the model due to the seperation between shared and general neurons. However, due to the sharing of sink neurons between articles, it can harm unique knowledge from other articles at high-levels of permanent deletion. At permanent deletion of 10 articles, we found on average the Truth Ratios on retained article facts declined less than 3%. On the other hand, as more documents are permanently deleted, the model converges to performance of the shared-only model.

On the other hand, as we mention in our paper, any number of sources can be deleted via routing with no degradation to remaining articles. Thus, we expect that permanent deletion can be leveraged for removing a small number of highly sensitive documents without harming retention of other documents.

Routing Accuracy

The real router is right only 73% of the time, and a wrong pick loses most article-specific knowledge (Fig. 9). How much does the deployed model actually know about articles that were not deleted?

Routing accuracy

Thank you for raising this. Under the deployed router, the model's truth ratio on facts from non-deleted articles is 2.65, compared with 2.75 under ground-truth routing. The 73% figure overstates how much knowledge is lost, for two reasons:

  1. It treats every fact as belonging to a single article. A routing decision counts as correct only if it selects the article from which we extracted the fact. Many facts also appear in related articles and are stored in the shared backbone (Appendix A.6), so a "wrong" pick often still answers them.
  2. Most errors are near misses. For Cloze prompts, which name the article, the router selects the correct article 73.1% of the time and ranks it in the top 5 for 85.9% and the top 10 for 89.1%, so nearly half of top-1 errors place the correct article among the top five candidates. For facts unique to one article, a miss does lose most of the knowledge (Fig. 9). This cost comes from the router, a modular retrieval component, rather than from how knowledge is stored: with ground-truth routing, the same model matches the standard transformer, and a reranker over the top ten candidates could raise accuracy to as much as 89.1% without changing the model.

Forgetting is unaffected: a deleted article's mask is withheld, so no routing decision can activate it.

Repeated Pretraining

The model sees each article for 7 epochs. In real pretraining most documents are seen once. Does the source-specific knowledge still collect in the sinks in that case?

Thank you for this question. It is correct that our Wikipedia experiments train for 7 epochs, so each article is seen seven times, and we do not isolate the single-exposure regime; we will state this explicitly.

The content that unlearning targets is mostly repeated. What a model retains about a specific source is driven by repetition: a training sequence present 10 times is regenerated roughly 1000× more often than one present once, and methods for detecting memorized sequences perform near chance on non-duplicated ones (Kandpal et al., 2022); factual accuracy likewise tracks how often relevant documents appear in pretraining (Kandpal et al., 2023). Real pipelines also deliberately repeat high-value sources (GPT-3 saw Wikipedia about 3.4 times; Brown et al., 2020), and repeating data for up to 4 epochs yields negligible loss differences compared with unique data (Muennighoff et al., 2023). In single-epoch web training, much repetition comes from duplicated and near-duplicated documents; assigning them a shared source identifier, as our cluster-level sources do, places that content in the multi-exposure regime we study.

For content seen once, we expect the same mechanism at smaller magnitude. Each sink is active on only about 1.25% of the training data (100 of 8,000 sinks per source), so it receives roughly 1/80 as many interfering updates as a backbone neuron, which is active on every token. Source-specific information that survives a single exposure should therefore survive preferentially in the sinks, while its copy in the backbone is overwritten by subsequent training. Because little survives a single exposure in any model, the gap between the trained model and one retrained without that document, which is what unlearning must close, is correspondingly small.

Kandpal, Nikhil, Eric Wallace, and Colin Raffel. "Deduplicating training data mitigates privacy risks in language models." International Conference on Machine Learning. PMLR, 2022

Brown, Tom, et al. "Language models are few-shot learners." Advances in neural information processing systems 33 (2020): 1877-1901.

Muennighoff, Niklas, et al. "Scaling data-constrained language models." Advances in Neural Information Processing Systems 36 (2023): 50358-50376.

Retraining Model

How was the retrained model behind "closely matches retraining" built?

The retrained model was trained with an identical architecture and hyperparameters as the main NULLs model, except with the unlearning target articles removed from the corpus. This follows the desiderata of unlearning, which is to simulate a model trained exactly in the same way, simply with the unlearning target removed from the corpus.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.