acceptodds
Under review as a conference paper at ICLR 2027

EMBER: Margin-Targeted Energy Reshaping for Mitigating Exact Memorization in Autoregressive Language Models

Abstract

Autoregressive language models can reproduce protected training passages verbatim. Training-time suppression reduces this behavior, but strong suppression can also degrade ordinary generation. We propose EMBER, a margin-targeted energy reshaping objective that controls how much pressure each protected continuation receives. Using average token negative log-likelihood as energy, EMBER applies a smooth margin penalty: low-energy protected sequences receive strong repulsion, and the pressure decays as their energy rises above a specified target. The margin sets the location of this transition, while a temperature controls its width. Our analysis characterizes this gradient attenuation and connects the energy target to conditional sequence probability. Experiments on BookMIA and a controlled Wikitext-2 split cover GPT-2 with full fine-tuning and Llama-3 and Qwen-3 with LoRA. On BookMIA, EMBER reduces exact match from 34.7% to 4.4% for Llama-3-8B and from 29.4% to 1.9% for Qwen-3-8B, while keeping perplexity and MAUVE close to the unprotected models. The results demonstrate that an explicit energy target provides a simple, effective way to balance exact memorization mitigation with generation quality.

Primary area:
alignment, fairness, safety, privacy, and societal considerations
Keywords:
Large Language Models, Memorization Mitigation, Copyright Protection

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_WMrbOpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
3
Soundness
3
Read review
Paper summary

This paper proposes EMBER, a training objective for reducing verbatim reproduction of designated protected passages while preserving ordinary language generation. It applies a smooth margin penalty to each protected continuation’s average token negative log-likelihood, attenuating suppression as its energy rises above a target, alongside ordinary-data likelihood training. The paper analyzes the resulting gradient weighting, local curvature, and probability interpretation, and evaluates the method on BookMIA and a controlled Wikitext-2 split across GPT-2 full fine-tuning and Llama-3 and Qwen-3 LoRA adaptation. Experiments over five seeds report substantial reductions in greedy exact continuation with relatively small changes in ordinary-text perplexity and MAUVE, supplemented by model-scaling and hyperparameter studies.

Critical strengths
  1. Consistent empirical suppression with limited measured utility loss. Across three model families and two data settings, EMBER substantially reduces exact continuation while largely preserving ordinary-text perplexity and MAUVE. Results over five seeds support the reproducibility of this trade-off in the evaluated settings.

  2. Clear analysis and explicit scope. The paper explains how the loss allocates suppression across examples and explicitly acknowledges that sequence-energy bounds do not guarantee reduced greedy reproduction. This makes the theoretical claims easier to assess and avoids overstating their guarantees.

Critical weaknesses
  1. Limited theoretical novelty and an unresolved connection to exact reproduction. Let Eθ(y∣s)=−∣y∣−1log⁡pθ(y∣s)E_\theta(y\mid s)=-|y|^{-1}\log p_\theta(y\mid s) denote the average token negative log-likelihood. EMBER’s protected loss is

    =\tau\log\left(1+\exp\left(\frac{m-E}{\tau}\right)\right) =-\tau\log\sigma\left(\frac{E-m}{\tau}\right).$$ However, the margin-based SimNPO objective already takes the form $$\ell_{\mathrm{SimNPO}} =-\frac{2}{\beta}\log\sigma\left( -\frac{\beta}{|y|}\log p_\theta(y\mid s)-\gamma \right) =-\frac{2}{\beta}\log\sigma(\beta E-\gamma).$$ Setting $\beta=1/\tau$ and $\gamma=m/\tau$ gives $$\ell_{\mathrm{SimNPO}}=2R_{\mathrm{EMBER}},$$ with corresponding gradients $$\nabla_\theta\ell_{\mathrm{SimNPO}} =-2\sigma(\gamma-\beta E)\nabla_\theta E =2\nabla_\theta R_{\mathrm{EMBER}}.$$ Thus, with identical scoring and loss reduction, the protected objectives are equivalent up to a constant factor that can be absorbed into the suppression weight. Although SimNPO defaults to $\gamma=0$, its original formulation explicitly includes a nonzero margin and examines its effects [1]. Consequently, EMBER’s central adaptive-gradient mechanism is not clearly distinct from existing work, and the additional analysis does not establish a new guarantee of reduced exact reproduction.In particular, its sequence-probability bound, $$E\geq m \quad\Longrightarrow\quad p_\theta(y\mid s)\leq e^{-|y|m},$$ neither bounds the first-ten-token energy nor guarantees a change in greedy token rankings. The paper acknowledges this limitation, but **omits a direct SimNPO comparison and does not sufficiently establish an independent theoretical contribution.**
  2. Limited evidence of applicability to contemporary model architectures. The experiments cover GPT-2, Llama-3, and Qwen-3. Our concern is whether the design remains effective under the architectures used by current leading models, rather than model recency alone. For example, mixture-of-experts models introduce parameter-sharing and routing patterns that could affect how protected-data suppression interacts with ordinary-data retention. Moreover, an absolute per-token NLL threshold may transfer differently across architectures, tokenizers, and model calibrations. Evaluation on a representative contemporary architecture, such as Qwen3.5[2], would help establish whether the reported suppression–utility trade-off extends beyond the tested configurations.

References

[1] Chongyu Fan, Jiancheng Liu, Licong Lin, Jinghan Jia, Ruiqi Zhang, Song Mei, and Sijia Liu. Simplicity Prevails: Rethinking Negative Preference Optimization for LLM Unlearning. arXiv:2410.07163, 2024. See Section 5, Equation (4), and Appendix A. Paper

[2] Qwen Team. Qwen3.5. Official model release announcement, 2026. Announcement

Questions for authors
  1. How does EMBER differ from margin-based SimNPO beyond reparameterization and scaling? This is a clarification request central to the novelty assessment. With E=−∣y∣−1log⁡pθ(y∣s)E=-|y|^{-1}\log p_\theta(y\mid s), setting β=1/τ\beta=1/\tau and γ=m/τ\gamma=m/\tau in SimNPO gives ℓSimNPO=2REMBER\ell_{\mathrm{SimNPO}}=2R_{\mathrm{EMBER}}. Please explain whether any implementation or objective differences invalidate this equivalence, clarify the independent methodological and theoretical contributions, and justify the omission of SimNPO from the related work and experiments.

  2. Does the suppression–utility trade-off persist on a contemporary hybrid architecture, such as Qwen3.5-9B-Base or the lower-cost Qwen3.5-4B-Base? This is a request for additional evidence relevant to the claimed applicability, rather than a suggestion for future work. Please evaluate EMBER alongside margin-based SimNPO under matched adaptation settings, reporting both the shared default configuration and validation-tuned parameters. This would clarify whether the method and its absolute NLL target transfer beyond the architectures currently evaluated.

  3. Does reaching the full-continuation energy target reliably correspond to reduced greedy EM@10? This is a request for mechanism validation. Since suppression can attenuate while the first ten tokens remain highly predictable, please report the relationship between full-continuation energy, first-ten-token energy, and greedy exact matching during training. A comparison with a prefix- or window-based penalty at matched ordinary-generation quality would help establish whether the chosen sequence-level objective is appropriate for the primary metric.

I would consider 2 (Weak Rejection) if the authors clarify the equivalence to margin-based SimNPO, revise the novelty claims, and provide a fair comparison. I would consider 3 (Weak Acceptance) if they additionally establish a substantive independent contribution supported by reproducible evidence, with validation on a contemporary architecture and a clearer link between the energy target and exact reproduction.

Reviewer_ZXyvOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper studies how to stop a language model from repeating protected text word for word, for example copyrighted book passages, while keeping it useful for ordinary text. During fine-tuning, the training data is split into an ordinary set and a protected set. The model learns the ordinary set as usual and is pushed away from the protected set.

The method, EMBER, treats the average per-token negative log-likelihood of a protected continuation as its "energy". It then applies a softplus penalty that is large when this energy is below a chosen target m and fades away once the energy rises above it. A temperature tau sets how sharp this fade is, and a weight lambda sets how strongly the protected term competes with ordinary training. In plain terms, each protected passage is pushed until it is "unlikely enough", and then left alone. Section 3 and Appendix A work out the gradient, the per-example weighting (a sigmoid of the distance to the target), a variational form with an entropy term, and a smoothness bound.

Experiments fine-tune GPT-2 (full fine-tuning) and Llama-3-8B and Qwen-3-8B (LoRA) on BookMIA and on a random 2:8 protected/ordinary split of Wikitext-2. Leakage is measured by greedy exact match of the first ten tokens (EM@10) and ROUGE-L on 1,000 prompts whose passages are in the protected training set. Utility is measured by perplexity and MAUVE on ordinary data. Against SFT, inverse regularization (IR), Goldfish, and Copyright Regression (CR), EMBER lowers BookMIA EM@10 from 34.7% to 4.4% on Llama-3-8B with a small perplexity increase (Table 1). There is also a scaling study over model sizes (Section 4.3), a one-at-a-time sensitivity study (Table 2), and a qualitative example (Table 3). Results are averaged over five seeds.

Critical strengths

S1. A simple, easy-to-use objective with clear and careful analysis. The loss is one softplus per protected example on top of standard training. It needs no reference model and adds nothing at inference time. Appendix A explains well what each control does: m places the transition, tau sets its width, and lambda scales its strength. The paper is also careful about scope. It states openly that a full-sequence energy target does not bound greedy EM@10 (Appendix A.6), and it lists what the evaluation does not cover (Section 4.5). This makes the claims easier to judge.

S2. Leakage and utility are reported together, with seeds. Every method in Table 1 is reported on both leakage (EM@10, ROUGE-L) and utility (PPL, MAUVE), with means and standard deviations over five seeds. This makes the trade-off visible. For example, IR reaches very low leakage but at a large utility cost. The study covers two datasets, two adaptation regimes, and a range of model sizes (Figure 3), and the trend that SFT memorization grows with model size is useful context.

Critical weaknesses

W1. The experimental setup does not show that EMBER solves a real problem better than simple alternatives. This is my biggest concern.

  • SFT is the only "unprotected" reference, and it does likelihood training on the protected set C (Section 4.1, Appendix D.1). EMBER never trains on C with likelihood. So most of the leakage that EMBER "removes" is leakage that SFT itself created during the same fine-tuning. The obvious baselines are missing: (a) the base model with no fine-tuning, and (b) fine-tuning on the ordinary set O only. Appendix D.3 mentions that SFT is "distinct from an ordinary-only adaptation and from the base model", but neither is reported. If simply leaving C out of training gives similar or lower EM@10 with equal or better PPL and MAUVE, the method has no clear advantage in this setting. For BookMIA, the base Llama-3 and Qwen-3 models may already have seen some of these books in pretraining. The base-model EM@10 is then the key number for knowing whether EMBER removes pre-existing memorization or only avoids adding new memorization.
  • The motivating use case is unclear. If a passage is known to be protected, a practitioner can drop it from fine-tuning. The more realistic case is removing memorization that a model already has, which is the unlearning setting. The paper states that it does not test forgetting from a trained checkpoint (Appendix D.6). So the setting where an explicit suppression objective is most needed is not evaluated.
  • The baselines do not treat C in the same way. Goldfish and CR still fit the protected text with a positive likelihood term (Appendix D.3), while EMBER and IR only repel it. Part of the gap in Table 1 may come from this difference rather than from the margin design.
  • Gradient ascent (Eq. 5) and NPO (Section 5) are discussed but not compared. These are the most natural baselines for a repulsion loss on designated text. Each baseline is also run at a single operating point (Appendix D.6), and the baseline hyperparameters (for example the Goldfish mask rate and the CR coefficient) are not reported. IR in particular looks badly tuned, with a very large perplexity increase. A fair comparison would trace the leakage-utility curve of each method by varying its main strength parameter, or compare at matched utility.
  • Leakage is measured only on the exact passages that were in the protected training set, with one fixed 40-token prefix and greedy decoding. There is no test of sampling, other prefix lengths, paraphrased prompts, or held-out protected passages. The paper notes this (Section 4.5), but it limits how far the claims about copyright protection can go.

W2. The novelty of the objective is limited, and the analysis is mostly standard softplus facts.

  • The loss is a log-sigmoid (softplus) margin on length-normalized NLL. This form is very close to existing unlearning losses. NPO already uses a log-sigmoid to make suppression fade smoothly (Zhang et al., 2024, cited). SimNPO (Fan et al., 2024, "Simplicity Prevails: Rethinking Negative Preference Optimization for LLM Unlearning") removes the reference model from NPO and applies a log-sigmoid with a margin to the length-normalized log-likelihood. As far as I can tell, this is close to Eq. 6 after a change of variables (m as the margin, tau as the inverse temperature). SimNPO is not cited or compared. The paper should explain what EMBER adds beyond it.
  • The theory in Appendix A (softplus bounds, the sigmoid gradient, the entropy form of softplus, Jensen for batch averaging, a smoothness bound) is correct as far as I checked. However, these are textbook properties of softplus. They describe the loss, but they do not connect it to the measured outcome. The paper itself shows that a full-sequence energy above m does not bound greedy EM@10 (Appendix A.6). So the analysis supports the shape of the loss, not the empirical claims.
  • The energy framing adds little. The "energy" is just the average token NLL, and nothing from energy-based modeling (partition functions, sampling) is used. Also, the default target m = 1 nat per token is lower than the ordinary-data energy implied by the reported perplexities. So at the target, protected passages may still be more likely than ordinary text. The "energy pit" picture (Eq. 4) does not quite match what the objective asks for.
  • Some claims are stronger than the evidence. The gap to CR on GPT-2 BookMIA is within the reported spread, and CR has lower EM there. The sensitivity study (Table 2) has no standard deviations, so the small differences across tau cannot be read as trends. The contribution of "consistent gains" across three model families rests on one configuration per baseline.
Questions for authors

Clarifications:

Q1. Missing reference points. Can you report EM@10, ROUGE-L, PPL, and MAUVE for (a) the base model with no fine-tuning and (b) fine-tuning on the ordinary set only? If EMBER clearly lowers leakage below both of these, especially below the base model on BookMIA, my main concern would be resolved. If ordinary-only training matches EMBER, please explain when one would prefer EMBER.

Q2. Unlearning setting. Can you run EMBER starting from the SFT checkpoint (or from a base model that already memorizes BookMIA passages) and compare it with gradient ascent, NPO, and SimNPO in that setting?

Q3. Relation to SimNPO. Please state the exact relation between Eq. 6 and the SimNPO loss. Is EMBER the same objective up to reparameterization? If not, what is different, and does that difference matter empirically?

Q4. Baseline tuning. How were the hyperparameters of IR, Goldfish, and CR chosen, and what values were used? Can you show leakage-utility curves for each method by varying its strength, or report results at matched PPL or MAUVE?

Q5. Utility evaluation. Is PPL computed on held-out ordinary text or on the ordinary training set? The Wikitext-2 perplexities in Table 1 are very low, and EMBER is sometimes lower than SFT. Please clarify the split and the tokenization used, and report utility on at least one standard downstream benchmark.

Q6. Stronger extraction. How does EMBER behave under sampling-based extraction, longer or shorter prefixes, and on protected passages that were not in the training set (for example a held-out part of the pre-2023 books)?

Minor:

Q7. Please add standard deviations to Table 2 and to the scaling numbers in Section 4.3.

Q8. With m = 1 and the reported ordinary perplexities, protected text may still be more likely than ordinary text at the target. Did you try setting m relative to the ordinary-data energy, and how was m = 1 chosen?

Q9. Will code, data splits, and the exact baseline implementations be released? The reproducibility statement does not mention a code release.

Reviewer_sGK4OpenReview ↗
Rating
2
Confidence
3
Informativeness
1
Presentation
3
Soundness
2
Read review
Paper summary

In this paper, the authors tackle the problem of mitigating exact memorization in autoregressive LLMs and present EMBER, which combines ordinary-data likelihood training with a softplus penalty on protected continuations whose length-normalized NLL falls below a target margin. The paper analyzes the resulting gradient weighting and curvature, and evaluates exact continuation and ordinary-generation quality on BookMIA and Wikitext-2. Experiments cover GPT-2 full fine-tuning and Llama-3/Qwen-3 LoRA adaptation, reporting lower greedy EM@10 than SFT with small changes in the reported utility metrics.

Critical strengths

The reported leakage–utility improvements are consistent across several settings. Over five seeds, EMBER improves all reported metrics relative to Goldfish in all six model/dataset settings, allowing ties in MAUVE and generally improving on Copyright Regression. I consider this more informative than the comparison against SFT alone, and it suggests a useful point for the evaluated objective.

Critical weaknesses
  1. My main concern is that the proposed protected-data objective is very close to existing work (see https://arxiv.org/pdf/2410.07163). Unfortunately, the paper does not cite it, and in my view it should clarify any substantive differences in retention normalization or training protocol and explain what its analysis or experiments contribute beyond this existing objective. The claimed methodological novelty is not evident to me.

  2. SFT positively trains on protected text, whereas EMBER replaces that fitting term with repulsion. Their difference combines withholding positive supervision with applying the penalty. Appendix D.3 itself notes that SFT "is distinct from an ordinary-only adaptation and from the base model before adaptation", yet neither the base model nor ordinary-only fine-tuning is reported? In my view, without these controls, the experiments do not establish the penalty's benefit over excluding protected text, or distinguish prevention of new memorization from reduction of existing memorization.

  3. The paper does not clearly show that ordinary evaluation text is held out from training. Appendix D.3 discusses possible Goldfish masks and Copyright Regression adaptations without fully specifying the implemented variants and their selected coefficients. I consider these details important when comparing leakage at similar utility, since a common training budget alone does not prove competitiveness.

Questions for authors
  • What substantive difference from the above cited paper's objective could justify the novelty of approach?
  • What leakage and held-out utility do the base model and ordinary-only fine-tuning achieve under the same evaluation protocol?
  • If you could specify the ordinary-data train/validation/test separation, implemented baseline objectives, and their tuning procedures, that would be useful and important.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.