- Rating
- 1
- Confidence
- 3
- Informativeness
- 1
- Presentation
- 3
- Soundness
- 3
Read review
Paper summary
This paper proposes EMBER, a training objective for reducing verbatim reproduction of designated protected passages while preserving ordinary language generation. It applies a smooth margin penalty to each protected continuation’s average token negative log-likelihood, attenuating suppression as its energy rises above a target, alongside ordinary-data likelihood training. The paper analyzes the resulting gradient weighting, local curvature, and probability interpretation, and evaluates the method on BookMIA and a controlled Wikitext-2 split across GPT-2 full fine-tuning and Llama-3 and Qwen-3 LoRA adaptation. Experiments over five seeds report substantial reductions in greedy exact continuation with relatively small changes in ordinary-text perplexity and MAUVE, supplemented by model-scaling and hyperparameter studies.
Critical strengths
-
Consistent empirical suppression with limited measured utility loss. Across three model families and two data settings, EMBER substantially reduces exact continuation while largely preserving ordinary-text perplexity and MAUVE. Results over five seeds support the reproducibility of this trade-off in the evaluated settings.
-
Clear analysis and explicit scope. The paper explains how the loss allocates suppression across examples and explicitly acknowledges that sequence-energy bounds do not guarantee reduced greedy reproduction. This makes the theoretical claims easier to assess and avoids overstating their guarantees.
Critical weaknesses
-
Limited theoretical novelty and an unresolved connection to exact reproduction. Let denote the average token negative log-likelihood. EMBER’s protected loss is
=\tau\log\left(1+\exp\left(\frac{m-E}{\tau}\right)\right) =-\tau\log\sigma\left(\frac{E-m}{\tau}\right).$$ However, the margin-based SimNPO objective already takes the form $$\ell_{\mathrm{SimNPO}} =-\frac{2}{\beta}\log\sigma\left( -\frac{\beta}{|y|}\log p_\theta(y\mid s)-\gamma \right) =-\frac{2}{\beta}\log\sigma(\beta E-\gamma).$$ Setting $\beta=1/\tau$ and $\gamma=m/\tau$ gives $$\ell_{\mathrm{SimNPO}}=2R_{\mathrm{EMBER}},$$ with corresponding gradients $$\nabla_\theta\ell_{\mathrm{SimNPO}} =-2\sigma(\gamma-\beta E)\nabla_\theta E =2\nabla_\theta R_{\mathrm{EMBER}}.$$ Thus, with identical scoring and loss reduction, the protected objectives are equivalent up to a constant factor that can be absorbed into the suppression weight. Although SimNPO defaults to $\gamma=0$, its original formulation explicitly includes a nonzero margin and examines its effects [1]. Consequently, EMBER’s central adaptive-gradient mechanism is not clearly distinct from existing work, and the additional analysis does not establish a new guarantee of reduced exact reproduction.In particular, its sequence-probability bound, $$E\geq m \quad\Longrightarrow\quad p_\theta(y\mid s)\leq e^{-|y|m},$$ neither bounds the first-ten-token energy nor guarantees a change in greedy token rankings. The paper acknowledges this limitation, but **omits a direct SimNPO comparison and does not sufficiently establish an independent theoretical contribution.** -
Limited evidence of applicability to contemporary model architectures. The experiments cover GPT-2, Llama-3, and Qwen-3. Our concern is whether the design remains effective under the architectures used by current leading models, rather than model recency alone. For example, mixture-of-experts models introduce parameter-sharing and routing patterns that could affect how protected-data suppression interacts with ordinary-data retention. Moreover, an absolute per-token NLL threshold may transfer differently across architectures, tokenizers, and model calibrations. Evaluation on a representative contemporary architecture, such as Qwen3.5[2], would help establish whether the reported suppression–utility trade-off extends beyond the tested configurations.
References
[1] Chongyu Fan, Jiancheng Liu, Licong Lin, Jinghan Jia, Ruiqi Zhang, Song Mei, and Sijia Liu. Simplicity Prevails: Rethinking Negative Preference Optimization for LLM Unlearning. arXiv:2410.07163, 2024. See Section 5, Equation (4), and Appendix A. Paper
[2] Qwen Team. Qwen3.5. Official model release announcement, 2026. Announcement
Questions for authors
-
How does EMBER differ from margin-based SimNPO beyond reparameterization and scaling? This is a clarification request central to the novelty assessment. With , setting and in SimNPO gives . Please explain whether any implementation or objective differences invalidate this equivalence, clarify the independent methodological and theoretical contributions, and justify the omission of SimNPO from the related work and experiments.
-
Does the suppression–utility trade-off persist on a contemporary hybrid architecture, such as Qwen3.5-9B-Base or the lower-cost Qwen3.5-4B-Base? This is a request for additional evidence relevant to the claimed applicability, rather than a suggestion for future work. Please evaluate EMBER alongside margin-based SimNPO under matched adaptation settings, reporting both the shared default configuration and validation-tuned parameters. This would clarify whether the method and its absolute NLL target transfer beyond the architectures currently evaluated.
-
Does reaching the full-continuation energy target reliably correspond to reduced greedy EM@10? This is a request for mechanism validation. Since suppression can attenuate while the first ten tokens remain highly predictable, please report the relationship between full-continuation energy, first-ten-token energy, and greedy exact matching during training. A comparison with a prefix- or window-based penalty at matched ordinary-generation quality would help establish whether the chosen sequence-level objective is appropriate for the primary metric.
I would consider 2 (Weak Rejection) if the authors clarify the equivalence to margin-based SimNPO, revise the novelty claims, and provide a fair comparison. I would consider 3 (Weak Acceptance) if they additionally establish a substantive independent contribution supported by reproducible evidence, with validation on a contemporary architecture and a clearer link between the energy target and exact reproduction.