- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 1
Read review
Paper summary
The paper introduces PUMA, a framework for multi-domain unlearning in large language models that generates low-rank adapters from prompts, enabling flexible unlearning without retraining for each domain. It combines forgetting objectives across domains and demonstrates improved performance over existing methods.
Critical strengths
- The paper proposes a novel framework, PUMA, which addresses the challenge of multi-domain unlearning by generating adapters from prompts, enabling flexible and efficient unlearning without retraining for each domain combination.
- The experiments are comprehensive, evaluating PUMA across multiple LLM backbones and domains, showing superior performance in balancing forgetting efficacy and retained utility compared to existing methods like Single, EUL and ALKN.
- The method’s efficiency is highlighted, as PUMA avoids retraining for each request and supports zero-shot deployment through a shared generator. This is a practical advantage for real-world applications.
- The paper incorporates structural embeddings and module-specific residual heads, which enhance the model’s ability to adapt to domain-specific updates, as demonstrated through ablation studies.
Critical weaknesses
- The theoretical analysis of PUMA’s parameter generation mechanism is limited. While the framework is empirically validated, the paper lacks a rigorous mathematical proof or derivation for the generator’s behavior.
- The experiments do not explicitly address the scalability of PUMA to domains outside the tested ones (e.g., unseen domains). The paper assumes that the generator generalizes well but does not provide evidence for this claim.
- The paper does not compare PUMA’s computational efficiency (e.g., memory usage or inference speed) with existing methods, which is critical for practical deployment.
Questions for authors
- PUMA’s parameter generator has only empirical support, without rigorous mathematical derivation to explain its behavior.
- The experiments do not verify whether PUMA can generalize to unseen domains, and the assumption of cross-domain transfer lacks supporting evidence.
- The paper omits comparisons of PUMA’s memory cost and inference speed against prior work, which restricts assessment of its practical usability.