Learning to Forget Together: Prompt-Conditioned Adapter Generation for Multi-Domain LLM Unlearning

Large language models (LLMs) can memorize and disclose sensitive information, making machine unlearning an important tool for mitigating privacy and safety risks. However, removing knowledge across multiple domains can cause interference between unlearning updates and degrade general model capabilities. The need to retrain for different combinations of target domains further complicates deployment. We introduce PUMA (Prompt-conditioned Unlearning via Multi-domain Adapter Generation), a framework that generates low-rank adaptation (LoRA) weights from prompts representing the knowledge to be forgotten. A shared parameter generator learns to translate instance-level prompts into unlearning adapters, enabling flexible requests expressed through examples. Once trained, PUMA supports instance-based, multi-domain unlearning without further optimization for each request, enabling zero-shot deployment. It combines forgetting objectives across domains within a single generated adapter, eliminating the need to train a separate adapter for every domain combination. Experiments show that PUMA achieves a better balance between forgetting efficacy and retained utility than existing domain-level unlearning methods while accommodating multiple target domains. These results highlight the potential of prompt-conditioned parameter generation for flexible and efficient LLM unlearning.