acceptodds
Under review as a conference paper at ICLR 2027

PULSAR: Who-Forgets-Whom Personalized Unlearning in Federated Large Language Models

Abstract

Federated unlearning (FU) has largely focused on producing a single unlearned global model, implicitly enforcing “forgetting for all” whenever any client requests deletion. In realistic federated deployments, however, data and model-sharing permissions are often client-specific and may change over time, requiring selective revocation of one client’s contribution from a subset of other clients. We propose PULSAR, the first who-forgets-whom personalized FU framework, which enables a source client to require a designated group of target clients to erase its contributions, while preserving the model utility of non-target clients that remain authorized or cooperative. Building on parameter-efficient federated fine-tuning, PULSAR constructs a portable, plug-and-play adapter-style knowledge filter from an unlearning operation and applies it at model dispatch to enforce client-specific forgetting. To mitigate utility degradation on target clients, we further introduce a joint optimization procedure to balance effective unlearning and utility retention. Single-round experimental results demonstrate that PULSAR achieves similar forgetting effect on targeted clients to Retrain baseline across multiple LLM backbones and FU settings, while maintaining similar model utility on non-target clients to the Finetune baseline.

Primary area:
foundation or frontier models, including LLMs
Keywords:
Federated Unlearning, Large Language Models, Personalized Unlearning

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_jBVoOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

In this paper, the authors study recipient-specific federated unlearning for large language models. They propose PULSAR, which allows different clients to forget different source clients’ data. The method uses source-specific filters and sends them only to the clients that need to forget the data. It also uses local optimization to reduce the loss in task performance. Experiments on TOFU and MUSE-News cover several LLMs, federated aggregation methods, and unlearning algorithms. The results show that PULSAR can achieve forgetting performance close to retraining while preserving reasonable utility for other clients.

Critical strengths
  1. The paper addresses an important and practical problem in federated unlearning. Existing methods usually produce one unlearned global model, so all clients are affected by the same forgetting request. PULSAR introduces a source–target relation and allows only selected clients to forget specific source data. This setting is useful when different clients have different data permissions.

  2. The proposed framework use different underlying unlearning algorithms and convert their effects into source-specific filters. The server then sends these filters only to the clients that need them. This design does not require a completely new unlearning algorithm and may be easier to integrate into existing federated learning systems.

  3. The method supports reversible and client-specific model updates. The filters are applied during model delivery, while relative updates are used for global aggregation. As a result, the forgetting operation does not need to be permanently written into the global model. This makes it possible to revoke a forgetting relation and restore the normal model for a client.

Critical weaknesses
  1. The paper mainly uses FTR, ROUGE, and model utility to measure forgetting. It does not report the original TOFU Forget Quality metric or sufficient privacy tests such as membership inference. In addition, multiple filters are combined by a weighted average, but this is not equivalent to unlearning the union of the corresponding datasets.

  2. The paper does not clearly explain how the source, target, and authorized clients are assigned or how many samples each client receives. It also does not provide enough information about random seeds, repeated runs, or statistical significance. Some FedAvgM and FedYogi results in Tables 5 and 6 are nearly identical across many settings, which may indicate a reporting or implementation problem.

Questions for authors
  1. The paper evaluates several unlearning algorithms within the PULSAR framework, but it does not provide direct comparisons with existing personalized or reversible federated unlearning methods, such as FUSED, ZeroFU, Mimir, or ConPFU. Why are these methods not included as baselines?

  2. Can the authors compare PULSAR with the following baselines under the same data split and computing budget: full retraining, global unlearning for all clients, local unlearning for each target client, and a standard personalized federated learning method without filtering?

  3. Tables 5 and 6 show almost identical results for FedAvgM and FedYogi in many settings. Are these values correct? Please provide the results for every seed and explain how the two aggregators were implemented.

  4. In PULSAR, each source client must run the underlying unlearning algorithm locally and upload a source-specific filter. What happens if the source client refuses to participate, is offline, or uploads an incorrect or malicious filter? Can the server generate or verify the filter without access to the source client’s data? If not, the selective forgetting guarantee appears to depend on source-client cooperation.

3 comments
AuthorsOpenReview ↗

Response to Reviewer jBVo: Experimental Validation and Clarifications

Thank you for the valuable review. We below provide the point-to-point responses. Revisions are marked in blue.


Q1. Personalized/reversible FU baselines. FUSED, ZeroFU, Mimir, and ConPFU target reversibility, adaptation, or forgetting intensity. PULSAR instead specifies who-forgets-whom contribution: designated targets forget while authorized recipients retain source knowledge. These methods lack this explicit source–recipient policy. We therefore add policy-aligned comparisons with FFA, target-local unlearning, and filter-free personalized FT (Sec. 5, p.10, Apps. F.10–F.11). These controls test recipient-specific dispatch under the intended policy.


Q2①. Full retraining and budgets. All controls use Split90. Retrain serves as a full-cost reference with 11 retain-only rounds. FFA and the target-local oracle match operator/FT epochs, while filter-free controls share the local-FT budget (App. F.11, p.44). Retrain is an effectiveness reference, not an equal-cost control.

Q2②. Global unlearning. FFA broadcasts the same source-derived filter to all recipients. Target FTR is comparable (PULSAR/FFA: 0.658/0.644 Llama, 0.647/0.651 Qwen). Authorized-client FTR increases from 0.565/0.569 to 0.641/0.649, and FOR falls from 0.509/0.465 to 0.449/0.434. MU changes by only −0.004/−0.002. PULSAR avoids unintended forgetting of knowledge still authorized for non-target recipients (Table 2(b), p.9, Table 16(a), p.43).

Q2③. Target-local unlearning. The target-local oracle receives source forget data and applies the same operator. At κ=1 with matched epochs, PULSAR/oracle MU/FTR is 0.541/0.636 vs. 0.561/0.635 (GradDiff), 0.546/0.618 vs. 0.547/0.612 (NPO), and 0.540/0.631 vs. 0.557/0.628 (SimNPO). PULSAR matches FTR without sharing source forget data, though the oracle has higher MU (Table 18, p.45).

Q2④. Personalized FT without filtering. Finetune and No-F+JDOO reach FTR 0.571 and 0.583, versus 0.658 for PULSAR (Table 2(a), p.9). The source-derived filter contributes forgetting beyond local personalization. Together, these controls substantiate policy-specific forgetting.


Q3①. FedAvgM/FedYogi results. The original near-duplicates arose from a duplicated warm-up checkpoint. We corrected the checkpoint, reran affected warm-ups, and regenerated Tables 3–4. Llama Finetune MU/FTR is now 0.535/0.493 (FedAvgM) versus 0.467/0.687 (FedYogi) (p.33).

Q3②. Per-seed evidence. Table 8 (p.37) lists seeds 42/43/44 for both aggregators and backbones. FedYogi's Llama SimNPO FTR exceeds FedAvgM's in each reported seed (App. F.2, p.32).

Q3③. Implementation. FedAvgM uses server momentum. FedYogi adds a sign-controlled second-moment update, implemented using squared server momentum (App. A.4.2, pp.20–21). The corrected results and per-seed comparisons distinguish FedAvgM from FedYogi, addressing the reported duplication.


W2①–②. Client roles and seeds. Under Split90, client 0 is the source (400 forget + 1,200 retain QA), client 1 the target (1,200 retain), and client 2 authorized (1,200 retain). Table 1 reports mean ± SD over unlearning seeds 42/43/44, averaging ten aggregators per seed. FedAvg per-seed results (Table 7, p.37) improve FTR and FOR against same-seed Finetune for all eight operators on both backbones. Roles, partitions, and seed-level reporting are now explicit.

W2③. Aggregator correctness. See Q3 and corrected Tables 3–4.


W1①. TOFU Forget Quality. Table 11 (p.39) adds KS-based FQ. For GradDiff, the KS test does not reject at p>0.05 in 15/22 Llama and 18/22 Qwen evaluations (App. F.5, p.35). This adds the benchmark's distributional test beyond FTR/ROUGE.

W1②. Membership inference. Table 11 adds Min-K/loss MIA AUC and PrivLeak. For Llama/GradDiff, AUC is 0.379/0.384 versus Retrain's 0.380/0.383. PrivLeak is −0.2%/−8.9% (Llama/Qwen) versus −63.1%/−59.0% for Finetune (App. F.5, p.35). These metrics add privacy evidence beyond FTR/ROUGE.

W1③. Multiple filters. Normalized composition is not equivalent to union unlearning. Tables 20–21 (p.49) report per-relation results and a union-data oracle. On T123, normalized ΔFTR/ΔFOR/ΔMU is +0.011/−0.015/+0.001 versus +0.025/−0.050/−0.015 for the oracle (App. F.13). Composition preserves more MU at weaker forgetting than the union oracle.


Q4①–③. Source availability and trust. Filter construction requires a cooperative source with forget data. Without it, the server cannot construct or semantically verify the filter. A cached filter remains usable while the source is offline (mean ΔFTR +0.085 versus +0.087 with refresh, Table 16(b), p.43). Malicious filters are outside the protocol-compliant-source assumption (Sec. 3.1, p.4).

AuthorsOpenReview ↗

Follow-up on Author Response

Dear Reviewer jBVo,

We hope our rebuttal and revisions have addressed your concerns. As the discussion deadline approaches, we would be happy to clarify any remaining questions.

If your concerns have been addressed, we hope that you may very kindly reconsider your rating for our paper, and perhaps champion our paper.

Best regards, Authors of Paper 7666

AuthorsOpenReview ↗

Summary of Revisions in Response to Reviewer jBVo

Dear Reviewer jBVo,

Thank you again for your careful review and constructive feedback. We have made several revisions and conducted additional experiments to address your concerns. The main changes are summarized below.


1. Baseline Comparisons and Evaluation Fairness.

(1) We further clarified the distinction between PULSAR and existing personalized/reversible FU methods (Response Q1). These methods do not directly implement the source–recipient authorization policy studied in our work and therefore are not directly equivalent baselines under the same task setting (Sec. 5).

(2) We also added comparisons with FFA, target-local unlearning, and filter-free personalized fine-tuning. In Table 2(b), PULSAR achieves comparable target forgetting to FFA (FTR: 0.658 vs. 0.644 on Llama-1B), while maintaining FinetuneFinetune-level performance for source (CS\mathcal{C}^{\mathcal{S}}) and authorized (CA\mathcal{C}^{\mathcal{A}}) clients, thereby avoiding the collateral forgetting caused by global unlearning. We further compare PULSAR with a target-local unlearning oracle (Table 18). Under GradDiffGradDiff, NPONPO, and SimNPOSimNPO, PULSAR achieves comparable or slightly higher FTR without requiring target clients CT\mathcal{C}^{\mathcal{T}} to access the source-side forget set Dm~\tilde{\mathcal{D}_m}. Appendix F.11 further details the operational budgets and data-access assumptions of these baselines.


2. Unlearning and Privacy Evaluation.

We added the original TOFU Forget Quality (FQ) metric and membership inference attack (MIA) evaluations (Table 11). For example, on Llama-1B, GradDiffGradDiff improves the log10 FQ from −11.77 (FinetuneFinetune) to −1.50. Its Min-K/loss MIA AUCs are 0.379/0.384, closely matching RetrainRetrain (0.380/0.383). We also added external-domain evaluations on MUSE-News (Tables 26–27). The target CT\mathcal{C}^{\mathcal{T}}'s MIA Min-K reaches 0.5141, close to RetrainRetrain (0.5075). Meanwhile, the authorized recipient CA\mathcal{C}^{\mathcal{A}}'s MIA Min-K and PrivLeak remain highly consistent with FinetuneFinetune, with mean absolute deviations of only 0.000081 and 0.016250, respectively, across eight unlearning operators U\mathcal{U} and two aggregation operators A\mathcal{A}.


3. Reproducibility and Experimental Consistency.

(1) We clarified the client roles and data partitioning in Appendix A.5. The source client CS\mathcal{C}^{\mathcal{S}} holds 400 forget QA pairs and 1,200 retain QA pairs, while the target CT\mathcal{C}^{\mathcal{T}} and authorized CA\mathcal{C}^{\mathcal{A}} clients each hold 1,200 retain QA pairs. The partition of the 3,600 retain QA pairs is kept fixed across the three unlearning seeds (42, 43, and 44).

(2) We also added results across three random seeds (Tables 7–8). Table 7 reports the mean and standard deviation under FedAvgFedAvg, while Table 8 provides per-seed results for FedAvgMFedAvgM and FedYogiFedYogi on both model backbones. In response to the previously observed similarity between the FedAvgMFedAvgM and FedYogiFedYogi results, we rechecked the corresponding experiments and corrected the affected results.


4. Filter Composition and Threat Model.

(1) We added a union-data oracle comparison (Table 21, Appendix F.13) to demonstrate that normalized filter composition is not equivalent to unlearning the union of forget sets. For T123, normalized composition achieves mean ΔFTR/ΔFOR/ΔMU of +0.011/−0.015/+0.001, compared with +0.025/−0.050/−0.015 for the union-data oracle. This shows that union unlearning achieves stronger forgetting but incurs greater model utility loss.

(2) We also clarified the source-client cooperation requirements and the limitations of filter verification under the current threat model in Section 3.1. A cached filter can be reused when the source client is temporarily offline (Table 16(b), Appendix F.10). However, initial filter construction still requires source-client cooperation, as the server cannot independently construct or semantically verify a filter without access to the source-side forget data.


We would appreciate your reviewing the revised manuscript and our responses before the discussion closes. If you find the revisions satisfactory, we hope to have your support for our work.

Thank you again for your input.

Sincerely,

The Authors of Paper 7666

Reviewer_Z6jLOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes PULSAR, a personalized federated unlearning framework that specifies which recipients should forget each source client's data. It converts unlearning-induced LoRA parameter differences into reusable filters, composes them at model dispatch, and aggregates updates relative to each client's initialization. A target-side proximal objective, JDOO, supports local adaptation. Authorized recipients receive unfiltered models. Experiments across several LLMs, primarily on TOFU, show selective behavioral forgetting, with supplementary MUSE-News and Non-IID evaluations.

Critical strengths
  1. The paper considers a practical case where a source wants certain clients to forget what they learned from its data, while allowing authorized clients to retain that knowledge.
  2. The modular LoRA design supports different unlearning operators. Evaluation across model families, MUSE-News, and Non-IID partitions provides useful evidence for recipient-specific behavior.
Critical weaknesses
  1. Although Appendix F.10 evaluates 100 clients, Tables 12 and 13 report only target-averaged results at one round. The source-target graph and results for individual source-recipient pairs are not provided. Since normalized filter composition in Equation (10) is acknowledged to differ from unlearning the union of forget sets, these averages may conceal failures on particular relations. More detailed evaluation would better substantiate the who-forgets-whom claim.
  2. In Table 10, adding JDOO reduces MU from 0.5458 to 0.5340 under the same filter. Table 11 likewise reports lower MU across all six operators, alongside stronger forgetting. Although multi-round curves show utility recovery, comparisons at matched forgetting strength and an anchoring-coefficient sensitivity analysis would better support the claimed utility benefit during active forgetting.
Questions for authors

same to weakness

3 comments
AuthorsOpenReview ↗

Response to Reviewer Z6jL: Additional Evidence and Clarifications

Thank you for the helpful review. We address both concerns with relation-resolved evaluations and matched-forgetting analyses. All revisions are marked in blue.


W1①. Source–target graph. We added a controlled 10-client topology (Llama-3.2-1B, FedAvg, seed 42, App. F.13, pp.45–49). Figure 10 (p.47) shows S1→T1, S2→T2, {S1,S2}→T12, and {S1,S2,S3}→T123. The three sources hold disjoint forget sets, and A1–A3 remain authorized. This directly tests whether forgetting follows the specified source–recipient relations.


W1②. Individual relations. Table 20 (p.49) reports each requested relation against Finetune for the same recipient and source forget set. For SimNPO, ΔFTR/ΔFOR is +0.031/−0.018 (S1→T1), +0.029/−0.035 (S2→T2), +0.019/−0.013 (S1→T12), +0.018/−0.012 (S2→T12), +0.010/−0.018 (S1→T123), +0.011/−0.018 (S2→T123), and +0.012/−0.009 (S3→T123). Across SimNPO, NPO, and GradDiff, all 21 requested comparisons improve FTR (+0.008 to +0.031) and reduce FOR (−0.050 to −0.008), whereas authorized recipients retain their Finetune behavior. Thus, the evidence supports relation-specific forgetting without spillover to authorized recipients, rather than a uniform forgetting strength across relations.


W1③. Composition vs. union unlearning. Equation (10) is a bounded parameter-space composition, not an exact substitute for unlearning the union of forget sets. Table 21 (p.49) compares it with a union-data SimNPO oracle. For T123, mean ΔFTR/ΔFOR/ΔMU across the three sources is +0.011/−0.015/+0.001 for normalized composition, versus +0.025/−0.050/−0.015 for the oracle. The oracle forgets more strongly but incurs greater utility loss. We explicitly clarify this distinction in Sec. 3.3 (pp.5–6): PULSAR prioritizes controllable recipient-specific composition, without claiming union-unlearning equivalence.


W2①. JDOO's role and utility cost. We revised Secs. 1, 3.4, and 4.4 to remove the claim that JDOO improves MU or the forgetting–utility frontier. We now position JDOO as a forgetting-preservation mechanism during target-side local adaptation. Its proximal term anchors updates to the filtered initialization, limiting fine-tuning from reversing the intended forgetting. For NPO/SimNPO, local drift decreases from 2.84/2.96 (SOO) to 0.78/0.83 (JDOO), relative to filter norms of 3.39/3.45 (Sec. 4.4, pp.8–9). Under the same filter, JDOO raises mean FTR by +0.014 and lowers FOR by 0.006 across six operators, at an MU cost of 0.009 (Table 15, p.43). Thus, its demonstrated contribution is stronger forgetting retention during adaptation, not higher utility.


W2②. Matched forgetting strength. To test whether the stronger forgetting merely reflects a shifted trade-off, we compared F+SOO and F+JDOO on Llama-3.2-1B (FedAvg, seed 42) with GradDiff, NPO, and SimNPO, sweeping κ∈{1,1.5,2,3}. FTR is matched by interpolation within the observed F+SOO range (App. F.12, Table 19, pp.45–46). Across nine matches, JDOO−F+SOO ΔMU ranges from −0.0099 to +0.0059, with no consistent JDOO advantage. Accordingly, we no longer claim an improved utility–forgetting frontier. Instead, JDOO and κ serve distinct intervention points: JDOO anchors target-side training, while κ sets filtering strength at dispatch.


W2③. Anchoring sensitivity. Figure 5(b) (p.9) provides the requested 4×4 μ–κ sweep: μ∈{0,0.1×,1×,10×} of its default and κ∈{1,1.5,2,3}. At κ=2, increasing μ from zero to its default raises FTR 0.657→0.672 and lowers FOR 0.451→0.434, while MU decreases 0.537→0.515. This confirms the distinct controls: μ anchors target-side adaptation, while κ sets filter strength at dispatch.

AuthorsOpenReview ↗

Follow-up on Author Response

Dear Reviewer Z6jL,

We hope our rebuttal and revisions have addressed your concerns. As the discussion deadline approaches, we would be happy to clarify any remaining questions.

If your concerns have been addressed, we hope that you may very kindly reconsider your rating for our paper, and perhaps champion our paper.

Best regards, Authors of Paper 7666

AuthorsOpenReview ↗

Summary of Revisions in Response to Reviewer Z6jL

Dear Reviewer Z6jL,

Thank you again for your careful review of our manuscript. We briefly summarize the revisions we made in response to your comments.


1. Source–target-specific evaluation. We added a controlled 10-client source–target graph (Fig. 10) and evaluated the seven requested relations individually (Table 20). Across three unlearning operators U\mathcal{U}, all 21 comparisons show improved forgetting on designated targets CT\mathcal{C}^\mathcal{T} over their corresponding FinetuneFinetune baselines, while authorized recipients CA\mathcal{C}^\mathcal{A} retain FinetuneFinetune-level performance. We also added a comparison with a union-data oracle (Table 21) to clarify the trade-offs of normalized filter composition.


2. JDOO. We added the requested μ–κ sensitivity analysis (Fig. 5(b)) and matched-FTR comparisons (Table 19). These results show that JDOO does not consistently improve utility at matched forgetting strength. We have therefore adopted a more cautious characterization of JDOO, focusing on its role in constraining local adaptation and limiting deviation from the filtered initialization. This refinement does not affect PULSAR's central contribution to recipient-specific forgetting.


We hope these additions and clarifications help resolve your concerns. We would greatly appreciate it if you could review the revised manuscript and our responses, and reconsider your score for our work.

Thank you again for your input.

Best regards,

The Authors of Paper 7666

Reviewer_xMzWOpenReview ↗
Rating
3(+1)
Confidence
2
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

This paper proposes PULSAR, which targets the "who-forgets-whom" problem in federated unlearning (FU). Existing FU methods, once any client issues a deletion request, remove the influence of that data from the global model and broadcast the unlearned model to all participants, so clients that still hold authorization also suffer utility loss.

In PULSAR, the source client runs an arbitrary unlearning operator locally and uploads the difference between its LoRA parameters before and after unlearning as a "knowledge filter". At dispatch time, the server subtracts this filter only for target clients whose authorization has been revoked, while source and authorized clients receive the unfiltered global model. Target clients additionally train with a proximal term (JDOO) to limit utility degradation. Aggregation uses only the increments relative to each client's dispatched initialization, so the filter never enters the global model and revocation simply stops applying it.

Experiments on TOFU cover six backbones, ten aggregation operators, and eight unlearning operators, with additional results on MUSE-News and non-IID splits. The paper reports that target-side forgetting metrics (FTR) approach the Retrain reference while utility on authorized clients is largely maintained.

Critical strengths
  1. The problem definition is valuable. Existing FU implicitly assumes forgetting-for-all; the paper points out that authorization is recipient-specific and may contract over time, and this blind spot genuinely exists. This is the main reason I do not recommend a clear rejection.

  2. The mechanism is simple and operator-agnostic. The filter is a parameter-level difference, so any unlearning algorithm can be plugged in; initialization-relative aggregation keeps the filter out of the global model, making revocation zero-cost; the filter is LoRA-sized and storage grows linearly with the number of source clients rather than source–target pairs. The experimental coverage is also broad (6 backbones, 10 aggregation operators, 8 unlearning operators, 3 splits, MUSE-News and non-IID stress tests, plus random-filter and JDOO ablations in the appendix).

Critical weaknesses
  1. No competing baseline, so the central claim is unverified. All eight rows of Table 1 are PULSAR instantiated with different unlearning operators; Finetune and Retrain are references, not competitors. The paper never runs a forgetting-for-all FU baseline (apply the same operator to the global model and broadcast it), so the authorized-side utility loss that motivates the whole paper is never quantified. It also does not compare against the trivial alternative of the server keeping two checkpoints and dispatching by role, even though Appendix E admits that in the single-source, time-aligned case PULSAR is equivalent to it. Without these comparisons the benefit over obvious alternatives cannot be assessed, which is the primary reason for my Soundness score.

  2. The main results come from a weak-signal, single-round regime, and the ablations undercut the proposed mechanism. In Table 1, Finetune's ROUGE-L on the forget set is 0.49 and Retrain's is 0.44, suggesting the model barely memorized the forget set after warm-up, so the subsequent differences of a few hundredths are hard to interpret. The main table reports three clients with full participation in a single round, while the multi-round curves (Figure 3, Appendix F.8) show target-side forgetting decaying from round 2 onward, because the source client keeps contributing updates that include the forget set and the filter is not refreshed. In Table 10, replacing the source-derived filter with an equal-norm random perturbation changes FTR by only 0.01, F+SOO is worse than No-F+JDOO, and JDOO (a utility-anchoring proximal term) increases FTR under every operator without explanation. In addition, no results are reported across random seeds (the ± in the tables is spread across aggregation operators), μ is fixed at 2 with no sensitivity analysis, and no code is provided.

Questions for authors
  1. In Table 10, PULSAR reports FTR = 0.624 on Llama-1B with FedAWA averaged over six operators, whereas the same six operators under FedAWA in Table 5 average about 0.66. What differs between the two settings? If they are identical, which number is correct?

  2. Section 3.3 states that the filter "dynamically updates and repeatedly applies across rounds", while Phase B(i) states that it is refreshed only when a source client issues or revokes a request. Which is actually implemented? In Figure 3, how many times was the filter refreshed during rounds 1–5?

  3. In Table 1, Finetune's forget-set ROUGE-L is 0.49 and Retrain's is 0.44. Does this mean the model did not memorize the forget set after warm-up? Could the authors report each client's training loss or ROUGE-L on the forget set at the end of warm-up, to show that there is memorized content to unlearn?

  4. Under the Table 1 setting, could the authors add a forgetting-for-all baseline (apply the same unlearning operator to the global model and broadcast it to all clients) and report the MU loss on authorized clients? This is the direct quantification of the paper's motivation and is currently missing.

These are the main issues I see. If the authors can address them, in particular 1 and 4, I would consider raising my score.

2 comments
AuthorsOpenReview ↗

Response to Reviewer xMzW: Clarifications and Additional Experimental Evidence

Thank you for the constructive review. We respond point by point below. Revisions are marked in blue.


Q1. Table 10 vs. Table 5. The original FedAWA FTR discrepancy (0.624 vs. ~0.66) came from runs using different configurations. We unified the settings and replaced the inconsistent results. Table 2(a) (p.9) now reports the seed-42 FedAvg/SimNPO run (MU/FTR 0.531/0.658) also used in Table 3 (p.33), which reports the three-seed mean (0.538/0.643). The remaining difference is due to seed-42 versus three-seed reporting, not conflicting runs.


Q4 and W1①. Forgetting-for-all (FFA) baselines. We added a global-broadcast control that applies the same source-derived unlearning filter to every recipient, rather than only to designated targets (Sec. 4.4, Table 2(b), p.9, and Table 16(a), p.43). The key result is preservation of authorized source knowledge. On Llama/SimNPO, authorized-client MU/FTR/FOR is 0.553/0.565/0.509 with PULSAR versus 0.549/0.641/0.449 with FFA. On Qwen (mean over three operators), it is 0.504/0.569/0.465 versus 0.502/0.649/0.434. Thus FFA causes unwanted forgetting at authorized recipients even though its MU difference is small. Target-side FTR remains comparable: PULSAR/FFA = 0.658/0.644 (Llama) and 0.647/0.651 (Qwen). This isolates the benefit of recipient-specific dispatch under the same filtering operation.


W1②. Two-checkpoint alternative. We acknowledge exact equivalence for one source with temporally aligned checkpoints (Sec. 3.3, p.6). During continued training, Table 16(b) (p.43) compares four schemes. Mean target ΔMU/ΔFTR versus same-round Finetune in rounds 2–5 is −0.019/+0.096 for a frozen checkpoint, −0.010/+0.085 for a cached filter, −0.008/+0.086 for a per-round regenerated checkpoint, and −0.006/+0.087 for PULSAR. The regenerated checkpoint needs five extra synchronous round-trips, while PULSAR uses none. The cached-filter result also demonstrates reuse without refresh. The distinction is support for evolving global models and revocation policies, not superiority over a two-checkpoint representation in the aligned single-source case (App. F.10, pp.42–44).


Q2. Filter refresh. An active source constructs its filter in round 1 and refreshes it in each subsequent round when available. If unavailable, its cached filter is reused. Figure 3 constructs/refreshes the filter in rounds 1–5 and deactivates it in round 6. Sections 3.2–3.3 (pp.5–6) and Algorithm 1 (p.24) now consistently specify this policy.


Q3 and W2①. Memorization after warm-up. ROUGE-L alone is not a reliable measure of memorization, as Retrain scores 0.42–0.44 without seeing the forget set. We therefore added per-client forget-set NLL, answer probability, and a longer warm-up control (App. F.6, p.40). After 10 warm-up rounds, source/target/authorized NLL is 0.835/1.118/1.126 on Llama and 0.799/1.045/1.056 on Qwen, versus approximately 1.9–2.0 for Retrain. The target's forget-answer probability is about 2.2× Retrain's. With 20 warm-up rounds, Finetune FOR rises to 0.544/0.537, yet PULSAR still reduces it by approximately 0.06–0.11 across the reported operators (Tables 12–13, p.40). These results confirm substantial cross-client memorization after warm-up and demonstrate that PULSAR remains effective even with stronger memorization.


W2②. Multi-round behavior. Absolute target FTR decreases from 0.651 to 0.624 during active rounds, but same-round Finetune also decreases from 0.567 to 0.527 as the source continues training on the forget set. The target retains a +0.082 to +0.098 FTR advantage relative to same-round Finetune throughout rounds 1–5 (seeds 42/43, Sec. 4.3, p.7, Table 14, p.42). This supports persistence of the relative forgetting effect, not monotonic absolute FTR.


W2③–④. Filter ablations. The source-derived filter yields FTR 0.658 versus 0.583 for an equal-norm random filter. F+SOO also exceeds No-F+JDOO in FTR (0.645 vs. 0.583) and lowers FOR (0.454 vs. 0.524). The filter contributes beyond local personalization (Table 2(a), p.9).


W2⑤. Why JDOO raises FTR. Equation (12) anchors local optimization at the filtered initialization. For NPO/SimNPO, parameter drift falls from 2.84/2.96 without JDOO to 0.78/0.83 with it (filter norms 3.39/3.45). Across six operators, JDOO adds mean +0.014 FTR at −0.009 MU (Table 15, p.43). We now describe JDOO as preserving forgetting during local adaptation, not as improving the matched-forgetting utility frontier (Sec. 4.4, p.9).


W2⑥–⑧. Seeds, sensitivity, and code. Table 1 (p.8) reports mean ± SD across unlearning seeds 42/43/44, after averaging the ten aggregators within each seed. Per-seed FedAvg results appear in App. F.2 (pp.32, 37). Figure 5(b) (p.9) sweeps μ at 0/0.1×/1×/10× and κ at 1/1.5/2/3. At κ=2, moving μ from zero to default changes FTR 0.657→0.672, FOR 0.451→0.434, and MU 0.537→0.515. The default is μ=2×10⁷ (App. A.5). Code will be released upon acceptance (p.11).

AuthorsOpenReview ↗

Follow-up on Author Response

Dear Reviewer xMzW,

We hope our rebuttal and revisions have addressed your concerns. As the discussion deadline approaches, we would be happy to clarify any remaining questions.

If your concerns have been addressed, we hope that you may very kindly reconsider your rating for our paper, and perhaps champion our paper.

Best regards, Authors of Paper 7666

Comments on the paper

1 comment

Dear Reviewers,

Thank you to those who have already contributed to the discussion.

As the discussion period draws to a close, I’d especially appreciate hearing from Reviewers Reviewer Z6jL, Reviewer jBVo, and Reviewer xMzW, who have yet to join the discussion. Please post a brief reply to the authors indicating which concerns their response has addressed and which, if any, remain. If all your concerns have been resolved, a short confirmation is sufficient.

Thank you for your time and input.

Best,
Your AC

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.