acceptodds
Under review as a conference paper at ICLR 2027

Neighborhood-Aware Unlearning for Generative Recommendation

Abstract

Generative recommendation predicts items by autoregressively generating semantic identifiers (SIDs), whose tokens are shared across semantically related items. With this paradigm gaining adoption, the question of how to support unlearning in it has become pressing. If training interactions must later be removed, such as injected spam or interactions with items a user no longer wants, this sharing lets an unlearning update unintentionally affect the predictions for the interactions' neighbors. We show that this effect is locally structured and hidden by global metrics: unlearning can leave recommendation utility on the whole test set nearly unchanged while substantially degrading it on neighborhoods of the forgotten interactions' SID. We propose Neighborhood-Aware Unlearning (NAU), which augments interaction-level unlearning with explicit control over the probability mass assigned to the forgotten items' neighborhoods. Across three datasets and four SID tokenizers, NAU restores spam-induced exposure toward the retrained reference while retaining more than 95% of retrained utility, and recovers the neighborhood utility that forget-and-repair unlearning loses at no global cost. We identify local probability redistribution as an important failure mode of unlearning in SID-based generative recommenders and show that explicitly controlling this redistribution can mitigate its effects.

Primary area:
alignment, fairness, safety, privacy, and societal considerations
Keywords:
Machine Unlearning, Recommender Systems, Generative Recommendation

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_QzxMOpenReview ↗
Rating
3(+1)
Confidence
3(−1)
Informativeness
3
Presentation
3
Soundness
2
Read review
Paper summary

This paper investigates interaction-level machine unlearning in Semantic ID-based generative recommendation. The authors identify a previously underexplored failure mode termed SID-induced unlearning spillover, where removing the influence of forgotten interactions can unintentionally affect neighboring items that share Semantic ID tokens. Importantly, these local effects may remain largely invisible under conventional global recommendation metrics. To address this issue, the authors propose Neighborhood-Aware Unlearning (NAU), which combines item-level forgetting, retain-set optimization, target separation, and neighborhood probability mass control. By adjusting the direction of neighborhood mass regularization, NAU aims to preserve legitimate neighboring items during spam removal or suppress semantically related items during unwanted-item removal. Experiments on three Amazon recommendation datasets evaluate NAU against general-purpose and recommendation-specific unlearning baselines. The evaluation covers multiple SID tokenization settings, two deletion scenarios, and both global and neighborhood-level recommendation behavior. The results demonstrate promising forgetting-utility trade-offs and improvements in local neighborhood utility, with substantially lower computational costs than retraining.

Critical strengths
  • The paper highlights an important limitation in existing recommendation unlearning evaluations: maintaining overall recommendation utility does not necessarily mean that the performance on items related to forgotten interactions is preserved. This concern is especially relevant to Semantic ID-based generative recommendation, where multiple items may share autoregressive token prefixes. The authors provide empirical evidence showing that recommendation quality within these local neighborhoods can change considerably, even when the overall NDCG remains largely stable.

  • NAU offers a straightforward way to control how probability mass is distributed among neighboring items during unlearning. The proposed objective is relatively simple, does not depend on a specific model architecture, and avoids second-order optimization or changes to the underlying Semantic IDs. The experimental evaluation is also fairly extensive. Beyond comparisons with several unlearning baselines, the authors examine two deletion scenarios, different SID tokenizers, the contributions of individual loss components, neighborhood utility, probability redistribution, and computational efficiency.

Critical weaknesses
  • The proposed objectives and evaluation metrics do not convincingly demonstrate interaction-level unlearning. My main concern is that the paper does not clearly distinguish removing the influence of forgotten training interactions from simply suppressing certain items or adjusting their recommendation exposure.

    • First, the forgetting objective applies gradient ascent to the negative log-likelihood of forgotten interactions, thereby reducing the probability of the corresponding target items under their original histories. However, lower prediction probabilities alone do not necessarily indicate that the influence of these interactions has been removed from the model.
    • Second, the spam-removal evaluation focuses on the difference in target exposure rates between the unlearned and retrained models. While this is a reasonable measure of whether the promotion effect introduced by poisoning has been reduced, similar aggregate exposure rates do not guarantee that the models produce comparable predictions for individual users.
    • Third, the unwanted-item removal setting appears to adopt a different definition of forgetting, which is closer to behavioral suppression. Specifically, the goal is to prevent requesting users from receiving recommendations of unwanted items, even when retraining without the deleted interactions would not necessarily lower their exposure. The paper should more clearly distinguish this objective from interaction-level unlearning.
  • The experiments do not adequately establish whether the observed neighborhood effects arise specifically from shared Semantic IDs or from more general neighborhood regularization. This raises questions about whether NAU directly addresses the structural dependencies introduced by shared SID tokens.

    • First, the neighborhoods used for training and evaluation are defined differently. Section 4 selects the top-8 most similar items based on cosine similarity in the pre-quantization embedding space, whereas the neighborhood evaluation considers items sharing the first SID token with the forgotten target. This inconsistency makes it difficult to assess whether the proposed objective effectively protects the neighborhoods most affected by SID sharing.
    • Second, the neighborhood mass objective does not explicitly preserve the probability distribution of the retrained model. Eq. (3) only regulates the total probability mass assigned to the selected neighborhood. With a positive objective weight, minimizing this term encourages the model to allocate more probability mass to neighboring items, but does not control how that mass is distributed among them.
    • Third, the current comparisons do not clearly isolate the benefits of the proposed neighborhood construction. Although Appendix H includes useful analyses of shared SID prefixes and restricted parameter updates, the paper lacks direct comparisons with simpler neighborhood regularization methods under comparable neighborhood sizes and training budgets. Such experiments would help determine whether the gains come from addressing SID-specific dependencies or from the broader effect of encouraging neighborhood probability mass.
Questions for authors
  • Q1. How do the authors distinguish the actual removal of forgotten training information from simply suppressing items associated with forgotten interactions? Could the authors provide additional comparisons between the unlearned and retrained models at the individual prediction or probability-distribution level, rather than focusing mainly on aggregate target exposure rates?

  • Q2. In the unwanted-item removal setting, are the deleted interactions also excluded from the historical contexts of subsequent retained training samples? If they remain in these histories, how can the retrained reference be considered free from the influence of the deleted interactions?

  • Q3. Why are the training neighborhoods defined using the top-8 most similar items in embedding space, while spillover is evaluated based on first-token SID sharing? Could the authors examine how much these two neighborhood definitions overlap and compare NAU with alternatives based on shared SID prefixes, random item selection, and embedding similarity, while keeping neighborhood sizes consistent?

6 comments
AuthorsOpenReview ↗

Author response: Part 1/3

We thank the reviewer for the precise framing. We agree that a lower probability alone does not show removal.

Summary

  • Removal vs. suppression (W1, Q1): among the methods that keep the model's utility, only NAU has a membership-AUC difference from retraining whose confidence interval includes zero on all three datasets. NAU repairs the model's behavior beyond the target, which output-side suppression cannot; and per query, NAU agrees with retraining on whether the promoted item is shown.
  • Sensitive item deletion as behavioral suppression (W1): we agree and define it in Section 2 as an interaction deletion plus a behavioral requirement; a new brand-takedown scenario covers the case where both coincide.
  • Deleted interactions in later histories (Q2): excluded both for retraining and for unlearning; we correct Appendix C, which misstated this.
  • SID specificity and Eq. 3 (W2, Q3): see Reviewer M2BR (W1); Eq. 3 controls only the total mass, the gain stays on the nearest items, k=8k=8 works best, and a simpler regularizer (rehearsal) costs global utility. All changes to the paper are listed at the end (comment 3/3).

Table, figure, and appendix numbers refer to the originally submitted manuscript unless explicitly stated otherwise.

W1 / Q1 (i). Are the deleted interactions still detectable?

Response. Under our loss-based membership inference attack, NAU is the only method retaining usable recommendation utility whose AUC difference from retraining has a 95% confidence interval containing zero on all three datasets. In contrast, Forget+Repair retains a significantly positive difference on Beauty and Toys.

New experiment: membership inference. A loss-based membership inference attack on sensitive item deletion. Members are the requesting users' training interactions with the category (62 / 94 / 56 interactions over five seeds), non-members the 174 / 272 / 156 held-out cases with the category as next item. Members and non-members come from different users and positions, so even a model that never saw the members separates the two groups to some degree. The right reference is therefore not 0.5 but the AUC of the retrained model, which never saw the deleted interactions; we report each model's AUC minus the retrained model's AUC (paired bootstrap 95% intervals):

BeautySportsToys
Before deletion+0.147 [+0.088, +0.207]+0.110 [+0.077, +0.143]+0.228 [+0.163, +0.299]
Finetune+0.150 [+0.089, +0.212]+0.120 [+0.085, +0.153]+0.240 [+0.178, +0.310]
Forget only-0.168 [-0.258, -0.078]-0.168 [-0.232, -0.106]-0.127 [-0.211, -0.034]
Forget+Repair+0.067 [+0.003, +0.132]+0.015 [-0.031, +0.057]+0.098 [+0.029, +0.174]
NAU-0.012 [-0.081, +0.055]-0.028 [-0.073, +0.015]-0.002 [-0.091, +0.092]

Finetune, SCIF, and TRACER barely change the model for the requesting users (they reach about as many of them as before deletion, see Reviewer FL1d, W1b) and keep the deleted interactions as detectable as before; Forget only overshoots, which makes the deletion itself detectable; Forget+Repair remains detectable on Beauty and Toys. Among the methods that keep the model's utility, NAU is the only one whose interval includes 0 on all three datasets. A failed attack does not prove removal, so we report it alongside the behavioral metrics.

W1 / Q1 (ii). Is the attack's influence removed beyond the target?

Response. Yes, it is removed to an extent that output-side suppression cannot reach. Blocking the promoted item at decoding time removes its exposure as completely as NAU but leaves utility at the poisoned model's level; NAU recovers 1.4 points of τP\tau_P more (33 of 45 runs). Part of this comes from the update on the retained data itself (Finetune, which does not forget, also gains utility), so we read it as evidence that unlearning changes the model's behavior beyond the target, which blocking cannot, rather than as proof of removal.

AuthorsOpenReview ↗

Author response: Part 2/3

W1 / Q1 (iii). Per prediction

Response. In short: In the evaluated Beauty mid-popularity case, NAU matches retraining on whether the promoted item is shown for every query. Per-query utility differences, however, are dominated by training randomness and do not clearly distinguish the methods.

New experiment: per-query agreement (Beauty, mid-popularity target): the poisoned model shows the promoted item where retraining does not on 0.8% of queries, NAU on none. For per-query NDCG (Beauty seed 2, three retrained models), a second retrained model differs from the reference by 2.95 to 3.17 points per query, and the poisoned model (3.15), Forget+Repair (3.10), and NAU (3.11) all lie at that level. At the probability level, Table 19 compares, per held-out history, the log-probability of the target and its SID neighbors with the retrained model; at our selected setting the target ends about 4 nats below its retrained value (some over-suppression, see Reviewer M2BR, W3).

W1, third point. Sensitive item deletion as behavioral suppression

Response. We agree and make this explicit in Section 2, which also renames the scenario "sensitive item deletion" (unwanted-item removal in the submission). It combines an interaction deletion, whose reference is retraining (Schelter et al., 2023), with a behavioral requirement from the request, that the user no longer be recommended the category. We state suppression for the requesting users as the goal and report the retrained exposure as a reference. With the refined evaluation (Q2), retraining lowers this exposure on Beauty and Sports but not on Toys, so the second part does not follow from the first.

New experiment: a scenario where deletion and suppression coincide. A brand takedown, in which all interactions with one Beauty brand (95 items, 933 users) are deleted for every user, so the retrained model never recommends it and any remaining exposure is distance to retraining. NAU, with its spam weights, comes closest to retraining at full utility (mean over 5 seeds; baselines with their sensitive item deletion settings):

Brand takedown (Beauty)Brand exposure, its buyers (UHF@10)Brand exposure, other users (UHR@10)τP\tau_P
Retrained00100
Before deletion34.6%2.82%101.7%
SCIF34.5%2.81%101.7%
Finetune23.3%1.15%100.6%
Kookmin3.96%0.04%77.9%
SEIF / Forget only / Fanchuan0026.9 / 0.0 / 14.8%
Forget+Repair1.12%0.04%96.4%
NAU0.14%0.01%96.4%

NAU leaves eight times less of the brand's exposure than Forget+Repair at identical utility (lower on 4 of 5 seeds, equal on the fifth), while the methods that remove the brand completely destroy the model.

Q2. Deleted interactions in later histories

Response. Yes, deleted interactions are excluded from the retained training data in both scenarios. For spam removal, all interactions of the fake users are forgotten. For sensitive item deletion, retraining uses the requesting users' sequences after removing the deleted interactions. On the unlearning side, all sensitive training examples of requesting users are in the forget set.

Correction to the evaluation. During our experimental audit, we identified that some sensitive-item evaluation runs still included deleted interactions in the requesting users' test histories. We corrected this consistently for all methods, including retraining, and updated the affected tables. Training, unlearning, and model selection are unchanged. The spam-removal experiments are unaffected.

AuthorsOpenReview ↗

Author response: Part 3/3

W2 / Q3. Are the effects SID-specific?

Response. Yes, see our response to Reviewer M2BR (W1): the SASRec contrast, the random-SID control on all three datasets, the 70 to 75% overlap of the two neighborhood definitions, and SID-prefix vs. random vs. embedding neighbors at equal size and budget (random: no gain; SID-prefix +10.2; embedding +23.5).

Eq. 3 controls only the total mass. You are right. Eq. 3 sets how much probability the neighborhood receives in total, not how this probability is spread over its items. Our new analysis shows what this means in practice: the added probability goes mostly to the eight items the term is trained on, which at k=8k=8 rise above their retrained level, while no statistically resolved additional effect is observed for the remaining neighbors (Reviewer M2BR, W1). The term thus counteracts the damage that the forget and separation terms cause through the prefix tokens the neighbors share with the target, for the nearest items, and does not reach bigger neighborhoods. A term that matches the distribution within the neighborhood is a natural extension, which we state as future work.

New experiment: neighborhood size. Larger neighborhoods do not help: on Beauty validation, the neighborhood gain peaks at k=8k=8 and falls for larger kk as the region becomes less local, which is why we use k=8k=8 (Beauty, 3 targets x 5 seeds; whole-set values all within the retraining noise floor):

kkWhole set ΔSHretr\Delta^{retr}_{SH} / τP\tau_P (test)Neighborhood gain over Forget+Repair (validation, points)
20.075 / 96.1+10.4 [+3.2, +17.9]
40.071 / 96.0+9.9 [+3.1, +17.0]
80.081 / 96.2+17.1 [+10.4, +24.5]
160.076 / 96.4+15.5 [+9.0, +22.0]
320.076 / 96.3+12.3 [+6.8, +18.3]
640.092 / 96.2+8.6 [+4.6, +12.9]

New experiment: a simpler neighborhood regularizer at equal size and budget (Beauty, 15 runs). In short: up-weighting the same neighbors through the retain data, the simplest alternative, cannot match NAU, because it costs global utility. We tested rehearsal, i.e., Forget+Repair whose retain batches are enriched with retained examples whose next item is one of the same 8 neighbors:

Whole set τP\tau_PΔSHretr\Delta^{retr}_{SH}
Rehearsal, 25% of each retain batch88.4%0.100%
Rehearsal, 50%82.1%0.158%
NAU96.2%0.081%

Rehearsal costs 8 to 14 points of global utility and ends only about as close to retraining as Forget+Repair (0.108%), while NAU's mass term acts without a global cost. Together with the random-neighbor variant (Reviewer M2BR, W1), this separates the contribution of the neighborhood construction from that of the regularizer.

Changes to the paper

  • Section 2: sensitive item deletion defined as an interaction deletion plus a behavioral requirement; scenario renamed.
  • Section 4 and Appendix C: clarified the forget/retain construction; corrected sensitive-item evaluation histories and updated affected results.
  • Section 7 and Appendix: Eq. 3 controls only the total mass, with the k sweep; distribution matching as future work.
  • Appendix: membership inference, per-query agreement, brand takedown, SASRec and random-SID controls, neighbor source, rehearsal.
Reviewer_M2BROpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
1
Soundness
2
Read review
Paper summary

This paper studies unlearning in generative recommendation and argues that shared semantic IDs can cause updates to hurt neighboring items, even when global recommendation quality remains stable. It proposes NAU, combining forget and retain optimization with target separation and neighborhood probability control. The method is evaluated on three Amazon datasets under spam removal and unwanted item removal scenarios.

Critical strengths
  1. The motivation is interesting and relevant. Local damage hidden by global metrics is a meaningful problem to study.

  2. Table 2 and Figure 2 provide useful evidence that neighborhood utility can change substantially while global performance remains almost unchanged.

  3. The method is straightforward to implement and keeps the model architecture and semantic IDs unchanged.

Critical weaknesses

1 I am not fully convinced that the gains specifically address SID sharing. Training uses embedding neighbors, while evaluation uses SID-prefix neighbors, and the connection between them is not clearly validated.

2 NAU considers neighborhood utility during model selection, while the baselines do not. This makes the neighborhood comparison less convincing. The same selection rule should be applied to all methods.

3 In Table 2, the retrained target exposure is already zero. Simply suppressing the target can therefore look successful. This does not show whether NAU removes spam influence while preserving legitimate recommendations.

4 The unwanted item removal results are inconsistent. On Toys, the neighborhood term increases unwanted exposure from 5.0% to 9.7%. On Sports, the resulting utility is only 92.4% of retraining, below the stated 95% threshold.

5 The paper is hard to follow without a method overview figure. The main neighborhood analysis for spam removal also focuses on one Beauty setting, leaving the broader applicability of the findings unclear. The appendix contains many similar tables with limited analysis, making it hard to identify the key findings. I would prefer a more focused set of experiments that clearly supports the motivation.

Questions for authors
  1. How do the results change when using SID-prefix neighbors or random neighbors instead of embedding neighbors?

  2. Does NAU retain its neighborhood advantage when Forget + Repair uses the same model selection criterion?

  3. When the retrained model still recommends the target item, can NAU recover that exposure while preserving legitimate recommendations?

  4. Why does the neighborhood term worsen unwanted exposure on Toys, and how can its effects on non-requesting users be limited?

4 comments
AuthorsOpenReview ↗

Author response: Part 1/4

We thank the reviewer for the concrete questions, which we answer with new experiments.

Summary

  • SID specificity (W1, Q1): two new controls (SASRec, random SIDs) show that the neighborhood damage follows shared SID tokens and that NAU's benefit is specific to them; the training neighbors must be related to the deleted item (random neighbors give no gain).
  • Same selection rule (W2, Q2): under NAU's own rule, Forget+Repair is selected exactly as in Table 1, so NAU's neighborhood advantage (+23.5 points) is unchanged.
  • Nonzero retrained exposure (W3, Q3): only partly: for popular targets NAU over-suppresses; we explain why and list possible countermeasures. NAU removes the attack's influence beyond the target, which decoding-time blocking cannot.
  • Sensitive item deletion (W4, Q4): with the refined evaluation the neighborhood term lowers the mean exposure on all three datasets, and a preliminary retain-side term limits the effect on other users.
  • Presentation (W5): method overview figure, reorganized appendix. All changes to the paper are listed at the end (comment 4/4).

Table, figure, and appendix numbers refer to the originally submitted manuscript unless explicitly stated otherwise.

W1 / Q1. Do the effects specifically come from SID sharing?

Response. Yes, we support this with evidence from two new controls.

New experiment (a): same objective on a backbone without shared IDs. We applied the identical objective, weights, and poisoned data to SASRec (independent item embeddings) and measured each term's effect on identical neighborhood test cases (45 runs per backbone: 3 datasets x 3 target popularities x 5 seeds, paired bootstrap, points of each backbone's retrained neighborhood NDCG@10):

Effect on the neighborhoodTIGERSASRec
Forget term (Forget+Repair minus Finetune)-16.3 [-21.3, -11.4]-0.5 [-1.5, +0.5]
Separation term (λs=0.1\lambda_s=0.1)-11.5 [-16.1, -7.0]-0.4 [-1.2, +0.4]
NAU minus Forget+Repair+13.1 [+6.8, +19.0]+1.1 [+0.3, +2.0]

Both the forget and separation terms cause substantial neighborhood damage in TIGER, while their effects on SASRec are negligible. NAU's neighborhood term counteracts this damage, highlighting the importance of explicitly controlling neighboring items in SID-based GenRec. In log-probability of the true next item the contrast is sharper still: the forget term costs TIGER's neighborhood 0.32 nats and SASRec's 0.003, the separation term 0.93 vs. 0.002 nats. With independent item embeddings there is almost no damage for the neighborhood term to repair (third row); the damage, and NAU's benefit, are specific to shared SIDs.

New experiment (b): same model with random SIDs. We retrained TIGER with the items randomly reassigned to the existing identifier tree, which keeps every code statistic and bucket size but destroys the semantic grouping (mid-popularity target, 5 seeds per dataset). If the damage follows shared codes, it should move from the semantic neighbors to the items that now share codes with the target. Change in log-probability of the true next item caused by the separation term (nats; paired bootstrap):

Original SIDs: semantic neighborsOriginal SIDs: random items sharing codesRandom SIDs: semantic neighborsRandom SIDs: items sharing codes
Beauty-0.67+0.03+0.03-1.26
Sports-0.74-0.01-0.01-0.87
Toys-0.86-0.03-0.02-0.92

The forget term shows the same pattern (e.g., Toys: semantic neighbors -0.27 nats with original SIDs, -0.01 with random SIDs). The damage therefore follows shared SID tokens, not semantic similarity or the deletion itself. (As intended for a control, random SIDs remove the semantic grouping the recommender relies on, so accuracy drops to 76%, 16%, and 54% of the clean model's NDCG@10; we therefore compare in log-probability.)

AuthorsOpenReview ↗

Author response: Part 2/4

W1 / Q1 (cont.). Training vs. evaluation neighborhoods; SID-prefix and random neighbors

Overlap. On average 70 to 75% of NAU's eight training neighbors share the target's first SID token (Beauty 70%, Sports 75%, Toys 73%), so the training neighborhood lies mostly inside the evaluated SID neighborhood. We train with embedding neighbors because SID prefixes give poorly sized neighborhoods: the first RQ-KMeans token is shared by 83 to 112 items on average, while the median item shares its first two tokens with only one or two items (Table 24).

New experiment: choice of neighbors at equal size and budget (Beauty, 3 targets x 5 seeds, everything else identical):

Neighbors in LnbrL_{nbr} (k=8)Whole set ΔSHretr\Delta^{retr}_{SH} / τP\tau_PNeighborhood gain over Forget+Repair (points of retrained NDCG@10), testvalidation
Embedding (NAU)0.081 / 96.2+23.5 [+13.1, +35.1]+17.1
SID-prefix (8 closest in SID space)0.072 / 96.2+10.2 [+2.3, +18.4]+11.9
Random items0.081 / 96.4+2.0 [-3.4, +7.5]+0.5

On the whole set the choice is within the retraining noise floor, as is Forget+Repair at 0.108 / 96.3. In the neighborhood the choice matters: random neighbors give no gain, while both related choices do, embedding neighbors at least as much as SID-prefix neighbors (test +13.3 [+3.5, +24.7], validation +5.3 [-1.6, +12.1]). Both lie mostly in the target's first-token region, where the shared-token damage occurs; random items do not. The benefit therefore comes from which items receive the mass, not from adding a probability-mass term.

New experiment: where the term acts. We split the SID neighborhood (test cases whose next item shares the target's first SID token) into two parts: cases whose next item is one of NAU's eight training neighbors (5.8 to 12.4% of the cases), and all other cases. The gain is concentrated on the first part: NAU raises these nearest items, on Beauty above their retrained level. On the other cases, NAU and Forget+Repair do not differ resolvably (+4.0, -1.8, -4.0 points on Beauty, Sports, Toys). The improvement is concentrated on the selected nearest neighbors, while NAU's additional effect on the remaining SID neighborhood is not statistically resolved. Per dataset, the first-token gain is resolved on Beauty (+23.5) and not on Sports (+3.2) or Toys (+12.0). We state the claim in this form: NAU recovers the items closest to the deleted one (on Beauty above their retrained level), without a statistically resolved additional effect on the remaining SID neighborhood.

W2 / Q2. Use the same selection rule for all methods

Response. Under NAU's own selection rule, Forget+Repair is selected in the same configuration as in Table 1, and NAU's neighborhood gain over it is unchanged. The rule uses the global (whole-set) criteria first, exposure distance and τP≥95%\tau_P \geq 95\%, and the local (neighborhood) criterion only to break ties.

New experiment. Applied to Forget+Repair over the forget weights of the submitted grid (λf∈{0.1,1.0}\lambda_f \in \{0.1, 1.0\}, 4 epochs as for NAU, Beauty validation), only λf=0.1\lambda_f=0.1 keeps τP≥95%\tau_P \geq 95\% (76.0% at λf=1.0\lambda_f=1.0), so the tie-break never applies:

Beauty, testΔSHretr\Delta^{retr}_{SH} / τP\tau_PNAU's neighborhood gain over it (points of retrained NDCG@10)
Forget+Repair, selected by NAU's rule (λf=0.1\lambda_f=0.1)0.108 / 96.3+23.5 [+13.1, +35.1]
NAU0.081 / 96.2

Forget+Repair (λs=λn=0\lambda_s=\lambda_n=0) is also one of the nine settings in NAU's own validation grid, so the identical rule already evaluated it and did not select it. Conversely, selecting NAU on the global criteria alone picks λs=0.1\lambda_s=0.1, which Table 1 reports (indented row): globally equivalent (0.072/0.139/0.072%), and on the same 15 Beauty runs still +12.8 [+2.2, +24.1] neighborhood points over Forget+Repair (validation +8.5 [+1.2, +15.5]). The local criterion thus selects between two NAU settings, both of which outperform Forget+Repair within the submitted search grid.

AuthorsOpenReview ↗

Author response: Part 3/4

W3 / Q3. Nonzero retrained exposure

Response. Only partly. When retraining still recommends the target (popular targets), NAU keeps the model's utility but over-suppresses the target: it ends within retraining noise on Beauty (0.6x the noise floor) and somewhat above it on Sports and Toys (1.7x, 1.3x; Reviewer FL1d, W1a). Table 2 shows the mid-popularity case, where retraining's exposure is zero. Table 1 pools all target popularities, where it is not (0.072/0.139/0.073%), and ΔSHretr\Delta^{retr}_{SH} penalizes over-suppression as much as leftover exposure.

Cause and remedies. We attribute the over-suppression to the forget term: gradient ascent lowers the target's probability under the fake users' histories, and for a popular target these histories resemble those of genuine buyers, so the probability also drops for them. Forget+Repair, which has only this term, shows the same gap, and a cap that stops forgetting once the target reaches an exposure level estimated from items of similar popularity did not bind. As future work we see three ways to address this: an anchor that holds the target's probability under genuine users' histories at its original level (the analogue of the retain-side term that already limits collateral effects in sensitive item deletion, W4), stopping each request once the target's validation exposure reaches an estimate of its retrained level, and target-specific forget and separation weights instead of one strength for all targets.

New experiment: influence beyond the target. NAU removes the attack's influence in a way that output-side fixes cannot:

Alternative (Beauty)Exposure removedτP\tau_P
Block the target at decoding (45 runs)allat the poisoned model's level; NAU +1.4 points (33 of 45 runs, p<10−4p<10^{-4})
Re-rank toward the user's history (15 runs; mild / strongest setting)34% / 80%93.6% / 80.1%
NAU (same evaluation, beam 50)all96.0%

W5. Presentation and scope

We add a method overview figure and reorganize the appendix around its findings. The SID controls above cover all three datasets; small supporting checks on Beauty: with five promoted targets (6 runs where the attack landed), NAU stays at 0.054% from retraining vs. 0.235% for Forget+Repair at equal utility; under a clone-and-append attack, 0.135% vs. 0.297%.

AuthorsOpenReview ↗

Author response: Part 4/4

W4 / Q4. Sensitive item deletion

Response. In short: the neighborhood term does not worsen exposure on average; the increase on Toys is within the noise of this small scenario, and with the refined sensitive item evaluation the term lowers the mean exposure on all three datasets while raising utility. Its side effect on other users can be limited with a new retain-side term.

We agree that the submitted results were mixed and that this scenario is limited by its size: with 11 to 17 requesting users per seed, the increase on Toys (5.0 to 9.7%) corresponds to 3 requesting users in total over five seeds. With the refined evaluation, which removes the deleted interactions from the requesting users' evaluation histories (Reviewer QzxM, Q2), the neighborhood term lowers mean exposure on all three datasets (per seed lower on 1, 3, and 2 of 5) and raises utility on all five seeds on Beauty and Toys. On Sports, every method that updates the model falls below 95% of retrained utility, Finetune included (Reviewer FL1d, W1b).

Without neighborhood term to NAU (λn=−1\lambda_n=-1); retrained in parenthesesBeautySportsToys
UHF@107.9 to 4.8 (8.5)15.4 to 5.6 (15.3)11.2 to 6.4 (16.2)
τP\tau_P93.2 to 95.891.6 to 92.392.0 to 95.2

Effect on non-requesting users. Unlearning also moves other users' exposure away from retraining (ΔUHRretr\Delta^{retr}_{UHR} 0.65 / 0.85 / 1.14 vs. 0.48 / 0.33 / 0.64 before deletion); Forget+Repair moves it as far on Beauty and Toys (0.74 / 0.22 / 1.13), and on Sports the neighborhood term, which acts on the category rather than on specific users, adds to it. We state this as a limitation. This issue is specific to sensitive item deletion: a spam target is removed for everyone, whereas here the request comes from a few users while the category stays legitimate for all others, so the suppression should not reach them.

New experiment (preliminary): retain-side term. With N\mathcal{N} the neighborhood of Eq. 3, Mθ(x)=∑j∈Npθ(j∣x)M_\theta(x) = \sum_{j \in \mathcal{N}} p_\theta(j \mid x) its probability mass given history xx, θ0\theta_0 the model before deletion, and R\mathcal{R} the retain histories (other users), the objective gains λa 1∣R∣∑x∈R(log⁡Mθ(x)−log⁡Mθ0(x))2,\lambda_a \, \frac{1}{|\mathcal{R}|} \sum_{x \in \mathcal{R}} \big(\log M_\theta(x) - \log M_{\theta_0}(x)\big)^2, so the neighborhood term can still lower the category for the requesting users, while any change of its mass for other users is penalized. Results (two weights tried, not selected on validation; cleaned evaluation, five seeds):

Beauty / Sports / ToysΔUHRretr\Delta^{retr}_{UHR} (other users)τP\tau_PUHF@10 (requesting users)
Before deletion0.48 / 0.33 / 0.64101.0 / 100.5 / 103.414.6 / 19.8 / 11.2
Retrained01008.5 / 15.3 / 16.2
NAU0.65 / 0.85 / 1.1495.8 / 92.3 / 95.24.8 / 5.6 / 6.4
NAU + anchor, λa=0.1\lambda_a=0.10.66 / 0.74 / 0.8196.6 / 93.4 / 96.76.3 / 6.7 / 6.4
NAU + anchor, λa=1\lambda_a=10.56 / 0.39 / 0.8397.9 / 95.0 / 99.86.3 / 8.3 / 6.4

At λa=1\lambda_a=1 the anchor brings other users closer to retraining on all three datasets (on Sports almost to the level before deletion), raises utility to at least 95% across all three datasets, and keeps requesting users' exposure at or below retraining. These results are promising but preliminary; we leave a user-specific form of the neighborhood term, with its weight selected on validation, to future work.

Changes to the paper

  • Section 4: we clarified the global-first selection rule, with neighborhood utility used only as a tie-break. We verified that applying the same rule to Forget+Repair leaves its selected configuration unchanged (Appendix B).
  • Section 5: addition of SASRec and random-SID controls; Appendix: where the term acts (the abstract states the narrowed claim).
  • Sensitive item deletion: refined evaluation, ">95%" claim removed, effect on other users as a limitation, retain-side term in the appendix as a preliminary result.
  • Section 7: popular-target over-suppression with its cause and future work.
  • Method overview figure; appendix reorganized (neighbor source, k sweep, rehearsal, decoding-time alternatives, other attacks).
Reviewer_FL1dOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
2(+1)
Soundness
2(+1)
Read review
Paper summary

NAU studies how to forget interactions without unnecessarily affecting nearby items in a semantic-ID recommender. It adds target separation and neighborhood probability-mass control to forget-and-retain training, without changing the SIDs. The experiments cover spam removal and unwanted-item removal, measuring target exposure, global and local utility, and runtime.

Critical strengths

Looking at the forgotten target's neighborhood, not just global NDCG, is useful. The paired comparisons show local damage that whole-test-set averages can hide.

Critical weaknesses
  1. NAU doesn't reliably deliver the forgetting-utility balance claimed in the abstract. In several popular-target blocks (Appendix D), it removes exposure that retraining retains and ends up farther from retraining than baselines. That's over-suppression, not better spam unlearning. For unwanted-item removal, Sports misses the 95% utility threshold, while Finetune achieves lower unwanted exposure on Toys at the same reported utility. The paper acknowledges these cases, but they limit the method on its intended objectives.
  2. I can't work out a consistent selection protocol from the paper. Appendix D says the other tokenizers use test-split selection; Tables 11-17 say baseline settings came from RQ-KMeans validation. The selected Sports NAU test result in Table 8 also doesn't match Table 22. These aren't cosmetic details: until they're reconciled, the reported comparisons aren't convincing.
Questions for authors
  1. Were baseline configurations for the other tokenizers selected on test, or transferred from RQ-KMeans validation selection?
  2. Why is the selected Sports NAU test UHF@10 12.68 in Table 8 but 11.144 in Table 22? Are these from different runs?
  3. There's too much blank space below Figure 1, which hurts the page's appearence. Could you tighten this spacing?
  4. Please use "AI Use Statement" rather than "AI Disclosure Statement" as the heading on page 10.
3 comments
AuthorsOpenReview ↗

Author response: Part 1/3

We thank the reviewer for the careful reading, in particular of the appendix.

Summary

  • Popular targets (W1a): the over-suppression is real but small: it is within retraining noise on Beauty and shared by every method that actually forgets. The methods closer to retraining barely forget there. We state it as a limitation and adjust the abstract.
  • Sensitive item deletion (W1b): under a refined evaluation, NAU recommends sensitive items to the fewest requesting users after unlearning compared with all methods that keep the model usable, and the Finetune Toys advantage disappears. On Sports, no method that updates the model in our main comparison keeps 95% utility, so we drop that claim for this scenario.
  • Selection protocol (W2, Q1): no configuration was selected on test; one outdated sentence in Appendix D said otherwise, which we apologize for and have corrected.
  • Q2 to Q4 are answered below; all changes to the paper are listed at the end (comment 3/3).

Table, figure, and appendix numbers refer to the originally submitted manuscript unless explicitly stated otherwise.

W1a. Popular targets

Response. We agree that for popular targets NAU removes some exposure that the retrained model keeps, and that ΔSHretr\Delta^{retr}_{SH} correctly counts this as error. However, we argue that this limitation does not undermine NAU's overall forgetting-utility trade-off, for three reasons.

  • (i) It comes from forgetting itself, not from NAU's added terms. Forget+Repair, which has neither the separation nor the neighborhood term, ends at practically the same distance with the other tokenizers (Beauty RQ-VAE 0.434 vs. NAU 0.436; TRACER 1.226 vs. 1.269; Sports RQ-VAE 0.035 for both; Toys RQ-VAE is the exception, 0.199 vs. 0.129). The gap is caused mainly by the gradient ascent on the target's interactions, meaning this is not mainly caused by NAU's added terms.

  • (ii) A smaller exposure gap on popular targets does not imply better forgetting overall. Where the attack dominates (unpopular and mid-popularity targets), Finetune, SCIF, and TRACER stay 0.36 to 2.48 percentage points from retraining on Beauty (RQ-VAE), against 0.004 to 0.020 for NAU. A closer distance on popular targets therefore does not mean a better forgetting-utility trade-off overall.

  • (iii) The gap is comparable to the variability across retrained models. (see the new experiment below).

New experiment: retraining noise floor. We retrained five models per dataset that differ only in the training seed. Their pairwise differences provide an empirical reference for the variability induced by retraining.

ΔSHretr\Delta^{retr}_{SH} (multiple of floor)BeautySportsToys
Retrained vs. retrained (floor)0.1180.0870.050
NAU0.081 (0.7x)0.138 (1.6x)0.074 (1.5x)
Forget+Repair0.108 (0.9x)0.137 (1.6x)0.195 (3.9x)
Finetune0.901 (7.6x)0.281 (3.2x)0.823 (16.5x)
SCIF1.309 (11.1x)0.467 (5.4x)1.232 (24.6x)
Popular targets only: floor0.350.250.11
Popular targets: NAU0.21 (0.6x)0.41 (1.7x)0.15 (1.3x)
Popular targets: Forget+Repair0.15 (0.4x)0.41 (1.6x)0.25 (2.3x)

We draw two conclusions from the table:

1. Across all targets, NAU is within or close to retraining noise, and it keeps the most utility. NAU is at 0.7x, 1.6x, and 1.5x the noise floor, while Finetune and SCIF stay 3 to 25 times above it. Because NAU removes almost all poisoned exposure (SH@10 0.010% on Beauty) and the retrained model's own exposure is small (0.07 to 0.14%), the methods that forget end at similar distances, and what separates them is what they keep: Forget only and SEIF have lower utility than NAU on every run, NAU keeps as much as Forget+Repair (no significant difference, 45 runs), and NAU in addition recovers the deleted item's nearest neighbors (Reviewer M2BR, W1). Blocking the target at decoding reaches a similar distance but leaves the attack's influence in the model (Reviewer M2BR, W3).

2. On popular targets, the over-suppression is systematic but small. NAU is within the noise floor on Beauty (0.6x) and somewhat above it on Sports and Toys (1.7x, 1.3x), where Forget+Repair is about as far or farther. The methods that are closer on Sports leave part of the attack in place (Finetune, SH@10 still 0.64) or lose utility (Kookmin, τP\tau_P 80.9%). No setting of NAU's weights removes the over-suppression (0.15 to 0.22 on Beauty for every setting with a forget term). We discuss its cause and possible remedies in our response to Reviewer M2BR (W3).

AuthorsOpenReview ↗

Author response: Part 2/3

W1b. Sensitive item deletion

Response. In short: NAU recommends sensitive items to the fewest requesting users after unlearning among the methods that keep the model usable, the reviewer's Toys observation does not hold once the evaluation inputs are cleaned, and the 95% utility claim does not hold on Sports for any method that updates the model, so we drop it for this scenario.

Refined evaluation. Only the evaluation protocol changed: training, the unlearning objective, and model selection are unchanged. Following Reviewer QzxM's question on deleted interactions in histories, we now also remove the deleted interactions from the requesting users' input histories at test time, for every model including the retrained one, as a real deletion requires. With 11 to 17 requesting users per seed, one user moves UHF@10 by 6 to 9 points, so we report pooled user counts (five seeds):

Requesting users still shown the categoryBeauty (of 61)Sports (of 72)Toys (of 61)
Before deletion9147
Retrained51110
Finetune9108
Forget+Repair7117
Forget only555
SCIF9147
NAU344

NAU reaches the fewest requesting users of all methods that keep the model usable on all three datasets (SEIF and Fanchuan reach none but retain less than 10% of retrained utility), fewer even than Forget only, which overshoots in the membership test (Reviewer QzxM). Given the small sample sizes, we interpret most differences as trends. The reduction on Sports, from 14/72 exposed users before deletion to 4/72 with NAU, is statistically significant under an exact paired McNemar test: depending on the overlap of exposed users, the two-sided pp-value lies between 0.002 and 0.031 for every pairing consistent with the counts. Finetune's Toys advantage noted by the reviewer no longer holds under the refined evaluation (8 vs. 4 exposed users).

Utility below 95% on Sports. The 95% threshold is missed on Sports by every method that changes the model, so it reflects the dataset rather than NAU: Finetune without any forget term reaches 92.9%, Forget+Repair 93.3%, Forget only 93.8%, and NAU 92.3%; the only methods above 95% (SCIF, TRACER) leave the model unchanged. Updating the model for this small request costs 7 to 8% of NDCG@10 on Sports and Toys, and only on Toys does the model before deletion start with enough headroom (103.4% of retrained) to absorb it.

AuthorsOpenReview ↗

Author response: Part 3/3

W2 / Q1. Selection protocol for the other tokenizers

Response. No reported configuration was selected on test, and we apologize for the confusion caused by the opening sentence of Appendix D. In early experiments we selected on the test split. We then switched to validation selection and re-ran the selection for all reported configurations, but did not update the sentence describing the old protocol. For RQ-VAE, LETTER, and TRACER SIDs, each baseline uses the configuration selected on the RQ-KMeans validation split of the same dataset (Table 7), and NAU uses its single selected setting.

Verification. We regenerated Tables 11 to 17 from the validation picks: Tables 12 to 17 are identical to the submitted ones, and only the TRACER rows of Table 11 and the Beauty Finetune, Forget+Repair, and Forget only cells of Table 1 change, because some seed-2 baseline runs had used a model trained on another data seed, which we found and fixed while auditing; NAU's results are unchanged. No selected configuration changes. NAU's spam weights also replicate when selected separately on the Sports and Toys validation splits (λf=0.1\lambda_f=0.1, λs=0.01\lambda_s=0.01, λn=0.01\lambda_n=0.01 on both). To conclude this point: reported configurations were all selected on validation sets and not test sets. We also audited the affected runs and updated the corresponding tables and comparisons; NAU's configurations and results remain unchanged.

Q2. Sports UHF@10: 12.68 (Table 8) vs. 11.144 (Table 22)

Thank you for catching this inconsistency. Table 22 inadvertently reported a result from an earlier execution rather than the selected run used in Table 8. We have corrected Table 22 and verified that the reported configurations and main conclusions remain unchanged. Separately, following Reviewer QzxM, the sensitive item tables now use the refined evaluation, so the revised Sports table reports 5.57% for this configuration, and the selection table reports validation results only.

Q3 / Q4. Presentation

We remove the blank space below Figure 1 and use the heading "AI Use Statement". Following Reviewer M2BR, we also add a method overview figure and condense the appendix (one summary table for the tokenizers, full tables at the end of the appendix).

Changes to the paper

  • Abstract: qualified for popular targets.
  • Section 4 and Appendix E (Appendix D of the submission): the selection of each configuration is stated in the text, not only in captions.
  • Section 5: retraining noise floor and the popular-target over-suppression; Section 7: stated as a limitation.
  • Sensitive item deletion: refined evaluation in all its tables; ">95%" claim removed for this scenario; pooled user counts with a paired McNemar significance analysis.
  • Tables 1 and 11: corrected baseline cells (seed-2 runs), NAU unchanged.
  • Figure 1 spacing, "AI Use Statement", method overview figure, one summary table for the tokenizers.
  • Table 2 and its Toys counterpart in the Appendix: the distance to retraining is computed per seed, as in all other tables (reference rows change only; NAU's values are unchanged).

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.