acceptodds
Under review as a conference paper at ICLR 2027

The Geometry of Unlearning in Semi-Bandits: Categories, Regret, and Exact Replay

18 views

Abstract

We study exact unlearning in combinatorial semi-bandits, where a learner selects several items each round and observes their individual rewards. After exploration, it chooses a final action and stops collecting feedback. Removing an observation can change that stopping decision. How can learning remain stable, and how can an erased run be reconstructed from feedback already collected? We assign each protected action category a budget for changes in public event probabilities. For product actions with a unique optimum and comparable reward gaps, we prove matching regret bounds up to logarithmic factors. The cost separates finding the best action from keeping decisions stable. Category arrangement controls the second term: categories with equal entropy can leave different sampling opportunities. Our learner attains the bound without knowing the gaps. Its completion envelope uses the largest stopping probability over possible missing rewards. Shared random draws ensure that erased exploration ends no later than factual exploration, so retained feedback suffices for exact replay of this missing-data rule. Synthetic and preference-derived experiments measure regret, feedback collection, and reconstruction together.

Primary area:
learning theory
Keywords:
machine unlearning, combinatorial semi-bandits, online learning, algorithmic stability, regret bounds, exact replay

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_PmywOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
1
Soundness
3
Read review
Paper summary

This paper studies exact unlearning of a single feedback row in product combinatorial semi-bandits, where the learner explores in doubling phases, commits to a slate (one block of items per position), and then stops collecting feedback. It requires category-wise stability: the public actions on the original and erased tables must be within total variation ρ\rho, conditional on a pre-feedback category label of the deleted row's action, with GG protected groups per position. With mm items in total, KK served per round, and gap Δ\Delta, it proves minimax pseudo-regret Θ~(min⁡{KΔT, m/Δ+KG/ρ})\widetilde{\Theta}(\min\{K\Delta T,\ m/\Delta + KG/\rho\}) on a class with a unique optimal slate and comparable gaps. Its Envelope learner attains this rate without knowing the gaps. In the phase containing the missing row, it uses the largest commitment probability over all fillings of the missing reward, so under a shared random draw the erased run stops no later than the factual one and can be replayed from collected feedback alone.

Critical strengths
  1. Exact replay from collected feedback. Under a shared random draw, the completion envelope makes the erased run stop no later than the factual run, so replay never needs uncollected feedback. The envelope is the smallest local rule with this property (Proposition B.1).

  2. Matching characterization of the stability cost. Theorem 1's lower bounds hold against adaptive learners and are attained by a learner that does not know the gaps, so the regret cost of category-wise stability is tight up to logarithmic factors. Corollary 1 adds that the arrangement of categories matters, not only their number. Two labelings with the same number of equally likely categories differ in cost by a factor of LL, the number of candidate blocks per position.

Critical weaknesses
  1. The main text is not self-contained. The central argument and the limits of the contribution are left to the appendix.

    • The body does not define some of the terms in its theorem statements, such as the "causal learners" of Theorem 1 and the "factual tape" of Theorem 2.
    • The key stability bound is asserted without explanation. Section 5 states that the best filling raises the certificate by at most eΛe\Lambda in expectation given the category, but it does not show that the category enters through the conditional inverse-propensity weight, where the constant ee comes from, or that the bound relies on reward-independent sampling. The ingredients (the coupling and the inflation factor 1/Qk,g1/Q_{k,g}) appear in separate paragraphs, and the reader must reconstruct the argument from Lemmas B.5–B.6 in a 44-page appendix.
    • The conclusion part (section 7) consists of three abstract sentences that state neither the results nor any limitation.
    • The restrictions that bound the contribution (a single request, gap regularity, and reward-independent probing) appear only as scattered appendix remarks.
    • The introduction cites no prior work, and the related work omits standard exact-unlearning references (Ginart et al., 2019; Bourtoule et al., 2021; Sekhari et al., 2021; Neel et al., 2021) as well as online learning with deletions (Hu et al., 2025).
  2. The paper has no narrative thread. Sections and paragraphs follow one another without a stated purpose or transition. The introduction has no roadmap and no list of contributions, so the reader is never told what each section establishes. Section titles name the paper's own constructions rather than what is shown, which could be confusing to the reader. For example, section 5, "Exact replay with a completion envelope", is titled after a coined term, yet it presents the main learner and the regret bound that attains Theorem 1 (Theorem 2), not only replay. Material also sits apart from where the titles place it. Section 3, "Setting and guarantees", omits the product action class, the protected groups and the reward model, which first appear in Section 4. Sections end without transitions. Section 4 closes with a paragraph on private learning and appendix pointers. Section 5 then opens on the "Envelope" learner, which has been named only once before, in passing. Within Section 5, the envelope (Eq. 10) is defined through a commitment probability whose ingredients (sampling rule, scores, certificate and gate; Eqs. 11–14) are given only afterwards. Each paragraph opens a new topic without linking to the previous one. The reader therefore cannot tell why each piece is introduced or how it serves the main results, which compounds W1.

Questions for authors
  1. Deletion is executed after training, when the factual run has already been served. What does the system output or store after unlearning? Why is reconstructing the full erased run, including a counterfactual earlier commitment, the right target? Can the construction handle online deletion, that is, a request arriving while the learner is still exploring, after which it must continue as if the row had been erased (as allowed by Nguyen-Tang and Arora, 2026)?
  2. The guarantee covers a single request. If the envelope is taken over the joint fillings of several missing rows, does the erased run still stop no later than the factual one? Does slope ρ/(eΛ)\rho/(e\Lambda) give one-step stability on inputs that already contain missing markers, so that Lemma A.2 yields rρr\rho for rr requests? If not, is the obstacle technical, or is there a counterexample?
  3. Theorem 2's upper bound carries a factor CΔC_\Delta that the lower bound lacks. What is the rate when gaps are not comparable, that is, for large CΔC_\Delta? Relatedly, the proof of Lemma B.6 uses the fact that the deleted row's action still has law qq given the rest of its phase, which holds only because sampling is fixed in advance. Is reward-independent sampling therefore necessary for category-wise stability itself, and not only for replay from collected feedback?
  4. Omit-only replay, which simply drops the missing reward, fails on 14 of 4608 synthetic requests and on none from real data. Under what conditions does it fail? Please characterize when the envelope is needed, or state that its value is a worst-case guarantee.
5 comments
AuthorsOpenReview ↗

New guarantees, stronger evidence and a clearer narrative

Thank you again for your detailed guidance on self-containment, narrative and the replay target. Alongside the point-by-point revisions in our main response, the wider discussion has led to additional results that make the paper's contributions easier to explain and assess.

1. A clearer distinction between the two learning guarantees.

Envelope learns when to stop collecting feedback; Gibbs uses a budget chosen in advance. Both provide category stability and exact replay from retained feedback.

Appendix K.19 now proves a sufficient Gibbs budget for near-optimal expected serving reward under the stated mean and probing assumptions. This budget does not grow with the number of later serving rounds.

The result also addresses your interest in multiple omissions: with the stated confidence, the serving-value bound holds simultaneously for every set of up to rr removed acquired rows. It averages over the final Gibbs slate draw. This is a fixed-budget utility result; Envelope's separate repeated-deletion conditions remain explicit in Appendix H.

2. A concrete answer to what replay must retain and recompute.

Appendix K.20 explains why original Envelope needs to reconstruct only the affected phase. If both executions continue, their score reset makes the later execution identical; otherwise the erased execution has already committed.

We also prove a certificate, based on public action counts, identifying completed phases whose responses can be discarded while preserving every separate single-withdrawal reconstruction. Across nine recorded commerce workloads, 5.43%–34.87% of acquired responses qualify.

This gives a precise retained-state rule: keep the required action/time metadata and saved randomness, while discarding responses certified unnecessary for this replay task. The percentages are certified response counts after collection.

3. Verified improvements to adaptive stopping.

Appendix K.21 adds adaptive refinements, including feedback reuse across phases, with proofs preserving category stability, exact single-erasure replay and the original regret order in the stated single-position setting.

  • Across 39 public-data model settings, three refinements each lower mean regret relative to original Envelope, with every pointwise paired 95% interval favoring the refinement.
  • On 4,096 fresh shared paths in a separate Criteo-conversion replication, geometric reuse reduces original Envelope's regret from 1,964.096 to 50.005—a 97.45% reduction. Collection falls from 100% to 2.084% of the horizon.

The best tested fixed-budget Gibbs achieves 40.394 regret here. Its stronger result illustrates the complementary fixed-budget contribution; complete Gibbs and Sparsemax comparisons remain available.

4. Broader evaluation and a stronger reproducibility record.

The evaluation now covers chronological recommendation logs, full catalogs, commerce, randomized email and advertising benchmarks. We added comparisons under matched category protection, collection-budget grids, file-backed withdrawal tests and optimized-cache baselines, with independent erased-reference checks. We retain the weaker regimes and modest cache gains (Appendix K).

The technical appendices also provide the bounded repeated-deletion extension and counterexample, reconciliation of the earlier lower bound, and diagnoses of the numerical reproducibility discrepancies with a pinned reference environment (Appendices H–J).

5. More substance with a clearer presentation.

Your concern that readers had to reconstruct the story from the appendix guided the revision. The main text now introduces the setup before the theorems, builds the learner in order, and explains the stability reasoning directly. The same running example connects these steps.

We also simplified the abstract and shortened the appendix by 16 pages, preserving the proofs and retaining extended plots and tables in a companion detailed record. The original structure, core guarantees and ten-page main text remain intact.

We hope you will consider these additions alongside our direct answers to your four questions. We respectfully ask you to reconsider your rating to 3, or 4 if the revised paper warrants stronger support. Thank you for helping us make both the argument and its practical implications clearer.

Dear Reviewer,

We have updated our point-by-point response and the follow-up “New guarantees, stronger evidence and a clearer narrative” in this thread.

The updates explain the changes to self-containment and narrative, answer all four questions, and summarize the additional contributions. We would be grateful for your reassessment and any remaining concerns.

Thank you for your time and consideration.

I thank the authors for the detailed responses and for the considerable effort they have put into revising the manuscript. The revision addresses the concerns raised in my review. However, the manuscript has since changed substantially. The current version changes the title, introduces a second learner (Gibbs) with a new serving guarantee, adds an adaptive refinement of Envelope with new proofs, and reports experiments on new datasets. The total length of paper expand from 55 pages at submission to 86 pages now. These are new contributions rather than further validation of the submitted results, and they cannot be adequately reviewed within the discussion period. I am keeping my scores unchanged. I believe the paper would benefit from resubmission, where its new contributions can receive a full review.

Reviewer_HypCOpenReview ↗
Rating
3
Confidence
3
Informativeness
3
Presentation
3
Soundness
3
Read review
Paper summary

The paper studies single-interaction unlearning in finite-horizon combinatorial semi-bandits. A learner explores by serving slates and observing their component rewards, then commits and stops collecting feedback. Erasure removes one interaction's feedback while retaining its calendar slot and served action. For covered product instances with a unique optimum and comparable gaps, category-wise stability yields matching minimax expected pseudo-regret bounds up to logarithms, separating identification cost m/Δ from stability cost KG/ρ. A gap-agnostic Envelope learner attains the bound. Its maximum-over-completions stopping gate and shared randomness ensure that the erased execution collects no feedback beyond the factual run, permitting exact reconstruction from the retained log. Equal-entropy examples, extensions, and synthetic and preference-derived experiments complement the main result.

Critical strengths
  1. A substantive characterization with an attaining learner. The category-dependent stability term has a lower bound covering causal, reward-adaptive exploration and an attaining gap-agnostic construction. Equal-entropy categories can have different regret costs because they preserve different sampling opportunities. This connects a matching learning law to a concrete geometric explanation.
  2. A constructive retained-feedback guarantee. The completion envelope establishes that the observations needed by each erased execution actually exist in the retained factual log. This goes beyond a distributional coupling that assumes unavailable counterfactual feedback. The simultaneous utility analysis is carefully separated from the stronger, unproved claim of repeated deletion.
Critical weaknesses
  1. The deletion contract needs a clearer operational motivation. Exactness is relative to the specified missing-feedback rule, including its modified stopping gate. The interaction's time slot and served action remain, later observations can be retained, and the guarantee concerns one interaction rather than a user's full history or sequential deletions. These are disclosed assumptions, not contradictions, but a concrete deployment example is needed to judge how the result meets an actual deletion request. The theoretical geometry remains meaningful even within this restricted model.
  2. The empirical advantages have a narrower interpretation than a general unlearning comparison. Figure 4 already compares mixed and uniform Envelope under the same category contract, which is a useful sampling ablation. The DP comparisons additionally change protection strength and collection behavior, so they cannot isolate the benefit of category geometry. Preference-derived experiments repeatedly sample fixed eight-item complete-vector cohorts, and the category separation uses very small stability budgets and long horizons. These choices illustrate the theory but leave its practical operating regime less clear. The completion envelope defines the missing-row execution; factual regret plots alone do not measure its counterfactual reconstruction benefit.
Questions for authors
  1. Can you give a concrete application where deleting one reward slice while retaining its action/time and using this marker-aware reference is the intended request? A small w>1 extension of Figure 1 mapping positions, blocks, arms, slates and categories would help. What prevents applying the same learner to a second deletion from an already-erased run?
  2. Beyond the existing uniform-versus-mixed Envelope ablation, can you compare an alternative stable acquisition policy under the same category contract and comparable collection budget? Please clarify which gains in Figure 4 come from sampling design versus the differing DP/collection contracts, and which practical budget/horizon regime motivates the displayed separation.
  3. At the scalar-calendar endpoint, how does the 1/ρ term relate to Nguyen-Tang–Arora's Appendix D L/ρ lower bound? Please distinguish adjacency, public output, request semantics and asymptotic quantifiers, particularly whether a pull-indexed reward-stack edit transfers to a fixed-calendar edit. This is a request to reconcile the contracts, not an established contradiction.
  4. Can you provide a reference environment or portability diagnosis? With pinned packages on macOS arm64, the original verifier fails the full-trace fingerprint for DP-KLUCB seed 0, cell 0, ρ=.0003, despite matching stored outcomes; some multinomial paths also differ. A platform-dependent cause is plausible but has not been established.
2 comments

First of all, Thank you for staying engaged with us through these revisions. We know that each response takes time, and we truly appreciate the care you have given this paper. Your questions have helped us make the presentation clearer and the work stronger. In particular, your concern about practical relevance prompted the stronger comparisons and adaptive-stopping results below.

1. Gibbs and Envelope make complementary contributions.

We have made the role of each learner explicit:

  • Gibbs lets us choose how much feedback to collect. It collects a budget BB fixed before the run and then serves without further feedback. Appendix K.19 bounds the budget needed for near-optimal expected serving reward, including after omissions. That sufficient budget is independent of the number of later serving rounds.
  • Envelope lets the observations determine when collection should stop. It makes this decision without knowing the reward gap. Its guarantee controls expected cumulative regret, including the maximum over the factual run and every separate single-erasure run.

Both satisfy the specified category-stability budget ρ\rho and support exact replay from retained feedback and saved randomness. Gibbs' strong practical results therefore support a contribution of the paper, alongside Envelope's adaptive guarantee.

2. We also improved the adaptive learner itself.

We took your concern about conservative stopping seriously. Appendix K.21 now provides new refinements, their proofs and a broader comparison:

  • Across 39 public-data model settings, three adaptive refinements each reduce mean regret relative to original Envelope. Every pointwise paired 95% confidence interval favors the refinement.
  • In a separate replication with 4,096 fresh shared paths and all 40 methods, geometric-half reuse reduces original Envelope's regret from 1,964.096 to 50.005, a 97.45% reduction. Feedback collection falls from 100% to 2.084% of the horizon.

The new analysis preserves category stability, exact single-erasure replay and the original regret order under the stated assumptions. The best tested fixed-budget Gibbs still achieves lower regret here, 40.394; the complete comparisons and unfavorable outcomes remain available. The optimized-cache timing results are unchanged.

3. We made the evidence easier to read and access.

We simplified the abstract and shortened the appendix by 16 pages, while preserving the proofs and the ten-page main text. Extended plots and tables remain in a companion detailed record. The updated empirical package includes protocols, complete new-study results and verification scripts, with a reading guide to the earlier empirical archives.

We would be very grateful if you could consider these contributions together and revisit the rating to 4. Your engagement has made a real difference to the paper, and we sincerely appreciate it.

AuthorsOpenReview ↗

Thank you for helping strengthen the pape

Dear Reviewer HypC,

Thank you for staying engaged with our paper throughout this discussion. We know that each careful response takes time, and we sincerely appreciate the attention you have given the work. Your questions have shaped several of its most useful improvements.

For your final assessment, we would like to bring together what changed in response to your feedback:

  • A concrete withdrawal model. We added the two-panel example, explained which feedback is removed and which historical information remains, and analyzed repeated deletion, including conditions for stability and a counterexample outside those conditions.

  • A fairer learning comparison. We added the category-stable Gibbs comparator and collection-budget comparisons under the same category contract. We separated sampling improvements from differences in DP protection and expanded the budget and horizon studies beyond the original small preference cohorts.

  • Lower-bound reconciliation and reproducibility. We distinguished the prior result’s contracts, proof-transfer step and asymptotic quantifiers. We also diagnosed reproduced scalar-math and multinomial discrepancies and provided a reference environment that passes the unchanged strict verifier.

  • Practical guarantees and replay measurements. We proved a sufficient Gibbs feedback budget independent of the later serving horizon and added a certificate for safely discarding completed-phase responses. We measured withdrawal processing across broader public-data workloads, including file-backed replay and optimized-cache comparisons, with independent erased-reference checks.

  • Improved adaptive stopping. Three refinements reduce mean regret relative to original Envelope across all 39 tested settings. In a separate replication using 4,096 fresh shared paths in the Criteo-derived conversion model, geometric-half reuse reduces regret from 1,964.096 to 50.005—a 97.45% reduction—while collecting feedback for 2.084% of the horizon. The analysis preserves category stability, exact single-erasure replay and the original regret order under its stated assumptions.

  • Clearer presentation. We simplified the abstract and explanations, clarified Gibbs’ fixed-budget role and Envelope’s adaptive role, and shortened the appendix by 16 pages while preserving the proofs.

The best tested fixed-budget Gibbs still achieves lower regret in that replication, and the optimized-cache timing results remain unchanged. These findings remain visible alongside the improvements.

Would you please consider whether the theoretical contribution, answers to your questions and additional verified evidence together now warrant a rating of 4? We would be very grateful for your reconsideration. Your engagement has made a meaningful difference to this paper, and we sincerely thank you for it.

Reviewer_bM6JOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
1
Soundness
3
Read review
Paper summary

The paper studies unlearning in combinatorial bandits with semi-bandit feedback. The authors propose a stability framework where actions are partitioned into protected categories and a replay mechanism that allows a learner to reproduce its behavior after deleting one observation. The paper provides theoretical regret bounds and experimental evaluation on synthetic settings and preference-derived datasets.

Critical strengths
  • interesting and novel problem formulation: the observation that missing feedback can change a commitment decision is an important for robust learning and leads to a problem that is different to most offline unlearning
  • strong theory: a lower bound that is matched up to logarithmic factors by the upper bound of the algorithm
  • categories lead to an interesting structure: insightful observation that two categories can have the same entrioy but lead to different stability costs
Critical weaknesses
  • problem setting is difficult to understand on a first read: many important variables are introduced quickly, without a domain and without an explanation or example. Their roles are primarily specified mathematically, e.g. for m, T, S_u, X_u, K, d, L, G, \Delta, \rho, categories. A running example might help. It is difficult to understand the unlearning in the setting: it is not clear to me how the missing values appear. Are the deletions arbitrary, according to a probabilistic model or adversarial? The category-wise stability is one of the most important notions but could be explained more intuitively. How can a bound as \rho=0.01 be interpreted? In addition, the category arrangement seem to drastically effect the stability cost, so how can categories actually be selected in practice? How to define appropriate protected groups? The practical interpretation of a protected category is unclear to me. How would you deciide in practice which category function \phi to use?
  • theoretical setting narrower than the motivating applications: the theory restricts to the assumption of one optimal slate, comparable gaps and a coverage. Later, matroids and heterogeneous gaps are mentioned, but its not fully clear which theorem is a general result and which relies on the product structure

I really like the core idea and find the theoretical results very interesting. I would therefore be happy to increase my score if the authors could improve the presentation by adding more intuition, explanations, and concrete examples. In my opinion, the technical results are strong enough to warrant acceptance, but the current presentation makes the paper unnecessarily difficult to understand.

Questions for authors
  • the most important point that is unclear for me is how the deletion model is defined? Is the missing value selected uniformly at random, according to a probabilistic model or adversarial? In your paper, it seems like you focus on one deletion only, I think in reality you either have zero forgetting or multiple missing values. How to deal with multiple deletions? In this case, I would expect an additional assumption that e.g. a fixed percentage of observations has to remain for each category and only a small part of the observations per category can be missing
  • I still doubt the practical applicability. Can you provide an assessment how to choose \rho, \phi, the categories and the protected groups in practice? How robust is your algorithm to bad choices of these variables?
4 comments
AuthorsOpenReview ↗

Further advances in feedback efficiency, exact replay and readability

Thank you again for recognizing the theoretical contribution. Alongside the changes detailed in our main response, the review discussion has helped us strengthen the paper around three practical questions: how much feedback must be collected, what must be retained, and how can withdrawal be handled exactly?

1. Two complementary ways to control feedback collection.

The revised paper makes the roles clear: Envelope learns when to stop, without knowing the reward gaps; Gibbs uses a feedback budget chosen in advance. Both provide category stability and exact replay from retained feedback.

For Gibbs, Appendix K.19 now proves a sufficient budget for near-optimal expected serving reward, including after omissions, under the stated mean and probing assumptions. Crucially, this budget does not grow with the number of later serving rounds.

The public-data evaluation illustrates this benefit. On Hillstrom's randomized email experiment at ρ=0.1\rho=0.1, Gibbs uses 799 rather than 51,159 responses, retaining 99.98% of full-budget estimated visit value on held-out data (Appendix K.17).

2. Substantial improvements to adaptive stopping.

We developed new adaptive refinements, including reuse of feedback across phases, with proofs preserving category stability, exact single-erasure replay and the original regret order in the stated single-position setting (Appendix K.21).

  • Across all 39 tested public-data model settings, three refinements each reduce mean regret relative to original Envelope; every pointwise paired 95% interval favors the refinement.
  • In a separate 4,096-path replication on the Criteo conversion model, geometric reuse reduces original Envelope's regret from 1,964.096 to 50.005—a 97.45% reduction. Collection falls from the full horizon to 2.084%.

The best tested fixed-budget Gibbs achieves 40.394 regret in that replication. Its stronger result supports the paper's complementary fixed-budget contribution; the full comparisons with Gibbs and Sparsemax remain available.

3. A guarantee for retaining less response data.

Appendix K.20 proves when responses from completed phases can be discarded while preserving every separate single-withdrawal reconstruction. The certificate uses public action counts; the required metadata remain.

Applied to nine recorded commerce workloads, it certifies 5.43%–34.87% of acquired responses as discardable after collection. This gives an explicit rule for reducing retained feedback while preserving exact reconstruction. The appendix also establishes local reconstruction costs and accounts for indexing, writing and persistence.

4. Broader evidence and stronger technical support.

The evaluation now extends beyond small preference cohorts to chronological recommendation logs, full catalogs, commerce, randomized email and advertising benchmarks. We added comparisons under matched category protection, collection-budget grids, independent erased-reference checks, and file-backed withdrawal benchmarks with optimized-cache baselines. The results retain weaker regimes and modest cache gains (Appendix K).

We also added a bounded repeated-deletion theorem and a counterexample defining its limit, reconciled the earlier lower bound, and diagnosed numerical reproducibility discrepancies with a pinned reference environment (Appendices H–J).

We hope these additions make the contribution both easier to understand and more concrete to evaluate. Alongside our direct responses to your presentation concerns, we respectfully ask you to reconsider your rating to 3, or 4 if you find the revised contribution warrants stronger support. Thank you for helping us improve the paper.

Dear Reviewer,

We have updated our point-by-point response and the follow-up “Further advances in feedback efficiency, exact replay and readability” in this thread.

The updates map your presentation concerns to the revised manuscript, answer both questions, and summarize the additional results. We would be grateful for your reassessment and any remaining concerns.

Thank you for your time and consideration.

AuthorsOpenReview ↗

A request to revisit the revised presentation

Dear Reviewer bM6J,

Thank you again for recognizing the core idea and for explaining what made the paper difficult to follow. Your statement that “the technical results are strong enough to warrant acceptance,” together with your willingness to reconsider after presentation improvements, gave us a clear direction for the revision.

We have worked directly on the points you identified:

  • A running example: the same two-panel recommendation example now connects the motivation, categories, collected feedback, learner and replay.
  • More accessible notation: quantities are introduced with their meanings and domains, and the learner is explained through sampling, scores, a stopping certificate and the decision to commit.
  • Practical interpretation: we explain what a withdrawal removes, what ρ=0.01\rho=0.01 means, and how choosing categories changes protection and learning cost.
  • Clearer scope and answers: we distinguish the assumptions behind the different results and answer both questions, including multiple withdrawals and the practical choice of protected groups.

The updated table and Q&A in our core response provide a compact guide to these changes and their manuscript locations. Additional analyses developed through the wider review discussion; our request here is to revisit the original contribution with the presentation improvements you asked for.

We would sincerely appreciate a brief response on whether these revisions resolve your concerns. If they do, would you please consider increasing your rating to 3? We have taken your feedback seriously, and would be very grateful to hear your assessment of the revised exposition.

Thank you again for your time and for helping us make the paper easier to understand.

Thank you for the extensive revision and for already addressing my comments in the revision. I appreciate the effort that has clearly gone into improving the manuscript, and I can already see that some of the concerns I raised in my review have been addressed, e.g. the missing domains and explanations of the variables.

However, I found it difficult to assess the revised manuscript efficiently, as the changes were neither highlighted in the manuscript nor clearly referenced in the rebuttal (e.g., by indicating the modified sections or lines). Comparing the two versions manually, I noticed that the revision goes far beyond addressing individual comments: the abstract and introduction have been substantially rewritten, sections and figures have been rearranged, entirely new sections and tables have been added, and the overall length increased substantially.

While I appreciate these improvements, the extent of the changes makes it difficult to determine whether all aspects of the revised manuscript are now sufficiently addressed without essentially reviewing and thus reading the entire paper again. A marked-up version or a more detailed change log would have been extremely helpful in this regard.

I wrote in my review that the results and the underlying contribution were strong, but that the presentation and overall quality of the manuscript did not meet the standards expected at a top-tier conference. I appreciate the apparent effort to address these concerns, but I am unfortunately unable to fully assess the extent to which the revised manuscript has resolved them.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.