acceptodds
Under review as a conference paper at ICLR 2027

Rethinking Demonstration Unlearning in Imitation Learning for Robotics

Abstract

Robot policies learn from demonstrations, some of which may later need to be withdrawn, for example when consent is revoked or an episode proves faulty. Demonstration unlearning edits the trained policy to remove their influence; ideally, the result matches a policy trained without them. Unlearning is often judged by isolated proxies such as restored task success, higher loss on withdrawn data, or membership-inference scores. When these proxies disagree with retraining, an edit can be credited with removing demonstrations whose influence remains detectable, or rewarded for overshooting what retraining would produce. We propose a retrain-calibrated audit that makes removal claims testable and shows whether an edited policy departs from retraining in behavior, data fit, or both. It compares the edited policy with independently retrained policies through actions executed at the same observations and prediction error (loss) on withdrawn episodes, including whether that loss separates them from never-used episodes; a symmetric rank test calibrates these differences against retraining variation. In physical cup insertion, an ACT edit raises success from 5/20 to 18/20 trials, yet its loss still perfectly separates audited withdrawn from never-used episodes and its actions remain far from retraining; a second policy class shows the same success-versus-evidence disagreement. In simulation, action and episode measurements detect complementary discrepancies; tests on fresh retain-only models and controlled data reintroduction reveal false rejections and sensitivity to checkpoint selection and queried states. Useful robot repair is thus not evidence of demonstration removal: unlearning claims should be checked against retraining in behavior and data fit.

Primary area:
applications to robotics, autonomy, planning
Keywords:
Machine Unlearning, Imitation Learning, Robotics, Manipulation

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_77GGOpenReview ↗
Rating
1
Confidence
1
Informativeness
1
Presentation
1
Soundness
1
Read review
Paper summary

None

Critical strengths

None

Critical weaknesses

Anonymity concern: The manuscript’s explicit linkage to a closely matching public preprint raises serious concerns for me about compliance with the double-blind policy. I decline to provide a substantive technical review pending clarification from the AC/PC.

In particular, §2 of the submission (p. 3, lines 138–145) explicitly refers to “The preprint of Li et al. (2026)” and states:

“This manuscript develops that study with fully symmetric candidate-inclusive calibration, independent retain-only nulls and controlled data reintroduction.”

The bibliography also provides the preprint’s author names and a direct link to its arXiv version (p. 11, lines 540–543).

Questions for authors

None

1 comment

We appreciate the scrutiny of the reviewer, and we are grateful for the efforts into the details. However, we would like to point out that by ICLR's author guidelines, related works of any authorship can be cited in the paper as long as used in third person and provide no explicit indication of authorship, including follow-up works. More detail and examples on the uniformity and variation of eligibility, requirements, and formatting of citing works of different authorship sources can be found in the Q&A session on submission of the author guidelines as well. We have reached out to AC already regarding the concern, and while we appreciate the endeavor and caution, we will also be grateful if technical reviews can be given after double checking the compliance of this paper with ICLR's double-blind policy.

Reviewer_f21qOpenReview ↗
Rating
2
Confidence
2
Informativeness
1
Presentation
1
Soundness
2
Read review
Paper summary

This paper evaluates demonstration unlearning in imitation learning, targeting policies retrained on retained demonstrations. The authors claim that task success, forgetting loss, and membership AUC are individually insufficient proxies for demonstration removal. They propose a retrain-calibrated audit that measures three coordinates against kk independent retrain-only references: (1) L1 distances between executed actions at matched observations; (2) membership AUC of withdrawn vs. never-used episode; (3) withdrawn-episode loss relative to the retrain. The three coordinates are combined into one score that is computed in the same way for the edited policy and for each of the kk retrain-only references. The rank of the edited policy's score among the K+1 scores gives a p-value for the null hypothesis (that the edited policy is indistinguishable), on these measurements, from one more retrain-only reference. The authors claim that task repair is not evidence of demonstration removal.

Critical strengths
  • Rigorous reporting standards and valid experimentation.
  • Evidences support the claim that task success and demonstration removal diverge.
Critical weaknesses
  • The audit procedure, listed as a contribution, is not clearly specified and its outputs vary across the paper;
  • The effect size for R200 varies across protocols without justification;
  • The audit, which is a proposed contribution, does not guarantee the removal is testable from the presented evidence.
Questions for authors

1- Which observation parameters would the paper recommend for the audit and what justifies it? 2- What are the original and retrain-only success rates on the corrected Can dataset across all the mentioned rollouts? 3- What result from the paper demonstrates that the audit is capable of detecting a valid edit and not only exclusively invalid edits? 4- How were the baseline methods tuned in Table 16?

5 comments

We appreciate the reviewer for the precise summary of the strength and weakness of this paper, which describes the method exactly and helped us gain profound insights in refining the paper for its audience and application. We agree that the audit and its primary settings must be specified more clearly, and every item below will be addressed in the revised version. The response is categorized into three parts corresponding to the four questions and three weaknesses raised by the reviewer, but not exactly under the same layout (despite being largely consistent), with the first two parts answering the questions and the third addresses the weaknesses.

Part 1

Regarding Questions 1 and 2, we agree that the paper should recommend default observation settings instead of only asking that the observation source be reported with every decision, and that Table 16 should report the success of the original and retrain-only policies alongside the edits, and we appreciate your valuable questions, which guide us to the information that a reader genuinely needs yet our submission did not provide, starting with the following detailed answers:

Q1. Recommended observation settings. We recommend the following default and will state it as such in Sec. 3.

  • Observations: frames of the demonstration data named by the request (the withdrawn episodes; on Can also the audit non-member episodes), held fixed for every candidate and reference. They are defined by the request alone, exist before any edit is made, and are identical for all models. When an edit targets a known part of the trajectory (the BRANCH window for redirect), that region is reported separately as an in-sample diagnostic.
  • Horizon: the actions the robot executes before it re-plans (the first 25 of ACT's 50). Later actions never reach the robot, and including them raises R200's arm-joint gap closure from 13.8% to 43.8%.
  • Coordinates: grouped by physical unit and reported per group (six arm joints; gripper separately).
  • Check: the same decision repeated on a bank of rollout observations from retain-only policies that are neither candidates nor references.

The justification is Table 2. With demonstration observations the joint rule rejects 2/20 fresh retrain-only models and detects 0/3, 2/3, 3/3 and 3/3 candidates at 5%, 25%, 50% and 100% reintroduction. With the independent rollout bank it rejects 0/20 and detects 0/3, 0/3, 1/3 and 3/3. The demonstration bank is the more sensitive instrument and the rollout bank the more conservative one. Neither dominates, so we recommend the first as primary and the second as a reported check, instead of choosing whichever is favorable. Neither bank covers states that only the edited policy visits, which remains a stated limitation (Sec. 5).

Q2. Original and retrain-only success on corrected Can. All values use the Table 16 protocol: 500 rollouts per policy, at the checkpoint used in the audit.

PoliciesnSuccess, mean ± SD (%)Range (%)
Original (trained with the 60 withdrawn episodes; the seeds used in Table 16)1235.5 ± 5.523.8-44.2
Retrain-only: the 19-reference audit panel1958.6 ± 9.741.0-75.6
Retrain-only: the 20 fresh nulls of Sec. 4.42054.5 ± 11.135.2-77.8

The originals and 13 of the references had already been evaluated. The six newer references and the 20 nulls were evaluated for this response, and no model was retrained.

Three readings:

  • Retraining without the withdrawn episodes raises mean success by about 23 points, so the request is consequential.
  • The most successful edit family in Table 16, our calibrated-ascent intervention (DIFT-pF), reaches 35.1 ± 6.3 on seeds 1-4 and 39.4 ± 9.3 on seeds 6-13, which is the level of the unedited originals. The single most successful edit (58.4%, expansion seed 9) reaches the retrain mean and is still rejected, through the loss coordinate.
  • Retrain-only success itself spans 35% to 78% across seeds of one recipe, so task success alone cannot distinguish an edit from retraining.

Moreover, we will also add the first two rows to Table 16.

PART 2

We are again grateful for the precious feedback carried by Questions 3 and 4, which helps us better comprehend, from a reader's perspective, what the paper leaves unanswered: while the evidence that the audit can pass a valid candidate is in the paper, it is not presented as an answer to this question, and how the edits in Table 16 were tuned is not described in Appendix F, both of which we guarantee to correct in the revised version. Here are our answers:

Q3. Does the audit pass valid candidates, or only reject invalid ones? It passes them, as shown in three ways, with one limit that we state.

  • Known-valid candidates pass. The only candidates known to satisfy the removal target are policies trained without the withdrawn data. We trained 20 such policies that were never used as references (Sec. 4.4, Table 13). The joint rule does not reject 18 of the 20. The two rejections exceed the largest reference score by factors of only 1.012 and 1.061, and both arise from the action coordinate.
  • The verdict follows exposure. When 5%, 25%, 50% and 100% of the withdrawn episodes are reintroduced into retrain-only training, the rule rejects 0/3, 2/3, 3/3 and 3/3 candidates. Policies that saw 3 of the 60 episodes pass, and policies that saw all 60 fail. This is independent of any editing algorithm.
  • The rule does not reject every edit mechanically. Four of the 86 corrected Can edits are not rejected jointly. We do not present them as valid removals, because each is rejected by at least one unadjusted marginal and their success is 0.2% to 26.0%.

The limit is twofold. None of the editing operators we tested produced a policy that we can certify as equivalent to retraining, so the positive controls are retrained policies and not edits. And two false rejections in 20 (one-sided 95% upper bound 28.3%) are too few to establish a 5% operating error rate. Sec. 4.4 states this, and a larger null cohort with an independent reference panel is the direct remedy.

Q4. Tuning of Table 16. All edits start from the same original checkpoints and share the update budget (10,000 gradient steps for the iterative methods, which is 5% of the retraining length; Adam; batch size 100) and the evaluation (500 rollouts).

  • Original comparison, seeds 1-4: the 13 baseline families at one default configuration each in our deterministic-BC adaptations (learning rate 1e-4; for example NPO β=50\beta = 50, SimNPO δ=2.5\delta = 2.5, SSD α=10\alpha = 10), and our calibrated-ascent intervention (learning rate 3e-5, anchor and proximity weights 3).
  • Tuned finalists, seeds 2-3: for six families we swept 16 configurations. NPO with β∈{10,50,200}\beta \in \{10, 50, 200\}, and β=50\beta = 50 with tether weight 3 or 10; SimNPO with δ=1.0\delta = 1.0, and δ=2.5\delta = 2.5 with tether 3; NegGrad+ with tether 0, 3 or 10; gradient ascent with tether 3 or 10; SSD with α∈{5,50}\alpha \in \{5, 50\}; Blindspot with action weight 1 or 50. The tether is the function-space proximity penalty of our own intervention (MSE to the original policy's actions on retained states), added so that each baseline has the same protection against utility collapse. Each configuration was run on two seeds and screened with 50 rollouts. The best configuration per family by task success, measured relative to the original and retrained policies, was kept (two for NPO), which gives the seven finalist rows. Selection used task success only and never audit outcomes.
  • Expansion, seeds 6-13: our intervention and retain fine-tuning on eight further seeds, without tuning.

Two limitations will be added to Appendix F. The sweep was run under the earlier removal request that Appendix F describes as superseded. The seven selected configurations were then re-run unchanged under the corrected request and were not re-tuned on it. In addition, seven families (retain fine-tuning, EU-k, RMU, Fisher, task vector, SCRUB, relabel) were evaluated at a single configuration. Table 16 is therefore a varied population of edits on which to test the audit, and not a ranking of published methods (Sec. 4.3).

PART 3

This last part covers the three weaknesses: the specification of the audit (W1), the R200 effect size across protocols (W2), and whether removal is testable (W3). We are profoundly grateful for these observations, especially as we have failed to make evident to the reader what the audit outputs, which of the reported numbers is primary, and what a rejection or a non-rejection establishes; therefore, we thank the reviewer for letting us revisit and specify more rigorously the audit that constitutes the core of our research:

W1. Specification of the audit. The procedure is the following.

  1. Before examining any candidate, fix the reference training recipe, the observations, the executed horizon, the coordinate groups, the member and non-member episode cohorts, the loss used as score, KK and α\alpha.
  2. Train KK retain-only reference policies (K=19K = 19).
  3. For the candidate and for each reference, compute three coordinates against the other KK models: mean executed-action divergence, absolute log-ratio of withdrawn-episode loss, and absolute difference in membership AUC (Eq. 3).
  4. Standardize each coordinate by the other models' mean and SD and take the maximum, which gives one score per model (Eq. 4).
  5. Compute pp as the rank of the candidate's score among the K+1K+1 scores and reject if p≤αp \le \alpha. Report the three marginal p-values and the raw effect sizes alongside.

The output of the audit is the decision in step 5, and it does not vary for R200. The joint p-value is 0.05 under both the arm-only and the pooled metric (Table 3), and all nine ACT candidates are rejected under every subsample, nonempty window and horizon perturbation (Table 12). What varies is the descriptive effect size under alternative settings, which Sec. 4.5 and Appendix D report as sensitivity analyses. We agree that the paper does not separate the two clearly. The revision adds these five steps as an algorithm box in Sec. 3 and labels every reported number as primary or sensitivity.

W2. R200 effect size across protocols. Gap closure is a ratio (Eq. 2), and each protocol changes what is averaged. All rows use R200 and the BRANCH observations.

SettingGap closure
Primary: first 25 actions, six arm joints, 3 retrains13.8%
Adding the gripper coordinate (all seven)24.9%
Gripper only97.2%
Arm joints, all 50 predicted actions43.8%
All seven coordinates, 19 retrains22.6%
All seven coordinates, window narrowed by one or widened by up to two banked observations per side4.4%-54.5%

The reasons are as follows:

  • The gripper nearly closes its own gap and has an undocumented unit, so pooling it with joint angles inflates the average.
  • Actions 26 to 50 are never executed, and that is where the original policy's distance from retraining grows (4.21 to 5.95) while the edited policy's barely changes (3.90 to 4.19), so averaging over the full prediction credits a correction the robot never performs.
  • The number of retrains changes the floor estimate.
  • The window changes which observations are compared (12 to 191 instead of 71).

The primary setting follows one rule: compare what the robot executes, in a common physical unit, on the window fixed in advance as the one the edit was trained on. We note that the arm-only reading is a retrospective decomposition of saved predictions (Sec. 4.1), and that all nine joint and marginal p-values are identical under it and under the pooled metric. The decision is the same in every setting. The magnitude depends on the convention, which is why Sec. 4.5 asks that the convention be reported with every closure value. The revision states the rule once in Sec. 3 and collects the other rows in a single sensitivity table.

W3. Whether removal is testable. We agree that the audit gives no guarantee that removal occurred, and the paper should not suggest otherwise. What it provides is a falsification test. A removal claim implies that the edited policy is indistinguishable from retrained policies on the measured actions and episode evidence, and the audit can refute that at a stated level. A rejection is a detected discrepancy. A non-rejection is the absence of a detected discrepancy with this panel and these observations, and it establishes neither equivalence nor removal (Sec. 3.3). The evidence that the test is informative in both directions is given under Q3. We will replace "makes removal claims testable" in the abstract with wording that states this scope, and add the Q3 evidence to the paragraph that introduces the test.

Reviewer_PRraOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
1
Soundness
2
Read review
Paper summary

The paper discuss an interesting topic of demonstration unlearning in imitation learning, which edits the trained policy to remove the influence of undesired data. The paper proposes a retrain-calibrated audit to make removal claims testable and better analyze the unlearning performance.

Critical strengths

The paper investigates an interesting problem of how to remove the influence of selected withdrawn training data from a learned policy, while preserving performance comparable to that of a policy retrained from scratch on the remaining data. Machine unlearning is valuable for removing corrupted or harmful data, but has been relatively under-explored in the context of imitation learning. Bringing this perspective into imitation learning could therefore be valuable and timely.

Critical weaknesses

My major concern is the writing, which makes the paper a little bit hard to follow.

  1. The paper consists of terminology and notations that are not well defined, for example: a) In the caption of Figure 1, what is "R200", what is "p=0.05", and what is "BRANCH observation"? b) In Lines 52-53, "Avoiding the characteristic failure of withdrawn episodes is a different goal, closer to policy correction. In robotics, the distinction between an edit and deletion is easily obscured. Suppressing an unwanted motion can repair a task...". I am not sure how these sentences should be connected as each of them seems independent. "Policy correction", "edit", and "suppressing an unwanted motion" seem all related to policy editing; are there any difference or are they the same thing? Also, what is the purpose of these sentences? c) In section 3, Line 149, the definitions of "D_r and D_f" are missing. d) Line 153, "Before evaluating candidates, fix the reference procedure, observation bank, executed horizon, coordinate aggregation, episode cohorts, score interface, and significance level." What are these terms and how are they related to the proposed retrain-calibrated audit? e) In Eq(1), Line 161, what is "d_r(\pi, \pi')" and what is its meaning? Also, in Line 162, "Here a_{\pi, h, g}(o;\xi) is coordinate g, from the specified group G, of the h-th of H executed action that \pi outputs at observation o; stochastic policies use paired sampling randomness \xi", what is coordinate, and what do you mean by "group" here? Also, how to get the "paired sampling randomness \xi" for \pi and \pi'? By using the same random seed or something else? f) In Line 172, how is the "f_r" defined and calculated? g) Around Eq (3), Line 203, what are the differences between "m_\pi" (defined in Line 184), "m_i", and "\bar{m}_i"? Also, what are the differences between "u_i" and "\bar{u}_i"?

  2. The paper includes results that are not well explained, which makes it difficult to understand the meaning of the results. a) In the caption of Figure 1, Lines 73-81, the paper lists a lot of results, but what conclusions could we draw from these results? Also, the detailed results should be discussed in the experiment section rather than in the caption of Figure 1. b) Another example: in Line 290-296, the paper again lists a lot of numbers, but the necessary discussion is missing and it is difficult for a reader to draw a conclusion from these numbers. c) In Abstract, Line 24-28, "In physical cup insertion, an ACT edit raises success from 5/20 to 18/20 trails, yet its loss still perfectly separates audited withdrawn from never-used episodes and its actions remain far from retraining". In this sentence, it's not straightforward to understand the issue of "perfectly separates audited withdrawn from never-used episode", the author could instead stress the edited ACT model doesn't behave like a ACT model trained from scratch without the withdrawn samples.

  3. Regarding the motivation, the paper briefly mentions that (L40-41) "As demonstration collections grow, so does the need to revise what a policy has learned when particular episodes are withdrawn or identified for removal." However, the paper does not explain why this is an important problem to investigate, especially considering demonstration unlearning is a relatively new topic for robot learning. A few concrete examples of real-world scenarios would make the paper clearly motivated.

  4. Things to improve: a) In Line 109: "Section 2 distinguishes the preprint formulation from ...". It's uncommon to end an Introduction section by introducing Related Work, the author can instead place this sentence at the beginning of Section 2.

Apart from the writing, the paper could benefit from more rigorous experiments, as currently there is only one task family, making the experiment relatively weak to support the claims.

I personally think demonstration unlearning is an interesting direction to investigate, while the presentation of this paper should be improved a lot to better present the core contributions.

Questions for authors

Apart from the questions in the weaknesses, I have one more question regarding the experimental setup. For the "executed-action consistency" in Sec 3.1, how to separate the effects caused by randomness inherited in the policy (e.g., Diffusion Policy) from the unlearning?

4 comments

PART 1

Thank you for the careful, line-by-line reading. We agree the presentation must improve, and every item below will be corrected in the revised version. The response is categorized into four parts corresponding to the four major areas of concern raised by the reviewer, but not exactly under the same layout (despite being largely consistent).

Regarding Question 1, we agree that the terminology across the paper might lead to confusion in understanding, and we appreciate your valuable feedback that guides us to a better presentation of it, starting with the following detailed clarifications of the issues you pointed out:

1a. Figure 1 caption.

  • "R200" is the policy obtained by applying the localized redirect edit to the original ACT policy for 200 gradient updates. Edits are named by type and update count: R = redirect, B = loss ascent, FT = fine-tuning on retained data.
  • "p = 0.05" is the p-value of the joint test in Eq. 4. The edited policy and 19 independently retrained policies each receive one discrepancy score, and p is the fraction of these 20 scores that are at least as large as the edited policy's. With 19 references the smallest possible value is 1/20 = 0.05. It means the edited policy is further from the retrained policies than any of them is from the others, so the audit rejects it at the 0.05 level.
  • A "BRANCH observation" is one policy input (camera images and joint state at one time step) taken from a withdrawn demonstration inside its BRANCH window: the interval from 5 frames before to 20 frames after the point where that demonstration's motion departs from the clean demonstrations. This is where the unwanted displaced-release behavior begins, and it is the window on which the redirect edit was trained. The action comparison uses 71 such observations.

1b. Lines 51-53. The three terms are different, and the passage is meant to separate them.

  • A policy edit is any update applied to an already trained policy. It is the general term.
  • Policy correction is an edit judged by behavior: the robot should stop producing an unwanted motion. "Suppressing an unwanted motion" is an example of correction.
  • Deletion (unlearning) is an edit judged by data: the result should match a policy trained without the withdrawn demonstrations.

The purpose of the passage is the premise of the paper. A correction can succeed, so the task is repaired, without being a deletion, because the policy still differs from one retrained without those demonstrations. Conversely, a correct deletion can lower task success when the withdrawn demonstrations were useful. We will rewrite the passage as these three definitions followed by that statement.

1c. Line 149. Df\mathcal{D}_f is the set of withdrawn demonstration episodes named in the removal request (the "forget" set). Dr\mathcal{D}_r is the set of retained episodes. D=Dr∪Df\mathcal{D} = \mathcal{D}_r \cup \mathcal{D}_f is the original training set. In the physical task there are 30 withdrawn and 100 retained episodes.

1d. Line 153. These are the settings of the audit. They are fixed before any edited policy is examined, so that the edited policy and the retrained references are measured in exactly the same way.

  • Reference procedure: the training recipe used to retrain on Dr\mathcal{D}_r (initialization, optimizer, length, checkpoint selection).
  • Observation bank: the fixed set of observations at which all policies are queried for the action comparison.
  • Executed horizon HH: how many predicted actions are compared, namely those the robot executes before it re-plans (the first 25 of ACT's 50).
  • Coordinate aggregation: which components of the action vector are averaged together (the six arm joints; the gripper separately).
  • Episode cohorts: the withdrawn episodes and the never-used episodes that are scored for membership.
  • Score interface: the per-episode loss used as the membership score.
  • Significance level: α=0.05\alpha = 0.05.

PART 2

We again appreciate the precious feedback on the presentation of the results, which helps us better understand how it could and should be improved from a reader's perspective to better deliver the significance and implications to the audience, which we also guarantee to better clarify in revised versions. Here are our clarifications:

1e. Eq. 1. dr(π,π′)d_r(\pi, \pi') is the mean absolute difference between the actions that two policies π\pi and π′\pi' output at the same observations. The average runs over the observations in the bank QrQ_r (region rr of the trajectory), the HH executed actions, and the coordinates in the group G\mathcal{G}. It answers the question "how differently would these two policies act in the same situation?" A coordinate is one component of the action vector; our arm has seven (six joint commands and one gripper command). A group is a set of coordinates with a common physical unit, here the six arm joints. Paired sampling randomness: a stochastic policy such as Diffusion Policy turns random noise into an action, so two calls differ even for one policy. We therefore give both policies the same noise at each observation by using the same random seeds. ξ\xi denotes this shared noise, and the expectation averages over repeated draws. ACT is decoded deterministically and needs no ξ\xi.

1f. Line 172. frf_r is the retrain floor: the average of drd_r over all pairs among the JJ independently retrained policies. It measures how much two policies trained on the same retained data differ from each other through training randomness alone, so it is the smallest distance an edit can meaningfully be expected to reach. br(π)b_r(\pi) is the average distance from a policy to the JJ retrained policies. The gap closure in Eq. 2 is the fraction of the way traveled from the original policy (0) to the floor (1).

1g. Eq. 3. mπm_\pi (line 184) is the mean loss of policy π\pi on the withdrawn episodes. In Eq. 3 the models are indexed i=0i = 0 for the edited policy and i=1,…,Ki = 1, \dots, K for the retrained references. mim_i is mπm_\pi for model ii, and mˉ−i\bar{m}_{-i} is the average of mjm_j over the other KK models. In the same way uiu_i is the membership AUC of model ii and uˉ−i\bar{u}_{-i} is the average AUC of the other KK models. Each model is therefore compared with all the others.

2a. Figure 1 caption. The single conclusion is that task success does not show removal. The same edit raises success from 5/20 to 18/20, yet (i) the policy's loss still ranks every withdrawn episode below every never-used one (AUC 1.000, where a retrained policy gives 0.639), (ii) only 13.8% of the gap between the original policy's executed arm actions and retraining is closed, and (iii) the joint test rejects it. We agree the caption carries too much detail. It will keep the explanation of the diagram and this one sentence, and the numbers will move to Sec. 4.2.

2b. Lines 290-296, read in order. Success of 18/20 against 20/20 for retraining means the task is nearly repaired. AUC of 1.000 against 0.639 means the withdrawn demonstrations remain perfectly recognizable from the policy's loss, whereas after retraining they are barely distinguishable from unseen episodes. Action divergence falling from 4.21 to 3.90, against a retrain floor of 2.00, means the edit moved the executed arm actions only 13.8% of the way toward a retrained policy (0.1% for fine-tuning with the same budget). The values 13.8%, 23.7% and 17.9% repeat this on three independently trained original policies. The conclusion is that the edit repaired the task but did not make the policy resemble one trained without the withdrawn data. We will add these interpretive sentences.

2c. Abstract. "Perfectly separates" means that every withdrawn episode has lower loss than every never-used episode, so the withdrawn data can still be identified from the edited policy. It is our honor to gain insights from your wording and state that the edited ACT policy does not behave like an ACT policy trained from scratch without the withdrawn demonstrations, in its actions or in its fit to those demonstrations, and we will adopt them in our future writing.

PART 3

Words fail to express our gratitude toward this part of the suggestions, especially as we have failed to consider the topic's significance for wider audiences who might not be exactly familiar with the data collection and training of imitation learning policies for robotics (like VLA or ACT); therefore, we thank the reviewer for letting us revisit and expand on the simple yet compelling motivation of our research:

  1. Motivation. Demonstrations are the only supervision an imitation policy receives, so their quality determines the quality of the policy. Faulty or inconsistent demonstrations do not average out. They show up on the robot as misjudged grasp or release positions, jitter, hesitation between conflicting strategies, and other unexpected motions, and they are usually identified only after training, when the defect is observed. Four concrete cases, each represented in our experiments:
  • An operator with a consistent bad habit. In our physical task, 30 of 130 teleoperated demonstrations release the cup in a displaced position. The policy trained on all 130 succeeds in 5 of 20 trials, failing mostly by swaying and knocking the cup over, against 20/20 for a policy trained on the other 100.
  • Lower-quality operators in a multi-operator dataset. On robomimic Can, policies trained with the 60 withdrawn episodes average 35.5% success, against 58.6% for policies retrained without them (500 rollouts each; these two numbers will be added to Table 16).
  • Corrupted recordings. With mirrored episodes in PushT, success is 57.9% against 72.6% after retraining without them.
  • A contributor or data owner revoking consent or a license. Here the withdrawn demonstrations may be useful, and success should fall after removal (80.8% to 62.5% in our benign PushT request).

Retraining from scratch after every such request is the clean answer, but it is costly for large policies and for repeated requests, which is why post-training edits are attractive. Their risk is that an edit hides the symptom without removing the cause: the robot passes the test trials while the faulty demonstrations still shape its behavior in situations the test did not cover. This is the question our audit addresses: after an edit, has the influence of the withdrawn demonstrations been removed, or only masked? We will add this paragraph to the Introduction.

PART 4

This last part covers the three remaining items, which are point 4, the task families, and the reviewer's question for the authors.

  1. Line 109. Agreed. The sentence will move to the start of Sec. 2.

Experiments ("only one task family"). The paper uses three task families, not one (Table 1):

  • Physical cup insertion: ACT and π0.5\pi_{0.5} on the robot with 200 blind-scored trials; Diffusion Policy offline.
  • PushT in simulation: one harmful and one benign removal request, 250 rollouts per checkpoint.
  • robomimic Can in simulation: 86 edits with 500 rollouts each, 20 freshly retrained policies, 12 controlled reintroductions.

The limitation stated in Sec. 5 is narrower: the hardware experiments cover one task family. We will name the three settings in the first paragraph of Sec. 4 so that this is visible.

Question for the authors (policy randomness versus unlearning). Three things separate them:

  • Paired sampling (1e): both policies receive identical noise, so sampling randomness is common to both and cancels in the comparison.
  • The retrain floor frf_r (1f): it measures how far apart two retrained policies are when nothing was edited, which is the contribution of training randomness, and every edit is read relative to it.
  • The rank test: the edited policy is rejected only if its discrepancy exceeds that of all 19 retrained policies, that is, only if it differs by more than randomness alone makes them differ.

For Diffusion Policy specifically, the gap between the original policy and the floor is not resolved relative to the spread among retrains, so we report its membership result and do not interpret an action gap closure (Sec. 4.3).

We are more than grateful for your constructive feedback for our paper, and will ensure that future versions would include the suggestions on clarity, organization, and presentation the reviewer provided to better deliver the paper's focus and contribution to demonstration unlearning, eventually contributing to the community via a clearer and more knowledgeable approach.

Area_Chair_a5mpOpenReview ↗

Reviewer PRra please respond to author's comments

Reviewer PRra please respond to author's comments

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.