acceptodds
Under review as a conference paper at ICLR 2027

Task Conditions Tell More Than the Task: Dataset-Property Leakage and Private Release in World Action Models

Abstract

Robot policies often compress private demonstrations into reusable task conditions. Even when trajectories remain hidden, a released condition may expose a dataset property such as collection-domain composition. We introduce \method, a release-time framework that measures dataset-property leakage against adaptive attackers and tests whether protected conditions still control robot behavior. Across multiple manipulation tasks, attacks reliably distinguish and estimate demonstration mixtures, and the signal persists across visual representations and an implicit VLA state. Common perturbation and linear erasure defenses either remain vulnerable after attacker retraining or damage control utility. We therefore construct a record-level differentially private release constrained by public task semantics. It reduces adaptive inference to near-chance behavior while preserving closed-loop task performance. Matched-state interventions further show that the protected condition retains task-specific control rather than merely accompanying success from the policy prior. Together, these results identify a privacy boundary in robot task conditioning and show that formal protection can coexist with behavioral utility.

Primary area:
alignment, fairness, safety, privacy, and societal considerations
Keywords:
World Action Models, Private Release, Dataset-Property Leakage

Reviews

Among:
Shape:
1234this paper, Top 4%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.33 over 3 reviews, above 96% of the others.
Reviewer_v3vQOpenReview ↗
Rating
4(+1)
Confidence
3
Informativeness
3
Presentation
3
Soundness
2
Read review
Paper summary

This paper introduces dataset-property leakage problem in aggregated demonstration task conditions of WAM. Experiments show that averaging demonstration features suppresses individual trajectory noise while amplifying broader collection-level distribution. This effect could enable adaptive adversaries to reconstruct private training-set compositions potentially.

Critical strengths
  1. Introduces a novel privacy attack problem in vector from released robotic task representations, showing that task conditions could cause an dataset properties leakage.

  2. Experiment shows that signal amplification under aggregation and demonstrates empirical vulnerabilities against adaptive, retrained attackers across multiple estimators across robot datasets.

  3. A differential privacy guarantee with extensive closed-loop robot rollouts and matched-state causal interventions, verifying that the protected condition retains functional task control.

Critical weaknesses
  1. While the paper conducts offline representation real-robot trajectories from BridgeData V2, the closed-loop policy rollouts are executed in simulation environments.

  2. Evaluating closed-loop execution on a physical manipulator could better understand such a property leakage problem.

Questions for authors

Overall, I think this is a good paper. But I have some questions that I need the author to answer.

  1. Robustness to Reference Anchor Quality: The public semantic ball B(a,C/n)\mathcal{B}(a, C/\sqrt{n}) relies on a reference anchor aa estimated from public data. How sensitive is policy success if there is substantial visual or domain mismatch between the public reference split and the target deployment environment?

  2. Threat Model for Closed-Loop Action Observers: Table 12 demonstrates that while OpenVLA's fused prefix exposes the collection environment (AUC 0.977), the predicted action output drops to near-chance (AUC 0.535). Does this imply that black-box observers monitoring only executed robot joint actions cannot reconstruct collection composition unless intermediate states are exposed?

  3. What is the runtime latency introduced by per-demonstration clipping, calibrated noise injection, and radial projection during online condition compilation?

I will further adjust my rating based on the author's response.

3 comments
AuthorsOpenReview ↗

Reply to Reviewer v3vQ [1/2]

Thank you for your time in reviewing our paper and for your constructive suggestions. We have carefully revised the manuscript and addressed all the raised concerns in our point-by-point response below.

W1 / W2 — Evaluation on real robots

The original Section 5.4, Fig. 7, and Appendix H, Table 12 use BridgeData V2 trajectories collected by real robots. The fused prefix exposes collection information more strongly than the predicted-action summary; the label and session controls do not reproduce that signal. The original results are summarized below. We have now added protection testing on the same real-observation features.

Observable / controlMean task AUCMinimum task AUC
Fused prefix0.9770.961
Action-decision state0.7060.503
Predicted action0.5350.505
Shuffled-label fused0.4990.499
Session-hash control0.4740.409

New real-data protection experiments. We apply the release to analysis aggregates of OpenVLA fused-prefix states extracted from real-robot BridgeData V2 trajectories. Reference, attacker-training, and evaluation roles are disjoint by collection session. All comparisons use the same target aggregates and matched raw baseline.

SettingValue
Real-robot trajectories / tasks / environments600 / 3 / 2
Ref. / attacker-train / eval trajectories (per task & env.)20 / 30 / 50
Original attacker-training pool (per task & env.)50
Target aggregate nn / ε\varepsilon / δ\delta20 / 1 / 10−510^{-5}
Sampling seeds / separately retrained attackers3 / 5
TaskRawBalanced ref. 40Balanced ref. 20Non-kitchen ref. 20Toy-kitchen ref. 20Largest protected seed/attacker AUC
Drawer PnP0.9390.5080.5160.5170.5210.537
Many skills0.8640.5150.5110.5090.5100.538
Stack blocks0.9820.5170.5240.5180.5130.541

The Raw and reference columns report the strongest three-seed mean AUC across attackers. Reference counts are per task; every reference column is a protected release. The final column reports the largest individual protected seed/attacker AUC across all reference choices. Balanced references draw equally from both environments. Reserving reference data reduces the original attacker-training pool, so the raw comparator is recomputed on the matched partition. These results support leakage reduction in real observations across the tested references. The aggregate is an offline analysis adapter, rather than a condition executed by OpenVLA's native controller.

This establishes that the measured protection extends to real robot observations. Physical closed-loop execution remains untested. We distinguish these two forms of evidence rather than equating offline real-data testing with operating a physical manipulator.

AuthorsOpenReview ↗

Reply to Reviewer v3vQ [2/2]

Q1 — Substantial reference mismatch

We tested smaller references, nominal-only and randomized-only references, and half/double projection radii while keeping protected target collections fixed. The privacy grid additionally includes a different-task reference.

Reference/radius settingsTasksSplit seedsStrongest pooled mean AUC rangeLargest seed/attacker AUC
9430.519–0.5220.553

The range summarizes the strongest attacker-family mean AUC in each setting, equally averaging tasks and seeds. The final column retains the largest individual task/seed/attacker value.

New control experiment. We vary the reference or radius while preserving the protected target collections, policy weights, and paired initial states.

TasksMechanism seedsPaired simulator seedsEpisodes / settingNew full-horizon episodes
4310120720

Changes below are relative to the original private release. Intervals use crossed-cluster bootstrap resampling.

Reference / radiusSuccess (%)Change and paired 95% interval (pp)
Original private release86.67Reference
20 reference episodes85.00-1.67 [-10.00, +5.00]
80 reference episodes85.00-1.67 [-8.33, +5.83]
Nominal-only reference88.33+1.67 [-10.00, +14.17]
Randomized-only reference85.83-0.83 [-6.67, +5.00]
Half projection radius82.50-4.17 [-12.50, +2.50]
Double projection radius84.17-2.50 [-9.17, +3.33]

All intervals include zero; this does not establish equivalence or non-inferiority. The domain-only settings also change reference size.

Equal-size reference comparison on real data. We use equally sized references drawn from both collection environments, non-kitchen only, or toy-kitchen only. Protected targets, data roles, and sampling seeds are unchanged; release parameters are recalibrated and attackers retrained.

Ref. traj. / taskTarget nnε\varepsilonδ\deltaSeedsRetrained attackers
2020110−510^{-5}35
Real-data taskBalanced referenceNon-kitchen onlyToy-kitchen onlyLargest seed/attacker AUC
Drawer PnP0.5160.5170.5210.537
Many skills0.5110.5090.5100.535
Stack blocks0.5240.5180.5130.541

These comparisons measure reference sensitivity within the evaluated tasks and collection environments. They do not establish control robustness to arbitrary changes in camera, lighting, embodiment, or task. DP holds for fixed target-independent parameters, but does not itself guarantee that an unsuitable reference preserves task execution.

Q2 — Does weak action-output leakage imply action-only privacy?

No. Appendix H, Table 12 and Fig. 7 show that the observed leakage depends on the exposed interface.

Exposed summaryMean task AUCMinimum task AUC
Fused prefix0.9770.961
Predicted action0.5350.505

The predicted-action result concerns the evaluated summary. It does not cover complete executed joint-action histories, longer state-action sequences, repeated queries, or adaptive interactions.

Q3 — Condition-compilation overhead

We added a CPU microbenchmark of clipping/averaging, noise generation, projection, and complete compilation, using precomputed synthetic features.

Measurement settingValue
InputSynthetic precomputed feature arrays; d=1024d=1024
Implementationfloat64 NumPy 1.26.4
CPU / threadsIntel(R) Xeon(R) Gold 6438Y+ / 1
Warm-up calls per operation200
Measured calls per operation2,000
Timing unitms
Aggregate sizeClipping + meanGaussian noiseProjectionTotal median / p95p_{95}
200.8990.1790.1071.218 / 1.247
401.6070.1800.1071.954 / 1.998
1004.0980.1800.1084.400 / 4.437

This cost is paid once per released aggregate, rather than at every robot action. Clipping/averaging costs O(nd)\mathcal{O}(nd), and noise generation/projection O(d)\mathcal{O}(d). The microbenchmark uses fixed synthetic CC and Gaussian-noise parameters and excludes visual encoding, reference calibration, policy inference, and network transfer. It measures release arithmetic, without claiming end-to-end robot latency or a hard real-time guarantee.

Ref:

[1] Gaussian Differential Privacy. Journal of the Royal Statistical Society: Series B (Statistical Methodology).

[2] The Algorithmic Foundations of Differential Privacy. Foundations and trends in theoretical computer science.

Thank you again for your valuable feedback. Please let us know if any concerns remain.

Thank you for the authors' response. I believe it has addressed my concerns well. After also considering the other reviewers' comments, I have decided to raise my score.

Reviewer_KqpxOpenReview ↗
Rating
3
Confidence
3
Informativeness
3
Presentation
2
Soundness
3
Read review
Paper summary

The paper introduces TASKLEAK, which is designed for dataset-property inference from released robot task conditions from LiLa-WAM. It shows that averaging demonstration transition features makes collection-domain composition identifiable as the aggregate size grows. The authors also evaluate some episode-disjoint attacks and conduct experiments on defense-aware attacks for private release.

Critical strengths
  1. The paper finds dataset-property leakage from robot task conditions and shows that aggregation can amplify this signal from private source demonstrations.

  2. The proposed release combines record-level differential privacy with closed-loop and matched-state evaluations to examine both privacy and task-control utility.

This paper is well-written and presented, with extensive experiments conducted to support its motivation.

Critical weaknesses
  1. I think there is similar success for the public anchor alone and the private release (Tab. 11), leaving it unclear whether the protected condition retains useful information from private demonstrations beyond the public task semantics.

  2. Although the proposed mechanism for protecting private demonstration-derived conditions is relatively simple, the idea itself is interesting and could inspire further research on robot privacy. However, the main experiments are conducted in simulation. While the real-robot data are used only for offline leakage analysis, leaving the defense’s effect on physical robot control untested.

Questions for authors
  1. What useful information from the protected target collection survives beyond the public anchor? Please report paired private-minus-anchor success intervals and test settings where a generic public task prototype is insufficient.

  2. How sensitive are privacy and control to the public reference? Please specify the reference collection’s size and domain composition, and vary its size, visual/domain mismatch, and the projection radius. This can help clarify deployment feasibility and whether the observed utility relies on a particularly favorable task prototype.

4 comments
AuthorsOpenReview ↗

Reply to Reviewer Kqpx [1/3]

We sincerely thank you for your time, effort, and constructive comments. We have updated our paper accordingly and address each of your concerns below.

W1 / Q1 — What survives beyond the public task prototype?

We added the requested paired private-minus-public intervals for every evaluated privacy budget, using the matched rollouts underlying Appendix G, Table 11 and Fig. 5.

SettingValue
Tasks / mechanism seeds / paired simulator seeds4 / 3 / 10
Episodes per condition and privacy budget120
Crossed-cluster bootstrap resamples20,000
Equivalence margin considered (pp)5
ε\varepsilonPrivate success (%)Public success (%)Private−Public\text{Private}-\text{Public} (pp)Paired 95% interval (pp)
0.184.1785.83-1.67[-7.50, +3.33]
0.385.8385.83+0.00[-7.50, +6.67]
186.6785.83+0.83[-5.00, +6.67]
385.8385.83+0.00[-5.00, +5.83]

Intervals resample mechanism and simulator seeds while preserving paired outcomes. They do not establish private-over-public superiority or equivalence within the stated margin.

The condition interventions establish dependence on task semantics.

ConditionSuccesses / episodesSuccess (%)
Raw99 / 12082.50
Private (ε=1\varepsilon=1)104 / 12086.67
Public103 / 12085.83
Wrong task (ε=1\varepsilon=1)13 / 12010.83
Zero51 / 12042.50

They do not establish useful private information beyond the correct anchor. The retained domain-adaptation and object-variant diagnostics likewise fail to demonstrate a setting where the generic prototype is insufficient. We therefore interpret the control result as preserved task execution, alongside suppression of measured dataset-property leakage, rather than evidence of private-information retention. If a public prototype suffices, releasing it directly is a valid option; the current utility results do not establish an added benefit from a private release in that setting.

AuthorsOpenReview ↗

Reply to Reviewer Kqpx [2/3]

Q2 — Reference specification and deployment sensitivity

The original independent references have the following composition.

TaskTotal referenceNominalRandomized
Adjust bottle18017163
Click bell18417167
Click alarmclock18417167
Turn switch17816162

We added smaller references, single-domain references, a different-task reference, and half/double projection radii. After recalibrating release parameters and retraining the attackers, measured leakage remains low across the tested grid.

Reference/radius settingsTasksSplit seedsStrongest pooled mean AUC rangeLargest seed/attacker AUC
9430.519–0.5220.553

The range summarizes the strongest attacker-family mean AUC in each setting, equally averaging tasks and seeds. The final column retains the largest individual task/seed/attacker value.

New control experiment. We vary the reference or radius while preserving the protected target collections, policy weights, and paired initial states.

TasksMechanism seedsPaired simulator seedsEpisodes / settingNew full-horizon episodes
4310120720

Changes below are relative to the original private release. Intervals use crossed-cluster bootstrap resampling.

Reference / radiusSuccess (%)Change and paired 95% interval (pp)
Original private release86.67Reference
20 reference episodes85.00-1.67 [-10.00, +5.00]
80 reference episodes85.00-1.67 [-8.33, +5.83]
Nominal-only reference88.33+1.67 [-10.00, +14.17]
Randomized-only reference85.83-0.83 [-6.67, +5.00]
Half projection radius82.50-4.17 [-12.50, +2.50]
Double projection radius84.17-2.50 [-9.17, +3.33]

All intervals include zero; this does not establish equivalence or non-inferiority. The domain-only settings also change reference size.

The domain-only control comparisons change available reference size as well as domain composition. We therefore added an equal-size comparison on real observations.

Equal-size reference comparison on real data. We use equally sized references drawn from both collection environments, non-kitchen only, or toy-kitchen only. Protected targets, data roles, and sampling seeds are unchanged; release parameters are recalibrated and attackers retrained.

Reference trajectories / taskTarget aggregate nnε\varepsilonδ\deltaSampling seedsRetrained attackers
2020110−510^{-5}35
Real-data taskBalanced referenceNon-kitchen onlyToy-kitchen onlyLargest seed/attacker AUC
Drawer PnP0.5160.5170.5210.537
Many skills0.5110.5090.5100.535
Stack blocks0.5240.5180.5130.541

Reference columns show the strongest three-seed mean AUC across attackers; the final column retains the largest individual seed/attacker value across reference choices. This removes the reference-size confound for the real-data privacy comparison; it does not measure control under reference mismatch.

A different-task reference is included only in the privacy grid. Control under severe semantic mismatch is not established. Deployment requires a target-independent reference with suitable task semantics; estimating it from the protected collection would require additional privacy accounting.

AuthorsOpenReview ↗

Reply to Reviewer Kqpx [3/3]

W2 — Real-world evidence and physical control

The original Section 5.4, Fig. 7, and Appendix H, Table 12 establish collection-property leakage using real-robot BridgeData V2 trajectories, under session-disjoint evaluation. We have now extended this real-data experiment from leakage analysis to protection.

New real-data protection experiments. We apply the release to analysis aggregates of OpenVLA fused-prefix states extracted from real-robot BridgeData V2 trajectories. Reference, attacker-training, and evaluation roles are disjoint by collection session. All comparisons use the same target aggregates and matched raw baseline.

SettingValue
Real-robot trajectories / tasks / collection environments600 / 3 / 2
Reference / attacker-training / evaluation trajectories per task and environment20 / 30 / 50
Original attacker-training pool per task and environment50
Target aggregate nn / ε\varepsilon / δ\delta20 / 1 / 10−510^{-5}
Sampling seeds / separately retrained attackers3 / 5
TaskRawBalanced ref. 40Balanced ref. 20Non-kitchen ref. 20Toy-kitchen ref. 20Largest protected seed/attacker AUC
Drawer PnP0.9390.5080.5160.5170.5210.537
Many skills0.8640.5150.5110.5090.5100.538
Stack blocks0.9820.5170.5240.5180.5130.541

The Raw and reference columns report the strongest three-seed mean AUC across attackers. Reference counts are per task; every reference column is a protected release. The final column reports the largest individual protected seed/attacker AUC across all reference choices. Balanced references draw equally from both environments. Reserving reference data reduces the original attacker-training pool, so the raw comparator is recomputed on the matched partition. These results support leakage reduction in real observations across the tested references. The aggregate is an offline analysis adapter, rather than a condition executed by OpenVLA's native controller.

Thus, the privacy evaluation now includes protected representations derived from real robot data. Joint privacy and task-success evidence remains the LiLa-WAM/RoboTwin experiment in Section 5.3 and Table 2. Physical closed-loop execution of protected conditions has not been tested.

We hope our responses have addressed your concerns, and we would be happy to provide further clarification if any questions remain.

Thanks for the author's detailed reply. I would like to keep my positive rating.

Reviewer_RUD6OpenReview ↗
Rating
3
Confidence
3
Informativeness
3
Presentation
3
Soundness
2
Read review
Paper summary

This paper investigates dataset-property leakage through reusable task-conditioning representations in world action models. The authors show that task conditions derived from private robot demonstrations can reveal aggregate properties of the underlying demonstration datasets, particularly nominal-versus-randomized domain composition. The proposed TASKLEAK framework evaluates property inference under different aggregation sizes, representation interfaces, and adaptive attacker settings. The paper further develops a record-level differentially private release mechanism constrained by public task semantics. Experiments assess privacy leakage, closed-loop task performance, and matched-state behavioral interventions, demonstrating that the protected representation can suppress the evaluated inference attacks while retaining task-dependent control behavior.

Critical strengths

1.The paper identifies a meaningful privacy risk in robot task conditions by demonstrating leakage of aggregate properties from private demonstration datasets. 2.The evaluation combines adaptive privacy attacks, differential privacy, and closed-loop control tests to assess both privacy protection and behavioral utility.

Critical weaknesses

1.The formal privacy guarantee does not fully align with the targeted dataset-property inference threat, since record-level differential privacy does not directly ensure protection of aggregate dataset properties when the compared collections differ in many records. 2.The evaluation of dataset-property leakage is relatively narrow, focusing primarily on nominal-versus-randomized domain proportions without examining more complex or task-correlated properties that may be harder to protect while preserving control utility. 3.Although the paper evaluates public-anchor conditions separately, it does not establish how inaccuracies or distribution shifts in the public semantic anchor affect the privacy-utility trade-off of the proposed projection. 4.The generalizability of the proposed private release mechanism remains uncertain, since its privacy-utility evaluation focuses mainly on LiLa-WAM, despite the paper demonstrating dataset-property leakage across additional representations and architectures.

Questions for authors

1.How sensitive is the privacy-utility trade-off to the quality and distribution of the public task-semantic anchor? 2.Can the proposed private release mechanism provide comparable privacy protection and task utility across other demonstration-conditioned VLA architectures?

5 comments
AuthorsOpenReview ↗

Reply to Reviewer RUD6 [1/5]

Thank you for your time in reviewing our paper and for your constructive suggestions. We have carefully revised the manuscript and addressed all the raised concerns in our point-by-point response below.

W1 — Record-level protection and dataset properties

We agree that record-level DP alone does not guarantee concealment of an arbitrary dataset property. Our claim has two parts. Section 4.3 guarantees record-replacement DP for a released condition. Sections 4.2 and 5.3, Table 2, and Fig. 4 separately evaluate resistance to dataset-property inference using attackers retrained on protected releases. The empirical result is not a consequence of record-level DP alone.

For collections differing in at most kk demonstrations, generic group privacy gives only

$ \left(k\varepsilon,;\delta\sum_{j=0}^{k-1}e^{j\varepsilon}\right)\text{-DP}. $

This guarantee weakens with the number of replacements[2]. Accordingly, we distinguish protection of an individual demonstration from concealment of a collection property. The reference, clipping threshold, and noise calibration must be fixed independently of the protected target collection.

A property-testing bound with explicit assumptions. We added a Gaussian testing argument in Appendix K.1 that relates protection to the number of changed demonstrations. Let nn be the collection size, kk the number of replacements, CC the clipping radius, and σ\sigma the Gaussian standard deviation. With the same target-independent public parameters, the clipped means of two fixed collections are separated by at most 2Ck/n2Ck/n. The optimal equal-prior testing accuracy therefore satisfies

$ \operatorname{Acc}^{\star} \leq \Phi!\left(\frac{Ck}{n\sigma}\right). $

Here Φ\Phi is the standard normal cumulative distribution function. Before projection, equal-covariance Gaussian testing gives this bound[1]; deterministic projection cannot improve the attacker's optimal accuracy.

The same bound applies to two collection-sampling hypotheses if they admit a coupling with at most the stated number of replacements. For our fraction contrast, a common random ordering of the same two fixed source pools gives nested samples with the required marginal distributions. Averaging the paired Gaussian total-variation bounds then bounds the mixture test. This supplies a direct, conditional connection to the evaluated property threat, rather than treating record-level DP as a universal property guarantee.

CalibrationValue
Record-level ε\varepsilon / δ\delta1 / 10−510^{-5}
Aggregate nn for the example mixture contrast40
Example fraction contrast / required replacements0% versus 10% / 4
Replaced records kkEqual-prior accuracy upper bound
10.54110
20.58176
40.66013
80.79549
400.99998

These are accuracy bounds, not AUC bounds, and explicitly illustrate how the formal bound deteriorates to vacuous as k→nk \to n. Thus, our supported conclusion is record-level DP together with measured suppression of the specified property attacks, rather than a universal guarantee for arbitrary collection shifts.

AuthorsOpenReview ↗

Reply to Reviewer RUD6 [2/5]

W2 — Properties beyond collection domain

We added a protection test for left-arm-use composition within randomized-domain episodes in Appendix K.2, Tables 20–21. Unlike the original collection-domain label, this property describes how the robot performs the task. Holding the collection domain fixed tests an operational property without making nominal-versus-randomized membership the prediction target. Raw and protected releases use the same episode-disjoint reference, attacker-training, and evaluation roles, with attackers retrained separately on each release distribution (aggregate n=40n=40, ε=1\varepsilon=1, δ=10−5\delta=10^{-5}, 3 split seeds, 5 retrained attackers).

TaskRaw AUCProtected AUCLargest protected seed/attacker AUC
Adjust bottle0.9990.5140.545
Click bell0.9430.5090.542
Click alarmclock0.8810.5120.547

AUC is the strongest three-seed mean across RF, MLP, ExtraTrees, histogram gradient boosting, and full-dimensional logistic attacks. The final column also retains the largest individual seed/attacker result, so the conclusion does not rely only on averaging. The raw release exposes this operational composition, whereas the protected release substantially reduces the evaluated attackers' discrimination in every tested task.

Closed-loop utility for the additional property. We additionally ran paired, full-horizon RoboTwin episodes using the arm-composition releases above (3 tasks, 3 condition seeds, 10 paired simulator seeds per task, 7 conditions; 630 full-horizon episodes total), fixing the mechanism before observing outcomes. Target collections contain randomized-domain demonstrations only; their arm-use composition varies, while the evaluation scene determines which arm must execute the task. Both original and protected conditions are compared with public, wrong-task, and zero conditions using the same initial states and policy randomness:

ConditionOverall success (%)Left / Right req.Bottle (%)Bell (%)Alarm (%)
Raw, 0% left-arm87.78 (79/90)35/39 / 44/5196.67100.0066.67
Protected, 0% left-arm93.33 (84/90)37/39 / 47/5196.67100.0083.33
Raw, 10% left-arm86.67 (78/90)34/39 / 44/5196.67100.0063.33
Protected, 10% left-arm93.33 (84/90)36/39 / 48/5196.67100.0083.33
Public reference91.11 (82/90)36/39 / 46/5196.67100.0076.67
Wrong-task reference27.78 (25/90)12/39 / 13/510.000.0083.33
Zero51.11 (46/90)21/39 / 25/510.00100.0053.33

Paired differences below use crossed resampling of condition and simulator seeds across the three tasks (retaining dependence from shared initial states, repeated vectors, and common Gaussian draws; single-release DP does not cover jointly publishing correlated alternatives):

ComparatorProtected (0% left-arm) diff [95% CI] (pp)Protected (10% left-arm) diff [95% CI] (pp)
Matched raw release+5.56 [-2.22, +13.33]+6.67 [+0.00, +16.67]
Public reference+2.22 [-2.22, +8.89]+2.22 [-4.44, +10.00]
Wrong-task reference+65.56 [+53.33, +76.67]+65.56 [+53.33, +76.67]
Zero+42.22 [+33.33, +52.22]+42.22 [+33.33, +52.22]

Since intervals vs. raw/public include zero and single-task control sensitivity varies (e.g., Click bell succeeds under zero input), we do not claim superiority over raw/public references or universal single-task condition dependence. These results verify high task utility under suppressed arm-use leakage rather than arm-preference imitation or hiding arbitrary task-essential attributes (see Appendix I for additional single-property tests and W4 for OpenCLAP/native-decoder results).

AuthorsOpenReview ↗

Reply to Reviewer RUD6 [3/5]

W3 / Q1 — Reference quality and distribution

We added reference-sensitivity tests in Appendix K.3 (Table 22) measuring both privacy and closed-loop success, rather than inferring utility from low AUC. The disjoint reference models an independently available non-private source (total/nominal/randomized: 180/17/163 Adjust bottle, 184/17/167 Click bell/alarmclock, 178/16/162 Turn switch); disjointness alone does not make data public, and source availability is a deployment assumption.

Privacy. Varying reference size, domain, task, and projection radius, we recalibrated releases and retrained attackers per setting across all tasks and split seeds (n=40n=40, ε=1\varepsilon=1, δ=10−5\delta=10^{-5}, 3 seeds, 5 attackers):

SettingMean AUCTask maxRun max
Full reference0.5220.5270.552
20 / 40 / 80 episodes0.5220.527–0.5300.552–0.553
Nominal-only0.5220.5280.551
Randomized-only0.5210.5240.551
Different-task0.5190.5290.552
Half / Double radius0.5220.5270.552

Columns show strongest attacker mean (averaged over tasks/seeds), largest task mean, and max single-run AUC (descriptive maxima, not population bounds). For any target-independent reference, record-replacement DP holds (noise recalibrates as clipping sensitivity changes), but DP does not ensure low property-inference AUC or useful control under an inaccurate reference (different-task is privacy-only and implies no task semantics). Half/double-radius AUCs coincide as every perturbed vector lies outside even the larger ball; changing a positive radius rescales releases around the known anchor, which standardization removes (verified via reconstruction). Radius controls proximity to the anchor, while DP comes from clipping and Gaussian noise.

New control experiment. Varying reference or radius with fixed target collections, policy weights, and paired initial states (4 tasks, 3 mechanism seeds, 10 simulator seeds; 120 eps/setting, 720 new eps), changes vs. the original private release use crossed-cluster bootstrap 95% CIs:

SettingPooled (%)Diff [95% CI] (pp)Bottle / Bell / Alarm / Switch (%)
Original private86.67Reference96.67 / 100 / 80.0 / 70.0
20 episodes85.00-1.67 [-10.0, +5.0]96.67 / 100 / 76.67 / 66.67
80 episodes85.00-1.67 [-8.33, +5.83]96.67 / 100 / 73.33 / 70.0
Nominal-only88.33+1.67 [-10.0, +14.17]100 / 100 / 83.33 / 70.0
Randomized-only85.83-0.83 [-6.67, +5.0]96.67 / 100 / 76.67 / 70.0
Half radius82.50-4.17 [-12.5, +2.5]96.67 / 100 / 73.33 / 60.0
Double radius84.17-2.50 [-9.17, +3.33]96.67 / 100 / 76.67 / 63.33

All CIs include zero (not equivalence; domain-only also changes size), leaving uncertainty around pooled changes. Task percentages expose radius sensitivity hidden by pooled means (e.g., Turn switch), supporting measured robustness within tested tasks rather than invariance to arbitrary reference errors.

Equal-size real-data privacy. Using equal-size references from both environments, non-kitchen only, or toy-kitchen only with unchanged targets, splits, and seeds (20 trajs/task, n=20n=20, ε=1\varepsilon=1, δ=10−5\delta=10^{-5}, 3 seeds, 5 attackers; strongest 3-seed mean and max single-run AUC):

Real-data taskBalancedNon-kitchenToy-kitchenRun max
Drawer PnP0.5160.5170.5210.537
Many skills0.5110.5090.5100.535
Stack blocks0.5240.5180.5130.541

This isolates reference composition from size on real data (without testing control mismatch). Overall, these results show empirical robustness across tested reference sizes, domain shifts, and radii, while confirming that an appropriate task reference remains necessary for useful control.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.