acceptodds
Under review as a conference paper at ICLR 2027

DAPI: Dual Adaptive Policy Improvement for Contact-Rich Manipulation

Abstract

Contact-rich manipulation can use action proposals ranging from sampling-based Gaussian proposals to learned Gaussian actors and generative priors. These proposals can all be improved at test time through model-based sampling and evaluation. Their performance can degrade, however, when the physical conditions at deployment differ from those assumed during planning, training, or data collection. Online model identification can mitigate this mismatch, but it creates a dual-control problem. Improving the model requires informative interactions, yet those interactions are selected by a policy that is itself affected by the current model mismatch. A task-driven policy may therefore fail to excite the uncertain parameters, leaving insufficient information for identification. The resulting uncertainty then continues to limit subsequent model-based policy improvement. We present DAPI, which augments task-driven policy improvement with an adaptive information-seeking channel. Among candidate action sequences that remain competitive for the task, DAPI favours actions whose predicted outcomes better distinguish the current physical hypotheses, and automatically removes this preference as parameter uncertainty contracts. A receding-window Stein update uses the resulting interaction data for online physical adaptation. The same candidate-reweighting mechanism applies without retraining to Gaussian sampling, frozen actors, and flow-matching proposals, with a noise-space update for the parameter-conditioned flow. Under substantial model mismatch in planar pushing, the receding-window update reduces median terminal error by 67%, whereas recursive Stein inference does not improve the nominal model. On lifting with an unknown center of mass, Gaussian sampling with online parameter adaptation alone succeeds in 0/12 rod groups and 0/24 plate cases, whereas adding DAPI’s information-seeking channel with the same proposal succeeds in 12/12 and 14/24.

Primary area:
applications to robotics, autonomy, planning
Keywords:
adaptive control, policy improvement, contact-rich manipulation, test-time adaptation, model predictive control

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_z1h8OpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper addresses the issue of contact-rich manipulation policies that perform poorly due to mismatches in physical models. Standard online adaptation techniques often fail because task-driven controllers tend to avoid the interactions needed to identify uncertain parameters. To address this problem, the authors propose Dual Adaptive Policy Improvement (DAPI). This approach enhances traditional task-driven policy improvement by incorporating an adaptive information-seeking component. DAPI assesses potential action sequences and prioritizes those that effectively distinguish among physical hypotheses, provided those actions remain effective for the primary task. Experimental results demonstrate that DAPI reduces the median terminal error by 67% in planar pushing tasks and improves success rates from 0/12 to 12/12 in rod-lifting tasks, where the center of mass is unknown.

Critical strengths
  1. The paper demonstrates both formally and empirically that task-driven adaptive control can bypass the necessary interactions for identifying unknown parameters. In the case of lifting, simply incorporating an information-seeking channel transforms what could be a near-total failure into a consistent success. As a result, the improvement can be clearly attributed to the proposed mechanism.
  2. The proposed method is training-free and works across different action proposals. The same reweighting step enhances Gaussian sampling, a frozen RL actor, and frozen flow-matching policies without retraining. This versatility makes it a valuable plug-in for adapting pretrained policies to account for physical mismatches.
Critical weaknesses
  1. The evaluation lacks depth and is overly simplistic to back up the claims made in the paper. The one more complex task (Rope-U shaping) shows that the method does not work well. While DAPI is presented as a general adaptation mechanism for handling contact-rich manipulation, it is evaluated only on four simulated settings with simplified actuation. There is no robot arm, gripper, or hardware. Most settings have few uncertain parameters, and the lifting tasks are purpose-built to induce the failure DAPI is designed to fix. In the higher-dimensional Rope-U shaping task, the belief hardly contracts, preventing the information-seeking channel from retreating. Consequently, DAPI underperforms compared to both adaptation without it and the true-parameter oracle. Realistic manipulation involves many coupled, partially identifiable parameters, and current evidence indicates that DAPI may worsen performance in this context.
  2. It is unclear whether the improvement from DAPI generalizes, since the true physical parameters vary only slightly. Each task uses only a few hand-picked parameter values rather than randomized ones. The seeds appear to change initialization rather than the underlying physics. As a result, the lifting outcomes rely on one specific adjustment for each task. It's unclear whether DAPI is beneficial when the adjustments are small (where task-focused control might already work well) or large (where trying different lifts may become risky). Without testing a range of parameter values, the evidence doesn’t confirm that DAPI’s advantages apply across all situations it claims to manage.
Questions for authors
  • Have the authors tested DAPI in a more realistic scenario, like a robot arm with a gripper and various uncertain parameters? In the rope task, DAPI underperformed compared to belief-only adaptation. Could a safeguard, such as capping α0\alpha_0 or disabling the information channel when beliefs stop contracting, restore the performance of belief-only adaptation in these cases?
  • Can the authors report results with randomized or swept true parameters? For example, COM offsets ranging from near zero to large in lifting, and mass and friction varied independently in pushing. In particular, does belief-only control already succeed for small offsets, and do DAPI’s exploratory lifts remain safe and efficient for large ones?
  • How does DAPI differ from two closely related prior works that the paper does not discuss?
    • Active Exploration for Robotic Manipulation. IROS 2022. It already combines a task reward with an information-gain term inside online MPC for contact-rich manipulation. How does DAPI’s task-competitive excitation differ from that formulation, and would a simple reward-plus-information baseline perform comparably on the lifting tasks?
    • Parameter-Robust MPPI for Safe Online Learning of Unknown Parameters. RAL 2026. It already couples SVGD-based online parameter learning with MPPI evaluated over parameter hypotheses, which resembles DAPI’s belief-only component. What does the receding-window update add beyond that approach?
8 comments
AuthorsOpenReview ↗

Response to Reviewer z1h8 (1/3): evaluation depth and generalization

Thank you for the pointed questions on realism, generalization and the rope, and for the two references. They led to new experiments (unknown friction, swept offsets, a gripper task and the suggested reward-plus-information baseline) and to a clearer statement of the limitation. We answer each point below.

W1. "The evaluation lacks depth and is too simple for the claims. Rope-U, the one complex task, shows that the method does not work well. There are only four simulated settings with simplified actuation: no robot arm, gripper or hardware. The settings have few uncertain parameters, and the lifting tasks are purpose-built to induce the failure DAPI fixes. In rope-U the belief hardly contracts, so the information channel never retreats, and DAPI underperforms both belief-only and the oracle. Realistic manipulation has many coupled, partially identifiable parameters, and the current evidence suggests DAPI may worsen performance there."

On more parameters and realism. We made friction unknown in addition to the center of mass, with the plant friction changing across seeds (0.4–1.2). Belief-only adaptation then never leaves the supports (rod 0/12, plate 0/24), while DAPI succeeds in 5/12 and 13/24 and recovers the friction in its successful groups (Table 8, Figure 11). We also added a gripper pick (Sec. 5.3, App. E.5): a bar with a hidden load is picked by a gripper whose grip can transmit only a limited torque, so the grasp has to be placed within a few millimetres of the center of mass, and the mass decides how far off it may be. With center of mass and mass both unknown, DAPI succeeds in 29 of 45 cases and belief-only adaptation in 15. Figure 12 and the video show the attempts. [Table 8, p. 24, l. 1275; Fig. 11, p. 25, l. 1321; Sec. 5.3, p. 9, l. 466; App. E.5, p. 26, l. 1361; Fig. 12, p. 26, l. 1389]

On the lifting tasks being purpose-built. We agree that they are constructed on purpose: they isolate a condition known in dual control as the turn-off of cautious control [2]: when the uncertain parameter decides the outcome of the task action, averaging the cost over the belief makes the controller avoid that action, and with it the data that would identify the parameter. The gripper pick with high pad friction removes this condition (high pad friction, where any grasp holds), and both methods then succeed in every case at the first attempt; the gain is tied to the condition, not to the scene. Where task motion is informative by itself, as in pushing, DAPI retreats within a few steps and its median error stays close to or below that of belief-only adaptation (Table 2).

On the number of cases. No result rests on single hand-picked cases; each number aggregates a grid fixed before the runs: 18 PushBox cases (2 motions × 3 mismatch levels × 3 seeds), 27 PushLetter cases (3 letters × 3 mismatch levels × 3 seeds), 12 rod groups and 24 plate direction–seed pairs, together with the offset sweeps, the two-parameter study, 36 pushing cases with independent mass and friction, and 45 gripper cases.

The rope limitation is addressed together with the safeguard question (Q1, cont.).

W2. "Generalization is unclear because the true parameters vary only slightly. Each task uses a few hand-picked values rather than randomized ones, and the seeds seem to change initialization rather than the physics. The lifting outcomes rely on one adjustment per task."

The center-of-mass offset is now swept (rod 1–5 cm, plate 0–5.5 cm in eight directions, Table 7), pushing varies mass and ground friction independently over 36 cases, and in the two-parameter study the seed also sets the plant friction. The particle figures (Figures 1, 4, 5 and 11) and the video show the belief at each attempt, so it can be checked that the same sequence, an exploratory lift followed by contraction and retreat, appears across these cases. [Table 7, p. 24, l. 1257; pushing, p. 24, l. 1294; Figs. 4, 5, 11]

AuthorsOpenReview ↗

Response to Reviewer z1h8 (2/3): rope safeguard and offset sweep

Q1. "Has DAPI been tested in a more realistic scenario (a robot arm with a gripper and various uncertain parameters)?"

The new gripper pick has two unknown parameters that both decide where to grasp (W1). DAPI succeeds in 29 of 45 cases, belief-only adaptation in 15, the true parameters in 15 of 15 and the nominal model in 3 of 15. In 23 of the 30 belief-only failures the bar is never raised by more than 6 mm in four attempts, so the belief cannot contract. DAPI first raises it by a few millimetres, the tilt and the wrist force identify both parameters, and the next grasp is placed at the center of mass. [Sec. 5.3, p. 9, l. 466; App. E.5, p. 26, l. 1361; Table 11, p. 26, l. 1395]

Q1 (cont.). "In rope-U, could a safeguard (capping or disabling the information channel when the belief stops contracting) restore belief-only performance?"

No. We tested this safeguard, which scales the channel weight down once the belief spread stops contracting; on rope-U it does not restore belief-only performance (median terminal error 49.3 mm vs 40.1 mm for belief-only, 24 seeds). We agree that the rope result is a limitation of the present retreat rule, and the revision states it as such. [App. E.3, p. 22, l. 1183]

The reason is the trade-off at the core of dual control. Every action has two effects: it advances the task, and it produces data about the unknown parameters. Seeking information is worth its task cost only while the parameters it reveals still matter for the task, and deciding when to stop is the central difficulty of dual control [1–3]. A spread that no longer contracts does not answer this question. In lifting, the spread stays high because the object has not been lifted yet; the next exploratory lift identifies the center of mass, so the channel must stay open. On the rope, the spread stays high although the rope keeps moving: the nine material parameters are coupled, and some of their combinations predict the same shape for the actions the proposal generates [4], so the belief cannot contract along them, while the predicted terminal shape the task needs is already identified (App. E.3). Here information seeking only costs task performance, and the channel should close. In adaptive-control terms, persistent excitation fails along these combinations, and in our runs no proposed action restored it (App. A). A safeguard that works in both cases therefore has to measure whether the remaining candidates can still excite the directions in which the belief is broad, not the spread itself; we state this as future work (Sec. 6).

[Revision: Sec. 6, p. 10, l. 511; App. A, p. 16, l. 811; App. E.3, p. 22, l. 1187.]

Q2. "Results with randomized or swept true parameters: COM offset from near zero to large in lifting, and mass and friction varied independently in pushing. Does belief-only already succeed for small offsets? Do DAPI's exploratory lifts remain safe and efficient for large ones?"

Yes, belief-only adaptation partly succeeds at small offsets (rod 1 cm: 3/6; plate 1.5 cm: 7/24), where a cautious lift is already almost level.As the offset grows it drops (rod 2, 4, 5 cm: 1/6, 2.3/12, 1/6; plate 3, 4.5, 5.5 cm: 3/24, 0.3/24, 2/24), while DAPI succeeds at every offset (rod 5/6, 5/6, 8.3/12, 4/6; plate 0 cm 7/8, then 24/24, 18/24, 11.7/24, 9/24; default offsets: mean over three seed sets, Table 3). No rod attempt was aborted at any offset. In pushing, with mass (0.31–0.58 kg) and ground friction (0.58–0.90) varied independently over 36 cases, adaptation improves on the nominal model in 30 (belief-only) and 31 (DAPI) cases; the median terminal error falls from 33.6 mm (nominal) to 9.9 mm (belief-only) and 14.2 mm (DAPI), and the worst case from 98.1 mm to 72.4 and 54.2 mm (Table 10). [Table 7, p. 24, l. 1257; p. 24, l. 1294]

AuthorsOpenReview ↗

Response to Reviewer z1h8 (3/3): related active-exploration MPC and references

Q3a. "Active Exploration for Robotic Manipulation (IROS 2022) combines a task reward with an information-gain term in online MPC for contact-rich manipulation. How does task-competitive excitation differ, and would a simple reward-plus-information baseline do comparably on lifting?"

We thank the reviewer for this reference. The suggested baseline does clearly worse on lifting: 2, 6 and 4 of 12 rod groups for bonus weights 0.3, 1 and 3, against 8.3 for DAPI on average (Table 9).

Schneider et al. [5] learn a dynamics model from scratch for a sparse-reward manipulation task, pushing a ball on a tilted table to a target unknown to the agent, and plan with an online MPC whose objective adds an information gain, estimated from an ensemble of learned models, to the task reward. After checking it, we agree that it shares with DAPI the idea of adding information to the task objective of an online MPC, and we now discuss it in the related work. The two differ in what the information is about and in when it is sought. There, the information term drives the exploration of an unknown model over training episodes. DAPI identifies a few physical parameters of a given surrogate within one deployment, rewards information only among candidates that are already competitive for the task, and removes this preference as the belief contracts.

The baseline above tests whether these two restrictions matter: it adds the same information score as a constant bonus to all candidates, without the task-competitive set and without the retreat. An unrestricted bonus also rewards lifts that are informative but bad for the task, and a constant bonus keeps exciting the object after the parameter is identified. [Related work, p. 3, l. 122; Table 9, p. 25, l. 1330]

Q3b. "Parameter-Robust MPPI for Safe Online Learning of Unknown Parameters (RAL 2026) couples SVGD parameter learning with MPPI over hypotheses, which resembles DAPI's belief-only component. What does the receding-window update add?"

The receding-window update changes how the belief is updated, and Table 2 isolates its effect; the information-seeking channel is what DAPI adds beyond PR-MPPI. Both are explained below.

PR-MPPI [6] and our belief-only variant follow the same principle: a particle belief over the parameters, updated by SVGD, is propagated into sampling-based planning. DAPI addresses what this principle leaves open: a task-driven planner may select interactions that do not excite the parameters, so the belief stops improving (Proposition 1). DAPI seeks information among task-competitive candidates and retreats as the belief contracts; with the same proposal and belief update, this raises lifting success from 2.3 to 8.3 of 12 on the rod and from 0.3 to 11.7 of 24 on the plate on average (Tables 3, 7, 8).

The receding-window update concerns how the belief is updated. At every step it re-evaluates the last W executed transitions under every particle, instead of absorbing each transition once when it arrives; because contact switches abruptly between sticking, sliding and separation, the belief follows the contact conditions of the recent interaction. Table 2 isolates this with planner and proposal fixed: the receding window beats the nominal model in 15/18 box and 19/27 letter cases, whereas per-transition Stein inference (DuSt [7]) does so in 6/18 and 3/27. [Related work, p. 3, l. 130; Table 2, p. 9, l. 432]

We hope these answers and the new experiments address your concerns, and we would be glad to discuss any remaining point.

References (z1h8)

[1] A. A. Feldbaum. Dual control theory I. Automation and Remote Control, 1960.

[2] K. J. Åström, A. Helmersson. Dual control of an integrator with unknown gain. Computers & Mathematics with Applications, 1986.

[3] A. Mesbah. Stochastic model predictive control with active uncertainty learning: A survey on dual control. Annual Reviews in Control, 2018.

[4] R. N. Gutenkunst, J. J. Waterfall, F. P. Casey, K. S. Brown, C. R. Myers, J. P. Sethna. Universally sloppy parameter sensitivities in systems biology models. PLoS Computational Biology, 2007.

[5] T. Schneider, B. Belousov, G. Chalvatzaki, D. Romeres, D. K. Jha, J. Peters. Active exploration for robotic manipulation. IROS 2022.

[6] M. Vahs, J. Choi, N. Schmid, J. Tumova, C. Fan. Parameter-Robust MPPI for safe online learning of unknown parameters. IEEE RA-L 2026.

[7] L. Barcelos, A. Lambert, R. Oliveira, P. Borges, B. Boots, F. Ramos. Dual online Stein variational inference for control and dynamics. RSS 2021.

Reviewer_2F3eOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
2
Soundness
3
Read review
Paper summary

The authors present an algorithm that accounts for information seeking during policy rollouts. By introducing and continuously updating a belief over physical parameters, the algorithm reweighs competitive action proposals based on exploration performance to better estimate that belief. The algorithm is evaluated in 5 simulation tasks: planar push-letter tasks (with unknown friction dynamics), lifting tasks (with unknown center of mass), and rope-U shaping task.

Critical strengths

The paper is mostly well-organized and well-written, and it was insightful to read. Although some sentences seemed AI-generated, it felt mostly human-curated and carefully crafted.

The methodology seems thorough and mathematically sound. It is rather long and convoluted, but the authors do a good job of guiding the reader through it.

Critical weaknesses

The paper's biggest weakness is that, because the methodology is complex and hyperparameter-dependent, performance may be brittle. The authors do not provide an extensive ablation study of method components or different hyperparameter choices to guide practitioners and show that the method works beyond the reported use cases. In fact, they provide a negative example (which is appreciated) but with limited insight and guidance on why and how to avoid this failure.

Questions for authors

Major Points

a. The way the method is put into context is too vague in some places. The Related Work section is also hard to follow in parts.

  • Lines 50-53 are very compressed, making it difficult to verify the logical consequence “What remains missing [...]”.
  • What is the difference between fixing model mismatch “online from interaction history (line 100) vs. determining “parameters during execution” (line 102)?
  • The connection between “seeking informative actions” (line 104) and “belief-space methods reasoning about their effect on future uncertainty” is also blurry: is uncertainty used to seek informative actions in these works? If so, instead of “while” say “by using”?
  • Only one reference is given for manipulation tasks, which does not sufficiently cover this very active field of research.
  • Similarly, lines 111-112 are rather vague.
  • Gaussian and actor-based proposals are introduced without reference in lines 123-124, and their connection with diffusion and flow policies is not explained. Also, “optimized in latent or noise space” (line 120) is too vague → optimized for what?
  • To make space for the above improvements, the whole section “Differentiable physical models” is more of an implementation detail than related work and can be moved to the Appendix.

b. Overall, it is not clear which parts of the proposed method use established prior methods and which ones are new. In Section 4, the authors are kindly asked to refer to related and prior work wherever possible.

c. The method relies on many engineering choices.

  • How is F_p, the differentiable surrogate, chosen (line 140)?
  • What are observation scales in (2), and how are they chosen?
  • The way the weighted belief is constructed is very task-dependent, and only described in Appendix B. In lines 192-194, instead of just deferring to the Appendix, could intuition and some guidance for how to construct this be given?
  • How are alpha_0 and tau chosen (see line 414)

d. For the parameter-conditioned flow, is G_\phi the same as for the randomized flow? It seems like that may not be, but the language in 307 (“we preserve the frozen generative structure”) is misleading. However, the parameter c is different in both cases, so it is not clear how that could be.

Minor Suggestions

  • The abstract could be substantially shortened. Lines 15-20 are convoluted and hard to parse. “it creates a dual-control problem” is stated as if that were an issue, but it is not clear why that would be a disadvantage.
  • In line 12, “generative priors” should be augmented by a qualifier to distinguish it from the learned Gaussian actors (e.g., “non-Gaussian”)
  • Line 27: add commas: applies, without retraining, to Gaussian sampling
  • Line 42 is missing a logical ending to the sentence “data collection.” like “and may defer from parameters at deployment time”
  • Figure 1 caption: the qualifier “red” is ambiguous: there are many red elements in the plot.
  • Figure 1: While this is generally a helpful figure, it is hard to understand what the input vs. output of the controller are, and what the task is (this is only explained much later in the paper). Also, it is hard to interpret what the right-hand plot is supposed to show (because sigma_1/sigma_0 are not defined at this point).
  • Inline 151, a_{i,t} is used while later a_i is used. The time subscript is dropped without any mention. Please try to be as consistent as possible with notation and remove the time subscript altogether if it is not necessary.
  • U_t and q_t+1 in line 157 are not properly introduced. It also seems that the introduced symbols T and C are not used later in the paper.
  • The authors distinguish x_k he state and y_k the measurements. Is this distinction helpful/necessary?
  • Line 197: “This reduces the persistent influence [...]” This sentence is not clear. Please consider dropping or expanding the explanation.
  • Equation (4): It would help to add “n=1, …, K” as a reminder.
  • Equation (5): S is not defined, and the notation w^T may be misleading as it looks like a transpose. Also, what is rho and how is it chosen?
  • Line 219: It seems like V is not defined the first time it is used here, only later in Proposition 1. Throughout this paragraph, it would be helpful to link it to Et\Epsilon_t, which is also only defined later in Proposition 1.
  • Similarly, in lines 223-225, this description presumes that the reader knows about the replanning step, but it seems like the whole pipeline of the methodology has not been explained yet?
  • The implications of the last part of Proposition 1 (After “Moreover, if the belief-averaged [...]”) are not sufficiently discussed.
  • The symbol k_\tau in line 256 may be confused with the kernel k. Is it really necessary to introduce a symbol for this?
  • The introduction of 4.4 is not helpful. What “proposal classes” are we talking about? These two sentences sound very stereotypically LLM-generated.
  • sigma_\xi in line 313 is not defined.
  • “Pushing” and “lifting” in the sentence on lines 364- 366 need to be defined.
  • Line 371: Letters-Planar-Pushing geometry -> benchmark? on a MuJoCo Plant -> in a MuJoCo simulation?
  • Line 407: “The gain over the nominal model [...]”: The gain by which method?
7 comments
AuthorsOpenReview ↗

Response to Reviewer 2F3e (1/5): hyperparameters and the rope failure

Thank you for the very careful, line-by-line reading and for your encouraging assessment. Your detailed comments improved Sections 2–4 throughout. We address each of them below and give the location of every change.

W1. "The method is complex and hyperparameter-dependent, so performance may be brittle. There is no extensive ablation of components or hyperparameters to guide practitioners or to show that the method works beyond the reported cases."

We agree that the dependence on hyperparameters has to be shown. The two hyperparameters of the information-seeking channel, α0 and τ, take the same values in all four tasks (α0 = 1, τ = 0.1; Table 4); they are not tuned per task. α0 = 1 gives the channel its full weight while the belief is still at its prior spread, and τ = 0.1 keeps it within the best tenth of the candidates. We varied them one at a time on the rod (12 groups) and on the plate (24 direction–seed pairs), Table 9. Success grows with α0: on the rod, α0 = 0 (belief-only), 0.25, 0.5 and 1 give 2.3 ± 1.5, 4, 4 and 8.3 ± 1.5 of 12 (belief-only and default: mean ± std over three seed sets, Table 3); on the plate, α0 = 0, 0.5 and 1 give 0.3 ± 0.6, 4 and 11.7 ± 5.1 of 24. For τ = 0.05, 0.3 and 0.5 the rod gives 6, 5 and 6; for τ = 0.05 and 0.3 the plate gives 5 and 14. Every setting is better than belief-only adaptation; the shared defaults are the best on the rod, and on the plate τ = 0.3 performs comparably.Removing the information score while keeping the task-competitive set and the retreat gives 3 of 12 rod groups, close to belief-only adaptation (2.3 of 12 on average), so the gain comes from the information score and not from restricting the candidates.

For the components, the reward-plus-information baseline in the same table removes both the task-competitive set and the retreat and adds a constant information bonus to all candidates. It reaches at most 6/12, because it also rewards informative lifts that are poor for the task and keeps exciting the object after the parameter is identified.

[Revision: Table 9, p. 25, l. 1330; Table 4, p. 16, l. 831; pointer in Sec. 5.3, p. 8, l. 419.]

W2. "The negative example is appreciated, but it gives limited insight into why the failure happens and how to avoid it."

Thank you for appreciating the negative example. The failure has a precise cause: persistent excitation, the classical condition for parameter convergence in adaptive control [15]: over each window, the interaction has to excite every direction of the parameter space, including combinations of parameters, and the belief contracts only along the directions that are excited. Since the information-seeking weight follows the belief spread, the channel closes only when every direction is excited.

The three tasks differ in how this condition is met. In pushing, the moving object excites all directions and the channel closes within a few steps. In lifting, the task-driven actions keep the object on its supports and excite nothing (Proposition 1); DAPI restores the excitation with an exploratory lift. On the rope, the nine material parameters are coupled, and some of their combinations predict the same shape for the actions the proposal generates [16]. Reweighting only shifts weight among the proposed actions, so it restores excitation only if some proposed action excites these combinations (App. A); on the rope we observed none, so the spread stays high and the channel stays open, although the predicted terminal shape, which is what the task needs, is identified (its spread under the particles falls from 60 to 21 mm, App. E.3). The spread cannot distinguish this case from lifting, where it is high only because the object has not been lifted yet.

To avoid the failure, the retreat has to measure whether the remaining candidates can still excite the directions in which the belief is broad, rather than the spread itself. We state this as the limitation of the present rule and as future work (Sec. 6); App. A gives the short derivation.

[Revision: Sec. 6, p. 10, l. 511; App. A, p. 16, l. 811; rope, App. E.3, p. 22, l. 1187.]

AuthorsOpenReview ↗

Response to Reviewer 2F3e (2/5): related work and contributions

(a) "Context is vague; Related Work is hard to follow."

We agree that the related work was too compressed. Each point is addressed below.

  • "Lines 50–53 are too compressed ('What remains missing')."

    The sentence now states what is missing and why: deployment-time adaptation of pretrained proposals that identifies the physical parameters online. Because the identification data come from the controller itself, its task actions should be informative while the uncertainty limits the task, and serve the task alone once it no longer does. [p. 1, l. 53]

  • "Line 100 vs line 102: what is the difference between fixing mismatch online from interaction history and determining parameters during execution?"

    Both use deployment data, but they estimate different things. Adaptation modules [1, 2] infer a latent context from recent history and condition the policy on it, without recovering physical parameters. Indirect adaptive control estimates the physical parameters themselves and controls with the adapted model. The paragraph now states this difference. [p. 3, l. 113]

  • "Line 104: the link between seeking informative actions and belief-space methods is blurry ('while' vs 'by using')."

    Both obtain informative actions, in different ways. Practical dual-control methods add an information term to the control objective, so information seeking is explicit [3, 4]. Belief-space methods plan over how the belief will evolve under each action, so information seeking appears implicitly [5, 6]. The sentence now reads "either explicitly … or implicitly …". [p. 3, l. 118]

  • "Only one manipulation reference is given."

    We added a survey of inertial-parameter estimation in manipulation [7] and Schneider et al. [8], who add an information-gain term to the reward of an online MPC. [p. 3, l. 122]

  • "Lines 111–112 are vague."

    Each method is now described by what it does: ensemble path-integral control averages the cost over sampled parameters [9], DuSt-MPC updates a particle belief once per new transition [10], and Parameter-Robust MPPI couples SVGD learning with MPPI [11]. These methods correspond to our belief-only baseline. [p. 3, l. 130]

  • "Lines 123–124: Gaussian and actor proposals are given without references, and their link to diffusion and flow policies is not explained. Line 120: 'optimized in latent/noise space' — optimized for what?"

    Sampling-based control draws candidates from a Gaussian proposal centred on the previous plan [12] or on the output of a learned Gaussian actor [13]. Diffusion and flow policies replace this Gaussian by a learned, possibly multimodal generator. Wagenmaker et al. [14] optimize the initial noise of such a policy for a task reward. In DAPI the flow also stays frozen, and the initial noise of the conditioned flow is updated with the same task and information weights as the Gaussian proposals. [p. 3, l. 139]

  • "Move 'Differentiable physical models' to the appendix."

    Moved to App. C. [p. 17, l. 875]

(b) "Unclear which parts are prior work and which are new; add citations throughout Section 4."

We agree that this separation should be explicit. The prior components are cited where they are used in Section 4: Stein variational gradient descent [17] in Eq. (3), the exponentiated reweighting of sampled candidates [12] in Eq. (5), the information proxy [18] in Eq. (7), and noise-space steering [14] in Sec. 4.4. The new elements are those listed in the contribution paragraph of the introduction: the receding-window belief update, the information-seeking channel restricted to task-competitive candidates with its adaptive retreat, and the noise-space update of the parameter-conditioned flow. [p. 2, l. 99]

AuthorsOpenReview ↗

Response to Reviewer 2F3e (3/5): method details

(c) "Many engineering choices need explanation: how F_p is chosen (line 140); the observation scales in (2) and how they are chosen; how the weighted belief is constructed (it is task-dependent and only in Appendix B, so give intuition at lines 192–194); how alpha_0 and tau are chosen (line 414)."

  • F_p. F_p only needs to be differentiable in θ and fast to evaluate for many sampled candidates. On a physical robot, f* is the real system, and F_p can be a differentiable simulator such as MJX, a learned dynamics model, or a smooth contact model; we use ContactSDF [19] for fast computation. [Sec. 3, p. 3, l. 158; Sec. 5.1, p. 8, l. 395]
  • Observation scales in Eq. (2). They are fixed per task at the order of the measurement resolution and the task tolerance (Table 4), are shared by the window energy (2) and the information score (7), and are the same for all compared methods. [p. 16, l. 846]
  • Weighted belief. All tasks use the same Stein transport toward the window-conditioned target; they differ only in the particle weights a_j. Recursively updated weights keep evidence from earlier windows; they are used where the informative interaction is short and intermittent (lifting), and equal weights where the recent window is informative by itself (pushing, rope). Sec. 4.1 now introduces a_j together with the other terms of Eq. (3), and App. B states this rule. [p. 4, l. 210; App. B, p. 16, l. 851]
  • α0 and τ. The same values in all tasks; see W1.

(d) "Is G_phi the same for the parameter-conditioned and the randomized flow? 'We preserve the frozen generative structure' (line 307) is misleading, since c differs."

The randomized and the parameter-conditioned flow are two separately trained networks. The conditioned flow additionally receives a parameter hypothesis in its condition, so the same initial noise is mapped to an action sequence suited to that hypothesis. The misleading sentence is replaced: its weights stay frozen and only its initial noise is updated. [p. 7, l. 357]

Reviewer_T6voOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This work demonstrates a framework that performs online system identification while manipulating objects with unknown physical parameters. It maintains a particle-based belief over these physical parameters, updates it using recent observations, and favors task-competitive actions whose predicted outcomes distinguish between competing physical hypotheses. When uncertainty decreases, it gradually reduces information-seeking behavior and shifts toward task completion, without retraining the underlying policy.

Critical strengths

The paper is addressing a meaningful problem - doing online sysid during manipulation. The solution, as shown in Algorithm 1, seems simple and elegant.

Critical weaknesses
  1. Presentation, especially image quality is an issue - Figure 1 and 7 are hard to understand because of the low resolution.

  2. The four tasks are too simple. A thorough study should have more tasks with higher complexity. In section 6 the authors discuss the advantages and limitation of DAPI v.s. baseline, based on only one task each. I would suggest more experimentations for a more solid conclusion.

Questions for authors

What does a successful policy look like? Would be nice to attach video demonstrations of these dynamic process so that readers can intuitively understand what's happening in the simulations.

2 comments
AuthorsOpenReview ↗

Response to Reviewer T6vo (1/2): additional evidence, figures and video

Thank you for your review, and in particular for asking for clearer figures and for videos. Both requests improved the paper: the redrawn figures and the supplementary video now show what happens directly. We answer each point below.

W1. "Presentation, especially image quality is an issue - Figure 1 and 7 are hard to understand because of the low resolution."

We agree that both figures were hard to read, and both are redrawn. Figure 1 now shows the rod lift as a few frames per attempt, with the particle belief at the start and the end of each attempt below each frame and the remaining uncertainty (belief spread relative to its initial value) on the right, so the failure of belief-only adaptation and the retreat of DAPI can be read directly. Figure 7 is re-rendered at full resolution. In the same style we redrew the task overview (Figure 2) and the other appendix figures (Figures 4, 5 and 8–12), including pushing. [Revision: Fig. 1, p. 2, l. 75; Fig. 2, p. 7, l. 353; Fig. 7, p. 21, l. 1120; Figs. 4, 5, 8–12 in App. E.]

W2. "The four tasks are too simple. A thorough study should have more tasks with higher complexity. In section 6 the authors discuss the advantages and limitation of DAPI v.s. baseline, based on only one task each. I would suggest more experimentations for a more solid conclusion."

We agree that a conclusion resting on a single task would be weak. The revision therefore adds a new task and more variation (listed below), and we also explain why the tasks are demanding in the sense that matters here.

Planar pushing and object lifting are the standard testbeds for contact-rich control and online adaptation [1–3]. The closest adaptive sampling-based controllers are evaluated without contact: DuSt-MPC [4] on an inverted pendulum, point-mass navigation and a ground vehicle, and Parameter-Robust MPPI [5] on a cart-pole and quadrotors. Each task isolates one condition of the same adaptive loop. In pushing, the task motion itself moves the object and excites the parameters. In lifting, the task-driven controller avoids the lift that would reveal the center of mass, so the parameter stays hidden. On the rope, the parameters are coupled, and some of their combinations are not excited by any proposed action. What makes the tasks hard is therefore not the scene, but whether the motion the controller produces excites the task-relevant parameters.

The first two conditions do not rest on one task each. Informative task motion is tested on a box and three letter shapes, and the non-exciting case on a rod and a plate with four proposal types, and now also on the gripper pick. The revision adds more variation and a new task:

  • Center-of-mass offset swept from 1 to 5 cm (rod) and from 0 to 5.5 cm (plate). Belief-only adaptation works only at the smallest offsets, while DAPI succeeds at every offset (Table 7).
  • Friction unknown in addition to the center of mass: belief-only adaptation never lifts the rod or the plate (0/12, 0/24), DAPI succeeds in 5/12 and 13/24 (Table 8, Figure 11).
  • Pushing with mass and ground friction varied independently over 36 cases: both adaptive variants improve on the nominal model in 30–31 cases (Table 10).
  • A gripper pick with unknown center of mass and mass (Sec. 5.3, App. E.5): DAPI 29/45, belief-only 15/45, with fewer attempts (147 vs 165).

[Revision: Fig. 2 (all tasks), p. 7, l. 353; gripper, Sec. 5.3, p. 9, l. 466 and App. E.5, p. 26, l. 1361; Table 7, p. 24, l. 1257; Table 8, p. 24, l. 1275; pushing, p. 24, l. 1294.]

Q. "What does a successful policy look like? Attach videos of the dynamic process so readers can understand intuitively what happens in the simulations."

The supplementary video shows, for each task, the scene, the particle belief and the information-seeking weight side by side: PushBox (nominal model, DuSt, DuSt with Stein MPC, receding window, DAPI), PushLetter, rod and plate lifting (belief-only vs DAPI), the rod with unknown center of mass and friction, the gripper pick, and the rope. In a successful lift, DAPI first makes one or two short lifts that tilt the object; the particles then collect around the true center of mass, the information-seeking weight falls, and the next lift is level. Belief-only adaptation keeps the bar under the object without lifting, so its particles never move.

We hope that the redrawn figures, the video and the additional experiments address your concerns. We are happy to clarify anything further.

AuthorsOpenReview ↗

Response to Reviewer T6vo (2/2): references

References (T6vo)

[1] K.-T. Yu, M. Bauza, N. Fazeli, A. Rodriguez. More than a million ways to be pushed: A high-fidelity experimental dataset of planar pushing. IROS 2016.

[2] F. R. Hogan, A. Rodriguez. Feedback control of the pusher-slider system: A story of hybrid and underactuated contact dynamics. WAFR 2016.

[3] M. Bauza, A. Rodriguez. A probabilistic data-driven model for planar pushing. ICRA 2017.

[4] L. Barcelos, A. Lambert, R. Oliveira, P. Borges, B. Boots, F. Ramos. Dual online Stein variational inference for control and dynamics. RSS 2021.

[5] M. Vahs, J. Choi, N. Schmid, J. Tumova, C. Fan. Parameter-Robust MPPI for safe online learning of unknown parameters. IEEE RA-L 2026.

Comments on the paper

1 comment
AuthorsOpenReview ↗

General response: new evidence, component ablation, and the rope limitation

We thank the reviewers for their careful reading. The reviews raise three groups of questions: whether the evidence goes beyond a few simple tasks (T6vo, z1h8), how the method depends on its components and hyperparameters (2F3e), and why DAPI does not help on the rope (2F3e, z1h8). The revision answers them as follows.

  1. More evidence. We swept the center-of-mass offset on rod and plate, made friction a second unknown parameter, varied mass and ground friction independently in pushing, and added a gripper pick in which both the center of mass and the mass of the object are unknown (Sec. 5.3, App. E.4–E.5). In every new lifting and gripper setting the information-seeking channel keeps its advantage over belief-only adaptation, and in pushing both adaptive variants improve on the nominal model.
  2. Hyperparameters and components. A one-factor study of α0 and τ, a reward-plus-information baseline that removes the task-competitive set and the retreat, and an ablation that keeps the task-competitive set but removes the information score (3 of 12 rod groups, against 8.3 for DAPI and 2.3 for belief-only adaptation on average; App. E.4, Table 9).
  3. Why the rope is different. The rope is explained by persistent excitation, the classical condition for parameter convergence in adaptive control: its coupled parameters leave combinations that the proposed actions do not excite in our runs, so the spread does not contract along them (Sec. 6, App. A).
  4. Presentation. Figures 1 and 2 and the appendix figures are redrawn (frames per attempt together with the particle belief), Sections 2–4 are revised following 2F3e's detailed comments, and a supplementary video shows the scene, the particle belief and the information-seeking weight together for every task.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.