- Rating
- 2
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper addresses the issue of contact-rich manipulation policies that perform poorly due to mismatches in physical models. Standard online adaptation techniques often fail because task-driven controllers tend to avoid the interactions needed to identify uncertain parameters. To address this problem, the authors propose Dual Adaptive Policy Improvement (DAPI). This approach enhances traditional task-driven policy improvement by incorporating an adaptive information-seeking component. DAPI assesses potential action sequences and prioritizes those that effectively distinguish among physical hypotheses, provided those actions remain effective for the primary task. Experimental results demonstrate that DAPI reduces the median terminal error by 67% in planar pushing tasks and improves success rates from 0/12 to 12/12 in rod-lifting tasks, where the center of mass is unknown.
Critical strengths
- The paper demonstrates both formally and empirically that task-driven adaptive control can bypass the necessary interactions for identifying unknown parameters. In the case of lifting, simply incorporating an information-seeking channel transforms what could be a near-total failure into a consistent success. As a result, the improvement can be clearly attributed to the proposed mechanism.
- The proposed method is training-free and works across different action proposals. The same reweighting step enhances Gaussian sampling, a frozen RL actor, and frozen flow-matching policies without retraining. This versatility makes it a valuable plug-in for adapting pretrained policies to account for physical mismatches.
Critical weaknesses
- The evaluation lacks depth and is overly simplistic to back up the claims made in the paper. The one more complex task (Rope-U shaping) shows that the method does not work well. While DAPI is presented as a general adaptation mechanism for handling contact-rich manipulation, it is evaluated only on four simulated settings with simplified actuation. There is no robot arm, gripper, or hardware. Most settings have few uncertain parameters, and the lifting tasks are purpose-built to induce the failure DAPI is designed to fix. In the higher-dimensional Rope-U shaping task, the belief hardly contracts, preventing the information-seeking channel from retreating. Consequently, DAPI underperforms compared to both adaptation without it and the true-parameter oracle. Realistic manipulation involves many coupled, partially identifiable parameters, and current evidence indicates that DAPI may worsen performance in this context.
- It is unclear whether the improvement from DAPI generalizes, since the true physical parameters vary only slightly. Each task uses only a few hand-picked parameter values rather than randomized ones. The seeds appear to change initialization rather than the underlying physics. As a result, the lifting outcomes rely on one specific adjustment for each task. It's unclear whether DAPI is beneficial when the adjustments are small (where task-focused control might already work well) or large (where trying different lifts may become risky). Without testing a range of parameter values, the evidence doesn’t confirm that DAPI’s advantages apply across all situations it claims to manage.
Questions for authors
- Have the authors tested DAPI in a more realistic scenario, like a robot arm with a gripper and various uncertain parameters? In the rope task, DAPI underperformed compared to belief-only adaptation. Could a safeguard, such as capping or disabling the information channel when beliefs stop contracting, restore the performance of belief-only adaptation in these cases?
- Can the authors report results with randomized or swept true parameters? For example, COM offsets ranging from near zero to large in lifting, and mass and friction varied independently in pushing. In particular, does belief-only control already succeed for small offsets, and do DAPI’s exploratory lifts remain safe and efficient for large ones?
- How does DAPI differ from two closely related prior works that the paper does not discuss?
- Active Exploration for Robotic Manipulation. IROS 2022. It already combines a task reward with an information-gain term inside online MPC for contact-rich manipulation. How does DAPI’s task-competitive excitation differ from that formulation, and would a simple reward-plus-information baseline perform comparably on the lifting tasks?
- Parameter-Robust MPPI for Safe Online Learning of Unknown Parameters. RAL 2026. It already couples SVGD-based online parameter learning with MPPI evaluated over parameter hypotheses, which resembles DAPI’s belief-only component. What does the receding-window update add beyond that approach?
8 comments
Response to Reviewer z1h8 (2/3): rope safeguard and offset sweep
Q1. "Has DAPI been tested in a more realistic scenario (a robot arm with a gripper and various uncertain parameters)?"
The new gripper pick has two unknown parameters that both decide where to grasp (W1). DAPI succeeds in 29 of 45 cases, belief-only adaptation in 15, the true parameters in 15 of 15 and the nominal model in 3 of 15. In 23 of the 30 belief-only failures the bar is never raised by more than 6 mm in four attempts, so the belief cannot contract. DAPI first raises it by a few millimetres, the tilt and the wrist force identify both parameters, and the next grasp is placed at the center of mass. [Sec. 5.3, p. 9, l. 466; App. E.5, p. 26, l. 1361; Table 11, p. 26, l. 1395]
Q1 (cont.). "In rope-U, could a safeguard (capping or disabling the information channel when the belief stops contracting) restore belief-only performance?"
No. We tested this safeguard, which scales the channel weight down once the belief spread stops contracting; on rope-U it does not restore belief-only performance (median terminal error 49.3 mm vs 40.1 mm for belief-only, 24 seeds). We agree that the rope result is a limitation of the present retreat rule, and the revision states it as such. [App. E.3, p. 22, l. 1183]
The reason is the trade-off at the core of dual control. Every action has two effects: it advances the task, and it produces data about the unknown parameters. Seeking information is worth its task cost only while the parameters it reveals still matter for the task, and deciding when to stop is the central difficulty of dual control [1–3]. A spread that no longer contracts does not answer this question. In lifting, the spread stays high because the object has not been lifted yet; the next exploratory lift identifies the center of mass, so the channel must stay open. On the rope, the spread stays high although the rope keeps moving: the nine material parameters are coupled, and some of their combinations predict the same shape for the actions the proposal generates [4], so the belief cannot contract along them, while the predicted terminal shape the task needs is already identified (App. E.3). Here information seeking only costs task performance, and the channel should close. In adaptive-control terms, persistent excitation fails along these combinations, and in our runs no proposed action restored it (App. A). A safeguard that works in both cases therefore has to measure whether the remaining candidates can still excite the directions in which the belief is broad, not the spread itself; we state this as future work (Sec. 6).
[Revision: Sec. 6, p. 10, l. 511; App. A, p. 16, l. 811; App. E.3, p. 22, l. 1187.]
Q2. "Results with randomized or swept true parameters: COM offset from near zero to large in lifting, and mass and friction varied independently in pushing. Does belief-only already succeed for small offsets? Do DAPI's exploratory lifts remain safe and efficient for large ones?"
Yes, belief-only adaptation partly succeeds at small offsets (rod 1 cm: 3/6; plate 1.5 cm: 7/24), where a cautious lift is already almost level.As the offset grows it drops (rod 2, 4, 5 cm: 1/6, 2.3/12, 1/6; plate 3, 4.5, 5.5 cm: 3/24, 0.3/24, 2/24), while DAPI succeeds at every offset (rod 5/6, 5/6, 8.3/12, 4/6; plate 0 cm 7/8, then 24/24, 18/24, 11.7/24, 9/24; default offsets: mean over three seed sets, Table 3). No rod attempt was aborted at any offset. In pushing, with mass (0.31–0.58 kg) and ground friction (0.58–0.90) varied independently over 36 cases, adaptation improves on the nominal model in 30 (belief-only) and 31 (DAPI) cases; the median terminal error falls from 33.6 mm (nominal) to 9.9 mm (belief-only) and 14.2 mm (DAPI), and the worst case from 98.1 mm to 72.4 and 54.2 mm (Table 10). [Table 7, p. 24, l. 1257; p. 24, l. 1294]
Response to Reviewer z1h8 (3/3): related active-exploration MPC and references
Q3a. "Active Exploration for Robotic Manipulation (IROS 2022) combines a task reward with an information-gain term in online MPC for contact-rich manipulation. How does task-competitive excitation differ, and would a simple reward-plus-information baseline do comparably on lifting?"
We thank the reviewer for this reference. The suggested baseline does clearly worse on lifting: 2, 6 and 4 of 12 rod groups for bonus weights 0.3, 1 and 3, against 8.3 for DAPI on average (Table 9).
Schneider et al. [5] learn a dynamics model from scratch for a sparse-reward manipulation task, pushing a ball on a tilted table to a target unknown to the agent, and plan with an online MPC whose objective adds an information gain, estimated from an ensemble of learned models, to the task reward. After checking it, we agree that it shares with DAPI the idea of adding information to the task objective of an online MPC, and we now discuss it in the related work. The two differ in what the information is about and in when it is sought. There, the information term drives the exploration of an unknown model over training episodes. DAPI identifies a few physical parameters of a given surrogate within one deployment, rewards information only among candidates that are already competitive for the task, and removes this preference as the belief contracts.
The baseline above tests whether these two restrictions matter: it adds the same information score as a constant bonus to all candidates, without the task-competitive set and without the retreat. An unrestricted bonus also rewards lifts that are informative but bad for the task, and a constant bonus keeps exciting the object after the parameter is identified. [Related work, p. 3, l. 122; Table 9, p. 25, l. 1330]
Q3b. "Parameter-Robust MPPI for Safe Online Learning of Unknown Parameters (RAL 2026) couples SVGD parameter learning with MPPI over hypotheses, which resembles DAPI's belief-only component. What does the receding-window update add?"
The receding-window update changes how the belief is updated, and Table 2 isolates its effect; the information-seeking channel is what DAPI adds beyond PR-MPPI. Both are explained below.
PR-MPPI [6] and our belief-only variant follow the same principle: a particle belief over the parameters, updated by SVGD, is propagated into sampling-based planning. DAPI addresses what this principle leaves open: a task-driven planner may select interactions that do not excite the parameters, so the belief stops improving (Proposition 1). DAPI seeks information among task-competitive candidates and retreats as the belief contracts; with the same proposal and belief update, this raises lifting success from 2.3 to 8.3 of 12 on the rod and from 0.3 to 11.7 of 24 on the plate on average (Tables 3, 7, 8).
The receding-window update concerns how the belief is updated. At every step it re-evaluates the last W executed transitions under every particle, instead of absorbing each transition once when it arrives; because contact switches abruptly between sticking, sliding and separation, the belief follows the contact conditions of the recent interaction. Table 2 isolates this with planner and proposal fixed: the receding window beats the nominal model in 15/18 box and 19/27 letter cases, whereas per-transition Stein inference (DuSt [7]) does so in 6/18 and 3/27. [Related work, p. 3, l. 130; Table 2, p. 9, l. 432]
We hope these answers and the new experiments address your concerns, and we would be glad to discuss any remaining point.
References (z1h8)
[1] A. A. Feldbaum. Dual control theory I. Automation and Remote Control, 1960.
[2] K. J. Åström, A. Helmersson. Dual control of an integrator with unknown gain. Computers & Mathematics with Applications, 1986.
[3] A. Mesbah. Stochastic model predictive control with active uncertainty learning: A survey on dual control. Annual Reviews in Control, 2018.
[4] R. N. Gutenkunst, J. J. Waterfall, F. P. Casey, K. S. Brown, C. R. Myers, J. P. Sethna. Universally sloppy parameter sensitivities in systems biology models. PLoS Computational Biology, 2007.
[5] T. Schneider, B. Belousov, G. Chalvatzaki, D. Romeres, D. K. Jha, J. Peters. Active exploration for robotic manipulation. IROS 2022.
[6] M. Vahs, J. Choi, N. Schmid, J. Tumova, C. Fan. Parameter-Robust MPPI for safe online learning of unknown parameters. IEEE RA-L 2026.
[7] L. Barcelos, A. Lambert, R. Oliveira, P. Borges, B. Boots, F. Ramos. Dual online Stein variational inference for control and dynamics. RSS 2021.
Response to Reviewer z1h8 (1/3): evaluation depth and generalization
Thank you for the pointed questions on realism, generalization and the rope, and for the two references. They led to new experiments (unknown friction, swept offsets, a gripper task and the suggested reward-plus-information baseline) and to a clearer statement of the limitation. We answer each point below.
On more parameters and realism. We made friction unknown in addition to the center of mass, with the plant friction changing across seeds (0.4–1.2). Belief-only adaptation then never leaves the supports (rod 0/12, plate 0/24), while DAPI succeeds in 5/12 and 13/24 and recovers the friction in its successful groups (Table 8, Figure 11). We also added a gripper pick (Sec. 5.3, App. E.5): a bar with a hidden load is picked by a gripper whose grip can transmit only a limited torque, so the grasp has to be placed within a few millimetres of the center of mass, and the mass decides how far off it may be. With center of mass and mass both unknown, DAPI succeeds in 29 of 45 cases and belief-only adaptation in 15. Figure 12 and the video show the attempts. [Table 8, p. 24, l. 1275; Fig. 11, p. 25, l. 1321; Sec. 5.3, p. 9, l. 466; App. E.5, p. 26, l. 1361; Fig. 12, p. 26, l. 1389]
On the lifting tasks being purpose-built. We agree that they are constructed on purpose: they isolate a condition known in dual control as the turn-off of cautious control [2]: when the uncertain parameter decides the outcome of the task action, averaging the cost over the belief makes the controller avoid that action, and with it the data that would identify the parameter. The gripper pick with high pad friction removes this condition (high pad friction, where any grasp holds), and both methods then succeed in every case at the first attempt; the gain is tied to the condition, not to the scene. Where task motion is informative by itself, as in pushing, DAPI retreats within a few steps and its median error stays close to or below that of belief-only adaptation (Table 2).
On the number of cases. No result rests on single hand-picked cases; each number aggregates a grid fixed before the runs: 18 PushBox cases (2 motions × 3 mismatch levels × 3 seeds), 27 PushLetter cases (3 letters × 3 mismatch levels × 3 seeds), 12 rod groups and 24 plate direction–seed pairs, together with the offset sweeps, the two-parameter study, 36 pushing cases with independent mass and friction, and 45 gripper cases.
The rope limitation is addressed together with the safeguard question (Q1, cont.).
The center-of-mass offset is now swept (rod 1–5 cm, plate 0–5.5 cm in eight directions, Table 7), pushing varies mass and ground friction independently over 36 cases, and in the two-parameter study the seed also sets the plant friction. The particle figures (Figures 1, 4, 5 and 11) and the video show the belief at each attempt, so it can be checked that the same sequence, an exploratory lift followed by contraction and retreat, appears across these cases. [Table 7, p. 24, l. 1257; pushing, p. 24, l. 1294; Figs. 4, 5, 11]