DAPI: Dual Adaptive Policy Improvement for Contact-Rich Manipulation

Contact-rich manipulation can use action proposals ranging from sampling-based Gaussian proposals to learned Gaussian actors and generative priors. These proposals can all be improved at test time through model-based sampling and evaluation. Their performance can degrade, however, when the physical conditions at deployment differ from those assumed during planning, training, or data collection. Online model identification can mitigate this mismatch, but it creates a dual-control problem. Improving the model requires informative interactions, yet those interactions are selected by a policy that is itself affected by the current model mismatch. A task-driven policy may therefore fail to excite the uncertain parameters, leaving insufficient information for identification. The resulting uncertainty then continues to limit subsequent model-based policy improvement. We present DAPI, which augments task-driven policy improvement with an adaptive information-seeking channel. Among candidate action sequences that remain competitive for the task, DAPI favours actions whose predicted outcomes better distinguish the current physical hypotheses, and automatically removes this preference as parameter uncertainty contracts. A receding-window Stein update uses the resulting interaction data for online physical adaptation. The same candidate-reweighting mechanism applies without retraining to Gaussian sampling, frozen actors, and flow-matching proposals, with a noise-space update for the parameter-conditioned flow. Under substantial model mismatch in planar pushing, the receding-window update reduces median terminal error by 67%, whereas recursive Stein inference does not improve the nominal model. On lifting with an unknown center of mass, Gaussian sampling with online parameter adaptation alone succeeds in 0/12 rod groups and 0/24 plate cases, whereas adding DAPI’s information-seeking channel with the same proposal succeeds in 12/12 and 14/24.