acceptodds
Under review as a conference paper at ICLR 2027

Adaptation and Probing Under Hidden Dynamics: Evidence from Planar Pushing

Abstract

Information about hidden dynamics can improve manipulation. Whether a dedicated probe adds to what ordinary task interaction already reveals is a separate question. We study this question in a controlled Push-T variant with an unobserved effective support-friction coefficient that spans more than a decade. Using diffusion policies with a shared backbone and training data, we compare true-parameter access, online estimation from task histories, history conditioning and per-tier specialists, and we measure the room left for a probe as the paired difference between a controller that knows the parameter from its first action and the same controller estimating it along the way. On held-out initial states, true-parameter access improves success by 5.5 percentage points on the evaluated training tiers and by 35.0 points at μ = 2.00, above the training range; the gain appears as closer approaches to the goal at low friction and as completed episodes at high friction. An estimator trained on ordinary task histories locates the friction within 64 steps, and a policy fed its estimate recovers the success-rate gain within the training range (recovered fraction 1.02 [0.76, 1.35]) and 58% of it at μ = 2.00. Parameter-input interventions change the amplitude of the first commanded action chunk and the timing of contact, and an incorrect value impairs success more than masking the input. The controller with the true parameter and the controller with the online estimate differ by 0.1 success points on the training tiers, with an interval that includes zero. These results establish ordinary task interaction as a strong adaptation baseline, and they set the reference against which dedicated probing should be evaluated: a controller that adapts during execution.

Primary area:
applications to robotics, autonomy, planning
Keywords:
hidden dynamics, adaptation, probing, system identification, diffusion policy, Push-T, friction, manipulation

Reviews

Among:
Shape:
1234this paper, Bottom 2%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.33 over 3 reviews, above 2% of the others.
Reviewer_iKudOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper studies the value of information about hidden dynamics in a modified Push-T environment with an unobserved friction coefficient. It compares diffusion policies with different information pathways, including friction-blind, true-parameter, estimator-conditioned, and history-conditioned policies, as well as per-friction specialists. The main finding is that online estimation from ordinary task interactions recovers nearly all of the success-rate improvement provided by true-parameter access within the training range. The paper interprets the remaining performance gap as the room for dedicated probing and contrasts this result with a one-push delivery task, where probing improves performance.

Critical strengths
  1. The paper makes a useful distinction between the value of knowing a physical parameter and the additional value of deliberately gathering that information. Its emphasis on comparing probing methods against a controller that already adapts during ordinary execution is relevant to experimental practice.

  2. The main policy comparisons use paired evaluation conditions and report uncertainty. The estimator is also evaluated on the trajectories generated by the estimator-conditioned policy itself, which strengthens the evidence that the adaptation mechanism works in closed-loop execution. The true, masked, and incorrect parameter-input interventions provide additional evidence that the controller uses the supplied information.

Critical weaknesses
  1. The broader insight provided by the empirical findings remains limited. The online-adaptation result in Push-T is useful, but the evaluation is limited to modified Push-T and a simplified delivery task. It remains unclear which task properties make ordinary interaction sufficient, and when dedicated probing provides additional benefit. Consequently, I found it difficult to determine what transferable guidance the study offers beyond the qualitative observation that probing is task-dependent. This limits the significance of the contribution as an empirical study.

  2. The experimental evidence supporting the probing conclusion is insufficient. Push-T does not directly evaluate a dedicated probing strategy; instead, the true-input versus online-estimate performance gap is treated as an upper bound, which needs further justification. The delivery comparison changes the controller and estimator and does not count probing costs. These experiments therefore do not isolate whether the opportunity to correct mistakes explains the different outcomes.

Questions for authors
  1. I am not sure why the performance gap between the true-input and online-estimate policies should be an upper bound on the benefit of probing. What assumptions are needed for this interpretation? Would it be more accurate to describe it as a reference gap for the particular controller evaluated here?

  2. Could you compare these two policies at shorter execution budgets using the stored trajectories? I wonder whether the small gap at 300 steps partly comes from having enough time to correct early mistakes.

  3. The Push-T and delivery experiments use different controllers and estimators, and the probing cost is not counted in the delivery task. I am not sure whether the different results can be attributed to the opportunity to correct mistakes. What evidence in the current results supports this explanation?

  4. The paper refers to Appendix C for the estimator details, but I could not find the training details there. Could you clarify where these are provided? Also, why do you use different numbers of training seeds for different variants? For variants evaluated with only one or two seeds, I find it difficult to judge whether the reported performance differences are robust across training runs.

Reviewer_NQdLOpenReview ↗
Rating
1
Confidence
2
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

Tries to evaluate whether probing for hidden dynamics in a modified friction version of the push-T task. Applied different levels of hidden friction and then evaluated the effect of adding friction awareness, history, as well as different kinds of information interventions to try to estimate how much value probing would give.

Critical strengths

The idea of investigating probing by evaluating the value of information is an interesting one and deserving of greater study.

Critical weaknesses

How the evaluation of the probing is done is very obfuscated: from reading the abstract and introduction, it is not clear how the probing is done, what assumptions are made, and what setting the probing would be done in, except for access to the true friction, which does not provide much context.

Some work on generating and evaluating hypothesis tests: Hypothesis Proposal and Evaluation, Granger causal skill chains, Hierarchical Experimentalist Agents: Learning Skills to Self-Improve In-Context via Experimentation, and these methods should probably get some citation since they evaluate similar concepts.

If the observations are the same, then wouldn't the observation be missing the velocity of the block, which was unnecessary in the current version but is not important for making the method not partially observed? Frame stack two frames also tends to perform poorly because of the high simulated velocity variance. As it is, we are evaluating a non-Markov system with a Markov model.

The formatting of the paper makes it hard to parse what the core objectives and evaluations are, by including details before they are clearly relevant and modifications in isolation without context. A rewrite of the paper clearly describing why an element is included would be desirable. In particular, it is unclear if the hidden friction is something unknown but held constant across evaluations, or if it is different at each episode. Overall, the writing gives the impression of an AI-generated report rather than a cohesive paper.

The evaluation outputs suggest that knowing the additional information is useful, but not whether the cost of probing can be quantified, which appears to be the purpose behind the paper. The use of the term "tier" is highly unclear, and there is no clear table of the tiers themselves in the main paper.

Questions for authors

See Weaknesses

Reviewer_Lzr4OpenReview ↗
Rating
1
Confidence
4
Informativeness
1
Presentation
1
Soundness
1
Read review
Paper summary

The paper studies a class of existing learning-based adaptation controllers and compare the performance of them with a specialist controller trained with privileged information on the push-T task. The authors would like to study the gaps between the other methods using estimated friction parameters compared to that using true friction parameters. The results show that those adaptation controllers are as good as the specialist.

Critical strengths

None

Critical weaknesses
  1. The paper studies the performance gaps between the existing adaptation methods and the specialist, but did not propose any new methods or ideas to address the gaps. The fact that these gaps exist is well known and expected (see for example Rapid Motor Adaptation, Kumar et al.)
  2. The paper studies exclusively the push-T tasks (with hidden friction coefficients), but there are more realistic tasks and other physics parameters required to experiment on to draw conclusions on this broad topic, for example, cloth folding or pick-and-place tasks. It’s not well motivated why only using a simple push-T task.
  3. The experiment scope is only one type of diffusion policy trained via imitation learning, and friction as a hidden physics parameter. The conclusions are potentially not general given the narrow experiment results.
Questions for authors
  1. Why only push-T, and not using different types of policies and different hidden physics parameters for different manipulation tasks?
  2. Why only imitation learning, and not studying the same problem in other types of policy and training methods? For example, policies trained using RL or Meta-RL methods.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.