- Rating
- 3
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 3
Read review
Paper summary
This paper proposes a new method by which a robot can infer the physical limitations of a teammate and adapt its behavior for future collaboration. In the proposed Watch, Infer, Coordinate framework, a helper robot first observes a physically constrained robot cooperating with another demonstrator, infers persistent low-level constraints such as joint position limits, velocity limits, or restricted motion directions from their joint behavior, and then replaces the demonstrator to coordinate with the same constrained robot on a new task. The key idea is that the constrained robot’s limitations affect not only its own actions but also how its partner compensates, making the joint behavior of both robots informative for capability inference. The inferred constraint model is then incorporated into planning for zero-shot coordination, leading to substantially better performance than capability-blind or imitation-based baselines.
Critical strengths
A key strength of this paper is its novel and interesting problem setting: inferring a robot partner’s hidden physical constraints from prior collaboration and using them for zero-shot coordination on new tasks. The whole pipeline is reasonably well-designed and works well in the simulated cases.
Critical weaknesses
My main concern is whether the proposed setting captures a sufficiently important real-world problem.
The paper assumes that a robot’s low-level physical constraints are unavailable to its partner and must instead be inferred indirectly from prior collaborative behavior. However, in many practical robotic systems, constraints such as joint limits, actuator degradation, or velocity restrictions could potentially be obtained more directly through diagnostics, system identification, or communication with the robot. The paper does not sufficiently motivate when such direct access is unavailable while prior demonstrations and detailed state-action observations are available.
This makes the practical applicability of the proposed setting unclear, especially given that the current evaluation is entirely simulation-based and relies on known dynamics, goals, full observations, and a predefined finite constraint set.
Questions for authors
- Why is inference from observation the right tool for this problem, and how often does this information asymmetry actually arise in practice?
- About reliability: What happens when the observed demonstrations are not sufficiently informative to distinguish between different constraints?
3 comments
Follow-up to rebuttal
Thank you again for your encouraging assessment and thoughtful questions! We have clarified the practical motivation for inference from observation and discussed how our approach handles insufficiently informative demonstrations.
We would greatly appreciate your thoughts on whether these clarifications address your concerns.
Thanks for your reply. I would keep my scores, but since this paper does not exactly align with my expertise, I'm not going to champion this paper.
Response to Reviewer F8yR
Thank you for your confidence in our work, and for acknowledging our paper's novel and interesting problem setting, and the well designed pipeline. We address your questions below:
Q1: Inference from observation: Imagine watching two people carry a heavy object. You notice that one struggles to raise an arm and their partner adjusts how they lift the object to compensate. Even without being explicitly told about the limitation, you can infer restricted shoulder mobility and account for it when you later collaborate with that person. We want to enable robots to gain this same ability to learn about a prospective partner from observing their previous physical collaborations. Our work studies the corresponding problem in robot teams, where a robot may be watching other robots collaborate but do not have access to their controller configuration and diagnostic interface. The only information available in our setting are both the agent's states and actions and goals. We also plan on extending this setting to human robot teams in the future.
Q2: Uninformative demonstrations: If the observed demonstrations are not sufficiently informative, then it would not be possible to distinguish between constraints. For example, if the agents acted completely at random, it's not possible to tell if the agent didn't take an action because it couldn't or because it chose not to. There are two ways we make sure that the observed demonstrations are sufficiently informative. We mention in our paper that the demonstrators are acting boundedly rational, which is similar to how humans make decisions, and ensures that the agents take actions that are not necessarily optimal but rather "good enough". We also ensure that in the evaluation, the constraints actually led to a change of behavior in the demonstrations, which ensures the demonstrations are informative.
We hope these clarifications and additional experiments address your main concerns.