acceptodds
Desk rejected at ICLR 2027

Actions are Experiments: Identifying Action Semantics for Prediction and Decision

Desk rejected at ICLR 2027, so left out of the paper lists and the map.

Abstract

The same numeric action can have different physical effects when a robot's controller, coordinate frame, or body changes. We study the resulting ambiguity in action-conditioned prediction: a model must identify what its commands currently mean. We represent this information as an action-semantic state inferred from short response queries, and instantiate it with local affine operators and temporal response kernels. The contribution is a controlled empirical test of this representation, using fixed predictors, held-out interfaces, and paired wrong-state controls. On a joint holdout of UR5e and action programs, after training on Panda and Sawyer, the response state reduces standardized future MSE from 1.7784 to 0.0203; replacing it with a wrong-body state gives 1.8799. A separate probe-then-decide test has zero normalized regret on 72 supported target cases. Official LIBERO-plus and RoboTwin simulations provide further evidence with distinct limits: static and temporal future prediction on RoboTwin both fall short of the prespecified accuracy threshold, while a separate role-goal decision test shows lower endpoint error. We also report the unmet ranking, support, and contact criteria and account for duplicated task dynamics. These results support response-derived action conditioning for tested local role-space futures and decisions. They do not establish benchmark task success, a visual world action model, or a universally sufficient state.

Primary area:
learning on time series and dynamical systems
Keywords:
Action Semantics, Action-Conditioned Prediction, System Identification, Cross-Embodiment Transfer, Robot Decision-Making

Related papers

Discussion (0)

Sign in to comment.