acceptodds
Withdrawn from ICLR 2027

ActAtlas: Learning Adaptive Execution Atlases for Long-Horizon Robotic Manipulation

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Long-horizon robot manipulation requires a policy to perform a sequence of con- nected operations. Diffusion policies can replan from new observations, but they often fail when two states look similar yet belong to different task stages and there- fore need different actions. We argue that a policy should know not only which local pattern the current state belongs to, but also how this pattern is connected to the other patterns of the task. To this end, we propose ActAtlas, which learns a data-adaptive graph of task stages from demonstrations and uses it to condition a diffusion policy, so that the robot generates actions with knowledge of both the current stage and how it connects to the rest of the task. Specifically, in the first stage, ActAtlas learns a set of execution anchors from demonstrations with self- supervision. Each anchor is a prototype shared by similar states across demonstra- tions, and the number of anchors is adjusted during training by growing, merging, and splitting. Transitions between anchors form a directed execution-flow graph. In the second stage, a graph attention network combines the current anchor with related anchors into a fixed-size context vector, which conditions a single diffu- sion policy. Because the graph weights are further trained end-to-end with the action denoising loss, these two stages allow ActAtlas to recognize the current stage of the task and use the learned stage-to-stage relations to choose the correct next action. Across four simulation benchmarks, ActAtlas improves the average success rate by 12.4% over the strongest baseline, with the largest LIBERO gain (16.2%) on the long-horizon suite. On four real-world tasks, it improves the aver- age success rate by 25.0 %. More broadly, our results suggest that learning how task stages connect to each other, rather than only what the current state looks like, is a simple and effective way to make generative robot policies more reliable on long, multi-step tasks.

Primary area:
applications to robotics, autonomy, planning
Keywords:
Long-Horizon Manipulation, Imitation Learning, Generative Control Policy, Self-Supervised Representation Learning, Diffusion Policy, Graph-Conditioned Control

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_dP1sOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

ActAtlas addresses long-horizon robotic manipulation, where visually similar states at different task stages can require different actions. It learns a data-adaptive set of execution anchors from demonstrations and builds a anchor transition graph to guide action selection. The author validates its effectiveness on four simulation benchmarks and four real-world tasks.

Critical strengths
  • The method fixes a specific failure pattern of receding horizon diffusion model.
  • The adaptive anchor and transition graph learning provide a structured way to capture stage-to-stage relationships.
  • The experimental evaluation includes extensive ablations and interpretability analyses.
Critical weaknesses
  • The anchor graph learning procedure is relatively complex and relies on a number of heuristically specified hyperparameters.
  • The learned transition graph is inherently tied to the ground truth trajectories, which may limit its applicability to unseen tasks or out of distribution states and failure recovery.
  • Without explicit goal conditioning, the anchor graph cannot resolve branching paths when distinct tasks share similar intermediate states.
Questions for authors
  • General VLA like π0.5\pi_{0.5} already outperforms ActAtlas in LIBERO tasks, what is the advantage of using anchor transition graph to track stage status instead of using natural language in a VLA?
  • Is the anchor graph learned seperately for each task? If so, how does it transfer to a multi task or open world scenario? Is there a general "skill" that can be shared between tasks like AutoCGP?
Reviewer_6pNxOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
1
Soundness
2
Read review
Paper summary

This paper partitions robotic manipulation into stages by constructing a directed graph, and then applies graph convolution to extract graph features that serve as stage-conditioning information for a diffusion policy, aiming to achieve stronger stage recognition and transition of actions.

Critical strengths

Strengths:

  1. Modeling robotic manipulation as a directed graph is an interesting attempt.
  2. Adaptively adjusting the number of concepts can indeed effectively mitigate the limitations of AutoCGP.
  3. Overall, the experimental results are solid, showing clear improvements over the baselines.
Critical weaknesses

Weakness:

  1. The core inference mechanism does not resolve the central problem raised in the paper. At inference time, the anchor is determined by nearest-neighbor retrieval on x_t alone, without incorporating any historical state information or Markov-chain transitions over the graph structure. This means that ambiguous actions would still be assigned to the same anchor.
  2. The method section is overly verbose and convoluted; the heavy notation and dense equations hinder readability, yet there is no description of the model architecture.
  3. In the Problem Statement section, the definitions of x_t and s_t within the demonstrations are missing.
  4. The real-robot experiments only include short-horizon tasks, which do not align well with the main problem the paper aims to address.
Questions for authors

Questions:

  1. How does ActAtlas resolve state aliasing in the headline results? In all main experiments, anchors are retrieved only by current state, and for a fixed g_t is a deterministic function of z_t.
  2. The authors should simplify the method section and provide either a model architecture diagram or a Preliminaries section.
Reviewer_A5V6OpenReview ↗
Rating
1
Confidence
3
Informativeness
2
Presentation
1
Soundness
1
Read review
Paper summary

The submission presents an algorithm for learning long-horizon robot manipulation sequences from demonstrations. The key to the approach is to identify a set of anchor state representations to encode task stages. Conditioning a diffusion policy on the stage anchor and the transition graph of anchors permits executing actions consistent with the current state and avoiding confusing similar states across diverse stages. Experiments are conducted in simulated and real-world tasks.

Critical strengths
  • The general idea appears novel and interesting. Indeed, informing the diffusion model of what it should be doing (the anchor) and how that fits with the rest of the task (the graph) is a reasonable way to enable short-horizon execution to be well contextualized in the rest of the task. Conceptually, the anchor is analogous to language instructions provided by hierarchical VLAs, but the graph provides additional context that is typically omitted in hierarchical methods. In particular, such context may help disambiguate how a particular stage of a task should be executed dependent on the remainder of the task stages.
  • The approach is demonstrated to work on a physical robot.
Critical weaknesses
  • The writing of the manuscript is far from sufficiently clear to understand the proposed approach. The explanations are terse and do not flow cohesively, and many (most) symbols are either not defined or not explained. As currently written, it would be impossible to replicate the approach after carefully reading through Section 3. See detailed notes below.

Sec 3.1

  • The problem statement needs significance revisions. Mostly, what is going on should be explained clearly in English, not only in the form of equations and symbols.
    • We are told that xtx_t supports anchor retrieval, but we are not told what it is
    • vtv_t is defined, but we don't know its dimensionality.
    • We are not explained what variables constitute the proprioception, nor why we need a history of the propriocetive state component but not the observation
      • My guess is that we are assuming that the external environment is static and then we only need to know how the robot has been moving. This should be stated
    • The choice of symbols vv and ss for exteroception and proprioception, and xx for something else is odd
    • All of the following are never defined/described: ztz_t, QQ, η\eta, zt−z_{t^-}, PtrP^\mathrm{tr}, Rϕ\mathcal{R}_\phi, gtg_t, K\mathcal{K}. Othe variables like $M=
      {\mu_k}$, V={νk}V=\{\nu_k\}, and BB are briefly defined but we are not explained what they represent.
      • I intuit that QQ is the anchor encoder, which receives some variable xx that I still cannot decipher and the previous anchor (I guess then that t−=t−1t^{-}=t-1?), plus the Atlas and the PtrP^\mathrm{tr} (which I also cannot decipher). Then R\mathcal{R} is the graph attention network that produces the context token gg that goes into the diffusion policy. I also cannot tell what active indices, retrieval keys, shared values, and directed scores mean. Are "active indices" the set of current anchors? Are retrieval keys and shared values learned anchor-specific attention key-values? And if so, would the queries be the z's? (This would likely explain the description of zt−z_{t^-} as the "previous query".)

Sec 3.2

  • The same happens here: qq, Norm\mathrm{Norm}, EηE_\eta, EuE_u, αu\alpha_u are not defined, it is not clear what an "offline query" or "online query" is. It seems like rtr_t are attention scores and z~t\tilde{z}_t is the assignment of each offline query to a given index kk. But the explanations here are not nearly sufficient to understand what is happening: "unit-norm queries and prototypes give [equation]", "apply [equation] backwards in stored slot order", "constant runs define segments", "soft assignments remain unchanged"... I can't piece together these short sentences/clauses without a lot more context and description. What are the slots here, what is a constant run, what soft assignments?
  • The overall flow of the algorithm here needs to be described a lot more precisely before diving into each individual computation.
  • After reading 3.1 and 3.2, the reader should know exactly what each symbol means, why it is introduced, and how it interacts with the rest of the process. The current manuscript does not provide that.

Sec 3.3

  • Same: what is NijN_{ij} (what does "adjacent assignments within demonstration windows" mean?)
  • What is the meaning of a "directed score"?
  • What does ii and jj index over?
  • What are the CC counts?

Sec 3.4

  • The same issues persist.
  • Overall, I don't believe that a human reader can understand what the algorithm defined throughout Section 3 does, and certainly could not replicate it. Therefore, it is impossible to extract meaningful insights from the contribution.
Questions for authors

Given the state of the manuscript at submission time, materially affecting my assessment would require a completely new reviewing round, which is out of scope during the discussion process.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.