- Rating
- 4(+1)
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 3
Read review
Paper summary
This paper proposes ATLAS, a RL framework for VLAs. The goal is to improve VLA performance on composite tasks, where VLAs tend to struggle. IE a VLA may be able to complete tasks A and B, but fail to complete A then B. They make two major contributions as far as I can tell. The first is a dense potential-based reward signal that is based on the probablity of completing a subtask. Then these are chained togethor so that they act basically as an indicator of if the agent will complete the task or not (w/ discounting). Then, by adding reward that subtracts the current score from the score at the next step, this encourages the agent to make progress on this potential, ie increase its probability of completing the task. They additionally create a regularizar that encourages the composite latent representations to match the subtask latent representations. They run experiments in both sim and real life and show good performance.
Critical strengths
This paper addresses a critical problem, long-horizon behaviors for VLAs. Their solution is intuitive and seems effective. Presentation is clear and easy to follow. Experiments look good and illustrate both the problem and their solution.
Critical weaknesses
Nothing major.
Questions for authors
- Does ATLAS stand for something?
- What is 33pp and 40pp in the introduction? Is this percent?
- For the alignment part, I get that you want to align the latents. However, it seems like you want to align the latent for the composite task A->B with the latent of task A only when task A is active in the composite task. Otherwise, you would just learn the mean of the latents A,B for A->B. Is this what you do? I do not see where you account for the active task in section 3.4. The text around line 90 seems to imply something like this is happening though.
- Why is there a significant difference in A,B performance in Table 4? Shouldnt this ablation only affect the composite tasks?
2 comments
Thanks for the clarifications.
Rebuttal for Reviewer ceJW
We thank the reviewer for the positive evaluation and the accurate summary of our method, and for recognizing that ATLAS addresses a critical problem for VLAs, that the approach is intuitive and effective, and that the experiments clearly illustrate both the problem and our solution in simulation and on a real robot. We address each question below.
ATLAS stands for Atomic-to-composite Transfer via Latent Alignment and potential Shaping, referring to our task-wise progress-potential shaping and phase-conditioned atomic-to-composite hidden-state alignment. We have added this explanation at the first mention of ATLAS in both the abstract and the introduction.
"pp" denotes percentage points, meaning absolute differences in success rates rather than relative percent increases. We have replaced "pp" with "percentage points" in the revision.
Yes. During training, the teacher for each composite observation is the atomic instruction of the currently active subtask, selected using nt, the number of completed phases. For the task A→B, the target is task A until task A is completed and task B afterward, so each sample has a phase-specific target rather than an averaged one. We clarified this in Sec.3.4.
ATLAS jointly trains atomic and composite tasks, and task-wise potential shaping is applied to atomic rollouts as well. Therefore, removing potential shaping affects atomic-task performance, while changes to alignment can indirectly affect atomic performance through shared policy updates.