acceptodds
Under review as a conference paper at ICLR 2027

LeapBot-JAC: Joint Action Chains for World Action Models

Abstract

Action-chunking policies predict future controls but execute only a prefix before replanning. How should a fixed action-token budget be organized to support ex- ecution? We propose LeapBot-JAC, which allocates sparse future-control refer- ences and dense execution targets within one shared action expert. Both chains are supervised by demonstrated controls and jointly denoised at aligned physical- time positions. Directed attention allows execution tokens to read reference to- kens subject to the host’s visibility constraints. JAC adds neither action tokens nor parameters and retains the host’s world-modeling process; both chains partic- ipate in inference, while only the execution chain drives the robot. On LIBERO, JAC reaches 98.3% success with FastWAM and LeapBot-WA, up from 97.0% and 97.3%. On zero-shot LIBERO-Plus, their success rates improve from 53.5% to 70.4% and from 73.1% to 77.3%, respectively. RoboTwin and two real-world tasks provide additional deployment evaluations. Single-training-seed compar- isons and FastWAM ablations support action-role organization as a practical de- sign choice under a fixed action-token budget.

Primary area:
applications to robotics, autonomy, planning
Keywords:
Robot Learning, World Action Models, Action Chunking, Joint Action Generation, Reference-Conditioned Control

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_fHF2OpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes LeapBot-JAC, a method for reorganizing a fixed action-token budget within existing world-action models. It replaces a sequence of 32 dense action targets with 16 sparse future-control references and 16 dense execution targets. The two chains share an action expert and are jointly denoised, using physical-time positional encoding and directed attention that allows execution tokens to read reference tokens. Only the execution prefix controls the robot. Evaluations on FastWAM and LeapBot-WA cover LIBERO, zero-shot LIBERO-Plus, RoboTwin, and two real-world tasks. The largest improvements occur on LIBERO-Plus, while measured model-call latency and memory remain nearly unchanged. Ablations investigate attention direction, reference-supervision weight, and reference stride.

Critical strengths

The paper addresses an interesting and practically relevant question: how to use a fixed action-token budget when only a prefix of the predicted sequence is executed. I appreciate the simplicity of the proposed design and the empirical evidence that reorganizing prediction targets can improve performance without expanding the model.

  1. A concrete and economical allocation of prediction roles. JAC combines dense near-term execution targets with sparse references spanning a longer future interval within a single shared expert. Physical-time alignment and directed visibility provide a clear implementation of this organization. The method preserves the action-token count, parameter count, and generation steps, making it a potentially useful modification to existing world-action models.
  2. Consistent robustness improvements across two hosts, supported by real-world evaluation. On zero-shot LIBERO-Plus, success improves from 53.5% to 70.4% for FastWAM and from 73.1% to 77.3% for LeapBot-WA, with improvements across all seven perturbation categories for both hosts. The two real-world tasks provide additional support, with 200 evaluation trials per model per task. These results are consequential because they extend the evidence beyond small improvements on standard LIBERO.
Critical weaknesses

Although the overall results are encouraging, the current evidence does not sufficiently isolate the contribution of the proposed reference–execution organization or establish the reliability of several reported improvements. The manuscript acknowledges important limitations, but these limitations still affect the strength of its central conclusions.

  1. The specific contribution of sparse references and directed reference-to-execution access remains insufficiently established. JAC changes target allocation, temporal coverage, positional encoding, attention visibility, and loss weighting together. The current controls do not separate these effects. In Table 3, the default reference-to-execution configuration achieves 98.3%, while reverse access achieves 98.2%, despite preventing execution tokens from directly reading reference tokens. These results leave the additional benefit of the proposed forward-conditioning mechanism unresolved; auxiliary supervision and shared-parameter optimization remain plausible explanations. Matched controls, such as a loss-reweighted single chain and a repeated-near-term reference variant, would help establish the contribution of sparse future references. This attribution is particularly important given related joint intention–action generation approaches such as LAWA [1].
  2. Training variability and incomplete deployment reporting limit the strength of the empirical conclusions. All local experiments use a single training seed, and success rates are reported without uncertainty estimates. Consequently, the small RoboTwin improvements and the narrow differences between attention or stride configurations do not establish reliable advantages. The four real-world evaluation rounds increase the number of trials but do not characterize training variability. Moreover, the strongest gains occur on LIBERO-Plus, whereas the mechanism ablations are conducted on LIBERO. The real-world evaluation is valuable, but important details—including the robot platform, demonstration counts, and control and replanning settings—are insufficiently specified for reproduction.

[1] Xiang Li et al. Latent Action as Intention Enables Efficient Future Imagination for World Action Models. arXiv preprint arXiv:2608.24882, 2026.

Questions for authors
  1. Could you provide matched controls that separate the effects of sparse future references from near-term loss reweighting and auxiliary supervision? In particular, comparisons with a loss-reweighted single chain and a reference chain that repeats near-term targets would help determine whether the longer reference span and distinct prediction roles are necessary.

  2. How consistently does the default reference-to-execution configuration outperform reverse or disconnected attention across independent training seeds? Could these comparisons also be evaluated on LIBERO-Plus, where the largest improvements are reported? Please provide uncertainty estimates for the main performance differences.

  3. Could you specify the robot platform, sensor setup, number of training demonstrations, control frequency, and replanning interval used in the real-world experiments? Please also clarify how initial conditions were selected and matched across methods and evaluation rounds, so that the reported deployment results can be reproduced and interpreted.

[1] Xiang Li et al. Latent Action as Intention Enables Efficient Future Imagination for World Action Models. arXiv preprint arXiv:2608.24882, 2026.

5 comments
AuthorsOpenReview ↗

Response to Reviewer fHF2

We thank the reviewer and respond point by point.

1. Isolating sparse future references. We agree that future coverage must be distinguished from near-term loss reweighting and auxiliary supervision. We added a loss-reweighted single-chain control built on FastWAM. It retains the original 32 consecutive Action Tokens, Action Expert, attention, and physical-time positional encoding, without an R chain. We weight the Flow Matching losses of the first 16 tokens by 1.2 and the remaining 16 by 1.0, using the original valid-token-count normalization and excluding padding tokens. The world-modeling objective, execution length, replanning protocol, and other training/evaluation settings remain unchanged. Results (%) are:

SettingLIBEROLIBERO-Plus
Single chain + near-term reweighting96.860.2
JAC, s=1 (R: 0–15)97.966.7
JAC, s=2 (R: 0–30)98.163.0
JAC, s=3 (R: 0–45)98.370.4
JAC, s=4 (R: 0–60)98.065.6

With s=1, R and A target the same near-term steps (0–15), providing a near-term-reference control while retaining two-chain joint denoising. Relative to this control, s=3 improves LIBERO-Plus by 3.7 points; relative to the reweighted single chain, by 10.2 points. On standard LIBERO, s=1 reaches 99.2%/100.0% on Spatial/Object but only 94.6% on Long, versus 97.2% Long at s=3. This pattern is consistent with shorter tasks benefiting from dense local reference and multi-stage tasks benefiting from broader future coverage. These comparisons support distinct reference roles beyond near-term reweighting alone. Unlike LAWA, which learns compact latent-action intentions via a separately pretrained tokenizer, JAC directly reorganizes existing control targets within one Action Expert, without additional annotations or a new tokenizer. We will report the control protocol and clarify the comparison and scope of these controls.

2. R→A interaction on LIBERO-Plus. On standard LIBERO, R→A/No-cross/R↔A/A→R score 98.3/97.7/97.5/98.2%, making directional benefits hard to distinguish. We therefore evaluated four attention configurations on LIBERO-Plus, fixing s=3, λR=0.2\lambda_R=0.2, and the Action Token budget:

FlowCameraRobotLang.LightBackg.NoiseLayoutTotal
R→A43.759.982.592.762.775.176.370.4
No cross36.650.080.287.558.368.872.264.8
R↔A34.547.380.690.759.063.569.863.6
A→R33.246.378.185.847.660.969.360.2

R→A leads across all seven perturbations and exceeds no-cross, bidirectional, and A→R by 5.6/6.8/10.2 points overall. Against A→R, the improvements are +13.6 Robot, +10.5 Camera, and +14.2 Noise. All variants retain the two-chain reference loss and shared Action Expert, so auxiliary supervision can help even when A cannot read R; this also helps explain A→R's 98.2% versus the 97.0% base on standard LIBERO. R→A additionally lets A access future reference features during joint denoising. Its advantage over disconnected and reverse-access controls supports a distinct contribution from directed conditioning, especially under perturbations. We will include these ablations and clarify the mechanisms.

3. Real-world reproducibility. We used a Franka robotic arm, a wrist-mounted first-person fisheye camera, and a third-person Intel RealSense L515. We collected 60 demonstration videos to fine-tune the deployed model. Cartesian commands ran at 20 Hz, with the underlying Franka service at 1 kHz. Each query executes K=10 actions before replanning. On an RTX 4090D, a serial replan-to-replan cycle takes approximately 0.7 s: 0.50 s execution and 0.20 s inference. Every trial starts from the same predefined robot home pose; objects are randomly placed within ±5 cm front/back and left/right offsets. All methods use the same bounded randomization protocol. We conduct four 50-trial evaluation rounds per model per task (200 trials total), reporting successes divided by 50 for each round. We will add these deployment and evaluation details for reproducibility.

Thank you for the substantive responses and additional experiments. The loss-reweighted single-chain and near-term-reference controls, together with the attention ablations on LIBERO-Plus, provide useful evidence for the contribution of sparse future references and directed reference-to-execution conditioning. The additional real-world deployment details also improve the clarity of the evaluation protocol. Please incorporate the new results, control settings, and deployment details into the revised manuscript.

AuthorsOpenReview ↗

Response to Reviewer fHF2

Thank you very much for your positive feedback and for recognizing the value of our additional experiments and responses. We have carefully considered your suggestions and incorporated the new experimental results, control settings, and real-world deployment details directly into the revised manuscript.

Specifically, we have added cross-chain attention ablations on LIBERO-Plus, extended the reference-weight and temporal-stride ablation analyses, and introduced a loss-reweighted single-chain control and a near-term-reference configuration to better isolate the contribution of sparse future references. Meanwhile, we have incorporated the motivation behind our real-world task designs, together with more comprehensive details of the robot platform, sensors, demonstration data, control frequency, replanning protocol, and initial conditions. In addition, we have conducted backbone transfer experiments on Flex-π and π0.5 to further investigate the transferability of JAC to other high-performance WAM and VLA architectures, and clarified the compatibility limitations of other candidate backbones under our current experimental configurations. We have also further refined our discussion of future predictions and the role of reference chains to clarify the methodological scope and applicability of JAC.

All these changes have been incorporated into the corresponding sections, tables, and figures of the revised manuscript. We sincerely appreciate your constructive suggestions and careful review, which have helped us substantially improve the experimental completeness, clarity, and overall presentation of our work. We hope that the revised manuscript adequately addresses your concerns, and we would greatly appreciate any further feedback or suggestions you may have.

Reviewer_rYmJOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper addresses an important problem in robotic control: action chunks are widely used for inference in robotics wherein an action horizon length of actions are executed while the rest of the prediction horizon is essentially discarded. The paper proposes JAC which divides an action chunk into two roles: dense execution and sparse reference, where the latter is used for predicting controls at strided future steps for better planning.

Critical strengths

The paper does not require additional requirements on top of the current training pipeline in the sense that it does not require additional annotations in data or extra forward passes during deployments. The design choices are well-motivated. In particular, allowing A to read R but not the converse is well-motivated and the implementation of RoPE is principled. The paper carries out a pretty rigorous set of experiments to justify the method. As Table 4 shows, the method does not induce additional cost as the p50 latency, peak memory and parameters are large the same as baselines’. The real world experiments are done over a large number of rollouts: “four rounds of 50 trials per task (200 trials per model per task)”

Critical weaknesses

The paper makes some claims that I do not find fully justified. Firstly, the pape treats prediction horizon beyond the action horizon as wasted capacity. This is not entirely true since methods like RTC and Bidirectional Decoding (BID) use those actions as guidance for inference. In particular, RTC does inpainting to allow the model to refine these prior predicts over time. They all rely on overlapping dense predictions between consecutive chunks.

Similarly, I do not understand how the method allows a model to use the sparse reference as an indication of “intent” so to speak. Concretely, the paper says that R is resampled from scratch at every query and then discarded. Looking at the choice of hyperparams, K = 10 and a 45-step reference, it seems that most of each reference is discarded before it is ever reached. So if the long-horizon intent is not kept consistent across replanning steps, I am not sure what the method is really bringing to the table.

The choices of various hyperparamters also undercuts the importance of the method. The choice of 45 step reference does not justify the method since that is merely a few seconds at typical control rates, whereas task-level intent and strategies span significantly longer time horizons. I am also not sure that uniform striding is a right choice since important actions like opening the gripper or contacts can fall between samples.

The results are somewhat dubious in my opinion. The ablations in table 3 don’t show exactly what lends strengths to the method. It seems like A -> R outperforms the base which is strange to me since in this paradigm, the execution chain cannot read the reference. In general, the ablation results are all within an extraordinarily short range. The performance on RoboTwin shows very marginal gains that are within noise.

Questions for authors

See weaknesses.

4 comments
AuthorsOpenReview ↗

Response to Reviewer rYmJ

We thank the reviewer and respond point by point.

1. Unexecuted predictions and prior work. We agree that describing predictions beyond the execution horizon as "wasted capacity" is inaccurate. RTC uses inpainting to refine overlapping predictions across successive action chunks, while BID exploits their temporal consistency. These methods demonstrate the value of unexecuted predictions, and we will correct our original wording.

JAC addresses a complementary question: how to organize a fixed action-token budget across different temporal scales. Rather than retaining 32 consecutive action tokens (32A32A), JAC reallocates the same budget into 16 dense execution tokens (16A16A) and 16 sparse reference tokens (16R16R), covering steps 0–15 and {0,3,…,45}\{0,3,\ldots,45\}, respectively. Both chains are jointly denoised by a shared Action Expert, with directed R→AR\rightarrow A attention allowing near-term actions to access broader future references. This provides extended temporal context without increasing the token budget or adding Expert parameters. Thus, JAC does not replace RTC/BID's prediction-reuse mechanisms but explores a complementary approach to organizing action predictions within each policy query.

2. Why R helps without persisting across replanning. We agree that R is regenerated at each query and is not retained across replanning cycles. Importantly, JAC does not aim to maintain a persistent task-level plan. Instead, R provides future control references within each policy query. During training, R is supervised by LRL_R to capture broader future controls, while A is supervised by LAL_A for near-term execution. Their shared Action Expert and directed R→AR\rightarrow A attention couple the two chains, allowing execution learning to benefit from reference information.

At inference, R and A are jointly denoised from the current observations, instructions, and available history. A accesses R during this same denoising process, so even references beyond the K=10K=10 execution prefix can influence the generation of current actions before being discarded. After execution, the next query generates fresh references based on the updated context. Thus, R serves as within-query future guidance rather than a persistent plan that must be carried across replanning cycles. We will clarify this distinction and avoid implying persistent task-level intent.

3. Reference horizon and uniform sampling. We agree that a 45-step reference horizon should not be interpreted as covering an entire task-level plan. JAC is designed to provide broader within-query control context beyond the immediate execution prefix, rather than represent long-term task intent. In our default setting, 16 reference tokens sampled at s=3s=3 cover steps {0,3,…,45}\{0,3,\ldots,45\}, while only the first K=10K=10 actions are executed before replanning.

Uniform striding provides a simple way to distribute a fixed number of reference tokens over a broader temporal span without additional event annotations. We acknowledge that sparse sampling may miss brief events such as gripper transitions or contacts. However, JAC retains dense consecutive actions in A over steps 0–15 for near-term control. As replanning proceeds, updated observations allow events approaching the execution window to be represented in subsequent dense action predictions. Thus, R provides coarse future context rather than precise event-level timing.

To examine the trade-off between temporal coverage and reference density, we evaluate four strides with λR=0.2\lambda_R=0.2:

ssR horizonLIBERO (%)LIBERO-Plus (%)
10–1597.966.7
20–3098.163.0
30–4598.370.4
40–6098.065.6

With s=1s=1, dense local references provide limited temporal coverage. Conversely, s=4s=4 extends the horizon but reduces reference density, resulting in a 4.8-point drop on LIBERO-Plus compared with s=3s=3. In particular, s=3s=3 outperforms s=4s=4 across all seven perturbation categories, with the largest differences on Robot (+10.3), Camera (+7.6), and Noise (+7.3). Overall, s=3s=3 achieves the best results on both benchmarks, suggesting a better balance between temporal coverage and reference density in our setting. We have clarified the temporal scope of JAC and expanded the stride analysis in the revised manuscript.

AuthorsOpenReview ↗

Response to Reviewer rYmJ

4. Contribution of cross-chain attention. We agree that the original ablation did not sufficiently isolate the contribution of cross-chain information flow. The A→R variant can still outperform the Base on LIBERO (98.2% vs. 97.0%) because both chains share the same Action Expert. Although A cannot directly access R, the reference loss LRL_R updates shared parameters during training and can therefore benefit A through parameter sharing. This is a training-time effect rather than direct reference access during inference.

To isolate the contribution of direct cross-chain access, we added a direction ablation on LIBERO-Plus, fixing s=3s=3 and λR=0.2\lambda_R=0.2:

FlowCameraRobotLang.LightBackg.NoiseLayoutTotal
R→A43.759.982.592.762.775.176.370.4
No cross36.650.080.287.558.368.872.264.8
R↔A34.547.380.690.759.063.569.863.6
A→R33.246.378.185.847.660.969.360.2

R→A achieves the best results across all seven perturbation categories, outperforming no cross-chain access, bidirectional access, and A→R by 5.6, 6.8, and 10.2 percentage points, respectively. In particular, the comparison with no cross-chain access isolates the additional benefit of allowing A to directly access R, beyond the shared-parameter effect of reference supervision. The results also show that bidirectional access is not necessarily beneficial; directing future-reference information toward execution is more effective than allowing unrestricted interaction. We will clarify the distinction between shared-parameter training effects and direct cross-chain guidance in the revised manuscript.

5. Reliability of RoboTwin results. Both RoboTwin baselines already achieve high success rates (91.02% for FastWAM and 91.49% for LeapBot-WA), leaving relatively limited room for further improvement. Under the same evaluation protocol, each Base/JAC pair was evaluated twice. Importantly, JAC outperformed its corresponding baseline in both evaluations for both backbones, and we report the averages: FastWAM improves from 91.02% to 91.24% (+0.22 pp), while LeapBot-WA improves from 91.49% to 92.00% (+0.51 pp). These consistent improvements across both evaluation runs and backbones support the effectiveness of JAC on RoboTwin.

The benefits are more pronounced on LIBERO-Plus, where JAC improves FastWAM by 16.9 pp and LeapBot-WA by 4.2 pp. Together with the cross-chain attention and stride ablations, these results further demonstrate the benefits of JAC, particularly under distribution shifts. Overall, JAC consistently improves performance across both RoboTwin hosts, with more pronounced gains on LIBERO-Plus.

Finally, we sincerely thank the reviewer for the careful review and constructive feedback. We have addressed the concerns through additional experiments, expanded ablation studies, and clarifications of JAC's design, temporal scope, and relationship to prior work. All corresponding revisions have been incorporated into the manuscript, with modified text, tables, and figures highlighted in blue. We hope that these revisions adequately address the reviewer's concerns and would greatly appreciate any further feedback or suggestions. We also plan to publicly release the code and relevant experimental resources upon publication to facilitate reproducibility and future research.

Reviewer_rYmJOpenReview ↗

Response to Authors

My concerns have been addressed and I have decided to increase my score.

Dear Reviewer,

We sincerely appreciate your thoughtful and constructive feedback, which has been very helpful in improving our work. We have carefully addressed your suggestions and incorporated the improvements into the revised manuscript. Thank you again for your time, consideration, and support.

Best regards, The Authors

Reviewer_xsP2OpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The authors work on the problem 'how to improve the organization of the action tokens within a fixed action-token budget'. The authors propose LeapBot-JAC, which introduces both sparse future control references and dense execution controls into one shared action expert. The authors verified their method on different simulation benchmarks and real world experiments. The method can improve the performances without adding extra computing cost under the same action-token budget compared with existed methods.

Critical strengths
  1. The authors propose the novel method of reallocation of the action tokens output which introduces sparse reference targets. It could help the action model to generate more reasonable actions conditioning on future control references.
  2. The method brings negligible inference overhead, which is very important for real-time deployment. The authors verified it on both FastWAM and LeapBot-WA.
  3. The authors designed different experiments to evaluate the impact of choices of access direction, reference supervision, and temporal coverage.
Critical weaknesses
  1. Although the method can bring large improvement on LIBERO-Plus, it doesn't show much improvement on Robotwin benchmark. The authors need to specify the reason and give more explanations.
  2. In Table 2, most of the best results lie on LingBot-VA or ImageWAM, the authors need to explain why they don't choose these as the baseline integration methods. Can LeapBot-JAC work together with LingBot-VA or ImageWAM?
  3. The authors mentioned 'Future constraints can affect the grasp' in Fig 1 (a), and said that's the motivation of the paper. While in the Introduction part, there's no explanation of related context when talking about the background of the paper. Can the authors provide more instructions to make readers better understand the realistic problems they aim at?
  4. Following the question 3), can the author explain how they decide on which real-world experiments to choose? (a) Sequential sorting (b) Drawer closing, are these two tasks related to the goal task 'Future constraints can affect the grasp'?
  5. For the ablation study of Reference stride s in Fig 4(b), pls explain why the performance drop when it is 4.
Questions for authors
  1. In Line 69, pls specify the meaning of λR before giving the default value of it
  2. In Fig 2, if the number of executable actions is K, the end of the actions list should be a K-1?
3 comments
AuthorsOpenReview ↗

Response to Reviewer xsP2

We thank the reviewer for the constructive comments and respond point by point.

1. RoboTwin gains. The reproduced RoboTwin baselines are already strong (91.02% for FastWAM and 91.49% for LeapBot-WA), leaving limited room for further improvement. With the same K=16K=16 execution protocol, JAC increases these results to 91.24% (+0.22 pp) and 92.00% (+0.51 pp), respectively. Each Base/JAC pair was evaluated twice under the same protocol, and JAC outperformed its corresponding baseline in both runs for each host. The reported success rates are the averages of the two evaluations. At the category level, FastWAM gains 0.56 points on Random, with a slight 0.12-point decrease on Clean, whereas LeapBot-WA improves on both settings (+0.54 and +0.48 points). These results show that JAC maintains and modestly improves strong RoboTwin performance, while its gains are more pronounced on LIBERO-Plus under distribution shifts (+16.9 and +4.2 pp for the two hosts). We have clarified these findings in the revised manuscript.

2. Backbone choice and transfer. LingBot-VA and ImageWAM use 16-token action sequences, whereas our main WAM configuration uses 32 tokens (16R+16A16R+16A) with K=10K=10 executed actions. A symmetric 8R+8A8R+8A allocation cannot preserve this execution prefix. Our preliminary experiments show that changing the number of actions executed before each replanning step substantially affects task success. Moreover, using different execution horizons for JAC and the base model would make their evaluation protocols inconsistent, confounding the attribution of performance differences to JAC itself. Therefore, we prioritize preserving the original execution and replanning protocols when integrating JAC. In addition to our main WAM evaluations, we further tested JAC on Flex-π\pi, a multi-stream WAM with a higher baseline success rate, to examine its transferability to another high-performing WAM architecture under the same fixed 32-token budget. Separately, we explored an expanded-token integration on π0.5\pi_{0.5}, retaining the original execution length while adding reference tokens, as an extension of JAC to VLA architectures.

ModelLIBERO (%)LIBERO-Plus (%)Action tokensParams (B)p50 (ms)p95 (ms)Peak mem. (GB)
Flex-π\pi Base98.387.132A12.512962.6983.826.65
Flex-π\pi + JAC98.9 (+0.6)88.5 (+1.4)16R+16A12.512965.3985.126.65
π0.5\pi_{0.5} Base96.485.210A3.353460.0461.269.18
π0.5\pi_{0.5} + JAC96.9 (+0.5)87.5 (+2.3)10R+10A3.353461.2962.969.18

Flex-π\pi provides a fixed-budget transfer evaluation. Despite its strong baseline, JAC improves LIBERO and LIBERO-Plus by 0.6 and 1.4 pp, respectively, while reallocating the same 32 tokens from 32A32A to 16R+16A16R+16A. Parameter count and peak memory remain unchanged. Both configurations use 10 flow-matching steps and CFG=1, verified by runtime instrumentation. The p50 and p95 latency increases are only 2.7 ms (0.28%) and 1.3 ms (0.13%).

For π0.5\pi_{0.5}, we retain 10 execution tokens and add 10 reference tokens, demonstrating an extension to VLA architectures rather than a fixed-budget comparison. JAC improves LIBERO by 0.5 pp and LIBERO-Plus by 2.3 pp, with unchanged parameter count and peak memory. The p50/p95 latency increases are only 1.25/1.70 ms (2.1%/2.8%), respectively, indicating a very small inference overhead despite the additional reference tokens. These results are reported in Appendix A.9.

3. Fig. 1(a) motivation. A grasp that succeeds immediately may leave an object poorly oriented for a subsequent insertion or constrained placement. This illustrates why manipulation decisions should account for future task requirements rather than only immediate execution. JAC addresses this issue by combining sparse longer-horizon control references (RR) with dense near-term execution (AA), allowing the execution chain to access future control information through directed R→AR\rightarrow A attention while retaining the fixed action-token budget. We have clarified this motivation and its connection to the proposed design in the revised Introduction.

AuthorsOpenReview ↗

Response to Reviewer xsP2

4. Real-world task selection. We selected two tasks to evaluate complementary aspects of the motivation. Sequential sorting involves two ordered grasp-and-place operations, requiring the policy to account for subsequent task requirements. Drawer closing involves contact-induced cabinet motion, requiring adaptation to changing environmental conditions during execution. Thus, the tasks examine temporal dependencies and dynamic interaction, respectively. Both are related to the broader motivation of considering future requirements and environmental changes, although they do not directly isolate the effect of future placement constraints on grasp-pose selection. We have clarified the rationale and scope of these tasks in the revised manuscript.

5. Reference stride s=4s=4. The stride controls the trade-off between temporal coverage and reference density. With 16 reference tokens, smaller strides provide denser near-term information but cover a shorter horizon, whereas larger strides extend the horizon at the cost of intermediate temporal details. To better illustrate this trade-off, we provide results for s=1,2,3,4s=1,2,3,4 on both LIBERO and LIBERO-Plus, with λR=0.2\lambda_R=0.2 fixed.

LIBERO (success rates, %):

ssSpatialObjectGoalLongAvg.
199.2100.097.894.697.9
298.2100.098.096.098.1
398.099.698.297.298.3
498.699.897.296.298.0

LIBERO-Plus (success rates, %):

ssCameraRobotLang.LightBackg.NoiseLayoutTotal
144.048.183.589.657.270.374.466.7
238.147.280.787.758.459.069.763.0
343.759.982.592.762.775.176.370.4
436.149.678.691.459.967.875.965.6

The results reveal a clear trade-off. With s=1s=1, dense near-term references yield strong performance on Spatial (99.2%) and Object (100.0%), but lower performance on Long (94.6%), compared with s=3s=3 (97.2%). This suggests that denser local references are beneficial for shorter tasks, whereas broader temporal coverage better supports longer-horizon manipulation.

Among the evaluated settings, s=3s=3 achieves the best average performance on both benchmarks (98.3% on LIBERO and 70.4% on LIBERO-Plus). Compared with s=4s=4, it improves LIBERO-Plus by 4.8 pp, with particularly notable gains on Robot (+10.3 pp), Camera (+7.6 pp), and Noise (+7.3 pp), while maintaining comparable performance on LIBERO (98.3% vs. 98.0%). This indicates that although a larger stride extends the reference horizon, excessively sparse references can omit important intermediate control information, particularly under distribution shifts. We have expanded the stride analysis in the revised manuscript to clarify why s=3s=3 provides a better balance between temporal coverage and reference density.

6. Definition of λR\lambda_R. We have clarified that λR\lambda_R denotes the auxiliary loss weight assigned to reference-chain supervision, and explicitly defined it before introducing its default value of 0.2.

7. Fig. 2 action indexing. The reviewer is correct. Under zero-based indexing, KK executable actions should be indexed from 00 to K−1K-1. We have corrected the endpoint in Fig. 2 accordingly.

Finally, we sincerely thank the reviewer for the careful review and constructive feedback. We have addressed the comments through additional experiments, supplementary analyses, and clarifications of the methodology and experimental settings. All revisions have been incorporated into the manuscript, with modified text, tables, and figures highlighted in blue. We hope the revised manuscript adequately addresses the concerns and welcome any further suggestions. We also plan to publicly release the code and relevant experimental resources upon publication to facilitate reproducibility and future research.

Dear Reviewer,

Thank you again for your valuable feedback and thoughtful review.

We have carefully addressed your comments in our rebuttal and hope that our responses have helped clarify your concerns. We would greatly appreciate it if you could kindly revisit our responses when you have a chance.

Please feel free to let us know if any questions or concerns remain. We would be happy to provide further clarification.

Thank you very much for your time and consideration.

Best regards, The Authors

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.