acceptodds
Under review as a conference paper at ICLR 2027

Mind the Execution Gap: Action-Semantic Mismatch in World-Model Control

Abstract

World-model controllers rely on action-conditioned dynamics for prediction and planning, yet real control systems often execute commands asynchronously due to communication delay, packet loss, reordering, and actuator buffering. We study how asynchronous execution changes the action semantics assumed within world-model controllers, rather than treating it only as an external control disturbance. Through controlled interventions, we identify two architecture-dependent failure modes: planning-based controllers such as TD-MPC2 suffer from a future-action timeline mismatch between imagined and executed action sequences, while recurrent world models such as DreamerV3 can attribute observed transitions to commands that were not actually applied. Our analysis shows that TD-MPC2 requires the correct future action sequence during latent dynamics rollout, whereas DreamerV3 requires timely attribution of each transition to the action that generated it. Based on these findings, we introduce two lightweight execution-consistent interfaces, Future-Sequence for TD-MPC2 and Applied-Action Feedback for DreamerV3, that correct these mismatches without modifying the pretrained world models. Experiments across delays, packet loss, reordering, multiple control domains, measured network traces, and a process-separated asynchronous stack consistently support both diagnoses and the corresponding architecture-specific corrections.

Primary area:
applications to robotics, autonomy, planning
Keywords:
world models, asynchronous execution, model-based reinforcement learning, action delay, latent dynamics, real-time control

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_fS4DOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper studies why world-model controllers like TD-MPC2 and DreamerV3 struggle with network delays, packet loss, and out-of-order messages when controlling real robots.

The key insight is that each model fails differently. TD-MPC2 plans a sequence of actions, so delays can make its plan inconsistent with what the robot actually executes. DreamerV3 relies on action-observation pairs to update its internal state, so attributing an observation to the wrong action can throw off its state tracking.

The proposed fixes are simple and don't require retraining. For TD-MPC2, send the remaining action sequence to the actuator. For DreamerV3, use feedback on the action actually executed to update the model's state.

Experiments across multiple benchmarks, real network traces, and a two-process UDP setup show that both fixes improve robustness to network issues. However, one interesting finding is that these fixes can sometimes hurt models already trained to handle fixed delays, suggesting a mismatch with what they learned during training.

Critical strengths
  1. Rather than treating network issues as generic delays, the paper identifies why TD-MPC2 and DreamerV3 fail for different reasons. TD-MPC2 relies on planned action sequences that may not match actual execution, while DreamerV3 can lose track of state when the executed action differs from the one used for its update. The authors isolate these failure modes through controlled experiments, making the diagnosis a strong contribution beyond simply proposing a robustness technique.

  2. The proposed fixes directly address each model's failure mode: TD-MPC2 sends the remaining action sequence to the actuator, while DreamerV3 uses feedback on the action actually executed. Both changes are lightweight and don't require retraining the model. Validation across benchmarks and a real two-process UDP setup also strengthens the practical relevance of the approach.

Critical weaknesses
  1. Table 2 shows that Future-Sequence improves performance for clean-trained models but can hurt models trained with fixed delays, with little benefit for random-delay training. This limits its practical value, especially when delay-aware training is already used. The paper acknowledges this issue but offers no guidance on when the fix helps or hurts, weakening the claim that it is a general solution.

  2. In Table 1, DATS and ACDA perform poorly even under clean execution, scoring 0.266 and 0.407 compared with the normalized baseline of 1.0. This raises questions about implementation quality, tuning, or evaluation fairness. The paper does not adequately explain this gap, making its claims of superiority over existing delay-aware methods less convincing.

Questions for authors
  1. Which checkpoints (command-trained or applied-action-trained) are used for each arm in Figure 6? Figure 4 shows benefits only for applied-action-trained models, raising questions about the “no retraining” claim. How do you isolate the interface effect from the training effect?

  2. Section 4 discusses results across TD-MPC, MetaWorld, ManiSkill2, MyoSuite, measured network traces, task success, and overhead, but provides no quantitative results. Where are these results reported, and which network traces were used?

  3. Table 4 shows Future-Sequence underperforming Single-Action at Fixed-2 and Fixed-3, yet Table 1 excludes fixed delays and Figure 7 tests only a one-period delay. Why were these conditions omitted, and how should the claim of robustness across adverse conditions be interpreted?

  4. Future-Sequence performs similarly to Receding Buffer in three of four Table 1 settings, while the packet-loss difference may be explained by the fallback policy (zero versus hold-last). Can you ablate the fallback policy to establish whether the gains come from future-sequence execution itself?

  5. Why does Oracle score lower than Future-Sequence under reordering (0.892 vs. 0.942), despite Finding 1? Can you also test the hypothesis that Fixed-2-trained models fail under adverse conditions because they effectively compensate for delay twice?

8 comments
AuthorsOpenReview ↗

Response to W1

Thank you for highlighting the applicability of Future-Sequence. Our method primarily targets existing world models pretrained under delay-free conditions. Unlike approaches that require delay-aware training, Future-Sequence improves robustness to asynchronous execution through a lightweight interface modification, without retraining or updating model parameters, thereby avoiding substantial additional training costs.

Models already trained under delayed execution have learned to accommodate their training-time action delays and may not require the same correction. Our goal is therefore not to further improve models that have already adapted to delay, but to enable efficient, low-cost asynchronous deployment of existing pretrained world models.

AuthorsOpenReview ↗

Response to W2

Thank you for raising this important concern about baseline fairness. To address the relatively low performance of DATS and ACDA in the original Table 1, we carefully revisited their training and evaluation settings and conducted extensive additional experiments.

We found that both methods can achieve strong control performance on certain tasks. After further training and evaluation, DATS achieves a clean normalized return of 0.982 on Walker, while ACDA achieves 0.981 and 0.988 on Walker and Cheetah, respectively. In contrast, Hopper and Dog remain challenging, contributing to lower aggregate performance. We therefore conduct task-matched comparisons under identical network conditions on tasks where both methods achieve strong baseline control, while further investigating the difficulties encountered on the weaker tasks.

(1) ACDA comparison. On Walker and Cheetah, we use matched initial states and the same 0–3-step network perturbations.

Method / TaskCleanRandomBurstyLossReorderAdverse avg.
ACDA / Walker0.9810.8940.8340.8410.8760.861
ACDA / Cheetah0.9880.6920.7030.6810.7110.697
ACDA / Avg.0.9850.7930.7680.7610.7930.779
Future-Sequence / Walker0.9960.8810.6230.6830.9600.787
Future-Sequence / Cheetah1.0860.8320.8180.6391.0360.831
Future-Sequence / Avg.1.0410.8560.7210.6610.9980.809

Both controllers perform well without perturbations. Under this evaluation setup, Future-Sequence has a slightly higher adverse average (0.809 vs. 0.779), with a larger advantage on Cheetah. ACDA is stronger in some individual conditions, so we do not claim uniform superiority.

(2) DATS comparison. On Walker, both methods have similarly strong clean performance and are tested under the same network conditions.

Method / TaskCleanRandomBurstyLossReorderAdverse avg.
DATS / Walker0.9820.2600.2890.2690.2870.276
Future-Sequence / Walker0.9960.8810.6230.6830.9600.787

Despite comparable clean performance (0.982 vs. 0.996), Future-Sequence achieves higher returns under all four perturbations (0.787 vs. 0.276 on average). Thus, its gains in this comparison cannot be explained simply by DATS failing to control Walker under clean execution.

(3) Additional training and failure analysis. We completed 21 additional training or continuation runs (12 DATS, 9 ACDA), totaling 11.4 million environment interaction steps. We examined planning horizons, candidate counts, batch sizes, action-history handling, and target-entropy settings, together with prediction and behavior diagnostics.

For Hopper, effective training signals are scarce: the agent must reach a standing-height threshold before receiving meaningful forward-motion rewards. Consequently, 83–90% of ACDA's rewards and about 99.8% of DATS's rewards are zero. This makes it harder for ACDA to learn useful value estimates and for DATS to rank action sequences using predicted rewards. We tested three ACDA configurations with three seeds each and one million steps per seed, without obtaining consistently strong control.

For Dog, high-dimensional coordination (223 observation dimensions; 38 action dimensions) makes accurate prediction difficult. ACDA's two-step pelvis-height MAE is 6.29 cm, versus 2.15 cm for a constant-height predictor. DATS's reward-prediction RMSE is 0.084, exceeding the mean observed reward of 0.043. Such errors undermine the reliability of planning and action selection.

For Cheetah, we evaluated 17 DATS training/planning candidates, including 400,000 steps of continuation training. The best validation score reached 0.742, but the independent clean-test score remained 0.590.

These diagnostics explain why some external baselines have low aggregate scores despite strong clean control on other tasks. The revised Table 1 therefore reports matched-task comparisons rather than relying only on aggregate figures. Future-Sequence adapts existing pretrained world models through a lightweight execution interface, without extra delay-aware training, reducing the cost of asynchronous deployment. Our central contribution rests on controlled experiments with frozen world models and the diagnosis of action-semantic mismatch, not on weak baseline scores. We thank the reviewer for prompting these clarifications.

AuthorsOpenReview ↗

Response to Q1 and Q2

Response to Q1: Checkpoint Usage and the No-Retraining Claim

Thank you for pointing out this ambiguity. For each task in Figure 6, CMD, RCP, and APP all use the same command-trained checkpoint, which was trained under zero-delay execution. Under these conditions, commanded and applied actions coincide. All model parameters remain frozen during evaluation.

Importantly, APP denotes the feedback interface used during evaluation, not a switch to an applied-action-trained checkpoint. When comparing CMD and APP, we change only the previous-action input to the recurrent state update, replacing the issued command with the action actually applied. This isolates the effect of the execution interface from the effect of training.

Figure 4 serves a different purpose. It uses separate checkpoints trained under random delays, with either commanded or applied actions as training inputs, to examine how training-time action semantics affect feedback correction. Thus, the improvements in Figure 6 require neither additional training nor a change of checkpoint.

We have clarified the training conditions, action inputs, and checkpoint usage in the figure captions and experimental details of the revised manuscript.

Response to Q2: Quantitative Results and Network Traces

Thank you for pointing out this omission. We have added quantitative results to Table 3 in Section 4.2 of the revised manuscript and documented the network-trace sources, replay settings, and overhead measurements in Appendix E.3.

For clarity, the table below reports raw episodic returns and task success rates. Table 3 in the manuscript instead normalizes performance by the clean Single-Action performance for the same task and checkpoint.

Evaluation and metricSingle-ActionFuture-Sequence
Original TD-MPC: Walker return474.4910.3
Original TD-MPC: Hopper return17.1210.3
Original TD-MPC: Cup return800.1958.3
MetaWorld success rate86.7%100.0%
ManiSkill2 success rate50.0%93.3%
MyoSuite success rate90.0%100.0%
Measured-latency replay: Dog return318.2488.9
Measured-latency replay: MetaWorld Pick-place success rate60.0%95.0%

The original TD-MPC and cross-domain results are averaged over three conditions: random delay, packet loss, and packet reordering, with five evaluation seeds per condition. The measured-latency replay results are averaged over four traces, with five evaluation seeds per trace. Across the three cross-domain benchmarks, the mean success rate improves by 22.2 percentage points.

The four measured network traces comprise office and library recordings from ACDA's public evaluation artifact, together with rural n8 and urban n78 road recordings from CICV5G. We replay these recorded latencies in simulation. The source links, original filenames, and replay settings are provided in Appendix E.3.

We also report the following deployment overhead measurements:

Overhead measurementResult
Additional runtime per step (NVIDIA L40S)0.284 ms
Additional communication per observation: ECT-Full261.8 bytes
Additional communication per observation: ECT-ID181.4 bytes

The runtime overhead was measured on an exclusively used NVIDIA L40S. Communication overhead represents the additional serialized JSON bytes introduced by execution-receipt fields and excludes UDP/IP headers. Detailed measurement procedures are provided in Appendix E.3.

These results provide quantitative evidence of the effectiveness of Future-Sequence across the evaluated controllers, task domains, and measured network-delay conditions. The improvements on MetaWorld, ManiSkill2, and MyoSuite are reflected in task success rates. We appreciate the reviewer's suggestion, which helped us improve the clarity and completeness of the experimental reporting.

Reviewer_jX9iOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper focuses on improving asynchronous execution for world-model controllers. The authors identify two architecture-dependent failure modes when adopting the asynchronous execution pipeline, focusing on two architectures: TD-MPC2 and DreamerV3. Through a thorough analysis of how these two architectures are affected, the authors propose solutions for these two models, respectively. Experiment results demonstrate the effectiveness of the proposed method.

Critical strengths

The problem the paper focuses on is important. I appreciate the author's analysis in identifying the different failure modes for different world-model architectures. The two findings are also intuitively reasonable and proved through experiments. Related-works also clarify the connection to other relevant methods. The included experiments also provide empirical evidence for the proposed training-free methods.

Critical weaknesses

My concerns for the paper is mainly in the following two aspects:

  1. It is unclear whether asynchronous execution is actually necessary for the cases and tasks the authors target for. I am also not sure for the compared baselines in the experiments, is there a synchronous execution baseline (apologies if I missed it)? Why won't synchronous execution suffice?

  2. The technical contributions of the proposed methods seem to be limited. To me, what the authors proposed is more like engineering tricks for the two types of world models they study.

  3. The current draft limits the studied models to TD-MPC2 and DreamerV3. While they are widely used models, the generalizability of the paper's analysis and method remains unclear.

Questions for authors
  1. How is the deployment environment set if everything is in simulation? For the delays, is it simulated as a delay in the number of steps/actions, or the delay in time (ms)? If it is the former, what is the purpose of setting the models to 10/20Hz? If the later, what does this time consist of?

  2. In Table 1, why the external methods (DATS and ACDA) perform so poorly? Discussion and clarifications would strengthen the paper's claims.

9 comments
AuthorsOpenReview ↗

Response to W1. Is asynchronous execution necessary for the evaluated tasks? Why is synchronous execution insufficient?

We thank the reviewer for raising this important question. To directly address it, we conducted two additional sets of experiments examining the effects of controller computation time and network delays.

We define synchronous execution as waiting for confirmation that an issued action has been applied before computing the next action, while the simulator continues running. To prevent indefinite waiting under packet loss, the controller is allowed to resume planning after a timeout of four execution periods. Asynchronous execution does not wait for execution confirmation; instead, the controller proceeds using the latest observation. Our asynchronous sequence-based approach additionally transmits future planned actions that the actuator can execute while awaiting a new plan. Within each task, all methods use the same frozen checkpoint, hardware, and per-planning-call configuration.

Experiment 1: Can computation time alone make synchronous execution insufficient?

We evaluated six tasks at their original control periods, without introducing additional network delays or artificially accelerating the simulator. We also tested a reduced-frequency planning baseline, which increases the planning interval according to the measured computation time while keeping the simulator running at its original rate.

The table reports mean episodic returns (higher is better), using one fixed trained checkpoint and five evaluation seeds per task.

Task (control period)SynchronousReduced-frequency planningAsynchronous sequence
Cheetah Run (20 ms)150.2130.2186.7
Dog Run (30 ms)33.033.641.5
Cup Catch (40 ms)293.0291.4952.6
Cup Spin (40 ms)560.2560.2831.5
Quadruped Run (40 ms)455.5455.5928.9
Finger Spin (40 ms)109.8109.8383.4

For the four tasks with a 40 ms control period, the measured controller computation time is approximately 45 ms, making it difficult to provide a new plan at every execution step. Even when the planning interval is increased to 80 ms, single-action execution still underperforms asynchronous sequence execution.

These results demonstrate that computation time alone can create a meaningful execution bottleneck, even without additional network delays. Under the evaluated hardware and task settings, neither waiting for execution confirmation nor reducing the planning frequency fully addresses this problem.

Experiment 2: Do network disturbances matter when computation is usually fast enough?

We further evaluated Walker at 20 Hz, corresponding to one action execution every 50 ms, with a measured controller computation time of approximately 44 ms. We used five independently trained checkpoints, each evaluated with five paired seeds. Asynchronous single-action execution sends only one action per update, whereas asynchronous sequence execution provides planned future actions.

Network conditionSynchronousAsynchronous single-actionAsynchronous sequence
No additional disturbance974.8971.1978.2
Fixed 1-period delay391.2625.2802.7
Random 0–3-period delay238.1467.9823.5
Random delay + 20% packet loss162.7382.1656.7
Packet reordering293.0570.3885.7

Without additional network disturbances, all three approaches achieve comparable performance. However, synchronous execution degrades substantially under delayed, lost, or reordered messages, while asynchronous sequence execution consistently achieves higher returns.

Together, these experiments clarify when asynchronous execution is beneficial: when computation or message delivery cannot reliably keep pace with the environment's execution schedule. Our results support asynchronous sequence execution under these evaluated conditions, rather than suggesting that asynchronous execution is necessary for every task or deployment setting.

AuthorsOpenReview ↗

Response to W2 and W3

W2. Are the technical contributions mainly engineering tricks?

Thank you for raising this concern. We acknowledge that the proposed interface modifications are relatively simple in implementation. However, our primary goal is to understand where and why pretrained world-model controllers fail under asynchronous execution, and whether these failures can be addressed by correcting action information without retraining the underlying models.

Figure 3 provides controlled evidence distinguishing different types of execution information and where they enter the planner. Incorporating future-action information into the latent dynamics rollout improves control, whereas using the same information only for trajectory rescoring does not yield comparable benefits. Figure 4 further shows that the effectiveness of applied-action feedback depends on the action semantics learned during training.

Guided by these findings, we introduce lightweight execution-side corrections while keeping all model parameters frozen during evaluation. Under the four perturbations in Table 1, Future-Sequence improves the average normalized return from 0.212 to 0.696 compared with Single-Action. Figure 11 in the revised manuscript further shows that, with matched fallback policies, Future-Sequence reduces the RMSE between planned and applied actions by approximately 40–44%. On an NVIDIA L40S, its average additional runtime is only 0.284 ms per step relative to Single-Action.

We believe the main contribution lies in identifying where execution-induced action mismatches affect world-model computation and demonstrating that targeted, low-cost corrections can address these failures. The simplicity of the interfaces also allows us to validate these findings without modifying the pretrained models and makes the corrections practical for existing controllers. We hope this analysis provides a useful way to diagnose similar failures in other world-model architectures and identify appropriate execution-side corrections.

W3. Can the findings generalize to other models?

Thank you for this valuable suggestion. We selected TD-MPC2 and DreamerV3 because they consume action information at different stages of computation: TD-MPC2 uses future action sequences to predict latent dynamics, whereas DreamerV3 uses the previous action to update its recurrent state. Studying these two architectures allows us to examine how the same execution mismatch can affect different components of world-model control.

To evaluate whether the benefits of Future-Sequence are specific to TD-MPC2, we additionally tested the original TD-MPC controller. The results are reported in Table 3 of the revised manuscript:

Original TD-MPC taskSingle-ActionFuture-Sequence
Walker0.4800.920
Hopper0.0420.520
Cup0.8250.988

These results provide additional evidence that Future-Sequence can be applied to another planning-based world-model controller. Experiments on MetaWorld, ManiSkill2, and MyoSuite further demonstrate its effectiveness across different control domains.

Importantly, our analysis is based on how a world-model architecture uses action information, rather than on the identity of a particular model. We hope this work offers a useful perspective for future research: examining whether the actions used for prediction or state updates correspond to the actions that actually generate environment transitions. The appropriate correction should then be determined by the model's action semantics, training conditions, and feedback timing.

AuthorsOpenReview ↗

Response to Q1

Q1. How are delays implemented, and what do 10/20 Hz represent?

Thank you for pointing out this ambiguity. We apologize for the insufficient explanation in the original submission. We have clarified the two experimental settings in Section 4.1 and Appendix E.2 of the revised manuscript.

First, the main comparisons and mechanism analyses use environment-step delays. Each step corresponds to applying an action and advancing the simulator. For example, a two-step delay means that an action packet issued at step 0 becomes available at step 2. These experiments allow us to investigate execution-related failure mechanisms under controlled conditions without imposing actual wall-clock waiting times.

Second, our UDP experiments use real wall-clock timing. The execution stack consists of three independent processes: the controller, the network relay, and the simulated actuator. The controller performs actual model inference and planning, while the actuator schedules action updates every 100 ms or 50 ms, corresponding to 10 Hz and 20 Hz, respectively. The network relay introduces additional delays, packet loss, and reordering. Importantly, the simulator continues running while the controller is computing or action packets are still in transit.

Thus, 10/20 Hz specifies the actuator update rate, not the model inference frequency. For example, at 20 Hz, a two-period network delay adds 100 ms of relay waiting time. If inference takes 45 ms, an action computed from an observation available at time 0 would arrive at approximately 145 ms, ignoring additional communication and scheduling overhead, and could be applied at the 150 ms update. Therefore, the specified 100 ms represents only the additional network delay. The total observation-to-execution latency also includes computation, message processing, and alignment with the actuator update schedule.

We further verified the timing implementation using runtime logs. In two representative runs, the measured median relay waiting times were 100.49 ms and 50.47 ms for configured delays of 100 ms and 50 ms, respectively. The actuator continued updating throughout these waiting periods.

These measurements confirm that the UDP experiments capture the intended asynchronous execution setting, in which the environment continues evolving while computation and communication are still in progress. The environment-step experiments, in contrast, provide controlled conditions for isolating the underlying failure mechanisms.

We appreciate the reviewer highlighting this distinction, and have clarified it in the revised manuscript.

Reviewer_5myWOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper investigates action-semantic mismatches in world-model control under asynchronous execution, including delays, packet loss, and packet reordering. Through controlled interventions, the authors identify two architecture-dependent failure modes: future-action timeline mismatch in TD-MPC2 and previous-action attribution mismatch in DreamerV3. They propose two execution-consistent interfaces, Future-Sequence and Applied-Action Feedback, to address these issues without modifying the pretrained world models during deployment. The paper evaluates these mechanisms across different execution conditions and controller architectures.

Critical strengths
  1. The paper addresses a practically relevant problem in deploying world-model controllers under asynchronous execution. Its emphasis on the action information consumed by learned models offers an interesting perspective beyond simply treating delays as external disturbances.

  2. The controlled experiments in Figures 2–4 are useful for investigating architecture-dependent failure mechanisms. In particular, the execution-information and insertion-point ablations in Figure 3 provide meaningful evidence about the role of action inputs in latent dynamics rollout.

  3. The proposed interfaces are simple and deployment-oriented. The paper also reports important negative results concerning the interaction between training-time action semantics and execution-time corrections.

Critical weaknesses

W1. Limited improvement over the strongest execution baseline.

In Table 1, Future-Sequence achieves an adverse-condition average normalized return of 0.733, compared with 0.717 for Receding Buffer. The two methods match to the reported precision under random delay, bursty delay, and reordering, with the difference arising from packet loss. Furthermore, Appendix D indicates that these methods differ in buffer management and fallback behavior. It remains unclear which component is responsible for the additional improvement. A controlled ablation separating these design choices, together with paired confidence intervals, would strengthen the evidence for the proposed interface's distinct contribution.

W2. Insufficient connection between the diagnostic mechanism and the deployable correction.

Figure 3 relies on exact future applied-action information obtained through a privileged execution-system copy. In contrast, the deployable Future-Sequence interface does not know future packet-delivery outcomes. Although both settings are motivated by action-timeline consistency, the paper does not directly quantify how much the proposed interface reduces the discrepancy between imagined and executed action sequences. Such an analysis would help establish whether the reported performance gains are specifically attributable to the diagnosed mechanism.

W3. The novelty of Future-Sequence relative to established packetized predictive control needs clearer positioning.

Transmitting planned future control inputs to an actuator-side buffer and updating the buffer as new plans arrive are established ideas in networked and packetized predictive control. The manuscript discusses related concepts but could better distinguish the novelty of its interface from these existing approaches. I see the most promising contribution as the architecture-specific analysis of action-semantic mismatches in learned world models, rather than the buffering mechanism itself.

W4. Several broader empirical claims need more detailed support or qualification.

Section 4.2 reports generalization to MetaWorld, ManiSkill2, MyoSuite, and measured network-delay profiles, but the submitted manuscript does not provide corresponding quantitative breakdowns. Section 4.1 also mentions deployment overhead and communication statistics without presenting the numerical measurements. Moreover, Figures 4 and 8 and Tables 2/4 reveal that the effectiveness of the corrections depends on training-time action semantics and feedback timing. These conditions should be stated more prominently when discussing the applicability of the proposed interfaces.

Questions for authors
  1. Can the authors report paired performance differences and uncertainty estimates for Future-Sequence versus Receding Buffer, and isolate the effects of fallback behavior and buffer-update rules?

  2. Can the authors directly measure the rollout–execution action mismatch before and after Future-Sequence, thereby connecting the deployed correction to the mechanism investigated in Figure 3?

  3. How do the authors distinguish the algorithmic novelty of Future-Sequence from established packetized predictive control schemes?

  4. Can the authors provide the quantitative cross-domain and measured-network results, communication overhead, and a clearer specification of the training-time action semantics required for the DreamerV3 experiments?

5 comments
AuthorsOpenReview ↗

Response to Q1 and Q2

We sincerely thank the reviewer for the constructive feedback and for recognizing the value of our mechanism-level analysis. We have added controlled ablations and direct measurements to clarify the respective contributions of future-sequence execution, fallback behavior, and buffer updates.

Q1. Buffer-update rules and fallback policies. Thank you for highlighting the need to isolate these design choices. We added matched-fallback ablations and additional task comparisons in Figure 10 and the Execution-buffer analysis. With fixed four-action packets, the four-task average normalized returns are:

FallbackReceding BufferFuture-SequencePaired difference
Hold-last0.5290.5290.000
Zero-action0.4680.4680.000

Under matched fallback policies, the two methods produce identical executed actions and returns episode by episode. Thus, their paired differences are zero (95% paired CI: [0, 0]). Although their internal update rules differ, they execute the same future actions in this fixed-length setting. The revised Table 1 also reports the same adverse-condition average (0.696) for both methods across 35 paired evaluations.

To understand fallback effects, we evaluated eight additional tasks and examined action trajectories. Zero-action performs better on six tasks, whereas hold-last performs better on two. For Reacher Easy/Hard, zero-action reduces the mean number of departures from the target region per episode from 7.3 to 4.3 and 11.2 to 8.7. For Walker Stand, hold-last reduces the proportion of time below the standing-height threshold from 17.4% to 10.2%. For Cheetah Run Backwards, hold-last achieves a higher mean backward speed (4.02 vs. 3.66 m/s). These diagnostics illustrate why the better fallback depends on the task.

Appendix E.1 also evaluates variable-length packets, where the update rules can produce different actions. FS and RB achieve 0.288 vs. 0.250 with zero-action and 0.378 vs. 0.376 with hold-last. These results establish that the main gain over Single-Action comes from preserving and executing future planned actions, a mechanism shared by both methods.

Q2. Direct rollout–execution mismatch measurement. Thank you for pointing out the need to connect the privileged diagnosis in Figure 3 with the deployable interface. We compared each selected four-action plan with the actions subsequently applied at its intended execution steps, using matched hold-last fallback and 70 paired episodes per condition:

Network conditionSingle-Action RMSEFuture-Sequence RMSE
Random delay0.7790.445
Bursty delay0.8180.462
Packet loss + delay0.8240.490
Reordering + delay0.7310.417

Future-Sequence reduces action mismatch by approximately 40–44% across these four conditions. Because this metric also includes differences from later replanning, we conducted a controlled offline comparison using identical source action packets and network-delivery records, zero-action fallback, and no new planning within each comparison window. On FS-recorded random-delay traces, the paired action-error difference (FS minus Single-Action) was −0.407.

We added the action RMSE analysis in Figure 11 and Appendix E. Together with Figure 3's insertion-point ablations, these results directly connect the deployable correction to the diagnosed future-action timeline mismatch, without requiring privileged future delivery information at deployment.

AuthorsOpenReview ↗

Response to Q3–Q4

We sincerely thank the reviewer for the thoughtful suggestions on novelty positioning and experimental completeness. We have revised the manuscript to clarify the relationship with prior control methods and provide the requested quantitative evidence.

Q3. Relation to packetized predictive control. We appreciate this important observation. Transmitting planned future actions, buffering them at the actuator, and updating them as new plans arrive are established ideas in packetized predictive control. We have clarified this lineage in the Introduction and added foundational references, including Quevedo et al. and Ljesnjanin et al.

We adopt this idea because TD-MPC2 already generates a sequence of future actions during planning, but conventional execution typically transmits only the first action. If the next command is delayed or lost, the actuator cannot use the future actions already computed by the planner. Transmitting the sequence enables the actuator to continue along an available plan without retraining the world model.

Our contribution is diagnosing how asynchronous execution alters the action-conditioned computations of learned world models and validating architecture-specific corrections. Figure 3 shows that TD-MPC2 requires the correct future-action timeline inside latent dynamics rollout, whereas DreamerV3 exhibits a distinct previous-action attribution mismatch. Future-Sequence applies an established control principle to pretrained world models, guided by this mechanism-level diagnosis. We appreciate the opportunity to clarify this distinction.

Q4. Quantitative results, overhead, and DreamerV3 training semantics. Thank you for noting these missing details. We added quantitative results to Table 3 in Section 4.2, with network sources, replay settings, normalization, and overhead methods documented in Appendix E.3. The table below reports raw episodic returns or success rates; manuscript Table 3 reports clean Single-Action-normalized scores.

EvaluationSingle-ActionFuture-Sequence
Original TD-MPC: Walker return474.4910.3
Original TD-MPC: Hopper return17.1210.3
Original TD-MPC: Cup return800.1958.3
MetaWorld success rate86.7%100.0%
ManiSkill2 success rate50.0%93.3%
MyoSuite success rate90.0%100.0%
Measured-latency replay: Dog return318.2488.9
Measured-latency replay: MetaWorld Pick-place success60.0%95.0%

Original TD-MPC and cross-domain results average three conditions (random delay, packet loss, and packet reordering), with five evaluation seeds per condition. The mean success rate across MetaWorld, ManiSkill2, and MyoSuite improves by 22.2 percentage points.

Measured-latency replay uses four recordings: ACDA office/library and CICV5G rural-n8/urban-n78, evaluated on Dog and MetaWorld Pick-place with five seeds per trace. Appendix E.3 provides source links, original filenames, and replay details. On an exclusively used NVIDIA L40S, Future-Sequence adds 0.284 ms/step. Execution receipts add 261.8 bytes/observation (ECT-Full) or 181.4 bytes/observation (ECT-ID) of serialized JSON, excluding UDP/IP headers.

We have also clarified DreamerV3's training conditions in the figure captions and Appendix C. Figure 6 compares CMD, RCP, and APP using the same frozen clean-trained checkpoint per task, where commanded and applied actions coincide during training; only the execution or feedback interface changes at evaluation. Figure 4 instead uses separate checkpoints trained under random delays, using commanded or applied actions as training inputs. All checkpoints remain frozen during evaluation. As Figures 4 and 8 show, Applied-Action Feedback must also be consistent with learned action semantics and arrive in time for the corresponding recurrent-state update. Thus, our no-retraining claim refers to deploying existing pretrained world models through lightweight execution interfaces.

We sincerely thank you the valuable feedback. :)

AuthorsOpenReview ↗

Gentle Follow-up Regarding Our Rebuttal

Dear Reviewer 5myW,

Thank you again for your constructive feedback. We would like to briefly follow up on the additional evidence provided in our rebuttal.

In particular, Figure 11 shows that Future-Sequence reduces planned-versus-executed action mismatch by 40–44% across four network conditions, strengthening the connection to the mechanism identified in Figure 3. We also added fallback-policy analyses across eight tasks (Figure 10), clarified the relationship with packetized predictive control, and provided quantitative cross-domain results in Table 3.

We would greatly appreciate hearing whether these additions have addressed your concerns. We look forward to any further feedback you may have.

Best regards.

Comments on the paper

1 comment
AuthorsOpenReview ↗

Paper Update

We sincerely thank all reviewers for their careful evaluations and constructive suggestions. We have addressed the concerns raised and incorporated additional experiments and clarifications into the revised manuscript. All modifications are highlighted in red in the revised PDF. The major updates are summarized below.

1. Mechanism analysis and technical contributions [Reviewers 5myW, jX9i]: Further clarified the two architecture-dependent failure mechanisms in TD-MPC2 and DreamerV3, and strengthened the positioning of our execution-consistent interfaces in relation to existing packetized predictive control.

2. Direct evidence connecting mechanism and deployment [Reviewer 5myW]: Added Figure 11 to directly quantify rollout–execution action mismatch, showing that Future-Sequence reduces action RMSE by 40–44% across four network conditions. Also added fallback-policy ablations and eight-task comparisons in Figure 10, together with variable-length buffer experiments in Appendix E.1.

3. Expanded empirical evaluation [Reviewers jX9i, 5myW]: Extended Table 1 with task-matched DATS and ACDA comparisons. Added quantitative results for original TD-MPC, MetaWorld, ManiSkill2, MyoSuite, and measured network-trace replay in Table 3. Average success across the three cross-domain benchmarks improves by 22.2 percentage points.

4. Deployment efficiency and training conditions [Reviewers fS4D, 5myW, jX9i]: Clarified DreamerV3 checkpoint settings in Figures 4 and 6, and added analyses of training–interface interactions and feedback latency. Further documented the process-separated UDP evaluation, measured network traces, and communication overhead. Future-Sequence introduces only 0.284 ms additional runtime per step on an NVIDIA L40S.

We sincerely appreciate the reviewers' valuable feedback, which has helped us improve the clarity and experimental completeness of this work. We hope the revised manuscript provides a clearer presentation of our contributions and their practical value.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.