acceptodds
Under review as a conference paper at ICLR 2027

SplineWAM: Adaptive Action Horizons for World Action Models via B-Spline Representations

Abstract

World action models (WAMs) are large embodied policies that jointly predict future video and the actions to execute, emitting a fixed-length action chunk per inference call. Such a policy allocates its computational budget uniformly in time, unable to execute for longer over free-space motion or to spend more inference on contact-rich manipulation, which limits the throughput a WAM can reach when served in the cloud. We present SplineWAM, which adaptively compresses the action trajectory into a fixed-size window of cubic B-spline parameters, fitting the knot times to the characteristics of the motion. One parameter budget then decodes into chunks of varying temporal resolution and duration, and both the executed span and the interval until the next policy call follow from the prediction itself. Aligning the video supervision to the fitted knot times of the demonstration rather than to a uniform grid concentrates the supervised frames where the action trajectory is complex. For asynchronous deployment we introduce Jacobian-Pullback Real-Time Chunking (JP-RTC), which imposes chunk continuity on the decoded raw actions the robot executes rather than on the spline parameters, and corrects the parameters through the decoder so that the executed prefix agrees with the actions already committed. On LIBERO-Plus and RoboCasa, SplineWAM improves success rate over an action chunking WAM by 8.2 and 4.4 points while cutting policy calls per episode by 22% and 26%. On three bimanual real-robot tasks under asynchronous execution, it leads or matches the baseline while decoding 1.2 to 1.6 times as much executed motion per call.

Primary area:
applications to robotics, autonomy, planning
Keywords:
Action Representation, World Action Model, B-Spline

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_pQHYOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

SplineWAM targets manipulation WAMs whose fixed action chunks impose a fixed replanning interval. It predicts fixed-size windows of adaptively fitted cubic B-spline parameters, using knot times to determine execution duration and future-video supervision. JP-RTC guides asynchronous predictions toward committed actions through the decoder Jacobian. Evaluation uses Fast-WAM on LIBERO-Plus and RoboCasa, with original-LIBERO and controlled-delay results in the appendices, plus three bimanual robot tasks with 30 trials per method and task.

Critical strengths

The integration addresses a concrete execution problem. Knot times coordinate video targets and execution duration without a separate horizon predictor (Sections 3.3–3.4, pp. 4–5). JP-RTC compares decoded actions, allowing successive windows to use different knot placements. Tables 7–8 provide empirical support for this choice over parameter-space RTC. This is a useful WAM integration of the spline representation credited to Han et al. in Section 2, rather than merely another compression format.

The synchronous comparisons support the full method. Backbone, demonstrations, and training budget are shared (Section 4.1, p. 6). Table 1 reports success gains of 8.2 and 4.4 points alongside fewer calls. Tables 3–4 show gains across six of seven LIBERO-Plus categories and all four RoboCasa families. Figure 4 also sweeps execution budgets for the baselines, making the comparison more informative than one chosen operating point.

Critical weaknesses

The benefit attributable to adaptive scheduling remains unclear. Figure 4 compares different representations as their execution budgets change, but does not compare fixed-step and row-budget execution using the same trained SplineWAM. The results therefore support the combined representation-and-alignment method more directly than the claim that adaptive execution improves the success–cost trade-off. This distinction matters because the spline representation and local-window construction are explicitly inherited (Section 2, p. 3). Video alignment is separately ablated, but knot-grid alignment’s additional gains over window alignment are only 1.2 and 1.6 points (Table 1), without training-seed variability; Appendix D explicitly reports single-run estimates (p. 19).

The asynchronous accuracy–cost claim needs stronger evidence. The headline call reductions come from synchronous Table 1. Under the eight-step delay, success advantages over action chunking shrink to 0.6 and 2.7 points, without accompanying asynchronous efficiency measurements (Tables 7–8, pp. 16–17). JP-RTC adds a Jacobian and solve at every denoising step, but runtime is not measured (Appendix E, p. 21). Table 2 does show longer robot chunks. However, its success gains correspond to one additional Bookshelf completion and three Whiteboard completions; Charge Earphone ties in success and loses 15.8 progress points. These observations support feasibility, but do not establish accuracy equivalence or measured cloud-throughput gains. The statement that the efficiency gain “costs no accuracy” (p. 9) is stronger than the evidence supports.

Questions for authors

Clarification: Which data selected the fitting tolerance and execution budget? Section 4.3 selects deployed settings from the sweeps, while Appendix D states there was no perturbation-specific model selection. Were the settings fixed independently of the reported test outcomes, and were baseline execution budgets selected by the same rule?

Targeted control: On one suite, can the same trained SplineWAM be compared under fixed-step and row-budget replanning at matched mean steps per call? This would directly test the scheduling contribution without changing representation or video supervision.

Targeted validation: At one asynchronous operating point, can you report success uncertainty across training runs, calls per control step, and measured accelerator time including JP-RTC? A reliable success–cost advantage would materially strengthen my assessment; an exhaustive latency sweep is not necessary.

3 comments
AuthorsOpenReview ↗

To Review pQHY: Part 1 of 2

We thank the reviewer for the clear and actionable questions. We ran both experiments requested in Q2 and Q3 as specified, and report them under W1 and W2.

W1. Benefit of adaptive scheduling; contribution of alignment.

Same trained model, fixed vs. row-budget execution. We executed the deployed SplineWAM checkpoints with a fixed number of steps per call, and compared them with the row budget at matched mean steps per call. The row budget improves over the fixed schedule at every matched length on both suites:

SuiteSteps per callRow budget − fixed (points)
LIBERO-Plus8.4 / 12.6 / 16.9 / 20.7+0.5 / +1.1 / +3.2 / +1.7
RoboCasa8.4 / 12.1 / 18.2 / 21.9 / 26.5+0.7 / +0.9 / +0.6 / +0.2 / +0.9

The improvement is larger on LIBERO-Plus, where knot spacing varies more within an episode than in RoboCasa's MimicGen demonstrations. With the schedule fixed, the same model still leads action chunking by 7.8 points at 8 steps per call on LIBERO-Plus; the row budget adds 0.5 points there and 3.2 at 16.9 steps. The fixed schedule removes only the adaptive replanning interval: the model still predicts, and is supervised on, the adaptive knot grid.

Contribution of alignment. Knot-grid and window alignment are two design variants within our method, and knot-grid placement is the better of the two on both suites (+1.2 / +1.6). Over three training seeds on LIBERO-Plus, this margin holds in the mean (74.3 ± 0.8 vs. 73.1 ± 0.5), a gap comparable to the seed variation. We therefore describe it as a smaller further gain, and measure the contribution of alignment itself against Naive B-Spline, which is the inherited representation trained on the same backbone without alignment. Against it, alignment gains +3.8 / +4.0 points at the Table 1 operating points, growing to +7.8 points at longer executed spans on LIBERO-Plus.

W2. Asynchronous accuracy–cost evidence; real-robot claims.

Asynchronous operating point. We evaluated all three seeds per method on LIBERO-Plus under the 8-step delay of Table 8:

Steps per callCalls per control stepSuccess (%)
Action chunking + RTC8.00.12544.9 ± 0.7
SplineWAM + JP-RTC, H = 128.40.11947.6 ± 0.8
SplineWAM + JP-RTC, H = 1612.70.07945.7 ± 0.8

On the real-robot computer (RTX 4090), a call takes 240 ± 10 ms for both methods. Guidance adds at most 10 ms per call for JP-RTC and at most 2 ms for RTC.

  • At H = 12, JP-RTC is 2.7 points more accurate at essentially the same cost, and every JP-RTC run is above every RTC run.
  • At H = 16, it needs 0.63× the calls per control step, i.e. 0.65× the accelerator time per control step including JP-RTC's overhead. Its success stays within seed variation of the baseline (+0.8).

Real-robot claims. We agree that "costs no accuracy" overstated the evidence. What the robot results do show is a consistent inference-cost advantage at comparable task performance:

  • On every task, SplineWAM executes 1.2–1.6× as many steps per call as action chunking with RTC, i.e. 0.61–0.83× the policy calls per control step.
  • Its success rate is comparable on all three tasks, and its progress is comparable or higher on two of them.
  • On Charge Earphone, the progress gap is concentrated at the socket insertion. Among the trials that pass that stage, JP-RTC does at least as well, including at the USB-C insertion. The evidence points to asynchronous continuity at this contact transition, although our robot runs do not separate it from the smoothness of the fit.
AuthorsOpenReview ↗

To Review pQHY: Part 1 of 2

Q1. Selection of ε and H. No separate validation set was used: both were chosen on the evaluation suites. The baseline's budget was chosen on the same data, and is its best point in the range we evaluated.

  • ε. We did not search for an optimal ε. We evaluated only four widely spaced compression ratios (raw actions per knot ≈1.0 / 1.5 / 2.0 / 3.0). The value 1.0 tests whether the B-spline representation is learnable at all, and 3.0 tests heavy compression of the action space. We do not claim that 1.5 is optimal, since we did not search around it. It is a rough value at which the method works on both suites, and it was carried over unchanged to the robot. The gain over action chunking does not hinge on this value: it is already +7.1 / +3.7 points at 1.0.
  • H. H needs no retraining, so we evaluated a range of budgets for action chunking, Naive B-Spline and SplineWAM (Fig. 4). Table 1 uses one row budget per suite for all spline variants, chosen so that SplineWAM executes about as many steps per call as the baseline's best point; for action chunking and SplineWAM this is also their best point. At matched executed length, SplineWAM leads at every budget, with the margin narrowing at the longest budgets.

The phrase "no perturbation-specific model selection" in Appendix D meant that ε and H were shared across all perturbation factors.

Q2. See W1 above. On the same trained model, the row budget improves over a fixed schedule at every matched length on both suites, by up to 3.2 points.

Q3. See W2 above. Over three training seeds, JP-RTC is better than action chunking with RTC at the operating point of Table 8 (47.6 ± 0.8 vs. 44.9 ± 0.7), and every JP-RTC run is above every RTC run. With H = 16, it needs 0.63× the calls per control step at comparable success. W2 also gives the measured per-call time, including JP-RTC.

Dear reviewer pQHY,

As the discussion period comes to an end, we would be grateful to hear whether our responses have addressed your main concerns. If there are any remaining questions or points that would benefit from further clarification, please let us know. We would be happy to provide additional information.

Reviewer_kCpTOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes SplineWAM, which replaces fixed-length action chunks in World Action Models with fixed-size windows of cubic B-spline knot times and control points. The model learns to predict spline parameters from demonstrations, allowing the decoded action horizon and replanning interval to adapt to motion complexity. The authors also align future-video supervision with the fitted knot times and introduce JP-RTC to maintain action continuity during asynchronous execution.

Experiments on LIBERO-Plus, RoboCasa, and three real-robot tasks show improvements in simulation success rates and fewer policy calls, with more modest gains under asynchronous real-robot execution.

Critical strengths
  1. Adaptive action representation. Using predicted spline knot times to adjust temporal resolution and execution horizons is an interesting way to reduce unnecessary policy calls, particularly for computationally expensive WAMs.

  2. Video–action temporal alignment. Aligning video supervision with spline knot times provides a reasonable way to allocate more supervision to complex motion segments, rather than uniformly sampling future frames.

  3. Asynchronous execution with JP-RTC. Applying continuity constraints in decoded action space rather than directly on spline parameters is a sensible design choice. The improvements over naive RTC support its usefulness for asynchronous execution.

  4. Extensive experiments and ablations. The paper evaluates two simulation benchmarks and three real-robot tasks, with comparisons across action representations, fitting tolerances, and execution horizons. These experiments provide useful evidence of the method's practical benefits and limitations.

Critical weaknesses
  1. Limited novelty over existing spline policies. Adaptive B-spline fitting, continuous action representations, and fixed-size spline windows have already been explored in B-Spline Policy and related work. The main additions seem to be video alignment and adaptive replanning, but their individual contributions are relatively limited. The novelty over existing spline-based methods is not sufficiently clear.

  2. Why specifically cubic B-splines? The paper does not justify why cubic B-splines are preferable to other representations, such as quintic splines, piecewise Hermite or Bézier curves, or frequency-based action representations. These alternatives offer different trade-offs between smoothness, local control, and precision. A comparison would help establish whether the gains are specific to the proposed representation or simply come from using continuous trajectories.

  3. Questionable connection between knot density and replanning. The method assumes that regions requiring more knots should also be replanned more frequently. However, reconstruction complexity does not necessarily reflect uncertainty or the need for feedback. A repetitive but rapidly changing motion may be predictable, while a smooth movement near an uncertain obstacle may require frequent replanning. This central assumption needs stronger validation.

  4. Adaptive horizon benefits are not properly isolated. The paper compares spline-based adaptive execution with conventional fixed-length action chunks, but these methods also differ in representation and training. A more direct comparison would execute the same trained spline policy using fixed versus adaptive horizons. Without this, it is unclear how much of the improvement actually comes from adaptive replanning.

  5. Limited contribution from video–action alignment. Knot-grid alignment improves over window alignment by only 1.2 and 1.6 points on the two benchmarks. It is unclear whether concentrating video supervision around action changes consistently improves world modeling or simply changes the training distribution. More direct evidence of its benefit is needed.

  6. Efficiency claims need stronger support. The paper emphasizes fewer policy calls, but does not directly establish improvements in end-to-end latency or multi-robot serving throughput. The spline representation also does not necessarily reduce the number of predicted parameters, and JP-RTC introduces additional computation. Fewer calls alone do not fully establish the claimed system-level efficiency benefits.

Questions for authors
  1. Given that adaptive B-spline fitting and fixed-size spline windows have already been explored in B-Spline Policy and related work, what is the main methodological novelty beyond video alignment and adaptive replanning?

  2. Why specifically choose cubic B-splines? Have the authors considered alternatives such as quintic splines, Hermite or Bézier curves, or frequency-based representations? How much of the improvement comes from B-splines specifically rather than continuous trajectory representations in general?

  3. What evidence supports using action reconstruction complexity to determine when replanning is needed? Does knot density actually correlate with prediction uncertainty, contact events, or closed-loop control errors?

  4. Could the authors compare fixed-step and adaptive-step execution using the same trained SplineWAM policy under matched inference budgets? Also, given the modest gains from knot-grid alignment, could they better isolate its contribution to future prediction and control performance?

  5. Could the authors report end-to-end inference latency, JP-RTC overhead, and system throughput rather than only policy calls per episode? Since the spline representation does not necessarily reduce output dimensionality, how much computational efficiency does the method actually provide?

6 comments
AuthorsOpenReview ↗

To Review kCpT: Part 1 of 3

We thank the reviewer for the detailed questions.

W1. Novelty over B-spline Policy (BSP). The main novelty of this work lies in the problem it poses: we use the action representation to control the inference cost of a large policy, so that how much each call covers follows the motion rather than a fixed clock, and we measure the resulting trade-off between success and policy calls (Fig. 4).

BSP's representation alone is not enough for this in a WAM. Our Naive B-Spline baseline is that representation on the same backbone. It gains 4.4 / 0.4 points over action chunking at the Table 1 operating points, and on LIBERO-Plus this margin is gone by about 16 steps per call (59.9% vs. 59.6%), while SplineWAM still leads by 8.2 points at 17 steps. The difference comes from three problems that a variable-duration window raises in a WAM, which BSP does not face or handles differently. Solving them is our methodological contribution:

  • What the video should supervise. With a variable-duration window, video sampled on a fixed time grid no longer matches the actions. We place the supervised frames on the fitted knot times. BSP has no video branch. This gains +3.8 / +4.0 points over Naive B-Spline, and +7.8 at 17 steps per call on LIBERO-Plus.
  • How much of each prediction to commit. We make the committed prefix an explicit row budget, counted in knots. The committed span then adapts to the motion, while its trade-off with policy calls stays controllable. BSP instead requests the next segment shortly before the current one runs out. On the same model, the row budget improves over a fixed schedule by up to 3.2 points (W4 below).
  • How to join consecutive predictions. Consecutive windows place their knots differently, so guidance on the spline parameters is ill-posed. JP-RTC imposes continuity on the decoded actions through the decoder Jacobian. With RTC applied to the parameters instead, a spline WAM falls below action chunking under delay, on both suites and in success on all three robot tasks; with JP-RTC it is at or above it (Tables 2, 7–8). BSP re-phases the new segment with a time shift, and disables this for delta actions.

We adopt BSP's representation, fitting and local windows, and in BSP, too, the predicted knots set how long a segment lasts. What prior work does not do is use this to control inference cost:

  • BSP targets faster execution through time rescaling, and does not measure inference cost.
  • Action compression methods such as FAST and BEAST reduce the tokens per fixed-length chunk, but not the number of calls.
  • Adaptive replanning methods add an auxiliary signal on top of a fixed representation.

W2. Why cubic B-splines? Our choice rests on two considerations.

  • Why a continuous curve rather than a frequency or fixed-length representation. Frequency-domain representations, such as the DCT in FAST, and other learned representations of fixed-length chunks always cover a fixed duration. This conflicts with letting the represented duration follow the motion. Table 1 reflects this: BEAST, a continuous B-spline over a fixed duration, does not share the gains of the adaptive-duration variants.
  • Why the cubic B-spline among continuous curves. We do not claim that every other curve family is worse, and we did not test them all. Our method does not require cubic B-splines: alignment, the row budget and JP-RTC need only a curve whose knot times are explicit parameters and whose decoding is differentiable. We validated it on the lowest-degree B-spline with continuous acceleration, and therefore do not count the choice of curve as a contribution.

We also trained linear (k = 1) and quintic (k = 5) SplineWAM at the same knot density. LIBERO-Plus success (%) at the same future knots per call, i.e. about the same steps:

Future knots per call5913
k = 168.667.959.3
k = 3 (paper)74.572.666.7
k = 571.771.265.7

Cubic is best at every length; quintic is close (1.0–2.8 points lower). Linear loses 4.7–7.4 points but stays 1.4–5.5 points above action chunking at equal steps per call. With adaptive knots, a linear spline is action chunking on a non-uniform time grid, suggesting that the adaptive grid gives part of the gain and smoothness the rest.

AuthorsOpenReview ↗

To Review kCpT: Part 2 of 3

W3. Knot density and replanning. Our method does not assume that dense knots mark uncertainty. It assumes only that placing policy calls by knot count uses a given number of calls better than placing them uniformly in time. W4 below tests this as an intervention: same model, same mean steps per call, and only where the calls fall differs. Placing them by knots is never worse and gains up to 3.2 points on LIBERO-Plus. On RoboCasa, where knot spacing hardly varies, it gains least (at most 0.9 points), as expected. Q3 below adds what the knots follow: contact in the demonstrations and failure at deployment. On your two examples:

  • A predictable but fast motion is replanned more often than needed, which costs calls, not success. This cost does not show in our closest case, the repetitive wiping of Clean Whiteboard: SplineWAM executes its longest chunks there (52.4 steps per call, 1.64× the baseline) and succeeds in 18 of 30 trials against 15.
  • A smooth motion near an uncertain obstacle is not flagged by knot density, a limitation of our design. A fixed chunk has the same blind spot, since it also commits a preset span there; ours can be shortened at deployment by lowering H, without retraining. Uncertainty-based replanning could set H online and is complementary to our method.

W4. Isolating the benefit of the adaptive horizon. We executed the deployed SplineWAM with a fixed number of steps per call, and compared it with the row budget at matched mean steps per call. The row budget improves over the fixed schedule at every matched length on both suites:

SuiteSteps per callRow budget − fixed (points)
LIBERO-Plus8.4 / 12.6 / 16.9 / 20.7+0.5 / +1.1 / +3.2 / +1.7
RoboCasa8.4 / 12.1 / 18.2 / 21.9 / 26.5+0.7 / +0.9 / +0.6 / +0.2 / +0.9

The improvement is larger on LIBERO-Plus, where knot spacing varies more within an episode than in RoboCasa's MimicGen demonstrations. With the schedule fixed, the same model still leads action chunking by 7.8 points at 8 steps per call on LIBERO-Plus; the row budget adds 0.5 points there and 3.2 at 16.9 steps. The fixed schedule removes only the adaptive replanning interval: the model still predicts, and is supervised on, the adaptive knot grid.

W5. Contribution of knot-grid alignment. Knot-grid and window alignment are two design variants within our method, and knot-grid placement is the better of the two on both suites (+1.2 / +1.6). The contribution of alignment itself is measured against the unaligned Naive B-Spline baseline: +3.8 / +4.0 points at the Table 1 operating points. This gain grows from +3.8 to +7.8 points on LIBERO-Plus as more of the window is executed. The aligned frames supervise the later part of the window, which is executed only at longer budgets. This points to an effect on control, not merely a change in the training distribution.

W6. Efficiency.

  • Where the saving comes from. The spline head does not reduce the output, and a call costs the same under both representations. The saving comes from executing more steps per call:
    • Synchronous: 2.1× (LIBERO-Plus) and 1.8× (RoboCasa) as many steps per call at action chunking's best success.
    • Asynchronous, at comparable success: 0.63× the calls per control step on LIBERO-Plus, i.e. 0.65× the accelerator time, JP-RTC's overhead included.
    • Real robot: 1.2–1.6× as many steps per call.
  • Latency and overhead. On the real-robot computer (RTX 4090), a call takes 240 ± 10 ms for both action chunking and SplineWAM. JP-RTC adds at most 10 ms per call, against at most 2 ms for RTC.
  • Throughput. With many robots per policy, accelerator time scales with calls per control step, so fewer calls mean proportionally more robots per accelerator.
  • Serving setting. This shared-model setting, which motivates our paper, has become concrete since submission. Two world action models were opened as public APIs in October 2026: Dream-1 (Dream Labs) and Poke-Aurora (PokeBot's RoboAPI). In both, a robot sends its observations and receives the next action chunk. Dream-1 returns 1.6 s of actions per call, i.e. 2,250 calls per robot-hour whatever the motion. These releases are early evidence that large embodied models can be served as an API, and they are the case our method is designed for: one model, many concurrent robots, and a compute cost set by the number of calls. An adaptive window lowers that number where the motion allows.
AuthorsOpenReview ↗

To Review kCpT: Part 3 of 3

Q1. See W1 above. Beyond video alignment and adaptive replanning, the novelty lies in using the action representation to control inference cost, together with JP-RTC, the third component this requires in a WAM.

Q2. See W2 above. The method is not tied to cubic B-splines, and among the degrees we tested, cubic gives the largest gain.

Q3. We answer the three parts in turn.

Contact events. Over all training demonstrations, taking grasp and release as contact events:

  • Density. Knots are 1.06–1.41× denser within 0.5 s of an event in every dataset, simulated and real-robot (82–100% of episodes; on LIBERO 1.23×, AUC 0.76).
  • Executed span. A call issued there commits less: 7.1 steps against 9.0 elsewhere on LIBERO, and 38.9 against 52.4 on Charge Earphone. On RoboCasa, whose MimicGen demonstrations are dense in knots throughout, the span barely changes.
  • Mechanism. A grasp or release is a sharp change of the commanded action, so an error-bounded fit must place knots there. On RoboCasa and the robot, the concentration comes from the gripper dimension; on LIBERO the arm motion adds to it (still 1.11× with the gripper dimension removed).

Closed-loop errors. Demonstrations contain no errors, so we compared the deployed model's own rollouts at the Table 1 operating point. A call executes up to a fixed number of predicted knots, so fewer steps per call mean denser knots and more frequent replanning. Within a task, a failed episode executes fewer steps per call than a successful one with probability 0.68 on LIBERO-Plus and 0.86 on RoboCasa (0.81–0.91 at other budgets), calling the policy 1.2× and 3.2× as often per control step. Over the first five calls the gap is small (0.57 / 0.54): the policy replans more once a rollout goes wrong, rather than forecasting it.

Prediction uncertainty. We did not measure it directly, and do not claim that knot density measures it (W3).

Q4. See W4 and W5 above. On the same trained policy, the row budget improves over a fixed schedule at every matched length. The contribution of alignment, measured against the unaligned baseline, grows with the executed span.

Q5. See W6 above. On the real robot, where every method runs asynchronously with RTC, the gains relative to action chunking with RTC are:

TaskSteps per callCalls per control stepAccelerator time per control step
Arrange Bookshelf32.0 → 38.6 (1.21×)0.83×0.86×
Charge Earphone32.0 → 45.7 (1.43×)0.70×0.72×
Clean Whiteboard32.0 → 52.4 (1.64×)0.61×0.63×

Accelerator time includes JP-RTC's overhead: at most 10 ms per call for JP-RTC, against at most 2 ms for RTC, on a call of 240 ± 10 ms.

Reviewer_3BxBOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper presents SplineWAM, which replaces fixed-length action chunks in world action models with fixed-size windows of B-spline control points and knot times, enabling adaptive execution horizons. It also aligns video supervision with the fitted knot times and introduces JP-RTC, an inference-time mechanism that promotes consistency between consecutive predictions in decoded action space through Jacobian-based parameter corrections. Experiments on two simulation benchmarks and three real-robot tasks demonstrate improvements in task performance and policy-call efficiency, although the gains vary across tasks and execution settings.

Critical strengths
  1. Adaptive horizons with a fixed parameter budget. The method decouples the size of the predicted representation from its temporal coverage. By encoding timing directly in the spline representation, it enables longer execution spans for compressible motion and shorter spans for more complex motion, offering a useful trade-off between task performance and policy-call efficiency.
  2. A well-motivated approach to asynchronous execution. JP-RTC addresses the mismatch between spline parameters and executable actions by enforcing consistency in decoded action space. This allows consecutive predictions to use different knot placements while encouraging agreement on committed actions. Comparisons with naive parameter-space RTC support this design choice.
  3. Controlled evaluation of the main components. The experiments use a shared backbone and training budget to examine action representation and video–action alignment, complemented by ablations on fitting tolerance and execution budget. Simulation and real-robot evaluations provide evidence for both the benefits and limitations of the approach.
Critical weaknesses
  1. Limited novelty over prior spline-based policies. B-spline action representations, including the prediction of knot times and control points in fixed-size local windows, have already been explored in prior work. The contribution therefore appears primarily to lie in integrating this representation with video supervision and asynchronous execution. Although these extensions are useful, the paper could more clearly establish the conceptual advance over its closest predecessor, B-spline Policy.
  2. Insufficient evidence linking trajectory compressibility to task complexity. The non-uniform timing targets are obtained through offline spline fitting driven by action reconstruction error. This criterion measures how well a trajectory can be approximated, rather than directly measuring task difficulty, contact sensitivity, or the need for feedback. The paper does not quantitatively establish that the resulting knot density is a reliable proxy for these properties. The single successful trajectory visualization provides limited support for this broader interpretation.
  3. Mixed evidence for the claimed benefits in precision-sensitive manipulation. On Charge Earphone, SplineWAM with JP-RTC matches the baseline’s success rate but reduces progress from 73.3% to 57.5%, with the degradation attributed to the socket-insertion stage. This weakens the motivating argument that adaptive temporal allocation preserves precision where finer control is needed. A more detailed analysis of contact-stage errors and failure cases would help clarify this limitation.
  4. Modest gains over standard RTC under asynchronous execution. While JP-RTC improves substantially over naive spline-space RTC, its gains over action chunking with RTC are only 2.7 percentage points on LIBERO-Plus and 0.6 points on RoboCasa. Moreover, SplineWAM still suffers substantial degradation relative to synchronous execution. Given the single-run simulation results, limited real-robot trials, and lack of a latency sweep or detailed runtime measurements, the practical advantage of JP-RTC over the standard baseline remains insufficiently established.
Questions for authors

Hyperparameter selection: How were the fitting tolerance and execution row budget selected for each benchmark and real-robot task? Were they tuned on a separate validation set, and how sensitive are the reported gains to these choices?

4 comments
AuthorsOpenReview ↗

To Review 3BxB: Part 1 of 2

We thank the reviewer for the careful review.

W1. Novelty over B-spline Policy (BSP). The main novelty of this work lies in the problem it poses: we use the action representation to control the inference cost of a large policy, so that how much each call covers follows the motion rather than a fixed clock, and we measure the resulting trade-off between success and policy calls (Fig. 4).

BSP's representation alone is not enough for this in a WAM. Our Naive B-Spline baseline is that representation on the same backbone. It gains 4.4 / 0.4 points over action chunking at the Table 1 operating points, and on LIBERO-Plus this margin is gone by about 16 steps per call (59.9% vs. 59.6%), while SplineWAM still leads by 8.2 points at 17 steps. The difference comes from three problems that a variable-duration window raises in a WAM, which BSP does not face or handles differently. Solving them is our methodological contribution:

  • What the video should supervise. With a variable-duration window, video sampled on a fixed time grid no longer matches the actions. We place the supervised frames on the fitted knot times. BSP has no video branch. This gains +3.8 / +4.0 points over Naive B-Spline, and +7.8 at 17 steps per call on LIBERO-Plus.
  • How much of each prediction to commit. We make the committed prefix an explicit row budget, counted in knots. The committed span then adapts to the motion, while its trade-off with policy calls stays controllable. BSP instead requests the next segment shortly before the current one runs out. On the same model, the row budget improves over a fixed schedule at every matched executed length, by up to 3.2 points.
  • How to join consecutive predictions. Consecutive windows place their knots differently, so guidance on the spline parameters is ill-posed. JP-RTC imposes continuity on the decoded actions through the decoder Jacobian. With RTC applied to the parameters instead, a spline WAM falls below action chunking under delay, on both suites and in success on all three robot tasks; with JP-RTC it is at or above it (Tables 2, 7–8). BSP re-phases the new segment with a time shift, and disables this for delta actions.

We adopt BSP's representation, fitting and local windows, and in BSP, too, the predicted knots set how long a segment lasts. What prior work does not do is use this to control inference cost:

  • BSP targets faster execution through time rescaling, and does not measure inference cost.
  • Action compression methods such as FAST and BEAST reduce the tokens per fixed-length chunk, but not the number of calls.
  • Adaptive replanning methods add an auxiliary signal on top of a fixed representation.

W2. Knot density as a proxy for task complexity. We agree that a reconstruction criterion does not directly measure task difficulty. Our method uses knot density as a replanning signal: where knots are dense, a call covers less time and the policy is called again sooner. Beyond the visualisation, we tested whether knots are dense in the two situations where a policy most needs to replan: around contact in every training demonstration, and when a deployed rollout goes wrong.

  • Around contact. Knots are 1.06–1.41× denser within 0.5 s of a grasp or release in every dataset, simulated and real (82–100% of episodes). A call issued there commits less: 7.1 steps against 9.0 elsewhere on LIBERO, and 38.9 against 52.4 on Charge Earphone.
  • Mechanism. A grasp or release is a sharp change of the commanded action, so an error-bounded fit must place knots there. On RoboCasa and the robot, the concentration within 0.5 s comes from the gripper dimension. On LIBERO the arm motion adds to it: knots remain 1.11× denser without the gripper dimension. On Charge Earphone it adds over the longer span a call commits, which still shrinks without the gripper (48.8 against 60.8 steps).
  • RoboCasa. Its MimicGen demonstrations are dense in knots throughout, so the committed span barely changes near contact (17.8 against 17.9 steps). This is also where the row budget gains least over a fixed schedule.
  • When a rollout goes wrong. Demonstrations contain no errors, so we compared the deployed model's rollouts at the Table 1 operating point. Fewer steps per call mean denser predicted knots and more frequent replanning. Within a task, a failed episode executes fewer steps per call than a successful one with probability 0.68 on LIBERO-Plus and 0.86 on RoboCasa, and calls the policy 1.2× and 3.2× as often per control step; the gap opens as the rollout goes wrong, not from the first calls.

Knot density therefore marks grasp and release in every dataset, follows the arm motion around them where it changes, as on LIBERO, and rises when a rollout goes wrong.

AuthorsOpenReview ↗

To Review 3BxB: Part 2 of 2

W3. Charge Earphone. Success is tied (8/30 each), and the progress loss is confined to one stage. JP-RTC fails at the socket insertion in 14 of 30 trials, against 1 for action chunking. Among the trials that pass that stage, it completes the second precision insertion at least as often (USB-C: 8/16 vs. 8/29). The socket stage is also where joining consecutive chunks matters most: with spline-parameter RTC, 20 of 30 trials fail there. This agrees with simulation: under synchronous execution SplineWAM leads on all four RoboCasa task families, and asynchrony costs it more than it costs action chunking (W4 below). We therefore read this as a loss at one contact transition, not a general loss of precision. The evidence points to the asynchronous join, which JP-RTC improves (20 to 14 failures) but does not remove, although our robot runs do not separate it from the smoothness of the fit.

W4. Gains over RTC under asynchronous execution. JP-RTC does not compete with RTC: for a per-step chunk it reduces to RTC. Its role is to make a spline policy usable under delay, so the comparison that isolates it is with RTC applied to the spline parameters: +8.1 / +1.9 points in simulation (LIBERO-Plus / RoboCasa), and 7 / 8 / 18 successes against 3 / 4 / 14 on the three robot tasks. Against action chunking with RTC, the comparison is between two systems; we answer each part in turn.

  • "Gains over action chunking with RTC are only 2.7 and 0.6 points." A continuity mechanism can only recover what asynchrony takes away, and asynchrony costs action chunking 21.5 points on LIBERO-Plus but 4.0 on RoboCasa. JP-RTC's gains over action chunking with RTC scale accordingly: +2.7 and +0.6. The advantage is not only in accuracy. With a longer budget (H = 16), SplineWAM needs 0.63× the calls per control step at success comparable to action chunking with RTC (45.7 ± 0.8% vs. 44.9 ± 0.7%).
  • "Substantial degradation relative to synchronous execution." Synchronous simulation charges nothing for inference latency, so any deployed policy loses accuracy when it must act while inferring. The baseline drops by 21.5 / 4.0 points as well. SplineWAM drops somewhat more (27.0 / 7.8), as Appendix F acknowledges, but with JP-RTC it still ends at or above the baseline on both suites.
  • "Single-run simulation results." We retrained action chunking and SplineWAM with two further seeds, and evaluated every run on all 10,030 LIBERO-Plus episodes. Under the protocol of Table 8, JP-RTC reaches 47.6 ± 0.8%, against 44.9 ± 0.7% for action chunking with RTC, and every JP-RTC run is above every RTC run.
  • "Lack of a latency sweep or detailed runtime measurements." We measured the deployed runtime on the real-robot computer (RTX 4090, 10 denoising steps). A call takes 240 ± 10 ms for both methods, and JP-RTC adds at most 10 ms, against at most 2 ms for RTC. At 30 Hz this stays within 8 control steps, the delay simulated in Tables 7–8, so the evaluated delay is the one the policy faces in deployment.
  • "Limited real-robot trials." With 30 trials per task, the confidence intervals overlap, so we claim comparable success rather than higher accuracy. What the robot shows consistently is the cost advantage. JP-RTC matches or exceeds the success of action chunking with RTC on all three tasks, while executing 1.2–1.6× as many steps per call.

Overall, the practical advantage of SplineWAM with JP-RTC over the standard baseline is comparable or higher success at a lower inference cost, both in simulation and on the robot.

Q1. Hyperparameter selection. ε and H were chosen on the evaluation suites, with no separate validation set; the baseline's budget is its best point on the same data.

  • ε. We did not search for an optimal ε. We evaluated only four widely spaced compression ratios (raw actions per knot ≈1.0 / 1.5 / 2.0 / 3.0). The value 1.0 tests whether the B-spline representation is learnable at all, and 3.0 tests heavy compression of the action space. We do not claim that 1.5 is optimal, since we did not search around it. It is a rough value at which the method works on both suites, and it was carried over unchanged to the robot.
  • Sensitivity to ε. At the four ratios, the gain over action chunking is +7.1 / +8.2 / +1.4 / −21.1 points on LIBERO-Plus and +3.7 / +4.4 / +2.7 / −1.6 on RoboCasa. It is stable up to 1.5 and is lost under heavier compression, so the practical rule is to keep the compression at or below 1.5.
  • H. H needs no retraining, so we evaluated a range of budgets for action chunking, Naive B-Spline and SplineWAM (Fig. 4). Table 1 uses one row budget per suite for all spline variants, chosen so that SplineWAM executes about as many steps per call as the baseline's best point; for action chunking and SplineWAM this is also their best point. At matched executed length, SplineWAM leads at every budget, with the margin narrowing at the longest budgets. On the robot, a call executes the whole window.

Thanks. The Naive B-Spline baseline isolates the representation from your three additions; the knot-density evidence around contact and in failed rollouts answers W2; the two extra seeds over all 10,030 episodes, with every JP-RTC run above every RTC run, plus the runtime measurement, settle W4. Q1's sensitivity curve reads as a plateau rather than a tuned spike, which I accept. W3 still reads as a hypothesis rather than a finding. I am raising my score from 2 to 3.

Thank you for taking the time to consider our response and share your updated assessment. We appreciate your encouraging feedback and are glad our clarifications were helpful.

Comments on the paper

2 comments
AuthorsOpenReview ↗

General Response and Revision Summary to Reviewers

We thank all reviewers for their careful and constructive reviews. We have revised the paper to address every comment, and all changed text is highlighted in blue. The main changes are:

  1. Contribution and relation to B-spline Policy. We state more sharply the problem the paper addresses, using the action representation to control the inference cost of a large policy, and which components we adopt from B-spline Policy and which are new (Sec. 1–2).
  2. New experiments.
    • The same trained model under fixed-step and row-budget execution, at matched steps per call (Fig. 4). The row budget is never below the fixed schedule, and is up to 3.2 points above it at longer executed spans.
    • Three training seeds of action chunking and SplineWAM on LIBERO-Plus, synchronous and asynchronous, including an asynchronous operating point with a longer budget (Table 8). Synchronously, SplineWAM reaches 74.3 ± 0.8% against 66.5 ± 0.8%. Under an 8-step delay, it reaches 47.6 ± 0.8% against 44.9 ± 0.7% at essentially the same cost, or 45.7 ± 0.8% with 0.63× the policy calls per control step.
    • Runtime measured on the real-robot computer, including the overhead of JP-RTC (App. E). A call takes 240 ± 10 ms under both representations, and JP-RTC adds at most 10 ms.
    • Knot density around contact events over all training demonstrations, simulated and real-robot, and in failed rollouts (App. G). Knots are 1.06–1.41× denser around grasp and release in every dataset, and failed rollouts call the policy 1.2× (LIBERO-Plus) and 3.2× (RoboCasa) as often per control step.
    • A spline-degree ablation with k = 1, 3, 5 (App. H). Cubic is the best degree; quintic is 1.0–2.8 points lower, and linear is 4.7–7.4 points lower but still above action chunking.
  3. Calibrated claims.
    • The real-robot results are now stated as comparable success at a lower inference cost.
    • What knot density represents, and its limitations, are clarified.
    • The gain of knot-grid over window alignment is described as a smaller further gain on top of aligning the video at all.
  4. Hyperparameter selection. How ε and H were chosen, on the evaluation suites and with no separate validation set, is now stated explicitly (Sec. 4.1).
  5. Presentation. Rows and the row budget are now defined earlier, in Sec. 3.2.
AuthorsOpenReview ↗

Discussion Period Summary

We sincerely thank the AC and all reviewers for their time and constructive feedback, which has substantially improved the paper. As the discussion closes, we would like to summarise the concerns raised and how we addressed them.

  1. Novelty over B-spline Policy (3BxB, kCpT). We clarified what is adopted and what is new. Its representation on our backbone gains +4.4 / +0.4 points over action chunking (LIBERO-Plus / RoboCasa), against +8.2 / +4.4 for the full method; 3BxB: this baseline "isolates the representation from your three additions".
  2. Is the adaptive schedule isolated? (kCpT, pQHY) We ran the requested control, the same model on a fixed schedule: at matched steps per call, the row budget is never below it (+0.5 to +3.2 points on LIBERO-Plus, at most +0.9 on RoboCasa).
  3. Knot density and replanning (3BxB, kCpT). Knots are 1.06–1.41× denser around grasp and release in every training dataset, and failed rollouts call the policy 1.2× / 3.2× as often. 3BxB: this "answers W2".
  4. Asynchronous evidence and runtime (all reviewers). Three seeds under an 8-step delay on LIBERO-Plus: 47.6 ± 0.8% vs. 44.9 ± 0.7% at equal cost, or 45.7 ± 0.8% at 0.63× the calls (0.65× the accelerator time with JP-RTC's overhead). A call takes 240 ± 10 ms either way. 3BxB: these "settle W4".
  5. Why cubic splines (kCpT). In a degree ablation, cubic is best; quintic is 1.0–2.8 points lower, linear 4.7–7.4 lower but still above action chunking.
  6. Hyperparameter selection (3BxB, pQHY). We state that ε and H were chosen on the evaluation suites and give the gain at all four tolerances, "a plateau rather than a tuned spike" to 3BxB.
  7. Real-robot claims (3BxB, pQHY). We withdrew "costs no accuracy" and claim comparable success at 0.61–0.83× the calls. The Charge Earphone progress gap is confined to the socket insertion (14/30 failures vs. 1/30); its cause "still reads as a hypothesis" to 3BxB and is stated as a limitation.

Status

  • 3BxB raised the rating from 2 to 3.
  • pQHY (3) has not yet replied; both experiments it requested (items 2 and 4) were run as specified.
  • kCpT (2) acknowledged "the reported performance gains" and remains unconvinced of "the proposed connection between action complexity, feedback needs, and better allocation of inference". We agree with the facts behind this concern: (i) much of the gain remains on a fixed schedule; (ii) the knot-density analysis is correlational, the causal evidence being the control in item 2; (iii) sampling video at knot times adds a smaller 1.2 / 1.6 points over window alignment. This limits the mechanism story (calls go where feedback is needed), not what the method delivers: higher success at equal cost, or comparable success with fewer calls.

We are grateful for the reviewers' engagement throughout the discussion and would be glad to provide any further clarification. Thank you again for your time and consideration.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.