acceptodds
Under review as a conference paper at ICLR 2027

Action-Consistent Attention with History Flow Matching

Abstract

Generative policies based on flow matching have achieved remarkable success in robotics by formulating action-chunk prediction as a conditional probability path over a sequence of time-steps. However, when predicting the conditional velocity field, existing methods typically treat the action chunk as a flattened vector sequence, overlooking two fundamental semantic structures: the temporal evolution of each individual joint across consecutive steps, and the inter-joint coordination required within each time-step for physically coherent control. In this paper, we propose AC-HFlow (Action-Consistent History Flow Matching), a novel paradigm that reduces multi-step inference to a few integration steps, with as few as one, without sacrificing performance, while capturing coherent action semantics. It employs Action-Consistent Attention (ACA) to explicitly reason over the intra-chunk joint–time grid, capturing the partial order of joint evolution across timestamps and the directed coupling among joints within the same step. Meanwhile, pooled historical action statistics serve as an informative source for flow matching, which mitigates the overhead introduced by random-noise initialization. Extensive experiments demonstrate that AC-HFlow delivers superior action consistency, strong generalization capability under visual and scene shifts, and effective few-step inference in which a single integration step already attains competitive success.

Primary area:
applications to robotics, autonomy, planning
Keywords:
Action-Consistent Attention, Flow Matching, History Information

Reviews

Among:
Shape:
1234this paper, Bottom 2%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.33 over 3 reviews, above 2% of the others.
Reviewer_wE3cOpenReview ↗
Rating
1
Confidence
2
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

This paper proposes AC-HFlow, a flow-matching policy for visuomotor action-chunk prediction that comprises two components: 1) Action-Consistent Attention (ACA) projects timestep features into a learned joint–time grid and applies a union mask combining causal temporal attention within each slot and directed attention between slots at each timestep. 2) History Flow Matching constructs a flow source using historical action means and standard deviations, followed by affine standardization.A positional-encoding variant, GRoPE, applies successive rotary transformations along the temporal and joint-slot indices.

Critical strengths
  1. A simple and well-motivated architectural idea. Explicitly exposing the joint–time structure of an action chunk to the velocity network is a reasonable question for chunked generative policies. ACA is a lightweight, zero-gated, and drop-in layer that does not change the training objective, so it would be easy to adopt in existing flow and diffusion policies.

  2. Transparent reporting of the experimental setup. The appendix provides per-task results for every step count, the LIBERO-Plus scene IDs and counts, and an explicit discussion of what the diagnostics do not claim. This makes it possible to check the claims in detail, and the authors are honest that the cross-family comparisons are not compute-matched.

Critical weaknesses

Weakness 1: The History Flow Matching source does not carry the history information it is credited with.

After the standardization in Eq. 11, the source simplifies to A0=(pσH⊙ϵh+(1−p)ϵn)/p2σH2+(1−p)2A_0 = \left(p\sigma_H\odot\epsilon_h + (1-p)\epsilon_n\right) / \sqrt{p^2\sigma_H^2+(1-p)^2}. If ϵh\epsilon_h and ϵn\epsilon_n are sampled per element, as the text suggests, then A0∼N(0,I)A_0\sim\mathcal{N}(0,I) exactly, and the source is identical to standard flow matching. If ϵh\epsilon_h is instead shared across the horizon, the history mean μH\mu_H still cancels out, and the only remaining effect is a per-control temporal correlation. In either case, the claims that the source is "aligned with the previously executed motion" and that it "shortens the transport distance" are not supported by the formulation. There is also a mismatch between training and inference: the buffer stores ground-truth history during training but the policy's own predicted chunks at test time.

The paper's own results are consistent with this concern. In Table 9, the union model reaches 78.1% at a single step both with and without history, and adding history to ACA improves the average by only 0.9 points (78.6 → 79.5). This weakens the second contribution, which credits the history-shaped source with enabling few-step inference.

Weakness 2: Unconvincing empirical evidence for the claimed advantages.

Much of the reported gain is measured against the FM-DiT baseline, which scores 0% on three of the seven tasks (24.3% average), while DDPM-DiT, from the same backbone family, reaches 73.8%. Every single modification (history only, joint-only ACA, time-only ACA, or union ACA) restores performance to roughly 78%. Given Weakness 1, this pattern points to an issue with the baseline configuration rather than to the proposed inductive bias. Likewise, the joint-only, time-only, and union variants perform almost identically, so the ablation can not show that the specific partial order matters.

For example, the paper lacks controlled comparisons between:

  • union-masked ACA and dense (unmasked) attention on the same J×TJ\times T grid;
  • the history-shaped source and plain Gaussian noise under an otherwise identical code path;
  • AC-HFlow and the baselines at the same small step budgets (e.g., DDIM at 10 steps rather than 90).
Questions for authors
  1. The second contribution centers on the history-shaped source. Could the authors clarify how ϵh\epsilon_h is sampled in Eq. 10 and state the resulting distribution of A0A_0? Would it be possible to report a variant that uses N(0,I)\mathcal{N}(0,I) as the source with an otherwise identical code path, to verify that the gain actually comes from the history statistics?

  2. The FM-DiT baseline fails completely on four tasks. Could the authors clarify whether it is exactly AC-HFlow with ACA and history removed (same normalizer, AdaLN conditioning, EMA, solver, and parameter count), and explain why it performs so much worse than DDPM-DiT? Reporting parameter and FLOP counts for all policies would also help.

  3. The paper does not compare the union mask against dense attention on the joint–time grid or against a parameter-matched extra attention layer. Would it be possible to report such comparisons, so that the effect of the proposed relational structure can be separated from the added capacity?

  4. The joint factors are learned linear projections of the hidden feature, and on LIBERO they factorize a 7-D OSC end-effector action rather than physical joints. Could the authors clarify why the slot ordering is meaningful, and whether performance changes when the order is randomly permuted or when JJ is set to the action dimension?

I would be willing to raise my score if the authors can adequately address my concerns.

Reviewer_KjFHOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes AC-HFlow, a flow-matching-based visuomotor policy designed to improve action consistency and inference efficiency. The method introduces two main components. First, Action-Consistent Attention represents action chunks as a joint-time grid and employs a union-masked attention mechanism to explicitly model temporal dependencies within individual joint factors and coordination among different joint factors at each timestep. Second, History Flow Matching leverages statistics from previously executed actions to construct a history-informed source distribution, aiming to provide a more informative initialization and enable efficient action generation with fewer integration steps. Experiments conducted on LIBERO, LIBERO-Plus, and a real robotic arm demonstrate the effectiveness of the proposed method.

Critical strengths
  1. Efficient few-step action generation. AC-HFlow achieves competitive performance with only a few flow integration steps, demonstrating its potential for efficient robotic action generation. The ability to maintain strong performance with reduced inference steps is particularly valuable for real-time robotic control.

  2. Structured attention design and comprehensive evaluation. ACA explicitly models temporal dependencies and inter-joint coordination within action chunks, providing a meaningful inductive bias for consistent action generation. Experiments on LIBERO, LIBERO-Plus, and a real robot demonstrate the effectiveness of the proposed method and its potential for generalization to different environments.

Critical weaknesses

Could the authors explain how the final source distribution retains historical information after Eq. (11)? Under the stated independent Gaussian assumptions, the normalized source appears to follow N(0,I), independently of the action history. Is there any additional coupling, conditioning mechanism, or implementation detail not captured by these equations?

Questions for authors

Could the authors clarify whether Eq. (11) is applied exactly as described during training and inference? If so, how can the history-conditioned initialization lead to the substantial performance improvement reported in Table 2, given that the normalized source distribution appears equivalent to standard Gaussian noise? Could the authors provide additional implementation details or ablation experiments to clarify the contribution of HFM?

Reviewer_9fjZOpenReview ↗
Rating
1
Confidence
3
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

The paper proposes AC-HFlow, a visuomotor flow-matching policy with two components. Action-Consistent Attention lifts horizon token representation into learned factors and applies a union mask over temporal and same-step factor relations, while retaining the backbone's ordinary horizon attention. History Flow Matching tries to extract statistics from the past action history to construct a meaningful prior. Validation includes LIBERO comparisons in 7 tasks, a LIBERO-Plus microwave robustness evaluation.

Critical strengths

Table 1 and 10 demonstrates that AC-HFlow policy works under limited inference budgets.

Critical weaknesses
  1. The most important issue lies in the construction of A0A_0 in section 3.3 History Flow Matching. Based on equations 10 and 11, it looks like A0∣H,p∼N(0,I)A_0\mid\mathcal H,p\sim\mathcal N(0,I) where no historical statistics were baked into this prior distribution. Unless there is a process missing, the claimed historical related prior is absent.
  2. The experiments do not clearly isolate the the physical action structure from ACA. It is good to see the ablation table 2. about the removing joint or control time branch, yet the scenario of keeping both branches with different masks is overlooked.
Questions for authors
  1. Explain how history dependency survives after the standardization. If this is properly addressed, I may raise my score.
  2. Does ACA improve the performance if there is denser visibility in the mask?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.