acceptodds
Under review as a conference paper at ICLR 2027

PRIM: Past-Action Regularization for Imitation with Flow Matching

Abstract

Behavior cloning commonly uses Flow Matching to learn distributions over future actions from expert demonstrations. Expert behavior can depend on temporal information absent from the current observation, making past actions a natural source of context for improving action prediction. However, strong temporal correlations in expert actions encourage policies to become highly sensitive to action history, allowing model-generated errors to be amplified when executed actions enter subsequent policy inputs. We introduce Past-Action Regularization for Imitation with Flow Matching (PRIM), which penalizes the velocity field’s Jacobian with respect to action history to reduce the sensitivity of generated actions to history errors. We show that strong conditional predictability from past actions requires history sensitivity, and derive the recursive propagation of action errors through environment dynamics and history inputs. Under sufficient conditions, controlling this sensitivity keeps accumulated errors bounded. Experiments on LIBERO-90 show that directly conditioning on past actions can lower success rates, while Jacobian regularization suppresses error accumulation and improves success.

Primary area:
applications to robotics, autonomy, planning
Keywords:
behavior cloning, flow matching, action chunking, non-Markovian control, Jacobian regularization, visuomotor policy learning

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_YCNjOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper studies error propagation in imitation policies conditioned on past actions. PRIM penalizes the Flow Matching velocity field’s sensitivity to action history, aiming to retain useful temporal information while reducing feedback from execution errors. The analysis relates temporal action correlations to history sensitivity and derives sufficient conditions for bounded error accumulation. Experiments on LIBERO-90 and four RoboMimic tasks evaluate task success, recursive history feedback, and robustness to perturbations.

Critical strengths
  1. PRIM provides a simple, targeted remedy for error propagation through action history. Results across four history lengths and three training seeds show substantial improvements on LIBERO-90, exceeding the no-history baseline after further training. Gains on ToolHang and Transport provide additional evidence of its usefulness beyond a single benchmark.
  2. The experiments provide convincing evidence for the proposed mechanism. Matched observations and sampling noise isolate history feedback, including propagation from model-generated initial errors. Simulator interventions further examine action deviations and task completion under shared perturbations. Together, these tests support the claim that PRIM reduces propagation of the tested errors.
Critical weaknesses
  1. The practical relevance of the theoretical guarantee remains unclear. The error bound relies on state-feedback contraction and a uniform relationship between the training penalty and sampling-path sensitivity. These assumptions are stated explicitly, but their validity for the evaluated policies is not established. The analysis therefore offers a conditional explanation without showing whether the bound is informative in the reported tasks.
  2. The comparisons leave uncertainty about PRIM’s advantage over alternative approaches. PALR and TCA use fixed auxiliary weights without a reported sensitivity study, and the TCA adaptation omits the original method’s information-bottleneck stage. The dropout control is also limited to one probability and one seed. Although the reported comparisons favor PRIM, broader tuning is needed to establish how much of the performance gap persists under other reasonable configurations.
Questions for authors
  1. Is there empirical evidence supporting the key conditions behind the error bound for the evaluated policies, particularly state-feedback contraction and the sampling-path comparison in Eq. (21)?
  2. How were the PALR and TCA weights selected, and have alternative values been evaluated? What motivated omitting TCA’s information-bottleneck stage?
  3. Could representative simulator executions and failure cases be provided to interpret the numerical gains? In particular, do they reveal fewer repeated errors, better recovery after deviations, or other changes in behavior?
5 comments
AuthorsOpenReview ↗

Response to Reviewer YCNj

Thank you for recognizing PRIM’s targeted design and our controlled experiments on history feedback.

1. Theorem 2 and empirical evidence

Theorem 2 gives sufficient conditions for bounded error accumulation. State feedback contraction leaves room to reduce the total error gain below one, and Eq. (21) assumes that at each relevant input the expected history Jacobian energy along sampling paths is bounded by CC times the training penalty. Under these conditions, reducing the history Jacobian limits the feedback of past errors even as each round introduces new errors.

PRIM’s mean history Jacobian energy was about 14% lower along sampling paths than on training interpolations. With observations and sampling noise matched, PRIM reduced cumulative action deviation energy by 54% over four replanning rounds and 72% over eight, showing that low sensitivity persists during generation and limits error accumulation across replans.

2. Baseline tuning and the information bottleneck

Auxiliary weights and dropout. The submission used fixed weights of 1 for PALR and 0.1 for TCA. We tested a tenfold increase and decrease and added PRIM and dropout settings. All runs use the same flow matching backbone, five past actions, 20k updates, and 450 matched LIBERO-90 rollouts. The training seed is 42 unless specified.

MethodAdditional settingsSuccess
PRIMWeight 0.01 / 0.1 / 1073.78% / 77.33% / 76.22%
PALRWeight 0.1 / 1058.44% / 45.56%
TCAWeight 0.01 / 160.00% / 19.56%
History dropoutProbability 0.05 / 0.1 / 0.263.78% / 67.33% / 58.22%
History dropoutProbability 0.1, seeds 42 / 43 / 4466.37% ± 1.00 pp

PRIM remains ahead of the tested PALR, TCA, and dropout settings with the same action history input.

TCA’s information bottleneck. The original comparison kept the deterministic representation to isolate TCA’s adversarial loss. We now add a Gaussian information bottleneck (IB) after 10k TCA updates, following Wen et al.’s appendix. The full representation uses an identity initialization for the Gaussian mean and scales the listed KL coefficients by 100/3584100/3584. Columns show total training steps.

Setting after 10k100D, 20k100D, 40k100D, 60kFull, 20k
Continue TCA44.00%40.22%39.33%60.00%
Gaussian layer, KL = 0———51.11%
IB, KL = 0.000134.67%38.44%34.67%44.00%
IB, KL = 0.00115.33%19.78%30.89%18.22%
IB, KL = 0.011.33%2.44%3.11%17.11%

Continuing TCA gave the highest success at each tested budget and representation. Among runs starting from the same full representation checkpoint and adding the Gaussian layer, success fell as the KL coefficient increased.

Single actions and action chunks. PALR and TCA address copycat behavior when predicting the next action from observation history. In our setting, the policy executes a chunk prefix and writes its latest executed actions into the next input, so prediction errors can recur and accumulate across chunks. PRIM directly limits sensitivity along this feedback path. With the same initial history perturbation and fixed observations, action deviations grew over the first three replans for PALR and TCA and shrank for PRIM, which may help explain its advantage in this setting. PALR · TCA appendix

3. Representative executions and failure cases

In the LIBERO-90 task “pick up the book and place it in the right compartment of the caddy,” both policies succeed without perturbation. At step 50 we perturb the continuous action history coordinates identically and keep gripper history unchanged. Both policies reach the compartment opening. During insertion, the unregularized policy tilts the book outward, and it slips outside and stays there. PRIM seats the book in the right compartment at step 164, versus step 163 without perturbation.

In a grasp transition with fixed recorded observations and 16 matched noise draws, PRIM predicts closing in all 16 samples at frame 64, when the first closing command enters history. The unregularized history policy keeps predicting opening until frame 66, and the policy without history still predicts opening at frame 68. We will label these events and the perturbation timing in the supplementary videos.

Thank you for the detailed response. The additional experiments address most of my concerns, particularly about the baseline comparisons. The written description of the book-placement example was helpful. If you already have recordings and it’s easy to include them in the supplementary material, could you make them available during the rebuttal period? It would be helpful to see how both policies respond to the perturbation during insertion, either in the book-placement example or in a similar comparison. An unannotated clip is fine.

Dear Reviewer YCNj,

Thank you for your interest in the book placement example. We have added a video and action traces to the supplementary material showing a LIBERO-90 task in which the robot places a book in the right compartment of a caddy. Both policies succeed with clean action history.

At step 50, we apply the same one-time perturbation to the three past arm actions for both policies, keeping the gripper history, simulator state, and sampling noise matched. The perturbation shifts the most recent vertical command upward. Relative to its clean run, the unregularized policy then shifts all ten vertical commands in its next plan in that same direction, by 0.192 to 0.347 controller units. For PRIM, each corresponding change is smaller than 0.0005 controller units. Both policies still bring the book to the compartment opening. During insertion, however, the unregularized policy opens its gripper and the book ends up outside. PRIM completes placement at step 164, just one step later than in its clean run. In this example, a single perturbation to action history influences an entire sequence of subsequent commands, and PRIM substantially reduces that response.

I hope this example addresses your question about what sensitivity to action history looks like during execution. If any aspect of the experiment remains unclear, I would be happy to clarify it before the rebuttal period ends.

Reviewer_tZQWOpenReview ↗
Rating
3(+1)
Confidence
2
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

This paper considers the problem of adding past actions as input for imitation policies. Adding action history can lower action prediction error, but it has been shown to lower task success. The authors propose PRIM, which is a flow matching IL policy that penalizes the velocity field's Jacobian with respect to the action history, so that the learned policy is less sensitive to changes in the past actions. The paper also has some theory on how temporal correlations in the actions leads to history sensitivity and provides some bounds on the error accumulation. Results on LIBERO-90 show that PRIM retains good success rates for various action history sizes and this pattern shows somewhat on the RoboMimic experiments.

Critical strengths

The paper identifies a clear problem and performs several diagnostic experiments (Section 4). The findings clearly show that the past states contain predictive information of current actions (over several delay values) and that the expert behavior is non Markovian on LIBERO-90. The proposed method PRIM seems like a reasonable way to combat sensitivity to past actions.

Critical weaknesses

The biggest weakness of this paper is that PRIM doesn't seem to actually use action history. In Table 3, the PRIM values suggest that different runs of the same policy with different action histories are so small that effectively it seems like PRIM doesn't use the action history. In Table 1, the velocity MSE values are very close to the no history baseline of λ=0\lambda=0.

Questions for authors
  • What happens if the history input is masked or shuffled? This would diagnose better if history matters at all.
3 comments
AuthorsOpenReview ↗

The usage of history

Thank you for your careful review and for recognizing the problem, our diagnostic experiments, and PRIM as a reasonable approach to history sensitivity. Your question identifies a distinction that our presentation did not make clear enough between sensitivity to a small history perturbation and dependence on the content of history. We appreciate your suggestion to mask or replace history and have conducted these experiments.

1. Does PRIM use action history?

We evaluated a frozen PRIM policy with three past actions (K=3K=3), trained for 60k updates, seed 42. We kept its weights fixed and compared three history conditions over 1,800 LIBERO-90 rollouts, matching the initial states and sampling seeds across conditions.

  • Executed history supplies the policy’s last three executed actions.
  • Replacement history supplies three consecutive actions from another held-out demonstration of the same task at each replan, keeping the history-validity flags enabled. Using a contiguous segment preserves the temporal structure within the replacement history while changing its correspondence to the current execution.
  • Masked history sets both normalized history values and history-validity flags to zero.
History conditionSuccessful rolloutsSuccess
Executed history1,488/1,80082.67%
Replacement history221/1,80012.28%
Masked history179/1,8009.94%

PRIM’s successful control depends strongly on the supplied execution history. Replacing its content or removing it substantially lowers success. Because the model is frozen, these differences arise from changing the history input during execution. We will add this comparison and its setup to the paper.

2. Why is the perturbation response in Table 3 so small?

Section 6.3 measures how an initial history perturbation propagates through eight replanning rounds, with observations, states and sampling noise matched between two branches. For this experiment, each policy predicts 20 actions, executes the first P=10P=10, and writes the last K=5K=5 executed actions into the next history.

We denote the initial history difference by

δHinit:=Hinitperturbed−Hinitclean.\delta H_{\mathrm{init}} :=H_{\mathrm{init}}^{\mathrm{perturbed}} -H_{\mathrm{init}}^{\mathrm{clean}}.

With action dimension dad_a and initial physical RMS 0.01, its energy per coordinate is

Einit:=E∥δHinit∥22Kda=(0.01)2=10−4.E_{\mathrm{init}} :=\frac{\mathbb E\|\delta H_{\mathrm{init}}\|_2^2}{Kd_a} =(0.01)^2=10^{-4}.

At round rr, the difference between the executed prefixes and its energy are

δra:=arperturbed−arclean,Er:=E∥δra∥22Pda.\delta_r^a:=a_r^{\mathrm{perturbed}}-a_r^{\mathrm{clean}}, \qquad E_r:=\frac{\mathbb E\|\delta_r^a\|_2^2}{Pd_a}.

Each branch feeds its own executed tail back into history. If RR selects that tail and JraJ_r^a is the history-to-action Jacobian at round rr, the local propagation satisfies

δ8a≃J8aRδ7a≃J8aRJ7aR⋯RJ1aδHinit.\delta_8^a \simeq J_8^aR\delta_7^a \simeq J_8^aRJ_7^aR\cdots RJ_1^a\delta H_{\mathrm{init}}.

Thus, the final energy includes both the initial perturbation scale and repeated attenuation. Normalizing by initial energy and propagation length gives

(E8Einit)1/8=(3.26×10−1510−4)1/8=0.0489.\left(\frac{E_8}{E_{\mathrm{init}}}\right)^{1/8} = \left(\frac{3.26\times10^{-15}}{10^{-4}}\right)^{1/8} =0.0489.

The equivalent per-round energy gain is 4.9%. This is the scale of repeated local error propagation. History can still change the generated actions when its content changes substantially, as the replacement and masking experiments show.

We agree that reporting the raw final squared value alone made this distinction difficult to interpret. We thank you for highlighting the omission and will report normalization by both initial energy and propagation length.

3. Does PRIM retain a predictive benefit from history?

We additionally compare final-action prediction MSE using matched 20k checkpoints across three training seeds, evaluated on the same 2,048 validation examples with correct expert history and shared Gaussian noise. Actions are generated using ten Euler steps, and MSE is measured over the first ten actions in physical action coordinates. Entries report means and sample standard deviations across training seeds.

PolicyFinal-action MSE
No history, K=0K=00.04077 ± 0.00024
Unregularized history, K=3,λJ=0K=3,\lambda_J=00.02588 ± 0.00022
PRIM, K=3,λJ=1K=3,\lambda_J=10.03493 ± 0.00034

PRIM reduces prediction MSE by 14.3% relative to the no-history baseline, with an improvement in all three training seeds. PRIM jointly minimizes prediction loss and history-Jacobian regularization, so it gives up some of the prediction-error reduction obtained by unrestricted history use while retaining a measurable predictive benefit.

Together, these results show that PRIM retains predictive benefits and dependence on execution history while suppressing the repeated propagation of small history errors and make this distinction explicit when discussing Tables 1 and 3.

I thank the authors for their response and I agree that these results support the fact that PRIM does use the action history. The masking and shuffled history inputs further support this. In light of this, I raise my score.

Thank you for your kind suggestions and appreciation!

Reviewer_nJj4OpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The authors find that on LIBERO-90, adding past actions to the policy input lowers prediction error, but the success rate drops substantially. The cause is that execution errors are written back into the historical inputs and amplified. The argument and method have four parts:

  • They use a “U-shaped curve of delayed-observation prediction” to argue that expert data is non-Markovian, and show that the U-shape disappears once the previous action is added.
  • They prove that, under stationarity and a Poincaré inequality, the stronger the temporal correlation of actions, the higher the lower bound on the Jacobian of the conditional-mean policy with respect to history (Theorem 1).
  • Methodologically, they use finite differences and Rademacher projection to penalize the Jacobian of the velocity field with respect to history (PRIM).
  • They prove that under closed-loop contraction conditions, the error is bounded (Theorem 2).

Experimental results: about 85% success on LIBERO-90, exceeding the current-state baseline, with improvements on RoboMimic as well.

Critical strengths

The problem is real (copycat behavior / error write-back), and the diagnostic design is well done: Exp. II uses separate predictors to rule out the effect of chunk position, and Exp. IV uses a Markovian policy as a control.

Critical weaknesses

PRIM effectively makes the policy nearly ignore its history. The Jacobian energy drops by 5–6 orders of magnitude, velocity MSE returns to the current-state level (0.088 vs. 0.089), and the perturbation response in Table 3 is 1e-15. This contradicts the motivation of “retaining historical information that has predictive value.” By Theorem 1, strong correlation requires high sensitivity, so suppressing sensitivity necessarily gives up the predictive benefit of history, yet the paper does not discuss this trade-off.

Where does the roughly 7 pp improvement over the K=0 baseline come from? It may come from the regularization or smoothing effect of the finite-difference penalty rather than from history. Key ablations are missing: setting or randomly replacing PRIM’s history at test time, and K=0 with an equally strong regularizer of another kind.

The theoretical novelty is limited: Jacobian regularization already appears in work such as Hoffman and Contractive Diffusion Policy. The conditions in Theorem 2, namely α<1 (closed-loop state-channel contraction) and the constant C relating the training distribution to the sampling path, are hard to satisfy in contact-rich manipulation tasks, and this is not verified.

Questions for authors

Please refer to the weaknesses above.

  1. Have you checked whether PRIM’s regularization actually preserves historical information that is predictive? Given that, where does the improvement over the baseline come from? You could set PRIM’s history to 0, or add other regularizers, to clarify this.

  2. Please clarify the use of Jacobian regularization in other papers and your theoretical contribution beyond it.

If both of these are addressed, I would be happy to raise my score.

6 comments
AuthorsOpenReview ↗

The Usage of History and Controlled Error Propagation

Thank you for recognizing our diagnostics and for your valuable questions.

1. The usage of history

1.1 Does PRIM really use history?

Section 6.3 matches observations, states and noise between branches. Their initial history difference is:

δHinit:=Hinitperturbed−Hinitclean.\delta H_{\mathrm{init}}:=H_{\mathrm{init}}^{\mathrm{perturbed}}-H_{\mathrm{init}}^{\mathrm{clean}}.

For KK actions of dimension dad_{a} and RMS 0.01, initial energy per coordinate is:

Einit:=E∥δHinit∥22Kda=(0.01)2=10−4.E_{\mathrm{init}}:=\frac{\mathbb{E}\|\delta H_{\mathrm{init}}\|_{2}^{2}}{Kd_{a}}=(0.01)^{2}=10^{-4}.

For P-action prefixes at round r, define:

δra:=arperturbed−arclean,Er:=E∥δra∥22Pda.\delta_{r}^{a}:=a_{r}^{\mathrm{perturbed}}-a_{r}^{\mathrm{clean}},\qquad E_{r}:=\frac{\mathbb{E}\|\delta_{r}^{a}\|_{2}^{2}}{Pd_{a}}.

Each branch writes its last K executed actions into history. R selects the tail; Jacobians are indexed by round:

δ8a≃J8aRδ7a≃J8aRJ7aR⋯RJ1aδHinit.\delta_{8}^{a}\simeq J_{8}^{a}R\delta_{7}^{a}\simeq J_{8}^{a}RJ_{7}^{a}R\cdots RJ_{1}^{a}\delta H_{\mathrm{init}}.

Normalize by initial energy and eight rounds:

(E8Einit)1/8=(3.26×10−1510−4)1/8=0.0489.\left(\frac{E_{8}}{E_{\mathrm{init}}}\right)^{1/8}=\left(\frac{3.26\times10^{-15}}{10^{-4}}\right)^{1/8}=0.0489.

The equivalent energy gain is 4.9% per round. Final energy includes initial scale and repeated attenuation, measuring local error propagation. We apologize for omitting normalization and will add it. Thank you for pointing this out.

1.2 Does PRIM retain predictive history information?

We evaluate frozen PRIM (K=3, 60k) on 1,800 matched LIBERO-90 initial states and noise seeds, varying history:

  • Executed history: the policy’s last three executed actions.
  • Replacement history: three consecutive actions from another same-task held-out demonstration at each replan, with valid history flags.
  • Masked history: normalized history values and validity flags set to zero.

Success is 82.67%, 12.28% and 9.94%, respectively. PRIM relies on its execution history for successful control while attenuating small local history perturbations.

2. History feedback and the Jacobian

2.1 How are history errors propagated?

With 20 predicted actions, ten executed and five in history:

Ht+10=at+5:t+9,Ht+10−Ht+10e=at+5:t+9−at+5:t+9e.H_{t+10}=a_{t+5:t+9},\qquad H_{t+10}-H_{t+10}^{e}=a_{t+5:t+9}-a_{t+5:t+9}^{e}.

With e denoting the expert, the next error is:

δt+10:t+19a≃εt+10:t+19+Jt+10sδt+10s+Jt+10aδt+5:t+9a.\delta_{t+10:t+19}^{a}\simeq\varepsilon_{t+10:t+19}+J_{t+10}^{s}\delta_{t+10}^{s}+J_{t+10}^{a}\delta_{t+5:t+9}^{a}.

The terms are fresh error, state feedback and history feedback; the executed tail enters history again. PRIM limits this repeated amplification.

2.2 How does PRIM use Jacobian regularization differently?

Hoffman et al. penalize the input-to-logit Jacobian to improve classification robustness against input perturbations.

CDP constrains the score Jacobian with respect to the noisy action being denoised. This promotes contraction of nearby sampling trajectories and reduces solver and score-estimation errors during action generation.

PRIM regularizes the velocity field with respect to executed action history H. With noisy chunk XτX_{\tau} and current condition S, its target is:

LJ=1KdaE∥δvθ(Xτ,τ,S,H)δH∥F2.\mathcal{L}_{J}=\frac{1}{Kd_{a}}\mathbb{E}\left\|\frac{\delta v_{\theta}(X_{\tau},\tau,S,H)}{\delta H}\right\|_{F}^{2}.

All three use Jacobians, but PRIM targets error feedback across successive action chunks. An executed error enters the next history and can distort the next chunk, whose tail is written back again. Our contribution connects temporal action correlation, history sensitivity and repeated error feedback. This motivates the history derivative as a regularization target; our analysis links its penalty to generated-action sensitivity and accumulated error.

2.3 Is the gain due to generic smoothing?

With matched 20k updates and 450 rollout initializations, best tested success is 77.33% for PRIM, versus 67.33% for dropout, 68.89% for second-order history regularization and 54.00% for current-state Jacobian regularization without history. Controls use calibrated parameter-gradient strengths.

3. Interpretation of the theory

Theorem 1 concerns conditional-mean optimality; PRIM trades prediction accuracy for lower error amplification through its joint objective. Finding the optimal task-specific trade-off is beyond this paper’s scope.

Theorem 2 gives sufficient conditions: α < 1 requires state-channel contraction; C bounds sampling-path history-Jacobian energy relative to the training penalty at each relevant local input. Under these assumptions, sufficiently small history sensitivity bounds accumulated error despite fresh errors each round.

Thanks for your detailed response. Most of my concerns have been addressed.

Remaining point:

The masked/replaced-history results show reliance on history, but they may also reflect out-of-distribution inputs. Do you have evidence that history provides a predictive benefit under PRIM, given that velocity MSE (0.088) matches the current-state model (0.089)?

Thank you very much for your understanding! We acknowledge that averaged velocity MSE alone provides an incomplete picture of the predictive benefit retained by PRIM. We therefore examine final-action prediction MSE, which compares the final actions generated by the Flow sampler with expert actions.

We evaluate 20k checkpoints across three training seeds on the same 2,048 held-out expert conditions, using correct action history. Sampling uses ten Euler steps, and action MSE is measured over the first ten generated actions in physical action coordinates. Entries report means and sample standard deviations across training seeds.

ModelVelocity MSESingle-sample action MSEAction MSE of 20-sample mean
Current-state baseline, K=0K=00.089220.04077 ± 0.000240.02829 ± 0.00015
Action history, K=3,λJ=0K=3,\lambda_J=00.059610.02588 ± 0.000220.01813 ± 0.00018
PRIM, K=3,λJ=1K=3,\lambda_J=10.087610.03493 ± 0.000340.02358 ± 0.00024

PRIM reduces single-sample action MSE by 14.3% and the MSE of the sample mean by 16.7% relative to the current-state baseline. Both improvements occur in all three training seeds. The unregularized K=3K=3 model achieves the lowest prediction error, while PRIM retains part of this predictive benefit under Jacobian regularization.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.