acceptodds
Under review as a conference paper at ICLR 2027

DAKF: Tracking Nonstationary Dynamics with Precision-Weighted Excitation

Abstract

Tracking changing dynamics requires deciding when to retain a model and when to adapt from noisy, dependent observations. We introduce the Dual-Arm Kalman Filter (DAKF), coupling retained-model filtering and adaptive identification through hard selection and majority synchronization. Precision-weighted persistent excitation (PW-PE) yields a finite-sample identification bound under calibrated conditional noise, with explicit filtering-bias and drift costs and a matching scalar precision lower bound. A filtered-response bias bound connects this guarantee to model mismatch, state uncertainty and selection. Experiments cover synthetic changes, variance diagnostics and recorded UAV braking, with covariance-matched classical baselines and an official KalmanNet implementation. Synthetic ablations support the architecture; the braking benchmark uses common ground-truth initialization, with cold-start sensitivity reported separately.

Primary area:
learning on time series and dynamical systems
Keywords:
Nonstationary System Identification, Kalman Filtering, Time Series

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_T3CsOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper studies PPO fine-tuning of flow policies using the change-of-variables density. For general velocity fields, it shares Hutchinson probes across the old and new policies to reduce noise in the log likelihood-ratio estimate, and uses stiffness-graded integration nodes to improve numerical stability. It also introduces EDC, where the Jacobian diagonal is a bounded network output, making divergence computation inexpensive. The split-EDC variant provides an invertible discrete map with exact density computation. Experiments on Robomimic state and pixel tasks and MimicGen state tasks show competitive success rates and sample efficiency against several on-policy baselines.

Critical strengths
  1. Sharing probes across the PPO ratio is a useful idea. The analysis explains how this makes the estimation noise depend on the Jacobian difference between the old and new policies, giving a clear reason why sharing probes can help during PPO updates.
  2. The paper provides both a general approach and an EDC parameterization that makes divergence computation cheaper. The experiments cover several manipulation tasks with state and pixel observations. The parameterization ablations and wall-clock comparison are also useful for assessing the learning performance and computational cost of the method.
Critical weaknesses

None.

Questions for authors
  1. In Section 4.1, what metric and evaluation setting did the authors use to select the BC checkpoints? How close were the initial success rates across methods?
  2. Figure 3 compares shared probes with the exact trace only on Square, and Figure 4 uses one seed per setting. Could the authors clarify whether the Figure 4 results hold across seeds and whether a single shared probe also performs comparably to the exact trace on other tasks?
  3. In Figure 8, split-EDC is labeled “n=1–3,” but the caption says three seeds for each method. Could the authors report how many seeds were used for each task?
  4. Is the stiffness-graded mesh in Appendix E fixed during RL fine-tuning? If so, did the authors check whether it remains suitable as the policy changes?
  5. As an additional validation, could the authors add experiments on real-world robot data to assess the numerical stability of log-density computation? Ask only questions whose answers could materially affect your assessment. Distinguish clarification requests from suggestions for future work.
1 comment
AuthorsOpenReview ↗

Kindly Reminder to Reviewer T3Cs

Dear Reviewer,

Due to your busy schedule, we understand that you may not have had sufficient time to look at our previous message yet.

We would like to kindly follow up on the possible review mismatch we raised earlier. As mentioned, the review discusses PPO fine-tuning of flow policies, Hutchinson probes, EDC, and split-EDC, whereas our submission focuses on dual-arm Kalman filtering for non-stationary dynamics.

Could you kindly double-check whether the review was correctly assigned to our submission and let us know?

We sincerely appreciate your time and understanding.

Best regards,

The Authors

Reviewer_ZwWmOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
2
Soundness
3
Read review
Paper summary

This paper proposes Dual-Arm Kalman Filter (DAKF) that simultaneously performs online state estimation and model updating in a non-stationary environment, such as UAV flight, where the system dynamics change over time. The DAKF simultaneously employs two distinct processing streams: estimation based on a reliable model and a new model estimation. It performs stationary filtering using the most recently synchronized model while detecting changes in the dynamics and switching (synchronizing) models to cope with sudden variations in the system. The proposed method is supported by theoretical analysis and experimental evaluation.

Critical strengths

The proposed method presents a certain degree of technical advancement. While conventional MMAE, IMM, and Switching Kalman filters employ multiple models in parallel, the proposed method utilizes an architecture in which “arms” with different roles operate in parallel. One of these arms estimates and learns unknown transition models online and supports switching to a more suitable model. This novel architecture offers the advantage of eliminating the need to prepare candidate models in advance.

Critical weaknesses

In the evaluation experiments, the proposed method demonstrated effectiveness only under conditions favorable to it. In other words, from a practical viewpoint, the situations in which the proposed method is effective may be extremely limited. Specific concerns include the fact that it functions effectively only when the initial velocity is known precisely; that its effectiveness is claimed based on synthetic simulations that use a specific section where an abrupt change occurs with additive noise; and that it performs worse than existing methods in evaluations involving scenarios where parameters change gradually. Furthermore, the ablation experiments do not sufficiently demonstrate the effectiveness of the proposed architecture. The authors explicitly mention these points, which adds credibility to this paper. However, from a practical viewpoint, the evaluation was conducted solely within a sandbox environment conveniently configured for the proposed method, and its effectiveness cannot be said to have been sufficiently verified.

Questions for authors

It is possible that situations in which the proposed method functions effectively may in fact be extremely important in practical applications. For example, in what sense is it important for the UAV case to follow a sudden deceleration? Does it occur frequently in practice, or, even if it is a rare event, does it lead to extremely serious consequences? Within the scope of what can be inferred from the current experiments and their results, how would you justify the importance of this method in practical terms?

2 comments
AuthorsOpenReview ↗

Response to Reviewer ZwWm

We thank the reviewer for recognizing DAKF’s model-bank-free adaptation. Our revisions are highlighted in orange in the revised manuscript.


Response to W1: Initial velocity.

Appendix C (Tables 15–16) shows tracking degradation under zero-velocity initialization. Our windows begin during mid-flight motion, with pre-event speeds of 5.71–14.54 m/s (Table 12). Zero initialization mismatches both state prediction and Arm 2's velocity regression. The ten-step Arm-1 warm-up addresses selection, not initialization.

He et al. [3] (CVPR 2023) estimate initial velocity from visual-inertial observations, reporting 0.09 m/s mean velocity RMSE on successfully initialized EuRoC sequences (Table 2). This supports the relevance of an initialized tracker facing later dynamics mismatch. Our common true initial state isolates DAKF's adaptation to the subsequent change.


Response to W2: Evaluation across flight events and gradual regimes.

We added four deceleration events from other indoor/outdoor UZH-FPV trajectories (revised Section 5.2; Appendix A.7, Tables 10–12 and Fig. 6). Events use recorded kinematics for selection and four-second windows with 20 paired noise trials and shared initialization.

State MAE (cm; event means; SD and RMSE in Appendix A.7):

EventKFSTFIMMOKF-SOKF-FRVFFKNetDAKF
Out-56.3691.1411.9811.1821.0336.2581.3850.851
In-69.7971.2122.7921.4481.1553.6911.3791.133
In-98.3921.1602.4191.2481.0511.4021.3761.048
In-106.7051.1482.0221.1961.0431.5531.3761.011
Out-45-112.4211.2383.3741.5461.1675.9961.3861.072
New four9.3291.1902.6521.3601.1043.1611.3791.066

DAKF has the lowest mean on all five events, with 0.003–0.182 cm margins over OKF-F. On the four added events, the mean is 1.066 versus 1.104 cm for OKF-F. Appendix B reports gradual synthetic DAKF/IMM MAE 0.645/0.631 and extended 3D UAV DAKF/Optimized-KF MAE 2.39/2.19 cm, delineating the abrupt-change regime addressed here.


Response to W3: Architectural ablations.

The ablations point to synchronization, rather than selection alone, as the source of architectural benefit. In synthetic abrupt tracking, adaptive-only / no-sync / full DAKF achieve 0.786 / 0.784 / 0.755 MAE; under low noise, 0.732 / 0.701 / 0.641 (Table 2).

The dual-arm design builds on Kalman filter optimality under correctly specified linear-Gaussian dynamics (Kalman, 1960). Synchronization lets Arm 1 use the updated transition while mitigating adaptation fluctuations. On the UAV maneuver, adaptive-only achieves 0.856 cm MAE, leaving limited room for DAKF (0.851 cm). Synchronization yields larger gains in synthetic experiments, especially under low noise, highlighting adaptive identification and stable model retention.


Response to Q1: Practical occurrence and consequences.

Abrupt deceleration occurs during obstacle avoidance, sharp turns, and replanning, especially in high-speed UAV flight. Our five real flight events (Fig. 6) exhibit speed changes of 2.02–5.46 m/s; Chen et al. [1] also report repeated stops during obstacle-driven replanning.

These maneuvers are safety-critical. With sudden deceleration, an outdated transition model causes velocity errors that propagate into position predictions and consume obstacle-clearance margins. Navigation measurements may also be degraded: GNSS jamming and spoofing can interrupt or corrupt positioning signals, motivating onboard alternatives [4]. VINS-Mono demonstrates real-time visual-inertial estimation and autonomous MAV flight [5]. Communication delays and packet dropouts are also established challenges for networked estimation and feedback [6]. Accurate prediction between delayed, missing, or corrupted observations is therefore essential: dynamics mismatch can cause delayed braking, failed avoidance, or collisions. Sun et al. [2] also show that dynamics mismatch can cause control-constraint violations. DAKF addresses transition adaptation at the estimation layer to avoid collision.


References

[1] Chen et al. An Autonomous Single-Actuator UAV with Omnidirectional Field of View, High Agility, and Collision Resistance. Nat. Commun. 2026

[2] Sun, He, Miao, Zou. Constrained Reinforcement Learning Under Model Mismatch. ICML 2024

[3] He, Xu, Ouyang, Li. A Rotation-Translation-Decoupled Solution for Robust and Efficient Visual-Inertial Initialization. CVPR 2023

[4] Morales-Ferre, Richter, Falletti, de la Fuente, Lohan. A Survey on Coping with Intentional Interference in Satellite Navigation for Manned and Unmanned Aircraft. IEEE Commun. Surveys Tuts. 2020

[5] Qin, Li, Shen. VINS-Mono: A Robust and Versatile Monocular Visual-Inertial State Estimator. IEEE T-RO 2018

[6] Hespanha, Naghshtabrizi, Xu. A Survey of Recent Results in Networked Control Systems. Proc. IEEE 2007

AuthorsOpenReview ↗

Kindly Reminder to Reviewer ZwWm

Dear Reviewer,

Due to your busy schedule, we understand that you may not have had sufficient time to look at our responses yet.

We posted our rebuttal a few days ago, including additional UAV deceleration events, further analysis of the initialization setting, and additional ablation studies to address your concerns.

Could you kindly check our rebuttal and let us know if our responses and the additional experimental results have addressed your concerns?

Thank you very much for your time and consideration.

Best regards,

The Authors

Reviewer_eJ1uOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper considers what is essentially an offshoot of the classical problem of filtering with potentially time-varying dynamics. This is of course well-motivated and well-studied both in theory and in application literatures. The core contribution of this paper is to introduce two arms into the filtering procedure, perform filtering with both arms, and then select one of them to get the state estimate. The proposed method is tested on synthetic and real data sets, showing an improvement in performance.

Critical strengths

In my view, the idea of using multiple filters and then selecting the output of one is quite elegant, sensible and at the same time (apparently) easy to implement. I certainly think this kind of "multi-arm" approach can be built on and extended further. It also addresses the relevant problem of learning online in non-stationary environments.

Critical weaknesses

The principal weakness of this paper is the unclear presentation of the basic method proposed. This makes it hard to implement the algorithm after a first read (which - as someone with an interest in the topic - I would potentially do). I note for instance that Step 7 of Algorithm 1 says k_t must be selected upon update of the loss scores, and this determines which filter output, of the two possible ones, should be used. However, k_t, or the hard selection procedure used to select it, is not defined precisely. One should not define variables in an algorithm that have not been seen in the paper prior to it. The introduction of the algorithm and its presentation must therefore be restated in a much clearer way for this paper to be of acceptable quality. And this is the most interesting part of the paper: the arm selection rule!

Questions for authors

Could the authors also comment on the relationship of the work with online learning in non-stationary environments more generally? There is a substantive literature on this as well, including one that utilizes Kalman filtering. The paper would be a good contribution to this literature as well.

2 comments
AuthorsOpenReview ↗

Response to Reviewer eJ1u

We thank the reviewer for the encouraging assessment and constructive suggestions. Our revisions to the algorithm and Related Work are highlighted in blue.


Response to W1: Arm selection and reproducibility.

Thank you for your careful reading and for your suggestion to clarify the arm-selection rule. We have revised Sections 2 (Related Work), 3 (DAKF), and Appendix A.1 accordingly. In the revision, Sec. 3 introduces both arms, explicitly defines selection and synchronization, and presents Algorithm 1 in causal order, referencing Eqs. (2)–(6).

Specifically, kt∈{1,2}k_t\in\{1,2\} denotes the selected arm, and Δt=rt(1)−rt(2)\Delta_t=r_t^{(1)}-r_t^{(2)}. Eq. (5) sets kt=2k_t=2 if Δt>τsel\Delta_t>\tau_{\rm sel}, kt=1k_t=1 if Δt≤0\Delta_t\leq0, and kt=kt−1k_t=k_{t-1} otherwise. The threshold τsel≥0\tau_{\rm sel}\geq0 provides hysteresis, with output x^t=x^t(kt)\hat x_t=\hat x_t^{(k_t)}.

The execution steps are:

(i) Adaptation. Before observing ztz_t, Arm 2 updates its transition using precision-weighted RLS (Eq. (3)) on completed filtered-state transitions s<ts<t. Each sample's precision is determined causally and fixed during replay. Arm 1 retains its reference transition between synchronizations.

(ii) Filtering. Both arms predict states and covariances, compute Kalman gains, and correct using the same ztz_t (Eq. (2)). Both remain active regardless of selection.

(iii) Scoring. Each arm updates its discounted performance score (Eq. (4)): rt(i)=γregretrt−1(i)+(1−γregret)[ℓt(i)+βVar⁡W(ℓ(i))]r_t^{(i)}=\gamma_{\rm regret}r_{t-1}^{(i)}+(1-\gamma_{\rm regret})[\ell_t^{(i)}+\beta\operatorname{Var}_W(\ell^{(i)})]. Appendix A.1 details benchmark-specific losses.

(iv) Selection. After warm-up, Eq. (5) determines ktk_t from the score difference and hysteresis; the selected posterior is returned as x^t\hat x_t.

(v) Synchronization. Let vt=W−1∑j=t−W+1t1[kj=2]v_t=W^{-1}\sum_{j=t-W+1}^{t}\mathbf{1}[k_j=2]. When vt≥τsyncv_t\geq\tau_{\rm sync}, Eq. (6) re-anchors Arm 1 to Arm 2 and resets scores for future predictions. Appendix A.1 details benchmark-specific implementations.

These revisions distinguish immediate output selection from sustained model synchronization and make the algorithm reproducible.


Response to Q1: Relationship to non-stationary online learning.

We agree that DAKF is a specific instance of online learning in non-stationary environments. Changing data-generating mechanisms and model mismatch are classical challenges in this area. Broadly, it includes (i) online optimization with drifting objectives [4], (ii) non-stationary bandits with changing reward distributions and exploration-exploitation [5], (iii) non-stationary RL with evolving transitions or rewards for policy optimization [6], and (iv) online prediction, identification, and filtering under evolving dynamics [1–3].

DAKF belongs to the fourth category: the unknown transition AtA_t changes, only noisy measurements zt=Hxt+vtz_t=Hx_t+v_t are observed, and the goal is hidden-state estimation, not reward or policy optimization. Its two arms are filters processing the same observation, not bandit actions.

Related studies address overlapping but distinct problems. CoSFan [1] adapts latent neural dynamics for non-stationary forecasting. PLOT [2] uses forgetting-based RLS to identify changing target dynamics for online tracking control, with target states sequentially revealed. More closely, Online Learning of the Kalman Filter [3] studies online prediction under partial observability, but assumes fixed system dynamics.

In contrast, DAKF learns time-varying transitions from noisy filtered-state estimates, coupling identification and filtering errors. Its retained/adaptive arms combine hard selection and majority synchronization without a predefined model bank. The PW-PE bound (Theorem 1) explicitly accounts for precision, filtering bias, and local drift; it does not establish dynamic regret for the full selector.

We have expanded Related Work (Sec. 2) by adding CoSFan [1] and online Kalman learning [3] and clarifying the existing comparison with PLOT [2].


References

[1] Missel & Wang, CoSFan, ICLR 2025. https://proceedings.iclr.cc/paper_files/paper/2025/hash/01946043eea3b55c4650e5dd07ae6cad-Abstract-Conference.html

[2] Tsiamis et al., Predictive Linear Online Tracking for Unknown Targets, ICML 2024. https://proceedings.mlr.press/v235/tsiamis24a.html

[3] Tsiamis & Pappas, Online Learning of the Kalman Filter With Logarithmic Regret, IEEE TAC 2023. https://doi.org/10.1109/TAC.2022.3207670

[4] Besbes et al., Non-Stationary Stochastic Optimization, Operations Research 2015. https://doi.org/10.1287/opre.2015.1408

[5] Besbes et al., Stochastic Multi-Armed-Bandit Problem with Non-stationary Rewards, NeurIPS 2014. https://proceedings.neurips.cc/paper_files/paper/2014/hash/91ba7292e5388b90b58d0b839a7f19ec-Abstract.html

[6] Cheung et al., Reinforcement Learning for Non-Stationary Markov Decision Processes, ICML 2020. https://proceedings.mlr.press/v119/cheung20a.html

AuthorsOpenReview ↗

Kindly Reminder to Reviewer eJ1u

Dear Reviewer,

Due to your busy schedule, we understand that you may not have had sufficient time to look at our responses yet.

We posted our rebuttal a few days ago, addressing your concerns regarding the algorithmic details, arm-selection and synchronization rules, and the positioning of DAKF within online learning in non-stationary environments.

Could you kindly check our rebuttal and let us know if our responses have addressed your concerns?

Thank you very much for your time and consideration.

Best regards,

The Authors

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.