- Rating
- 1(−1)
- Confidence
- 3
- Informativeness
- 1(−1)
- Presentation
- 3
- Soundness
- 1
Read review
Paper summary
This paper proposes K-PRISM, a neural-augmented Kalman-style filter for partially known dynamics, unknown observation models, and time-varying sensor reliability. The prediction combines a known physics prior with a learned residual that sees only position-independent quantities. The gain is a feedforward MLP conditioned only on velocity features and the current innovation, which the authors argue yields translational invariance and single-timestep adaptation. For multiple sensors, a joint gating network computes normalized per-sensor, per-dimension weights from the context of all sensors. The paper also gives a convex-combination interpretation of the update, a Lipschitz bound on the sensor heads, and an error-contraction result under a lower bound on the total gain. Experiments on the Lorenz attractor, NCLT, and EuRoC show accuracy comparable to or better than KalmanNet with far fewer parameters and more stable training.
Critical strengths
1. K-PRISM achieves accuracy comparable to or better than KalmanNet (29K-107K parameters) with only about 0.5K-1.4K parameters. This is appealing in terms of parameter efficiency and training stability.
2. The authors openly acknowledge that K-PRISM does not surpass the published KalmanNet result, that it is inferior to KalmanNet in the Lorenz steady-state setting, and that most of the performance gain comes from the residual. They also report their KalmanNet reproduction results per seed, which increases the credibility of the comparison.
3. On the data-limited NCLT setting, the paper compares against LSTM and Transformer baselines at multiple capacities. K-PRISM substantially outperforms them even at the same parameter count, which illustrates the benefit of retaining the Kalman structure.
Critical weaknesses
1. Structural issue with the multi-sensor normalization (Eq. 12).
Since , for small we have unless all are close to zero. The update then reduces to a weighted average of the sensor proposals, and the physics prior is effectively ignored. Figure 3, where the two weights sum to roughly 1, is consistent with this.
However, the learned gains on NCLT fusion are 0.05-0.10 (Table 13), which is difficult to reconcile with Eq. 12. Please clarify the value of and which update equation is actually used in each experiment.
2. Translational invariance does not hold for the multi-sensor mechanism.
The gating context (Eq. 11) includes the encoder features . For a sensor that observes absolute position, such as GPS, these features depend on position. The claim that translational invariance holds for the full multi-sensor mechanism is therefore inconsistent with Definition 1.
3. Core claims are not validated on real data.
The only evidence for cross-modal gating is the synthetic two-sensor Lorenz study. In NCLT fusion, GPS is suppressed by the dataset's validity mask rather than by the learned gate. Likewise, the "unknown observation model" setting is not really tested, since all observations are low-dimensional and nearly identical to the state.
4. The contribution of the gain mechanism is unclear.
On NCLT GPS-only, the scalar gain (20.35 dB) and the fixed 0.5 gain (20.82 dB) perform similarly to K-PRISM (20.64 dB), and even the fixed gain outperforms KalmanNet (22.20 dB). On EuRoC, the gap between KalmanNet+res and K-PRISM is also small. The improvement appears to come mainly from the physics prior and the learned residual rather than from the proposed gain.
5. Missing related work.
Lee et al. (IROS 2020, "Multimodal Sensor Fusion with Differentiable Filters") address multimodal sensor fusion within differentiable filters. Recent works also approximate the Kalman gain with a learned gating network (e.g., arXiv:2602.11183). Neither line of work is discussed, which makes it difficult to assess the novelty of the proposed method.
Questions for authors
Q1. What value of is used in Eq. 12? Could you show the distribution of after training, to indicate how far it deviates from 1 in practice? In addition, please clarify whether the NCLT fusion experiment uses Eq. 9 or Eq. 12. If GPS is the only correcting sensor, how can the learned gain converge to 0.05-0.10?
Q2. The gating context includes encoder features from sensors that observe absolute position. In what sense is the multi-sensor mechanism translationally invariant? Do you have results with these features removed from the context?
Q3. How does K-PRISM differ from the multimodal differentiable filter of Lee et al. (IROS 2020) and from recent works that approximate the Kalman gain with a learned gating network (e.g., arXiv:2602.11183)?
2 comments
I thank the authors for their detailed response and transparent corrections. However, the response has reinforced several of my concerns.
-
Lack of real-world validation of the multi-sensor architecture. The authors have confirmed that the NCLT fusion experiment uses the single-sensor gain (Eq. 9) rather than Eq. 12. Consequently, the joint gating mechanism (Eqs. 11–13), which is central to both the title and the proposed methodology, has been evaluated only on synthetic Lorenz experiments.
-
Lack of controlled evidence for translational invariance. The clarification that the comparison in Appendix C was not parameter-matched removes the evidence previously used to rule out model capacity as a confounding factor in the NCLT "w/o invariance" ablation. As a result, there is no remaining controlled evidence supporting the claim that translational invariance improves generalization.
-
Withdrawal of the NCLT GPS-only results. I consider it a serious concern that the reported KalmanNet results were obtained under a different observation setting and that the GPS data loader used ground-truth information from the entire session before splitting the data. Since these results supported a central claim in the abstract and the comparisons against LSTM and Transformer baselines, all affected results should be removed or recomputed without data leakage.
-
Reduced scope of the contribution. After these corrections, the remaining contribution is a compact estimator combining an explicit physics prior, a learned residual, and a feedforward gain. This combination substantially overlaps with existing work, and the supporting real-world evidence is limited to NCLT fusion and EuRoC, with most of the performance improvement on EuRoC attributable to the learned residual.
I appreciate the authors' transparency throughout the rebuttal process. However, given the withdrawal of several central results and the limited novelty of the remaining contribution relative to prior work, I have decided to lower my rating.
Response to reviewer zCn7
Thank you for highlighting the compactness of K-PRISM and the points requiring clarification. We have revised the paper to distinguish its compact physical-state estimator from the broader properties that the experiments do not establish.
Q1. Normalization, epsilon and the NCLT update
The normalized two-sensor Lorenz gate uses ε = 10⁻⁶ in Eq. (12), separately for each state coordinate. Its direct prior coefficient is
1−∑iwt,k(i)=∑iαt,k(i)+ϵϵ.
Thus, when summed raw confidences are appreciable relative to epsilon, the direct prior weight is nearly zero. Section 3.3 and the discussion of Proposition 1 now state this explicitly. We have removed the claim that this normalization preserves a meaningful prior contribution or establishes rejection of all unreliable sensors. No post-training weight distribution is reported, so we do not infer the requested quantiles or below-0.9 frequency from the algebra or the plotted average traces.
NCLT fusion instead uses the unnormalized sigmoid gain of Eq. (9) with the prediction-correction update in Eq. (10), not Eq. (12). Wheel/IMU measurements drive prediction; GPS is the correcting observation, and the supplied GPS-validity mask suppresses missing updates. Table 13 reports constant per-coordinate gains from the scalar-gain fusion variant, which uses the same unnormalized, masked GPS correction. Nothing forces these gains to sum to one, so values of 0.05–0.10 are consistent with that variant (Appendix F.5). Sections 3.3 and 4.2.2 and Appendix F make this distinction explicit.
Q2. Translational invariance and encoder features
We have corrected the unconditional multi-sensor invariance claim. In Eq. (11), an encoder of absolute-position measurements can retain position information. Omitting a separate position input therefore does not establish invariance of the generic joint gate.
The narrower position-independent input construction concerns the specified NCLT/EuRoC implementations. The original synthetic position-toggle comparison keeps encoded observations in both variants and changes parameter count. Appendix C and Table 6 now describe it as the effect of an additional explicit position input, not a parameter-matched feature-removal experiment. We do not claim that removing the encoder features is cost-free; no such matched ablation is reported.
Q3. Relationship to the cited methods
Lee et al. (2020) learn crossmodal weights for combining unimodal filter estimates/covariances or particle-filter measurement contributions. K-PRISM instead corrects a common predicted physical state using sensor proposals, without propagating a posterior covariance. We have added this work and the distinction in Section 2.2; crossmodal weighting itself is not claimed as new.
NeuroKalman uses a feedforward sigmoid elementwise gain with a recurrent prior and visual-memory measurements in a latent navigation representation. K-PRISM emphasizes an explicit nominal physics prior plus a learned residual in a compact physical-state estimator. Section 2.2 now discusses this overlap and distinction. We do not claim novelty for feedforward gains alone or superiority over either cited method.
Scope of the real-data and gain-contribution claims
The contribution is the compact combination of an explicit nominal physics prior, learned residual and physical-state correction. We have narrowed the abstract, contributions, conclusion and real-data claims accordingly. The real-data tests use known observation structure, and NCLT supplies a validity mask; they do not establish autonomous joint reliability estimation. We also no longer present the existing comparisons as isolating the gain architecture’s independent advantage. The GPS-only values cited in the review are now confined to historical appendix diagnostics (Appendix F.3), not matched-accuracy evidence: the cited KalmanNet reference uses odometer-velocity observations, whereas the GPS loader used full-session ground truth before splitting (Appendix F.6). No new GPS comparison is included.