acceptodds
Withdrawn from ICLR 2027

Calibration-Aware Temporal Representation Learning under Feature Estimation Noise

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Temporal models typically treat the representation available at each time step as deterministic, although in practice intermediate features may be estimated under sensor noise, stochastic encoding, missing observations, or limited measurement precision. Unlike static prediction, representation error in a temporal model can alter the hidden state and subsequently propagate through all later transitions. We study this trajectory-level effect and its consequences for probabilistic prediction. Our analysis separates local uncertainty injection from downstream dynamical amplification: finite-perturbation bounds characterise worst-case error propagation, a local Jacobian analysis describes how feature covariance is transformed through the hidden trajectory, and a second-order expansion shows that even zeromean feature noise can induce systematic hidden-state bias through nonlinear curvature. Motivated by this analysis, we propose Calibration-Aware Temporal Representation Learning (CaTRe), whose core mechanism is an uncertaintyconditioned gate that attenuates unreliable hidden-state updates before their perturbations propagate through time. We evaluate CaTRe on Room Occupancy Estimation and Occupancy Detection with GRU and liquid time-constant (LTC) backbones under clean evaluation, additive representation noise, channel missingness, and temporal thinning. The empirical effect is strongly regime dependent. When GRU baselines are already near saturation or have low probabilistic error, CaTRe produces limited gains and can trade off across metrics. In contrast, the largest improvements occur for LTC settings with high baseline probabilistic error: on Room Occupancy, CaTRe improves clean accuracy from 0.241 to 0.807, reduces NLL from 11.855 to 0.737, and reduces ECE from 0.753 to 0.203. Ablations further show that uncertainty gating is the dominant contributor, while auxiliary probabilistic and trajectory-stability losses are not always complementary. These results support a trajectory-level view of representation uncertainty: its effect depends not only on local noise magnitude, but also on how temporal dynamics transform that uncertainty before prediction.

Primary area:
unsupervised, self-supervised, semi-supervised, and supervised representation learning
Keywords:
Temporal representation learning, Feature estimation noise, Uncertainty propagation, Reliability diagrams, Robustness

Reviews

Among:
Shape:
1234this paper, Bottom 1%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.00 over 3 reviews, above 0% of the others.
Reviewer_pwrVOpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
2
Soundness
2
Read review
Paper summary

The paper proposes a gating mechanism to make the temporal representation learning robust. The motivation is that noisy data, errors, and missing observations could induce representation errors that could be amplified. Some theory is provided but is not solid enough.

Critical strengths

The motivation is clear, and the problem is valuable. Some theoretical analysis is provided.

Critical weaknesses

The literature review of the robust temporal model is limited. Some notations are not clearly explained. The theoretical analysis is not deep enough to support the gating design.

Questions for authors
  1. The literature review in terms of the hidden state errors seems to be limited. I am not so sure if this is the first work to observe this phenomenon and deal with this problem.
  2. Fig. 1 seems to be AI-generated: too many elements that are trivial and non-technical. It’s not suitable for a top-tier conference like ICLR.
  3. What do \mathcal{H}{hh,t}:P{t-1} and \mathcal{H}{hh,t}:\Sigma{t-1} mean in Eq. (12)? It’s not common to use colon to connect two matrix and produce a vector (since A_tb_{t-1} is a vector).
  4. The bold lowercase p is not defined in Eq. (14). What is binned ece?
  5. The theoretical analysis in Section 3.2 is not deep enough and doesn’t strongly support the gating mechanism in Eq. (3). E.g., in theory, dose the gate really control the error propagation?
  6. The experiments are limited. First, the author should compare the method with robust temporal models that can similarly deal with this problem with sufficient literature and citations. Second, the backbone (GRU/LTC) is not representative enough, given that we have so many sequence models. Third, more realistic datasets with noise and errors should be tested.
  7. I am not so sure if the three degradations in Eq. (19) are well-known and common in this domain.
Reviewer_v2c4OpenReview ↗
Rating
1
Confidence
2
Informativeness
1
Presentation
2
Soundness
1
Read review
Paper summary

This paper formulates feature-estimation uncertainty in temporal models using an additive noise model (with zero mean) and studies how these errors propagate through hidden states and affect predictions. It analyzes hidden-state perturbations, covariance propagation, and bias induced by nonlinear dynamics. Based on this motivation, they propose CaTRe, which uses an uncertainty-conditioned gate to control hidden-state updates, together with two auxiliary losses. Experiments on two occupancy classification datasets with GRU and LTC backbones show mixed results.

Critical strengths
  1. The paper considers a relevant problem that the errors in temporal inputs can affect both the current hidden state and later predictions.
  2. The experiments cover not only the additive Gaussian noise setting, but also missing sensor channels, and missing time points.
Critical weaknesses
  1. I couldn't find a detailed description of the gating mechanism in section 3, even though the introduction presents it as the core component of CaTRe. It is only briefly shown in figure 1, without explanation of the notation or implementation. This is the biggest issue of this paper. It is unclear how the gating is realized, and thus difficult to evaluate how it addresses the problem studied in the paper.

  2. The authors did not explain why it is good to minimize the LHS of (14). Specifically, I think there is no guarantee that the clean prediction is well calibrated or must lead to higher accuracy.

  3. The experiments only use GRU and LTC backbones, without testing on a Transformer baseline. Although the analysis in section 3.2 does not directly apply to Transformers, I think including a Transformer baseline would help assess the practical value of CaTRe, given the strong performance of Transformers in multivariate time-series classification.

  4. The choice of the full CaTRe model is not well justified. The results in section 5 show that the two auxiliary losses do not consistently improve performance, and the gate-only variant outperforms the full model in the LTC ablations.
    Why are both auxiliary losses included in the main version of CaTRe?

Questions for authors
  1. Are feature pertubations applied only during test time, or also applied in the training data? If applied in training, do they share the same noise distribution with test time data?
  2. What is n in table 2?
  3. Symboals in sec3.3 are not defined and hard to read. What are p_{yn} and P?
  4. Table 2 only reports changes in metric values. Without the absolute values, we cannot compare the performance of GRU and LTC under perturbation.
  5. In tabe 1, LTC CaTRe on Room Occupancy accuracy is 0.807 \pm 0.131, but in table 3 it is 0.891 \pm 0.021.

Overall, this paper appears incomplete, with issues in its writing and organization that likely be AI generated and lack sufficient human checking.

Reviewer_9TGsOpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
2
Soundness
1
Read review
Paper summary

Paper summary:

  • This paper studies time series modelling with uncertain inputs
  • Their main goal is to show that when input features are noisy, this can result in biased unrolled trajectory, in addition to the standard noise
  • They analyze this by locally linearizing the trajectory around the true inputs (assuming gaussian input noise)
  • The propose CaTRe, which combines two things: three things, a brier loss function, a loss function to encourage stability under input noise, and a gating term which gates the function updates
Critical strengths
  • The analysis is sound, although fairly rudimentary and not particularly insightful
  • The problem of studying time series biased unrolls under input noise is promising
Critical weaknesses
  • Limited motivation for the auxiliary losses. The loss terms in Eq. (18) are insufficiently motivated. What does the Brier loss provide beyond standard cross-entropy, which already optimizes a proper scoring rule? The perturbation distribution used to construct the paired trajectories for the stability loss is also unclear. A more explicit connection between this regularizer and the uncertainty-propagation analysis would help justify its inclusion.

  • Incomplete exposition of the method. The gate appears to be the main source of improvement, yet its mathematical form is confined to Figure 1 rather than explained in the methodology. Crucially, the “uncertainty summary” (u_t) is not defined: its contents, dimensionality, and construction during training and evaluation remain unclear. These details are essential to understanding and reproducing the method. Limited empirical support for the full objective. In the highlighted LTC ablations, the gate-only variant outperforms full CaTRe, sometimes substantially, while gains over the GRU baselines are generally small or mixed. These results support the additional gate more strongly than the proposed combination of gating and auxiliary losses. The paper should better justify retaining the full objective and explain when, if ever, the auxiliary terms provide a benefit.

  • Insufficient controls to identify the source of improvement. The gate receives additional information through (u_t), but the experiments do not isolate the benefit of this information from the benefit of additional gating. Relevant controls include supplying (u_t) directly to the backbone’s update network or existing gates, and adding the proposed gate without (u_t). These comparisons would clarify whether uncertainty-conditioned gating is necessary to obtain the reported gains.

  • Limited explanatory value of the theoretical analysis. The perturbation analysis does not address the tradeoff between attenuating noise and preserving useful information. Setting the gate identically to zero would eliminate input-induced trajectory perturbations while preventing the model from incorporating observations. Although the classification objective discourages this solution, the theory does not explain when gating improves robustness without sacrificing predictive performance.

  • Limited dataset coverage. Evaluation is restricted to two closely related occupancy datasets, providing limited evidence for the method’s applicability across temporal-learning tasks. Evaluation on distinct domains, such as PhysioNet Challenge 2012 and PAMAP2, would strengthen the empirical support and test the approach under different patterns of missingness and sensor reliability.

Questions for authors

Please see weaknesses section

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.