- Rating
- 1
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
The paper aims to address the difficulty of maintaining causal states when perceptual access changes. It introduces Temporal Causal Observability and analyzes when observations provide reliable evidence for state updates. A method named RME combines predictions and observations by considering different possible access conditions. Experiments on frozen causal representations demonstrate reduced state estimation errors when some observations are replaced.
Critical strengths
- The paper distinguishes changes in causal states from changes in perceptual access. The controlled replacement experiments make it possible to study observation changes without altering the underlying causal trajectory.
- The paper compares different update rules under the same observation conditions and includes ablation experiments on scale calibration and uncertainty propagation.
Critical weaknesses
-- The experimental assumptions limit the applicability of the proposed method. The method accumulates evidence within observation blocks whose boundaries are known in advance, and assumes that different views lose access independently. These assumptions make it easier to determine which observations should be trusted. However, access may change unpredictably or be lost across several views at the same time, making the available evidence less informative. The experiments mainly examine partial replacement while valid observations remain available, so they do not adequately establish performance under sustained loss of access. Further evaluation under less restrictive observation conditions is necessary to support the applicability of the method.
-- The reliability of the uncertainty estimation is not sufficiently demonstrated. The proposed method uses uncertainty to determine how much the predicted state and the current observations should influence each update. Although uncertainty propagation improves the experimental results, the estimated variance does not adequately reflect the growth of prediction error over time. This raises concerns about whether the method can maintain an appropriate balance as estimation errors accumulate. The assumption of independent measurement errors is also inconsistent with the strong correlations between the views used in the experiments, which may lead to excessive confidence in repeated evidence. The influence of these limitations on continuous state estimation needs further analysis.
-- The comparison with existing filtering methods does not fully support the advantage in continuous state estimation. The paper compares the proposed method with filtering baselines when updating from a clean previous observation. However, its main objective is to maintain the state through repeated updates, where earlier estimation errors can affect later predictions. Under this setting, the comparison mainly focuses on whether to average over possible access conditions or select a single condition. This explains the benefit of the proposed decision rule, but does not show whether its advantage over the filtering baselines persists as errors accumulate. Comparing these methods under the same recursive setting is therefore necessary for a more complete evaluation.
-- The contribution of causal information is not sufficiently distinguished from the design of the observation replacements. The grouping experiment shows that the learned causal-factor groups perform better than rearranged groups. However, the observation replacements are also organized by causal factor, so coordinates in the same group share the same access condition by construction. Rearranging the groups breaks this correspondence and makes the shared-access assumption less appropriate. Consequently, the performance difference may reflect the match between the groups and the replacement mechanism, without establishing an advantage specific to causal information. A comparison with groups learned from observational dependence would help clarify whether causal grouping provides benefits beyond capturing shared access conditions.
Questions for authors
Please see weaknesses.
3 comments
Response to Reviewer gtDA — Part 2/2
To check the role of the prior, we reused the runs with assumed p from 0.6 to 0.95 (reply to fHSU, W1). As the prior mass on the all-invalid case falls from 6.4% to 0.0125%, the harm grows steadily in all four settings:
| Setting | p = 0.6 (6.4% prior) | p = 0.8 (0.8%) | p = 0.95 (0.0125%) |
|---|---|---|---|
| Pong, independent donors | +0.0662 | +0.0840 | +0.1090 |
| Pong, coherent donors | +0.0381 | +0.0607 | +0.0892 |
| Voronoi, independent donors | +0.1034 | +0.1431 | +0.2065 |
| Voronoi, coherent donors | +0.1248 | +0.1620 | +0.2048 |
Values are the MSE increase from updating on frames with no valid view; Appendix F.11 gives all five values of p with intervals. So the rare all-invalid prior contributes to the failure, but a larger prior does not remove it. We keep p = 0.8 for the reported method. A prior that allows correlated loss of access, and an explicit regime for shared replacements, are natural next steps.
W2. Reliability of the uncertainty. We agree. The propagated variance does not track the slow growth of error over long horizons, and the paper now says plainly that better state estimates are not calibrated long-horizon uncertainty. We made two additions. First, a common-error channel reduces the over-counting of repeated evidence that you describe, since three agreeing views no longer count as three independent measurements. The ordering holds under it (Voronoi: 0.0427 against 0.0576 for hard validity, 5/5). Second, Appendix F.8 reports coverage, NLL and dispersion by sequence age: on Pong, squared error reaches 3–4 times the predicted variance after 512 frames.
W4. Causal grouping versus the replacement design. We ran the comparison you suggested. We learned two alternative groupings from observational dependence on training encodings, without access labels: mutual information (MI) between coordinates, and MI between their increments. We also include one regime shared by all coordinates and the eight size-matched shuffles of the original ablation. Mean , lower is better:
| Setting | Learned causal | MI | MI on increments | One shared regime | Shuffles (mean of 8) |
|---|---|---|---|---|---|
| Pong, independent donors | 0.2901 | 0.2868 | 0.3023 | 0.4294 | 0.3004 |
| Pong, coherent donors | 0.2972 | 0.2987 | 0.3182 | 0.4416 | 0.3110 |
| Voronoi, independent donors | 0.0424 | 0.0454 | 0.0504 | 0.1732 | 0.0609 |
| Voronoi, coherent donors | 0.0514 | 0.0559 | 0.0593 | 0.1859 | 0.0728 |
The learned causal grouping has the lowest error in every setting but one. MI comes closest: on Pong the two are within about 1% (MI slightly lower with independent donors); on Voronoi the learned grouping's error is 6–8% lower. Against the other alternatives the learned grouping is lower in every setting: by 4–16% against MI on increments, 32–76% against one shared regime, and 3–4% on Pong and about 30% on Voronoi against the shuffles. So observational dependence can come close to the causal grouping, but none of the alternatives improves on it across settings. Section 5.3 now reports these controls (Appendix F.9).
In short: under the original masks, RME has lower hidden-state error than both recursive filtering baselines in every representation; its advantage over hard validity holds without known boundaries on both benchmarks; and the learned causal grouping holds up against the data-driven groupings you suggested. The limits you pointed to (shared evidence, no valid view, long-horizon calibration) are now measured and stated in the paper. We hope this fuller comparison addresses your concerns, and we would be grateful if you would reconsider your assessment in light of these results.
Short follow-up before the discussion closes
Thank you again for your review. Since the discussion closes tomorrow, here is the short answer to your four points.
W3: under the same recursion, RME's mean hidden-state error is 30–78% lower than a χ² gate's and 17–72% lower than a Student-t update's, and it is lower in every representation.
W4: grouping by mutual information on the encodings comes within about 1% on Pong, and the causal grouping's error is 6–8% lower on Voronoi; MI on increments, one shared regime and size-matched shuffles are worse in every setting.
W1: without known boundaries, RME keeps its advantage over hard validity in every representation, and with correlated loss and separate donors it also holds. Shared erroneous evidence and frames with no valid view are now stated as limits in Section 6.
W2: a common-error channel reduces the over-counting of agreeing views, and the ordering holds under it; long-horizon calibration remains a stated limit.
We would be grateful to know whether this addresses your concerns, or which concern remains.
Response to Reviewer gtDA — Part 1/2
Thank you for four specific requests. We ran each of them. The filtering baselines, the boundary-free test and the grouping controls support the method. The tests with shared replacement and with no valid view and the calibration analysis confirm your concerns, and the paper now states these limits.
W3. Filtering baselines under the same recursion. The χ2 gate (99%) and the Student-t update (ν = 4) now run recursively. Each feeds back its own mean and variance, with the same 64 Monte Carlo draws as RME.
RME has lower EH in every representation. Recursion widens the gap: on Pong with independent donors, the χ2 gap grows from 0.034 in one-step updates to 0.142. The reason is lock-out. Once its prediction drifts, the χ2 gate keeps rejecting valid views. In the correlated-loss test (8-frame blocks), in the first two frames after access returns, it rejects a valid view in 31% of frames on Pong and 8% on Voronoi, against 6% and 0.3% for RME. Student-t has slightly lower all-valid error on Pong (0.244 against 0.253), a trade-off we report; on Voronoi, RME is lower on both measures.
W1. Known boundaries. They are not needed for the advantage over hard validity. With a fixed hazard of 0.25 instead, RME beats hard validity in all 3 Pong and all 5 Voronoi representations, for both donor types (on Pong, hard-minus-RME EH is +0.021 [0.016, 0.026] and +0.021 [0.011, 0.031]). On both benchmarks RME stays 19–67% below median fusion (reply to fHSU, Q1).
W1. Views losing access together, and sustained loss. You were right that this reduces the evidence. We added masks in which views share one access state with probability 0.5 or 1, with blocks of 4, 8 and 16 frames, and with separate or shared donors (Appendix F.7).