acceptodds
Withdrawn from ICLR 2027

CausalGap: Benchmarking Representation Responses in Traffic Prediction under Structural Interventions

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Traffic prediction is a cornerstone of intelligent transportation, yet model reliability under sudden structural changes remains poorly understood. Existing evaluations quantify predictive degradation under structural interventions, but provide limited evidence on whether model representations register those changes or remain rigid when performance deteriorates. We introduce CausalGap, a diagnostic framework centered on the Representation Response Landscape, a two-dimensional state space defined by Dual-Metric Intervention Sensitivity (DMIS) and Relative Degradation (RD), which measure representation responsiveness and normalized post-intervention error increase, respectively. A constructive error-only impossibility example motivates joint analysis of response and degradation within a four-regime landscape, including pathological rigidity, characterized by high degradation and low responsiveness at a declared representation interface. To reduce structural-access confounding, CausalGap introduces a Dual-Track Structural Access Interface separating native and controlled access. The primary falsifiable test asks whether weak-intervention response provides incremental information about stronger unseen degradation beyond prespecified weak-error trajectory summaries and controls, complemented by an information-matched post-hoc steering test of explicit DMIS parameterization. The benchmark protocol supports topology-input stress testing and intervention-consistent response-failure analysis across heterogeneous traffic predictors and multiple observational, controlled, and simulation-based settings, while treating real-world disruption evidence separately when representation response is unavailable. Matched-error comparisons, invariance checks, and alternative response measures assess measurement stability and diagnostic consistency across model families and settings. CausalGap provides a diagnostic benchmark for representation-response analysis under structural interventions. Resources and evaluation instructions are available at https://anonymous.4open.science/r/CausalGap-80CB.

Primary area:
datasets and benchmarks
Keywords:
Traffic Prediction, Representation Response, Structural Interventions, Model Robustness, Pathological Rigidity

Reviews

Among:
Shape:
1234this paper, Bottom 2%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.33 over 3 reviews, above 2% of the others.
Reviewer_kpWMOpenReview ↗
Rating
1
Confidence
2
Informativeness
1
Presentation
1
Soundness
1
Read review
Paper summary

The paper is somehow hard to follow. Under my understanding, this paper proposes two new metrics that can investigate the topological response. The two metrics are DMIS and RD based on latent representations.

Critical strengths
  1. This problem is novel. Except errors, this paper notices the change in topology. This is a new perspective.
  2. The experiments are comprehensive from multiple perspecgtives.
Critical weaknesses
  1. This paper exhibits extensive traces of AI related content. Including paper writing and figure generation. The paper is really hard to follow. For example, "which measure representation responsiveness and normalized post-intervention error increase", I cannot understand these sentences without the help of GPT. The figure is generated by AI.
  2. In my understanding, the coordination in Fig1.c is the main contribution. However, this seems to be an theoretical contribtuion. What're the thresholds of this coordination? Is this thresholds effecient for other tasks?
  3. The metrics are based on latent representations. How to prove these representations are continous and robust? Will the affect of noise or drift?
  4. In my opponion, papers in this field is to propose some good metrics to divide different data, just like a regression or PCA. However, the data comes from controlled synthetic dynamics. Authors need to use naturalistic data to validate.
Questions for authors

See weaknesses.

Reviewer_qChzOpenReview ↗
Rating
1
Confidence
2
Informativeness
1
Presentation
1
Soundness
1
Read review
Paper summary

This paper is about a methodology to evaluate the resilience (generalizability) of traffic prediction models under disruptions. The core concept is that, with a structural disturbance, we have potentially a causally different problem. I.e. a disturbance corresponds to an intervention. And such an intervention may have implications in the internal representation of the model (e.g. the graph changed) or generally provide poor results. This is a well known fact, but the authors propose this framework to measure the "gap" between the original (unperturbed) model and the resulting one.

Critical strengths

This is an extremely important problem in transport research. Not only in traffic prediction, but transport modeling in general (e.g. OD prediction, incident prediction, etc.).

The general approach of inferring the causal gap is novel and extremely relevant.

Critical weaknesses

This article was very very painful to read:

  • Already from the beginning DMIS, RD and Representation Response Landscape are new concepts. Problem formulation is incomplete (X, A and Y are not defined!!).
  • f= g ◦z has no apparent meaning… I mean g and z are not defined, nor are f...
  • The number of (new?) concepts at every paragraph is overwhelming. “Pathological-Rigidity”, “centered topology-controlled pair”, “Error-Response Measurement Impossibility”, “Average Relative Degradation (ARD)” (BTW, ARD relates to a completely different and well known concept in ML…). None of this is ever accompanied with an intuition or example.
  • I checked the github, and the explanation is also quite insufficient

None of these concepts seem to build on existing literature, which is completely puzzling…

I confess I got half way and quit reading more, because this is not a paper for a normal human to read.

Questions for authors

I strongly recommend rewriting this paper from scratch, avoiding the use of LLMs. It is not written for a human to understand (maybe unless this human has been working very very closely with the topic...).

Reviewer_NCw1OpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper introduces CausalGap, a diagnostic framework and benchmark for analyzing how traffic forecasting models’ selected representations respond to structural interventions and how that response co-occurs with predictive degradation. The framework proposes DMIS, a dual-metric intervention sensitivity score, and RD, a relative degradation score, and uses their joint landscape to define four operational regimes: stable invariance, adaptive response, over-reactive instability, and pathological rigidity. It further introduces a Dual-Track Structural Access Interface (DSAI) to separate native structural access from a shared controlled topology interface, and tests weak-to-strong incremental value (H1) and post-hoc representation steering (H2). Experiments cover 21 models in five families and eight traffic datasets, with controlled synthetic dynamics used for confirmatory H1/H2, an abstract SUMO replication for rank/regime replication, and observational datasets documented only as non-counterfactual contexts.

Critical strengths
  1. The measurement framework is carefully formalized and audited.
    DMIS has explicit invariance properties, the paper separates Track A and Track B structural access, and the evidence roles are unusually clearly specified: controlled synthetic for confirmatory tests, SUMO only for replication, observational datasets excluded from response-failure analyses. The reproducibility appendix and released artifacts are strong.

  2. The paper is honest about weak and specification-sensitive results.
    The authors do not overclaim that DMIS is a linear proxy for RD, and they explicitly report that H1 is small and sensitive to the weak-history specification and that H2 is a modest, configuration-specific parameterization effect. This honesty is valuable, and the joint response-degradation view is a useful diagnostic lens for the traffic-forecasting community.

Critical weaknesses
  1. The confirmatory evidence is too weak to support the paper’s main diagnostic claims.
    H1 reduces macro-MAE only from 0.001288 to 0.001262 (2.6×10−52.6\times10^{-5}), and the direction reverses under the mean-only history specification; the RSA alternative also reverses the H1 direction. H2 improves error reduction from 0.0582% to 0.1007%, but exceeds the shuffled-DMIS placebo by only 0.0103 percentage points. The four operational regimes are threshold-calibrated rather than empirically recovered: GMM ARI is only 0.135 in controlled synthetic and 0.346 in SUMO. These results may be interesting as negative or exploratory findings, but they do not yet establish that the proposed landscape provides robust incremental diagnostic value.

  2. The “causal” framing and cross-architecture comparison remain limited.
    The paper explicitly states that CausalGap does not identify latent causal mechanisms, and DMIS measures response at a selected interface rather than a causal effect. This is an important clarification, but the name and framing still invite causal reading. In addition, DSAI Track A and Track B produce substantially different landscapes (mean DMIS 0.2256 vs. 0.8768; regime agreement 66.1%), so cross-architecture conclusions depend heavily on the Track B encoder and interface. Real observational datasets are excluded from the response-failure analyses because counterfactual outcomes are unavailable; SUMO is only an abstract replication. This limits both the causal interpretation and the real-world relevance of the benchmark.

Questions for authors
  1. Can the authors report confidence intervals, permutation tests, and per-family results for H1? How should readers interpret the mean-only reversal and the RSA direction change? Is H1 still a confirmatory result if the direction is not stable across prespecified specifications?
  2. What is the operational value of the four regimes if GMM recovery is weak and thresholds are calibration medians? Would a continuous landscape be more appropriate than discrete labels? Can the authors provide a decision-relevant example where regime assignment changes a practitioner’s conclusion?
  3. How sensitive are the main conclusions to the Track B encoder capacity and interface choice? Since Track A and Track B disagree substantially, which reported conclusions depend on Track B, and can the authors report an encoder-capacity or adapter sensitivity analysis?
  4. Why are real observational datasets completely excluded from response-failure analyses? Could the I-405 closure or another documented event be used as a non-confirmatory case study to illustrate the diagnostic on real data?
  5. Given the small H2 placebo margin, should H2 be framed as exploratory rather than as evidence for explicit DMIS parameterization utility? Can the authors add more seeds/models or report a power analysis?
  6. How sensitive are the family-level regime patterns to the Family II/III boundary and to individual model inclusion? A leave-one-model-out or alternative family grouping analysis would strengthen the claim.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.