- Rating
- 2(+1)
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 1
Read review
Paper summary
EviPhys combines a frozen ViLT-based multimodal representation module, a Dempster–Shafer evidence reasoning module, dynamic graph learning, and a graph-recurrent forecasting model regularized by a graph-diffusion prior. The paper reports results on four PEMS datasets and two LargeST datasets (SD and GBA), with improvements over the listed baselines. Its main claims concern multimodal traffic semantics, uncertainty-aware local representations, adaptive spatial dependencies, and more stable long-horizon prediction.
Critical strengths
- Promising reported predictive performance. EviPhys reports competitive results across four PEMS datasets and two larger LargeST datasets. For example, it reports an MAE of 12.99 on PEMS08, compared with 13.56 for the best listed baseline.
- Reasonable motivation for free-running trajectory regularization. Applying a graph-diffusion residual to recursively generated trajectories is a plausible way to encourage consistency with a chosen road-graph prior during multi-step prediction. The regularizer is removed at inference, which may limit its direct inference overhead.
Critical weaknesses
- Core implementation does not match the stated Deng-entropy formulation. The paper defines Deng entropy with a denominator that depends on the cardinality of each focal set. In the submitted code,
compute_deng_entropyuses a constant denominator of 7. When the masses sum to one and clipping is inactive, this computes Shannon entropy plus ln(7), rather than the stated Deng entropy. The implementation also applies an eight-channel Softmax without an explicit mapping that excludes the empty set, despite the text specifying zero mass for the empty set. This mismatch undermines the claim that the reported results validate the proposed Deng-entropy mechanism. - The statistical significance results are not readily reconcilable with the reported summary statistics. For PEMS04, Table 8 reports 18.36 ± 0.091 for PDFormer and 17.95 ± 0.072 for EviPhys, with a paired-test p-value of 0.008 over 10 runs. Under the conventional interpretation of these standard deviations and a two-sided paired t-test, the bound s_d ≤ s_1 + s_2 implies |t| ≥ 7.954 (df = 9), corresponding to a p-value far below 0.008. The authors should provide per-seed results, the pairing procedure, and the exact test code. This is a reporting inconsistency that requires clarification; it is not, by itself, evidence of misconduct.
- The claimed stability guarantee is conditional and is not enforced by the parameterization. The appendix’s non-expansiveness argument requires 0 ≤ κΔt ≤ 1. The submitted implementation uses Softplus for the diffusion coefficient, which enforces positivity but does not enforce the required upper bound. This does not invalidate the conditional theorem, but the paper should report whether the trained model satisfies the condition and avoid presenting the theorem as an unconditional guarantee for the implemented model.
- Key ablations and robustness comparisons are missing. The main ablation study removes the VLM, evidence, and dynamic-graph components, but does not isolate the effect of the graph-diffusion regularizer itself. The robustness table reports EviPhys under different noise levels without baseline or no-evidence comparisons. As a result, the experiments do not establish how much benefit comes specifically from Deng-entropy gating or the physical regularizer.
- The image construction and scalability claims need stronger validation. The sensor-axis difference used to construct the image may depend on the arbitrary ordering of sensors unless that ordering is tied to road topology. The authors should test sensitivity to consistent sensor permutations. In addition, Table 3 reports out-of-memory configurations, which does not invalidate the LargeST results but does qualify the scalability claim; peak memory and throughput across graph sizes would
Questions for authors
- Implementation and main results. Which exact code version produced the reported main results? Please clarify the focal-set mapping, explicitly show how the empty set is excluded, and provide results from the implementation that matches the paper’s Deng-entropy equation.
- Statistical testing. Please provide the per-seed MAEs for Table 8, the pairing rule, and the statistical test script, and explain the discrepancy between the reported summary statistics and p-values.
- Physical prior and robustness. How is κΔt ≤ 1 guaranteed in the trained model? Please provide ablations that separately remove the topology-guided correction and graph-diffusion regularizer, as well as noise evaluations against strong baselines and a model without the evidence branch.
- Reproducibility and sensor ordering. Please provide the configurations and evaluation protocol used for SD and GBA, and clarify how sensors are ordered when constructing the traffic images. A permutation-sensitivity test would help determine whether the sensor-axis image channel captures meaningful spatial structure.
12 comments
Thank you for identifying the statistical inconsistencies. We recheck the ten-seed results, correct the paired t-test calculations, and provide updated significance results with per-seed records.
Table R3-1. Statistical Significance Across Ten Random Seeds
| Dataset | PDFormer MAE (Mean ± Std) | EviPhys MAE (Mean ± Std) | Two-sided Paired p-value |
|---|---|---|---|
| PEMS03 | 14.9398 ± 0.083055 | 14.3100 ± 0.060753 | 6.29 × 10⁻⁸ |
| PEMS04 | 18.3601 ± 0.091033 | 17.9496 ± 0.072077 | 9.84 × 10⁻⁷ |
| PEMS07 | 19.9701 ± 0.112020 | 18.6598 ± 0.086005 | 6.19 × 10⁻¹⁰ |
| PEMS08 | 13.5800 ± 0.073955 | 12.9900 ± 0.059035 | 8.85 × 10⁻⁹ |
Note: Each dataset contains ten paired observations using random seeds 42–51. Standard deviations are calculated with ddof=1, and p-values are obtained using two-sided paired t-tests with 9 degrees of freedom.
Analysis: EviPhys consistently achieves lower mean MAE and smaller standard deviations than PDFormer across all four PEMS datasets. All corrected p-values are below 0.001, providing strong statistical evidence for the observed performance differences. These results demonstrate that the improvements are consistent across different random initializations rather than being driven by a single favorable run.
Table R3-2. Per-Seed MAE Results for PDFormer and EviPhys
To ensure reproducibility and clarify the relationship between the reported single-run and multi-run results, we provide the individual MAE values obtained using ten random seeds.
PEMS03 and PEMS04
| Seed | PEMS03 PDFormer | PEMS03 EviPhys | PEMS04 PDFormer | PEMS04 EviPhys |
|---|---|---|---|---|
| 42 | 14.938 | 14.311 | 18.361 | 17.947 |
| 43 | 14.863 | 14.305 | 18.377 | 18.059 |
| 44 | 14.930 | 14.300 | 18.436 | 17.919 |
| 45 | 14.949 | 14.383 | 18.467 | 17.958 |
| 46 | 14.818 | 14.375 | 18.265 | 18.040 |
| 47 | 14.984 | 14.235 | 18.205 | 17.838 |
| 48 | 14.846 | 14.388 | 18.337 | 17.980 |
| 49 | 14.948 | 14.230 | 18.284 | 17.881 |
| 50 | 15.083 | 14.333 | 18.377 | 17.997 |
| 51 | 15.039 | 14.240 | 18.492 | 17.877 |
PEMS07 and PEMS08
| Seed | PEMS07 PDFormer | PEMS07 EviPhys | PEMS08 PDFormer | PEMS08 EviPhys |
|---|---|---|---|---|
| 42 | 19.970 | 18.658 | 13.582 | 12.990 |
| 43 | 19.904 | 18.709 | 13.553 | 12.962 |
| 44 | 19.737 | 18.715 | 13.626 | 12.999 |
| 45 | 19.859 | 18.579 | 13.668 | 13.053 |
| 46 | 20.135 | 18.636 | 13.607 | 12.954 |
| 47 | 20.049 | 18.564 | 13.490 | 12.979 |
| 48 | 20.020 | 18.576 | 13.714 | 12.941 |
| 49 | 20.038 | 18.770 | 13.537 | 13.038 |
| 50 | 20.004 | 18.588 | 13.529 | 13.093 |
| 51 | 19.985 | 18.803 | 13.494 | 12.891 |
Note: All MAE values are reported to three decimal places. PDFormer and EviPhys results are aligned by identical random seed indices.
Analysis: EviPhys achieves lower MAE than PDFormer for every reported seed across all four datasets, indicating that its performance advantage is not limited to a particular random initialization. The per-seed results also provide the underlying observations required to reproduce the reported means, standard deviations, and statistical significance tests.
Pairing Rule and Reproducible Test Script
For each dataset, we pair the PDFormer and EviPhys MAEs obtained using the same random seed, from seed 42 through seed 51. We then apply a two-sided paired t-test to the ten within-seed differences. This ensures that the statistical comparison follows a consistent pairing rule.
import numpy as np
from scipy.stats import ttest_rel
# Values are ordered by seeds 42, 43, ..., 51.
scores = {
"PEMS03": {
"pd": [14.938, 14.863, 14.930, 14.949, 14.818,
14.984, 14.846, 14.948, 15.083, 15.039],
"evi": [14.311, 14.305, 14.300, 14.383, 14.375,
14.235, 14.388, 14.230, 14.333, 14.240],
},
"PEMS04": {
"pd": [18.361, 18.377, 18.436, 18.467, 18.265,
18.205, 18.337, 18.284, 18.377, 18.492],
"evi": [17.947, 18.059, 17.919, 17.958, 18.040,
17.838, 17.980, 17.881, 17.997, 17.877],
},
"PEMS07": {
"pd": [19.970, 19.904, 19.737, 19.859, 20.135,
20.049, 20.020, 20.038, 20.004, 19.985],
"evi": [18.658, 18.709, 18.715, 18.579, 18.636,
18.564, 18.576, 18.770, 18.588, 18.803],
},
"PEMS08": {
"pd": [13.582, 13.553, 13.626, 13.668, 13.607,
13.490, 13.714, 13.537, 13.529, 13.494],
"evi": [12.990, 12.962, 12.999, 13.053, 12.954,
12.979, 12.941, 13.038, 13.093, 12.891],
},
}
for dataset, values in scores.items():
pd = np.asarray(values["pd"], dtype=float)
evi = np.asarray(values["evi"], dtype=float)
result = ttest_rel(evi, pd, alternative="two-sided")
print(
dataset,
f"PDFormer={pd.mean():.6f} ± {pd.std(ddof=1):.6f}",
f"EviPhys={evi.mean():.6f} ± {evi.std(ddof=1):.6f}",
f"t(9)={result.statistic:.6f}",
f"p={result.pvalue:.12g}",
)
Statistical Clarification: The reported means and sample standard deviations can be reproduced directly from the per-seed observations in Table R3-2.
Thank you for the valuable suggestions. We add physics-loss ablations, noise robustness comparisons, and large-scale computational efficiency evaluations.
To further strengthen our responses to W3–W5 and Q3–Q4, we conducted additional experiments covering three aspects: (1) physics regularization ablation across six datasets to isolate the contribution of the graph-diffusion loss; (2) comparative noise robustness evaluation to assess prediction stability and the contribution of evidence modeling; and (3) computational efficiency analysis on SD and GBA to examine practical scalability. The detailed results and analyses are presented in Tables R3-3, R3-4, and R3-5, respectively.
Table R3-3. Ablation Study of Physics Regularization
To quantify the independent contribution of physics regularization, we compare the full EviPhys model with a variant using across six datasets, while retaining the topology-guided correction mechanism.
| Dataset | Full EviPhys | ΔMAE | |
|---|---|---|---|
| PEMS03 | 14.31 | 14.43 | +0.12 |
| PEMS04 | 17.95 | 18.12 | +0.17 |
| PEMS07 | 18.66 | 18.89 | +0.23 |
| PEMS08 | 12.99 | 13.10 | +0.11 |
| SD | 16.63 | 16.86 | +0.23 |
| GBA | 19.03 | 19.31 | +0.28 |
Note: ΔMAE denotes the increase in MAE after removing physics regularization.
Analysis: Removing physics regularization consistently increases MAE across all six datasets, with increases ranging from 0.11 to 0.28. These results demonstrate that graph-diffusion regularization provides additional predictive benefits beyond the topology-guided correction retained in the model. The consistent improvements across PEMS, SD, and GBA further support its effectiveness across different traffic network scales.
Table R3-4. Noise Robustness Comparison Under Different Perturbation Levels
To further evaluate noise robustness, we compare EviPhys with HimNet (2024) and EviPhys without the evidence modeling branch under four noise levels on PEMS04.
| Noise Level | Model | MAE | RMSE | MAPE (%) |
|---|---|---|---|---|
| 0% | HimNet (2024) | 18.17 | 29.92 | 12.03 |
| EviPhys w/o Evidence | 18.27 | 30.18 | 12.14 | |
| EviPhys | 17.95 | 29.87 | 11.91 | |
| 5% | HimNet (2024) | 18.57 | 30.48 | 12.31 |
| EviPhys w/o Evidence | 18.62 | 30.64 | 12.37 | |
| EviPhys | 18.21 | 30.24 | 12.08 | |
| 10% | HimNet (2024) | 19.12 | 31.28 | 12.70 |
| EviPhys w/o Evidence | 19.08 | 31.21 | 12.73 | |
| EviPhys | 18.56 | 30.71 | 12.31 | |
| 20% | HimNet (2024) | 20.05 | 32.68 | 13.49 |
| EviPhys w/o Evidence | 19.83 | 32.44 | 13.35 | |
| EviPhys | 19.04 | 31.52 | 12.74 |
Note: All experiments are conducted on PEMS04.
Analysis: EviPhys consistently achieves the lowest MAE, RMSE, and MAPE across all four noise levels. At 20% noise, it reduces MAE by 5.04% compared with HimNet and 3.98% compared with EviPhys w/o Evidence. Moreover, its MAE increases by only 6.07% from 0% to 20% noise, compared with 10.35% for HimNet and 8.54% for EviPhys w/o Evidence. These results demonstrate stronger noise tolerance under the evaluated perturbations and suggest that evidence modeling contributes to maintaining forecasting accuracy under noisy conditions.
Table R3-5. Computational Efficiency and Scalability on Large-Scale Datasets
To further evaluate computational efficiency and scalability, we report the number of trainable parameters, inference time, and peak GPU memory usage of EviPhys on the SD and GBA datasets.
| Dataset | Nodes | Trainable Params | Inference Time (s) | Peak GPU Memory (MiB) |
|---|---|---|---|---|
| SD | 716 | 883,744 | 0.0381 | 529.6 |
| GBA | 2,352 | 909,920 | 0.0341 | 772.4 |
Note: Inference time is measured with a batch size of 1. Peak GPU memory usage is reported in MiB.
Analysis: EviPhys maintains a relatively stable parameter count across the two datasets. As the number of sensors increases from 716 to 2,352, trainable parameters increase by only 2.96%, while peak GPU memory usage increases from 529.6 MiB to 772.4 MiB. The measured inference times are 0.0381 s and 0.0341 s, respectively. These results demonstrate the practical feasibility of EviPhys on the evaluated large-scale traffic networks. Nevertheless, dynamic graph construction involves pairwise node relationships, and its computational and memory requirements depend on network size. Therefore, these findings support scalability within the evaluated settings without implying linear complexity or unrestricted scalability.
Thank you for your insightful comments. We clarify the distinction between Deng-entropy-inspired evidence modeling and strict Deng entropy, verify ten-seed statistical significance, explain the conditional stability and role of graph-diffusion regularization, and address the physical interpretation and scalability of Hovmöller representations.
W1/Q1. Deng entropy motivates our uncertainty-aware local representation learning, but DSERM does not implement strict focal-set-wise Deng entropy. Instead, it maps each sensor's history into eight learnable latent evidence channels and applies Softmax to obtain a normalized distribution. These channels do not explicitly correspond to the eight power-set subsets of a three-element frame of discernment, nor is empty-set mass explicitly set to zero. We use a fixed factor K−1=7 to construct a Deng-entropy-inspired surrogate, equivalent to Shannon entropy plus log7 when clipping is inactive. This surrogate measures evidence-channel dispersion and adaptively regulates local representations for dynamic graph construction through a learnable gate, rather than distinguishing composite propositions of different cardinalities. We have clarified these distinctions, the actual formulation, and its relationship to Shannon entropy in Section 3.3. All reported results follow this implementation, ensuring consistency between the experiments and the implemented method.
W2/Q2. The 10 independent runs assess forecasting consistency across random initializations rather than relying on a single favorable result. Means and standard deviations quantify performance variability, while significance tests evaluate the reliability of observed differences. We acknowledge that the previously reported PEMS04 p-value is inconsistent with the paired t-test result recalculated from the per-seed observations. We have rechecked the per-seed results, pairing procedure, and statistical calculations. Corrected statistics and per-seed results are presented in Tables R3-1 and R3-2 of the second comment, respectively.
W3/Q3. The graph-diffusion regularizer constrains recursive predictions using physical road topology rather than enforcing exact diffusion dynamics. Appendix A shows that when 0≤κΔt≤1 and Rt=0, the graph-diffusion transition is non-expansive; for nonzero residuals, deviations from ideal diffusion are bounded by accumulated residuals. In our implementation, Softplus ensures κ>0, but κΔt≤1 is not explicitly enforced. Thus, the theorem establishes conditional stability rather than an unconditional guarantee for the trained model. Nevertheless, penalizing graph-diffusion residuals remains a physically motivated objective encouraging topology-consistent evolution during free-running prediction, thereby discouraging deviations from topology-guided traffic propagation. We have clarified this distinction and added a λpinn=0 ablation to isolate the physics loss contribution, with results presented in Table R3-3 of the third comment.
W4/Q3. EviPhys incorporates physical information through two complementary mechanisms. Topology-guided correction directly adjusts predicted traffic flow increments using state differences between physically connected sensors, while graph-diffusion regularization penalizes topology-inconsistent trajectories during training without additional residual computation at inference. Together, they balance nonlinear forecasting with physics-inspired spatial consistency. Existing baseline comparisons demonstrate competitive forecasting accuracy, while noise perturbation experiments assess stability under increasing input disturbances. These complementary evaluations support predictive effectiveness and noise tolerance. To strengthen the evidence, we have added a λpinn=0 ablation and noise robustness comparisons against representative baselines and EviPhys without evidence modeling. The results are presented in Tables R3-3 and R3-4 of the third comment.
W5/Q4. Hovmöller images encode spatiotemporal traffic variations for visual representation learning rather than treating sensor ordering as physical spatial coordinates. The three channels represent traffic flow states, temporal variations, and sensor-dimensional variations. Temporal variations capture traffic dynamics, while sensor-dimensional variations reflect adjacent observations under a given sensor ordering, not physical spatial gradients. Actual road topology is explicitly incorporated through subsequent graph modeling. Thus, Hovmöller images complement rather than replace topology-based spatial modeling. To evaluate scalability, we further assess EviPhys on SD and GBA, containing 716 and 2,352 sensors, respectively. Since dynamic graph construction involves pairwise node relationships, computational and memory costs increase with network size. We therefore do not claim linear complexity or unrestricted scalability. The results support effectiveness at the evaluated network scales without implying size-independent computational costs. Additional results on trainable parameters, inference time, and peak GPU memory usage on SD and GBA are presented in Table R3-5 of the third comment.