- Rating
- 2(−1)
- Confidence
- 3
- Informativeness
- 3
- Presentation
- 3
- Soundness
- 3
Read review
Paper summary
EviScope studies whether local radar evidence should condition spatial and temporal query sampling for long-range camera-radar 3D detection. The method constructs a parameter-free BEV evidence field and uses a 5,450-parameter shared conditioner to predict a radial sampling scale and temporal weights. On MAN TruckScenes, the authors first establish a strong RaCFormer training recipe and then evaluate the module with seven matched four-epoch pairs, one eight-epoch pair, allocation probes, substitution tests, and an internally pre-specified held-out seed test. The recipe is competitive, and the module reliably learns evidence-associated allocation, but its incremental detection benefit remains statistically unresolved (+0.31 +/- 0.36 NDS and -0.27 +/- 0.22 mAP over seven pairs).
Critical strengths
- The empirical methodology is unusually careful. Matched seeds, uncertainty estimates, a held-out confirmation test, and an evidence-blind control prevent the paper from overstating attractive but non-replicating findings. This is consequential because it turns an inconclusive module result into a credible lesson about evaluation in a noisy 3D-detection regime.
- The proposed conditioner is technically lightweight and well motivated, and the supplementary package is detailed enough to inspect the key implementation choices and reproduce the analyses. The strong module-free recipe is itself practically useful.
Critical weaknesses
- The central architectural claim is not supported by a reproducible end-to-end gain: the paired mAP estimate is negative and the NDS estimate is smaller than its standard error. Therefore acceptance rests mainly on the diagnostic methodology and negative result, not on EviScope as an improved detector.
- The evidence-specific mechanism is not isolated from learned geometry/temporal conditioning under the full schedule. The paper identifies the needed geometry-only matched-seed control but does not report it, leaving the causal value of query-local radar evidence open.
Questions for authors
- Can the authors provide full-schedule matched-seed results for a geometry-only or evidence-shuffled conditioner with the same capacity and learned temporal branch? This would materially determine whether local radar evidence, rather than added adaptive capacity, is useful.
- How sensitive are the paired conclusions to checkpoint selection and to evaluation on a second split or dataset? Even a limited cross-dataset check would affect how broadly the negative/diagnostic conclusion should be interpreted.
2 comments
Follow-up to Reviewer kiSa: Completed Three-Seed Controls (W1, Q1)
Thank you again for your careful review. Since our initial reply to your review—which reported two seeds each for the shuffled-evidence control (W2/Q1) and the eight-epoch baseline comparison (W1)—seed 2 has now completed for both experiments (with the shuffled-evidence update first noted in our 15 November comment, and the third eight-epoch pair just completed on the same node):
| Matched comparison | Completed seeds | EviScope − reference (mAP) | EviScope − reference (NDS) |
|---|---|---|---|
| Geometry-only conditioner (4 ep, W2/Q1) | 0–2 (3/3) | +0.87 ± 0.21 (+1.12, +1.04, +0.46) | +1.25 ± 0.72 (+1.62, +2.26, −0.14) |
| Shuffled-evidence conditioner (4 ep, W2/Q1) | 0–2 (3/3) | +0.86 ± 0.34 (+1.28, +0.20, +1.11) | +1.75 ± 0.74 (+2.89, +1.99, +0.36) |
| Module-free baseline at 8 epochs (W1) | 0–2 (3/3) | +0.13 ± 0.13 (+0.26, −0.13, +0.27) | +0.66 ± 0.31 (+0.67, +1.19, +0.12) |
Values are mean ± SE in percentage points:
- Causal isolation within the conditioner (W2/Q1): Both four-epoch controls retain the exact 5,450 parameters, learned radial and temporal heads, identity initialisation, and auxiliary target. At seed 2, the shuffled control scores
24.66 / 37.37against EviScope's25.77 / 37.73(+1.11mAP,+0.36NDS), so both mAP and NDS improve in all three shuffled pairs, and all three shuffled runs also fall below the module-free baseline on both metrics (−0.65 / −1.18,−0.17 / −0.60,−1.51 / −0.02). - Matched baseline comparisons (W1): At seed 2 (8 epochs, both arms trained and evaluated on one node), EviScope reaches
26.65 / 39.27against the baseline's26.38 / 39.15(+0.27mAP,+0.12NDS). Across all three eight-epoch pairs, NDS is positive in every pair (+0.66 ± 0.31), alongside the seven four-epoch pairs (+0.31 ± 0.36NDS,−0.27 ± 0.22mAP; Table 7).
Could you please let us know whether these completed three-seed controls address your W2/Q1, and which remaining concerns—whether the four-epoch end-to-end margin (W1), single-benchmark scope (Q2), or points raised elsewhere in the discussion—weigh most heavily in your current assessment? We would greatly appreciate the opportunity to address them directly.
Response to Reviewer kiSa
Thank you for identifying the full-schedule controls that can test the contribution of local radar evidence. We report geometry-only controls at three seeds, shuffled-evidence controls at two seeds, and the sensitivity analyses below. All ± values are mean ± SE over matched seeds.
W1. Reproducible gain over the module-free detector. The original seven four-epoch pairs remain +0.31 ± 0.36 NDS and −0.27 ± 0.22 mAP. Our new evidence distinguishes the value of the input within the conditioner from the module's total gain:
W2 and Q1. Full-schedule geometry-only and shuffled-evidence controls.
Q2. Checkpoint sensitivity and another split/dataset.