- Rating
- 3
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 3
Read review
Paper summary
This paper proposes partitioning the voxel grid at a query frame by time of first observation: Robs (resolved at query time t), Rfuture (hidden at t but resolved by later ego LiDAR sweeps transformed back to the query pose), and Rnever (never resolved in the log). Later sweeps are ray-cast to produce free/occupied labels in Rfuture, turning ego motion into delayed physical supervision for geometry that was occluded at t. A matched intervention - changing only the loss mask of an occupancy learner from Robs to Robs + Rfuture - raises Rfuture F1 by +0.1156 on Occ3D-nuScenes (five-seed scene-clustered CI [+0.103, +0.122]) without degrading standard metrics, and transfers to Waymo, multiple encoders, cross-domain evaluation, and a second, depth-free lifting architecture. The paper additionally (i) profiles six released models on a coverage-precision frontier in Rfuture that standard Occ3D scoring cannot see, and (ii) uses the Rfuture-supervised occupancy model as a geometric verifier of counterfactual rollouts from the Cosmos 3 Edge driving world model, where no reference image exists, showing about 2x stronger action-conditioned separation than an observed-only reader and a measurable selection gain among stochastic rollouts.
Critical strengths
-
Clean and well-motivated idea, executed as a controlled experiment. The core intervention changes only the loss region; architecture, schedule, seeds and labels are held fixed. The "full" arm (asserting free on Rnever) and the 90-degree rotated-label control directly address the two most obvious objections (more targets; any extra supervision). This level of experimental control is rare.
-
Unusually honest and rigorous statistics. Matched operating points instead of argmax comparisons, scene-clustered bootstrap intervals, five-seed replication with per-seed and per-scene breakdowns, pre-specified success criteria for the second architecture (with a candid admission that "pre-specified" is not externally verifiable), and negative results reported rather than hidden (Waymo hallucination non-replication, path-proximity loss weighting doing nothing, the withdrawn angle-divergence explanation, the planner probe that cannot discriminate). Appendix G's disclosure of two near-miss statistical errors in their own code is commendable.
Critical weaknesses
-
The headline gain is partly by construction, and the residual evidence is on a self-defined metric. Training on region X and evaluating on region X is close to tautological; the paper's real claims are (a) standard metrics do not decline, (b) the gain is not mere conservatism or blob suppression, and (c) transfer. (a) holds but the margins on standard metrics are small (matched RayIoU +0.0051); the strongest evidence is (b)-(c), which is solid but should be foregrounded even more honestly as the load-bearing result.
-
Dynamic-object label noise is under-specified. A voxel occupied at t by a hidden object that moves away before ever being observed will be labelled free by later sweeps - a systematic error mode. Section 6.4 evaluates on objects "hidden at t and revealed after they move" (recall 0.716 to 0.856), so such voxels evidently exist in the evaluation; but the construction of their labels (which sweep, which classifier, what happens when the object is never seen while stationary) is never spelled out in the main text. This is the single most important methodological detail for a reader to trust Rfuture labels.
Questions for authors
-
Exactly how are voxels labelled that were occupied at t by an object that moves before first being observed? Which sweep provides the occupied label, and what fraction of Rfuture labels does this affect? Is a moving-but-never-seen-while-present object labelled free?
-
What future horizon is used to construct Rfuture by default, and how was it chosen? (Doubling the horizon moves free-space error by 0.0006, but the base horizon is not stated in the main text.)
-
Does Rfuture supervision change semantic mIoU on the standard Occ3D protocol, even on the small models? If it cannot be tested, why not - could a released semantic checkpoint provide pseudo-labels on train?
Thank you for the positive and careful review.
W1. Which result carries the claim. We agree. Because
obs_futureis trained on the region it is scored on, part of the gain is expected by construction. The evidence we rely on is that the gain is not explained by conservatism or by suppressing errors near the ego, and that it transfers. On standard metrics we claim only "no decline". The introduction now says this directly. We also add a comparison with Occ3D labels (general response, point 1), which shows that much of the gain comes from accumulating later sweeps.W2/Q1. A voxel occupied at t by an object that leaves before it is seen. It is labelled free, because no later sweep sees it occupied. A hidden moving object gets an occupied label only if a sweep sees it before it leaves. Against nuScenes boxes, such voxels are 0.45% of R_future, and 67% of them are labelled free. Removing all voxels near annotated objects, the F1 gain is +0.128 (point 2 of the general response). The rule is now stated in Sec. 3.1.
Q2. Default horizon. 10 s: the next 20 key frames on nuScenes (plus the sweeps between them) and the next 100 frames on Waymo. It was fixed before any training. Doubling it to 20 s changes the free-space error by only 0.0006, so the never-observed region is not an artefact of the window. On nuScenes we also use a later observation only within 15 m of the sensor, because farther away a 0.4 m voxel can fall between two LiDAR beams. Both are now in Sec. 3.1.
Q3. Semantic mIoU. We could not test this during the discussion period. A semantic arm needs per-class labels for every training frame, and the semantic teacher outputs we have (from a released FlashOcc) cover only the validation split. The closest evidence is the class-stratified binary result (Sec. 6.4): with one matched threshold, the R_future gain is positive for every class group, including vehicles (+0.102) and vulnerable road users (+0.108). These are binary results, not semantic accuracy.