- Rating
- 1
- Confidence
- 4
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 3
Read review
Paper summary
This paper revisits LSS-based view transformation. The authors observe that standard lifting assigns the same image feature to different depth bins along a camera ray, which results in a rank-one representation. Based on this observation, they propose RayLift. The main component is Multi-Feature Lifting (MFL). MFL generates multiple feature heads from image feature and uses optimal transport to route different mixtures of these heads to different depth bins. The paper further introduces Causal Ray-Radial Convolution (CRRC) for near-to-far contextual propagation in ego-centric polar space and visibility-conditioned temporal fusion for incorporating historical features. Experiments on Occ3D-nuScenes, SurroundOcc, and Occ3D-Waymo demonstrate the state-of-the art performance of RayLift.
Critical strengths
The observation that conventional LSS-based lifting produces highly constrained per-ray representations is intuitive and well motivated. This paper also makes an effort to connect this observation to the proposed architecture.
Critical weaknesses
-
The main improvements are not well aligned with the claimed occlusion-recovery mechanism. RayLift is motivated by recovering farther occluded semantics along multi-surface rays, but RayIoU evaluates only the first occupied surface along each ray. In Table 2, the gains are more pronounced in RayIoU than in mIoU, which does not directly support the claimed improvement on farther occluded objects.
-
The current ablations do not isolate the necessity of the proposed optimal-transport-based MFL design. MFL introduces several changes simultaneously, including multiple learned feature heads, geometry-conditioned routing and so on. The current ablations mainly show that the complete MFL module improves performance, but do not clarify which of these ingredients is actually responsible for the gain.
-
The analysis is performed on GT-selected multi-class rays, where different occupied depths have different semantic labels. This setting naturally favors the paper’s hypothesis that features at different depths should become less similar. However, not all depth variation corresponds to semantic variation. A single object can occupy several adjacent depth bins along the same ray, in which case maintaining similar representations across these depths may be beneficial. Since the proposed balanced routing explicitly encourages the use of multiple feature heads, it is unclear whether MFL selectively introduces diversity only when the underlying semantics change, or whether it also introduces unnecessary variation within the same object.
Questions for authors
-
Why are the gains more pronounced in RayIoU if the main motivation is to recover farther occluded semantics?
-
How necessary is the optimal-transport-based routing in MFL?
-
Does MFL preserve feature consistency across depth bins belonging to the same object or semantic class?