- Rating
- 4
- Confidence
- 4
- Informativeness
- 3
- Presentation
- 3
- Soundness
- 3
Read review
Paper summary
This paper proposes MapLightning, which replaces the dense BEV grid with a compact set of 1D learnable map tokens. Compared with dense BEV representations, the proposed representation uses fewer tokens, requires less memory, and enables faster inference. It also allows the map decoder to use full attention for better global context and long-range dependencies, while avoiding reliance on camera projection parameters and thus being robust to camera-extrinsic perturbations. Extensive experiments demonstrate strong accuracy and efficiency on nuScenes and Argoverse 2. The method is also generalizable to object detection and downstream trajectory prediction.
Critical strengths
(+) Novel design with strong accuracy and efficiency. Most works in the mapping community follow MapTR-style design using BEV as an inefficient representation, whereas MapLightning is the first work to replace dense BEV with compact 1d tokens, achieving state of the art accuracy and efficiency, and avoid camera-extrinsic perturbations which is common for real-world autonomous driving.
(+) Comprehensive experimental evaluation. The paper provides extensive comparisons across two datasets, three image backbones, different perception ranges and dataset splits, together with additional experiments on uncertainty-aware mapping, trajectory prediction, and 3D object detection.
(+) The paper is well motivated, clearly organized, and easy to follow.
Critical weaknesses
(-) Adaptation to end-to-end frameworks is not fully discussed. The authors mention in the limitations section that the compact 1D token design may require additional adaptation for end-to-end frameworks such as MapFormer in UniAD, but they do not explain what this adaptation would look like. Since autonomous driving is moving toward unified end-to-end architectures rather than modular pipelines, addressing this compatibility issue would make the proposed representation more broadly useful.
Questions for authors
-
Why does the proposed method outperform other baselines by a much larger margin when using smaller real-time backbones such as MobileNetV3-Large or ResNet-18?
-
How are the 1D map tokens initialized? Are they simply randomly initialized learnable embeddings, and are the same token embeddings shared across all scenes?
Response to Reviewer 4DLy
Thank you for the careful and comprehensive review! Our response is:
W1: Adaptation to end-to-end frameworks is not fully discussed. (e.g., MapFormer in UniAD)
We thank the reviewer for this suggestion and agree that integrating MapLighting into an end-to-end architecture would expand its utility. However, we deliberately scoped the paper to mapping: it lets us isolate the effect of the intermediate representation under a controlled comparison, which is not possible when the change is entangled with several jointly trained downstream modules. While it is beyond the scope of the current paper, we sketch out a potential adaptation below and leave its full implementation and analysis to future work. We will include this discussion in the camera-ready.
End-to-end frameworks such as UniAD connect TrackFormer, MapFormer, MotionFormer, OccFormer, and the Planner downstream of BEV features B. The key integration step is to treat the multi-view tokens X = SelfAttention(I), where I are image tokens, as a substitute for BEV features B. Because all downstream modules share one compact token set, this might lower the cost of the shared representation in end-to-end pipelines. MapFormer can be then replaced by MapLightning directly: map tokens M are injected into X via further self-attention blocks in the mapping branch, and the decoder uses full cross-attention on M to predict vectorized map elements, whose queries QM are passed to MotionFormer.
This change from BEV features B to 1D multi-view tokens X also affects other modules. Similar architectural changes can be made to TrackFormer, MotionFormer, and the Planner. Further research needs to be done to see the benefit of 1D multi-view tokens for OccFormer.
Q1: Why does the proposed method outperform other baselines by a much larger margin when using smaller real-time backbones such as MobileNetV3-Large or ResNet-18?
We hypothesize that lightweight backbones produce weaker image features, and that dense BEV projection (e.g., MapTRv2) may be affected more because it lifts image features in a single pass without refining them. In contrast, our mapper jointly refines image and map tokens through multiple self-attention layers (illustrated in Figure 3), potentially compensating for weaker features.
Q2: How are the 1D map tokens initialized? Are they simply randomly initialized learnable embeddings, and are the same token embeddings shared across all scenes?
The map tokens are randomly initialized learnable embeddings, shared across all scenes.