acceptodds
Under review as a conference paper at ICLR 2027

MapLightning: Online Vectorized HD Map Construction with 1D Map Tokens

Abstract

Online vectorized HD map construction is essential for scaling safe autonomous driving and requires accurate, real-time inference. Prior methods typically rely on dense bird's-eye-view (BEV) grids as the intermediate representation. We propose MapLightning, which replaces the dense BEV grid with a compact set of 1D learnable map tokens. To construct map tokens from image features, we choose self-attention over vanilla cross-attention because it enables joint interactions and contextual aggregation among image and map tokens. Our transformer-based mapper concatenates map and image tokens, applies full self-attention, discards the image tokens, and retains the updated map tokens for decoding. This design offers three advantages. First, our representation is efficient, using fewer tokens, consuming less memory, and running faster. Second, the lightweight design allows the map decoder to use full rather than deformable cross-attention for better global context. Third, unlike BEV-based methods, our network does not use camera projection parameters, making it robust to camera-extrinsic perturbations. MapLightning uses up to 16.7 fewer intermediate tokens than dense BEV-based methods and achieves state-of-the-art accuracy and efficiency on nuScenes and Argoverse 2. Its lightweight variant surpasses MapTRv2 by +10.1 mAP on nuScenes and +16.2 mAP on Argoverse 2, while delivering 1.73 faster inference (40+ FPS) with 53% less memory. We further show improvements on uncertainty-aware map construction and trajectory prediction. Code and models will be released.

Primary area:
applications to robotics, autonomy, planning
Keywords:
Vectorized HD Map Construction, Trajectory Prediction, Token Compression, Bird's-Eye-View, Autonomous Driving

Reviews

Among:
Shape:
1234this paper, Top 4%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.33 over 3 reviews, above 96% of the others.
Reviewer_4DLyOpenReview ↗
Rating
4
Confidence
4
Informativeness
3
Presentation
3
Soundness
3
Read review
Paper summary

This paper proposes MapLightning, which replaces the dense BEV grid with a compact set of 1D learnable map tokens. Compared with dense BEV representations, the proposed representation uses fewer tokens, requires less memory, and enables faster inference. It also allows the map decoder to use full attention for better global context and long-range dependencies, while avoiding reliance on camera projection parameters and thus being robust to camera-extrinsic perturbations. Extensive experiments demonstrate strong accuracy and efficiency on nuScenes and Argoverse 2. The method is also generalizable to object detection and downstream trajectory prediction.

Critical strengths

(+) Novel design with strong accuracy and efficiency. Most works in the mapping community follow MapTR-style design using BEV as an inefficient representation, whereas MapLightning is the first work to replace dense BEV with compact 1d tokens, achieving state of the art accuracy and efficiency, and avoid camera-extrinsic perturbations which is common for real-world autonomous driving.

(+) Comprehensive experimental evaluation. The paper provides extensive comparisons across two datasets, three image backbones, different perception ranges and dataset splits, together with additional experiments on uncertainty-aware mapping, trajectory prediction, and 3D object detection.

(+) The paper is well motivated, clearly organized, and easy to follow.

Critical weaknesses

(-) Adaptation to end-to-end frameworks is not fully discussed. The authors mention in the limitations section that the compact 1D token design may require additional adaptation for end-to-end frameworks such as MapFormer in UniAD, but they do not explain what this adaptation would look like. Since autonomous driving is moving toward unified end-to-end architectures rather than modular pipelines, addressing this compatibility issue would make the proposed representation more broadly useful.

Questions for authors
  • Why does the proposed method outperform other baselines by a much larger margin when using smaller real-time backbones such as MobileNetV3-Large or ResNet-18?

  • How are the 1D map tokens initialized? Are they simply randomly initialized learnable embeddings, and are the same token embeddings shared across all scenes?

1 comment
AuthorsOpenReview ↗

Response to Reviewer 4DLy

Thank you for the careful and comprehensive review! Our response is:

W1: Adaptation to end-to-end frameworks is not fully discussed. (e.g., MapFormer in UniAD)

We thank the reviewer for this suggestion and agree that integrating MapLighting into an end-to-end architecture would expand its utility. However, we deliberately scoped the paper to mapping: it lets us isolate the effect of the intermediate representation under a controlled comparison, which is not possible when the change is entangled with several jointly trained downstream modules. While it is beyond the scope of the current paper, we sketch out a potential adaptation below and leave its full implementation and analysis to future work. We will include this discussion in the camera-ready.

End-to-end frameworks such as UniAD connect TrackFormer, MapFormer, MotionFormer, OccFormer, and the Planner downstream of BEV features BB. The key integration step is to treat the multi-view tokens X = SelfAttention(I), where I are image tokens, as a substitute for BEV features B. Because all downstream modules share one compact token set, this might lower the cost of the shared representation in end-to-end pipelines. MapFormer can be then replaced by MapLightning directly: map tokens M are injected into XX via further self-attention blocks in the mapping branch, and the decoder uses full cross-attention on M to predict vectorized map elements, whose queries QM{Q}_{M} are passed to MotionFormer.

This change from BEV features BB to 1D multi-view tokens XX also affects other modules. Similar architectural changes can be made to TrackFormer, MotionFormer, and the Planner. Further research needs to be done to see the benefit of 1D multi-view tokens for OccFormer.

Q1: Why does the proposed method outperform other baselines by a much larger margin when using smaller real-time backbones such as MobileNetV3-Large or ResNet-18?

We hypothesize that lightweight backbones produce weaker image features, and that dense BEV projection (e.g., MapTRv2) may be affected more because it lifts image features in a single pass without refining them. In contrast, our mapper jointly refines image and map tokens through multiple self-attention layers (illustrated in Figure 3), potentially compensating for weaker features.

Q2: How are the 1D map tokens initialized? Are they simply randomly initialized learnable embeddings, and are the same token embeddings shared across all scenes?

The map tokens are randomly initialized learnable embeddings, shared across all scenes.

Reviewer_AK5nOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes MapLightning for online vectorized HD map construction. Instead of lifting multi-view image features into a dense BEV grid, the method concatenates image tokens with learnable map tokens and jointly updates them through a self-attention mapper. The image tokens are then discarded, and the compact map tokens serve as the intermediate representation for a MapTRv2-style decoder with full cross-attention. Learned positional, camera, and temporal embeddings replace explicit camera projection and ego-pose alignment.

The paper evaluates the approach on nuScenes and Argoverse 2, primarily using the Near-Extrapolation splits to reduce geographic overlap between training and validation. It reports improvements in map accuracy and inference efficiency, compares self-attention with a cross-attention variant, and includes ablations on token count, mapper depth, and temporal context. Additional experiments examine uncertainty-aware mapping, downstream trajectory prediction, 3D object detection, and sensitivity to perturbations of camera calibration parameters.

Critical strengths
  1. The paper provides convincing empirical evidence that a compact, non-BEV intermediate representation can support accurate vectorized mapping.

The contribution is supported by more than the default multi-frame comparison. In the single-frame setting of Table 1, the self-attention mapper improves over dense BEV projection from 26.7 to 30.1 mAP on nuScenes and from 53.1 to 58.3 mAP on Argoverse 2 with ResNet-50. It also outperforms the cross-attention variant across all three backbones. These controls show that the gains cannot be explained solely by using additional temporal context. The default configuration further achieves 35.3 and 63.3 mAP on the two datasets. This is a useful architectural result for efficient autonomous-driving perception.

  1. The evaluation is broad and addresses a meaningful generalization issue.

Using the Near-Extrapolation splits as the primary benchmark reduces the influence of geographic memorization, and the paper retrains several available baselines on these splits. Results on additional splits, lightweight backbones, and downstream trajectory prediction provide complementary evidence of practical utility. The reported MobileNetV3-Large configuration reaches 40.2 FPS while improving mapping accuracy, and Appendix B.4 provides a component-wise efficiency breakdown. Together, these results make the proposed representation relevant beyond a single accuracy comparison, subject to the measurement clarifications below.

Critical weaknesses
  1. Important details needed to interpret and reproduce the efficiency claims remain unspecified.

Appendix B.4 provides a useful component-wise breakdown, but the retained map-token count does not describe the full sequence processed by the mapper. Section 3.3 applies full self-attention to the concatenation of image and map tokens, so its computational cost depends on the image-token count as well as the map-token count. FlashAttention reduces attention-memory requirements but does not remove the quadratic attention computation.

The manuscript does not clearly specify the feature-map resolution or stride, the resulting image-token count, and the mapper width and attention-head configuration. The reported timing and memory measurements also need an explicit inference batch size, numerical precision, synchronization procedure, and treatment of cached temporal features. In particular, it is unclear whether the memory totals in Table 10 represent peak end-to-end allocated memory or a sum of component measurements.

These omissions matter because the accuracy-efficiency trade-off is a central contribution. I am not disputing the reported measurements, but the current description is insufficient to independently interpret and reproduce them. Providing the exact configuration and profiling protocol would substantially address this concern.

  1. The camera-extrinsic robustness result supports a narrower claim than robustness to changes in camera geometry.

Appendix E and Figure 8 perturb the numerical camera extrinsics. Since MapLightning does not consume these parameters, unchanged predictions are expected when the input images remain fixed. This establishes independence from erroneous calibration metadata, which is useful, but it does not establish robustness to a physical camera displacement, a changed viewing direction, or transfer to a different camera rig.

The distinction is consequential because the model uses learned camera and image-position embeddings, which may encode properties of the training rig even without explicit projection matrices. The paper should clearly limit the robustness claim to the tested calibration-parameter perturbations, unless evidence involving actual image or viewpoint changes is available. This concern limits the deployment interpretation rather than invalidating the reported mapping accuracy.

Questions for authors
  1. Clarification of the mapper configuration. What are the feature-map resolution or stride, total image-token count, embedding dimension, number of attention heads, and FFN width used for the default nuScenes and Argoverse 2 experiments? Please state the full joint sequence length processed by self-attention, rather than only the number of retained map tokens. These details would help interpret the reported mapper runtime.

  2. Clarification of the efficiency measurements. For Table 10, what inference batch size, numerical precision, warm-up and synchronization procedure, and attention implementation were used? Does the reported memory include model parameters and cached features from previous frames, and is it measured as peak end-to-end allocation or assembled from component measurements? Please also clarify which settings were shared with the baseline implementations. This is a request for the existing profiling protocol, not a new benchmark.

  3. Clarification of the robustness claim. In Appendix E, are only the calibration matrices perturbed while the images remain unchanged? If so, would the authors explicitly distinguish robustness to incorrect calibration metadata from robustness to physical camera or viewpoint changes in the abstract and conclusions? If any existing experiment addresses the latter, please identify it. I am not requesting a new camera-rig evaluation during rebuttal.

1 comment
AuthorsOpenReview ↗

Response to Reviewer AK5n

Thank you for the careful and comprehensive review! Our response is:

W1: Important details needed to interpret and reproduce the efficiency claims remain unspecified.

  • W1.1: the retained map-token count does not describe the full sequence processed by the mapper; its computational cost depends on the image-token count as well as the map-token count.
    • Both methods use the same image encoder to extract image features. The difference lies in the intermediate representation stage: MapTRv2 uses dense BEV projection, while our mapper applies self-attention to both image and map tokens. As shown in Table 10, our mapper is faster (1.7 vs. 11.6 ms) than BEV projection, despite including the computation for both image and tokens in self-attention.
  • W1.2: The manuscript does not clearly specify the feature-map resolution or stride, the resulting image-token count, full-joint sequence length and the mapper width and attention-head configuration.
    • The full mapper configuration for nuScenes and Argoverse 2 is provided in the table below.
SettingnuScenesArgoverse 2
Stride3232
Feature-map resolution (H×W)15×2520×20
Image tokens per image375400
Image tokens N (cameras × 4 frames)9,00011,200
Map tokens1,2001,400
Joint sequence length10,20012,600
Mapper width256256
Heads8 (32 dims per head)8 (32 dims per head)
FFN width10241024
  • W1.3:The reported timing and memory measurements also need an explicit inference batch size, numerical precision, synchronization procedure, and treatment of cached temporal features.
    • For both ours and MapTRv2, we use the same GPU and input resolution, PyTorch attention, a batch size of 1, and FP16 precision by default. We run 10 warmup iterations, call torch.cuda.synchronize() before and after the timed loop, and report the mean latency over 50 iterations. Past-frame image features are cached (Section 3.4), so each timed iteration runs the backbone on the current frame only.
  • W1.4: whether the memory totals in Table 10 represent peak end-to-end allocated memory or a sum of component measurements.
    • The memory totals in Table 10 are the sum of each component’s peak allocated memory (backbone, mapper, decoder), measured in isolation, not the end-to-end peak. End-to-end peak memory is largely dominated by the common image backbone, and is thus similar across methods. We have clarified this distinction in the revised abstract and introduction.

W2: The camera-extrinsic robustness result supports a narrower claim than robustness to changes in camera geometry. (Appendix E and Figure 8)

  • Thank you, we agree that Appendix E and Figure 8 demonstrate independence from calibration metadata (because they are not input to our method), rather than learned invariance to physical camera or viewpoint changes. In our experiments, we train and test separate models on nuScenes and Argoverse 2, each with a fixed camera rig. We do not claim transfer to unseen rigs. We have revised the paper accordingly.

Q1: Clarification of the mapper configuration.

  • Please see W1.1 and W1.2

Q2: Clarification of the efficiency measurements (Table 10).

  • Memory: The reported memory includes model parameters but not the cached past-frame image features, which take about 3 MiB (3 frames × 1.1 MiB), under 1% of the reported memory. Please also see W1.4.
  • Protocol: Please see W1.3

Q3: Clarification of the robustness claim.

  • Q3.1: In Appendix E, are only the calibration matrices perturbed while the images remain unchanged?
    • Yes, only the extrinsic parameters are perturbed; images are unchanged. Our method does not use extrinsics as input hence is not influenced by the errors in calibration.
  • Q3.2: explicitly distinguish robustness to incorrect calibration metadata from robustness to physical camera or viewpoint changes in the abstract and conclusions.
    • Please see W2
Reviewer_aq4ZOpenReview ↗
Rating
3
Confidence
4
Informativeness
3
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes MapLightning for online vectorized HD map construction from surround-view images. Instead of lifting image features into a dense BEV grid, the mapper flattens multi-view and multi-timestep image tokens, concatenates them with a small set of learnable map tokens, refines both jointly through four self-attention layers, discards the image tokens, and feeds the map tokens to a MapTRv2-style decoder with full instead of deformable cross-attention. The mapper reads no camera intrinsics, extrinsics, depth, or ego-pose; camera identity, time, and within-image position enter through learned embeddings. Experiments run on nuScenes and Argoverse 2 under the leakage-controlled Near-Extrapolation split against retrained public baselines. The self-attention mapper reaches 35.3 mAP on nuScenes and 63.3 mAP on Argoverse 2 at 18.9 FPS, outperforming MapTRv2 and prior methods while using about 1.2k map tokens against roughly 20k BEV cells. The paper additionally reports uncertainty-aware maps, downstream trajectory prediction, 3D detection, extrinsic robustness, runtime and memory breakdowns, and token-count, depth, and timestep ablations. The public-benchmark evidence and controlled aggregation comparison are solid, but the conceptual contribution assembles known token-pooling and DETR components, the absence of camera and pose geometry is not fully analyzed, and the detection, trajectory, and robustness claims rest on under-matched protocols

Critical strengths

1.The sparse-output motivation is well grounded. Map elements occupy only 4.27% of the nuScenes raster range and 2.65% on Argoverse 2, so a dense BEV grid allocates most capacity to empty space; a fixed budget of learned map tokens matches the task sparsity more directly.

  1. compare cross-attention and self-attention for constructing map tokens from image features, and show that self-attention is more effective for online vectorized HD map construction.

3.The empirical gains are consistent across datasets, splits, and backbones. On the Near-Extrapolation split, self-attention reaches 35.3 mAP on nuScenes and 63.3 mAP on Argoverse 2, ahead of the strongest retrained baseline by about 7.8 mAP on nuScenes and of MapTRv2 by about 10.2 mAP on Argoverse 2, while using far fewer tokens and running at 18.9 FPS versus MapTRv2's 14.1 FPS. Gains persist on the original and StreamMapNet splits and with MobileNetV3-Large and ResNet-18 backbones.

Critical weaknesses
  1. The extrinsic robustness claim is mechanically weak. Figure 8 shows a flat curve because the model never reads extrinsics, so perturbing a parameter it does not consume leaves its input unchanged; this is not evidence of learned invariance. A meaningful test would perturb calibration or camera rig at inference, vary intrinsics, or compare methods under matched training. The robustness wording in the abstract should be qualified.

2.The reported results are encouraging, but the experimental analysis and baseline comparison are not clear enough to substantiate the claimed accuracy gains and geometric reliability. Without intrinsics, extrinsics, depth, or ego-pose alignment, the image-to-map mapping relies on learned camera, time, row, and column embeddings, yet the paper reports no AP-by-distance breakdown, no embedding-removal ablation, no near-versus-far error analysis, no class-imbalance or error taxonomy for spurious and missing elements, and no ablation over the 50 instance queries plus the 300 one-to-many queries; the fixed budget of 50 tokens per camera per frame appears only as a tuning choice in the limitations. On the baseline side, per-method epochs, temporal frames, input resolution, augmentation, and query budgets are not documented, and it is unclear whether the MapTRv2 baseline in Table 2 uses a single frame or the same four-frame history as the proposed model. These gaps leave it unclear how much of the gain comes from the mapper design, matched training conditions, or query budget, and how the method behaves at far range, under class imbalance, or on unseen camera rigs.

Questions for authors

1.How are map coordinates recovered without intrinsics, extrinsics, depth, or ego-pose? Please report AP by distance bin, ablate the camera, time, row, and column embeddings one at a time, and test unseen camera rigs or viewpoint changes.

2.How are past-frame features registered to the current ego frame without pose alignment? Please add a pose-information ablation and a cross-frame continuity or junction-consistency metric

3.The flat extrinsic curve follows from not reading extrinsics. Please test calibration noise, intrinsic changes, and unseen rig configurations at inference, match training across compared methods, and clarify what robustness means in this setup.

4 comments
AuthorsOpenReview ↗

Response to Reviewer aq4Z (1)

Thank you for the careful and comprehensive review! Our response is:

W1: Flat extrinsic curve is not evidence of learned invariance; perturb calibration or camera rig at inference, vary intrinsics, or compare methods under matched training; qualify the abstract wording

  • Thank you, we agree. Figure 8 perturbs the extrinsic parameters while the images remain unchanged. Since MapLightning does not consume intrinsic or extrinsic parameters, its predictions remain unchanged. We have revised the abstract, introduction, and Appendix E, replacing ‘robust to extrinsic perturbation’ with ‘does not use calibration metadata as input and is hence not influenced by calibration errors’ to clarify the scope of our claim, and added this scope statement to the limitations.

W2: Additional analyses and baseline details

  • W2.1(a): AP-by-distance breakdown and near-versus-far error analysis
    • We report per-class AP and mean AP (mAP) across three longitudinal distance bins (0–10, 10–20, and 20–30 m, combining front and back), assigning each ground-truth map element to a bin based on its centroid.
    • As shown in the table below, our self-attention model consistently outperforms MapTRv2 across all distance bins on both datasets, with particularly large improvements at far range (20–30 m): +11.9 mAP on nuScenes and +21.1 mAP on Argoverse 2, suggesting better long-range context modeling.

nuScenes

RangeMethodAP_divAP_pedAP_boundmAP
0-10 mMapTRv221.336.642.233.3
0-10 mOurs (cross-attn)16.052.740.236.3
0-10 mOurs (self-attn)29.955.947.144.3
10-20 mMapTRv222.818.323.521.5
10-20 mOurs (cross-attn)23.627.424.025.0
10-20 mOurs (self-attn)36.041.131.436.2
20-30 mMapTRv210.29.06.78.6
20-30 mOurs (cross-attn)11.018.412.013.8
20-30 mOurs (self-attn)17.729.314.520.5

Argoverse 2

RangeMethodAP_divAP_pedAP_boundmAP
0-10 mMapTRv249.461.862.657.9
0-10 mOurs (cross-attn)67.569.762.266.4
0-10 mOurs (self-attn)73.879.867.073.5
10-20 mMapTRv240.748.055.248.0
10-20 mOurs (cross-attn)66.756.155.659.5
10-20 mOurs (self-attn)71.065.661.966.2
20-30 mMapTRv232.728.323.328.1
20-30 mOurs (cross-attn)50.941.334.442.2
20-30 mOurs (self-attn)60.251.236.249.2
  • W2.1(b): embedding-removal ablation
    • Removing any one of the four embeddings (camera, time, row, or column) causes training to fail to converge: the loss does not decrease, and mAP stays close to zero. We will add this observation to the camera-ready.
AuthorsOpenReview ↗

Response to Reviewer aq4Z (2)

  • W2.1(c): class-imbalance or error taxonomy for spurious and missing elements
    • The ground-truth counts are imbalanced: pedestrian crossings are the rarest class (14.3% of nuScenes and 22.0% of Argoverse 2 instances).
    • We report class-wise TP, FP, and FN counts using a confidence threshold of 0.4 and a Chamfer distance threshold of 1.0 m.
    • As shown in the table below, our self-attention model achieves more TPs and fewer FPs and FNs overall on both datasets compared with MapTRv2. The gains are largest on pedestrian crossings, the rarest class. On nuScenes, spurious predictions drop from 57.5% to 34.0%, and missing elements drop from 74.7% to 46.5%. On Argoverse 2, spurious predictions drop from 34.0% to 15.0%, and missing elements drop from 45.4% to 26.9%.

nuScenes

MetricMethoddivpedboundTotal
GT Count (% of Total)18,439 (47.7%)5,548 (14.3%)14,707 (38.0%)38,694 (100%)
TP (% of GT) ↑MapTRv223.8%25.3%39.4%29.9%
TP (% of GT) ↑Ours (cross-attn)26.2%44.1%40.2%34.1%
TP (% of GT) ↑Ours (self-attn)33.0%53.5%48.5%41.8%
Missing, FN (% of GT) ↓MapTRv276.2%74.7%60.6%70.1%
Missing, FN (% of GT) ↓Ours (cross-attn)73.8%55.9%59.8%65.9%
Missing, FN (% of GT) ↓Ours (self-attn)67.0%46.5%51.5%58.2%
Spurious, FP (% of preds) ↓MapTRv257.0%57.5%50.5%54.0%
Spurious, FP (% of preds) ↓Ours (cross-attn)66.2%37.0%54.3%57.6%
Spurious, FP (% of preds) ↓Ours (self-attn)49.3%34.0%44.9%45.1%

Argoverse 2

MetricMethoddivpedboundTotal
GT Count (% of Total)25,183 (43.7%)12,667 (22.0%)19,720 (34.3%)57,570 (100%)
TP (% of GT) ↑MapTRv240.6%54.6%58.5%49.8%
TP (% of GT) ↑Ours (cross-attn)64.9%66.4%61.9%64.2%
TP (% of GT) ↑Ours (self-attn)70.2%73.1%66.2%69.5%
Missing, FN (% of GT) ↓MapTRv259.4%45.4%41.5%50.2%
Missing, FN (% of GT) ↓Ours (cross-attn)35.1%33.6%38.1%35.8%
Missing, FN (% of GT) ↓Ours (self-attn)29.8%26.9%33.8%30.5%
Spurious, FP (% of preds) ↓MapTRv226.0%34.0%33.0%30.9%
Spurious, FP (% of preds) ↓Ours (cross-attn)28.6%24.1%29.7%28.0%
Spurious, FP (% of preds) ↓Ours (self-attn)22.8%15.0%25.3%22.0%
  • W2.1(d): ablation over the 50 instance queries plus the 300 one-to-many queries
    • We retain the same decoder query configuration as MapTRv2 (e.g., 50 instance queries and 300 one-to-many queries, 350 in total during training) for a fair comparison. Our changes focus on the intermediate representation and its associated attention mechanism, rather than on the decoder query configuration.
  • W2.2: the fixed budget of 50 tokens per camera per frame appears only as a tuning choice in the limitations.
    • Thank you, we agree. We provide a token-budget ablation in Table 11: 0.9k, 1.2k, and 1.5k total tokens achieve 33.3, 35.3, and 34.9 mAP, respectively. Our default of 50 tokens per camera per timestep (1.2k total) achieves the highest mAP among these settings. However, we acknowledge that a deeper analysis of the token budget is needed. To this end, we also discuss adaptive token allocation as future work in the Limitations section.
AuthorsOpenReview ↗

Response to Reviewer aq4Z (3)

  • W2.3: On the baseline side, per-method epochs, temporal frames, input resolution, augmentation, and query budgets are not documented.
    • Ours mainly follows MapTRv2’s settings (we have added minor clarifications in Appendix F.1), and all baselines with public code are retrained using their official code on both datasets. We have added a table below listing each method’s training epochs, temporal frames, input resolution, augmentation, and query budgets. The asterisk (*) indicates that MapTracker additionally uses propagated track queries, one per tracked instance, whose number varies by scene.
MethodNS epochsAV2 epochsTemporal framesNS input (H×W)AV2 input (H×W)AugmentationInstance queries
MapTR24241450×800, pad 480×800614×614, pad 640×640Photometric distortion, GridMask50
StreamMapNet2430Streaming480×800608×608Photometric distortion100
MapTracker58355 (1 current + 4 history)480×800608×608Photometric distortion100*
MapTRv22461450×800, pad 480×800614×614, pad 640×640Photometric distortion350
DAMap2461450×800, pad 480×800614×614, pad 640×640Photometric distortion350
AugMapNet2430Streaming480×800608×608Photometric distortion, BEV aug.100
Ours2464 (1 current + 3 history)450×800, pad 480×800614×614, pad 640×640Photometric distortion350
  • W2.4: whether the MapTRv2 baseline in Table 2 uses a single frame or the same four-frame history as the proposed model.
    • MapTRv2 uses a single frame in Table 2 and throughout the paper, as it has no temporal modeling mechanism. We provide a matched (apples-to-apples) single-frame comparison in Table 1, where ours achieves higher accuracy and speed than BEV projection (MapTRv2’s design). In Table 2, our four-frame model also outperforms temporal baselines, including StreamMapNet and MapTracker.
AuthorsOpenReview ↗

Response to Reviewer aq4Z (Questions)

Q1

  • Q1.1: Explain how map coordinates recovered without intrinsics, extrinsics, depth, or ego-pose.

Thank you for this query. Our statement on L210-215 may have caused a miscommunication prompting this question. Our crucial insight is that the network itself does not need these values as inputs. The network's predictions are always relative to the ego-vehicle in normalized coordinates (x^,y^∈[0,1]\hat{x},\hat{y}\in [0,1]), vehicle at the center looking forward) at any timestep. Then the predictions are scaled to metric scale using the known perception range (e.g., 60x30 m). Our prediction space follows the same protocol as MapTR/MapTRv2: we assume a metric, ego-vehicle-centred birds-eye-view frame.

The key insight is that the ground truth, which is the map expressed in the ego-vehicle frame, already provides the regression target for the network and is consistent-by-construction with the multi-view images throughout the dataset. Camera Intrinsics, extrinsics, depth, or ego-pose become necessary inputs to the network only when it is enforced by an explicit intermediate representation such as a BEV grid by other approaches like MapTRv2.

The predictions can further be placed in a global coordinate frame using the current ego-pose through a rigid transform. For example, in our sequence-level reconstruction visualization, we follow MapTracker (MapTracker’s Appendix C, Algorithm 3) to accumulate per-frame map predictions over a whole scene as a post-processing step, using ground-truth ego poses to align predictions and per-instance track IDs to group and merge polylines across frames. For methods without tracking, such as ours and MapTRv2, these track IDs are obtained using MapTracker’s post-hoc track extraction algorithm (MapTracker’s Appendix B.2). Ego poses are used in this visualization, but are not part of the network. We have revised L210-215 to make this distinction explicit and added these additional details in the manuscript.

  • Q1.2: report AP by distance bin, ablate the camera, time, row, and column embeddings one at a time
    • Please see W2.1
  • Q1.3: test unseen camera rigs or viewpoint changes
    • In our experimental setting, each dataset like nuScenes or Argoverse 2 has one fixed camera rig, and we train and test on that configuration, so the learned camera/row/column embeddings implicitly capture rig-specific geometry. We do not claim transfer to unseen rigs.

Q2

  • Q2.1: How are past-frame features registered to the current ego frame without pose alignment?
    • Past-frame features are not explicitly registered to the current ego frame (that is, we do not warp them using ego-pose). Instead, the mapper (Section 3.3) uses self-attention to fuse current and cached past-frame image features (Section 3.4), distinguished by learned time embeddings (Section 3.2), and learns temporal relationships implicitly from the ground-truth map supervision.
  • Q2.2: add a pose-information ablation
    • Our model deliberately avoids ego-pose inputs (Section 3.2). Evaluating the benefit of pose information would require modifying the architecture and retraining, rather than ablating an existing component, so we leave it to future work.
  • Q2.3: a cross-frame continuity or junction-consistency metric
    • Unlike standard mAP, which evaluates each frame independently, C-mAP from MapTracker (MapTracker’s Section 4.2 and Appendix B.3) also penalizes predictions whose track IDs change for the same ground-truth map element across frames. We reuse the post-hoc extracted track IDs (Please see Q1.1) to compute C-mAP.
    • As shown in the table below, our self-attention model achieves the highest C-mAP on both datasets (27.2 vs. MapTracker’s 19.2 on nuScenes, and 54.3 vs. 43.1 on Argoverse 2), indicating more temporally consistent predictions.

nuScenes

MethoddivpedboundC-mAP
MapTRv28.314.822.815.3
MapTracker12.122.922.619.2
Ours (cross-attn)9.623.417.716.9
Ours (self-attn)18.034.429.327.2

Argoverse 2

MethoddivpedboundC-mAP
MapTRv229.037.039.035.0
MapTracker40.345.743.243.1
Ours (cross-attn)44.447.245.045.5
Ours (self-attn)54.957.150.854.3

Q3

  • Q3.1: test calibration noise, intrinsic changes, and clarify robustness
    • Please see W1
  • Q3.2: unseen rig configurations at inference
    • Please see Q1.3
  • Q3.3: match training across compared methods
    • Please see W2.3

Comments on the paper

2 comments
AuthorsOpenReview ↗

General Response

Dear AC and reviewers,

We sincerely thank everyone for their careful and constructive reviews. All three reviewers recommend acceptance, and we are grateful for their assessment of the work and happy to note that some reviewers felt that they gained a substantially deeper understanding of the problem.

MapLightning is the “first work to replace dense BEV with compact 1D tokens, achieving state of the art accuracy and efficiency,” (4DLy) with a “comprehensive experimental evaluation” (4DLy) providing “convincing empirical evidence that a compact, non-BEV intermediate representation can support accurate vectorized mapping” (AK5n). We are happy that our “sparse-output motivation is well grounded” (aq4Z) and that “the empirical gains are consistent across datasets, splits, and backbones”. (aq4Z) We hope that our “useful architectural result for efficient autonomous-driving perception” (aq4Z) would drive progress forward in this direction.

The reviewers raised four main concerns, which we addressed as follows.

1. Scope of the robustness claim (aq4Z, AK5n). Both reviewers pointed out that our extrinsic-robustness claim should be rewritten. We thank them for pointing this out and agree with their assessment, and we have revised the abstract, introduction, Appendix E, and limitations to state that the model does not take calibration as input and is therefore unaffected by calibration errors. We also explicitly wish to state that we do not claim transfer to unseen camera rigs and leave that as future work.

2. How the model behaves without explicit geometry (aq4Z). We clarified that predictions are regressed directly in the ego frame in normalized coordinates, supervised by the ground truth map that is consistent with the input images, and that past frames are fused implicitly through learned time embeddings. The predictions can further be placed in a global coordinate frame using the current ego-pose through a rigid transform. We backed this with the requested analyses: accuracy by distance (the advantage holds at every range and is largest at far range), an error breakdown (fewer spurious and fewer missing elements, with the largest gains on the rarest class), a temporal-consistency metric (highest among compared methods on both datasets), and showing all four learned embeddings are necessary.

3. Fairness and documentation of baselines (aq4Z). We clarified that MapTRv2 is a single-frame method and added a table of per-method training settings. The matched single-frame comparison in Table 1 isolates the gain from the representation itself, and our multi-frame model also outperforms temporal baselines in our system-level comparison tables. The decoder query configuration is kept identical to MapTRv2.

4. Reproducibility of the efficiency claims (AK5n). We provided the full mapper configuration, including the joint image-plus-map sequence length, the complete profiling protocol, and a clarification of how memory is measured.

Further points. Following Reviewer 4DLy recommendation to discuss extension to end-to-end architectures, we have sketched one we could replace BEV features in an end-to-end framework such as UniAD, and answered the questions on lightweight backbones and token initialization. Two requests from Reviewer aq4Z, a pose-input ablation and unseen-rig evaluation, require architectural changes or new data settings, we acknowledge both as limitations and future work.

AuthorsOpenReview ↗

Closing Note: Summary of contribution

As the discussion period closes, we would like to briefly restate the core contribution, since it is easy to lose among the individual responses.

MapLightning proposes a change in the intermediate representation itself, rather than a new module within the existing pipeline. Since MapTR, most online mapping methods have relied on a dense, calibration-driven BEV grid and improved the components built on top of it. We show instead that a compact set of learned 1D map tokens, constructed without intrinsics, extrinsics, or ego-pose as inputs to the model itself, is more accurate, faster, and uses less component-wise memory. We isolate this critical change in representation, and the gain holds across backbones, datasets, and splits.

We remain available for the rest of the discussion period (until Nov 18 AoE) and would be glad to answer any further questions or clarify anything that remains open.

We thank the AC and reviewers again for their time and engagement.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.