acceptodds
Under review as a conference paper at ICLR 2027

MirageDrive: Semantically Aligned Appearance Transfer with an Exemplar for Multi-View Driving Scene Generation

Abstract

Controllable driving-scene generation provides scalable annotated multi-view data for autonomous driving, but its appearance diversity is often constrained by the training distribution. Expanding this diversity requires incorporating new appearance cues while preserving scene geometry and annotation alignment. We introduce MirageDrive, an exemplar-guided framework that transfers the appearance of a single reference image to multi-view driving scenes under geometric and semantic conditions. Semantic Region Control supplies dense target-layout guidance, while Exemplar-Guided Appearance Transfer combines class-wise AdaIN pseudo-target supervision with Region-to-Region Attention. The pseudo targets introduce exemplar-derived appearance statistics while retaining the target layout, providing spatially aligned supervision for jointly learning structural control and appearance transfer. Region-to-Region Attention directs reference features to matching semantic regions, coordinating appearance guidance with the target structure. Once trained, MirageDrive accepts new exemplars without additional model adaptation. Experiments on nuScenes demonstrate that MirageDrive achieves semantically aligned appearance transfer, improves controllability, and benefits downstream vision tasks through its generated data.

Primary area:
generative models
Keywords:
Driving scene generation, Semantic apperance transfer, Autonomous driving

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_nJeqOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

MirageDrive transfers appearance from a single exemplar to six-view driving scenes using semantic-map conditioning, class-restricted attention, and class-wise AdaIN pseudo-targets. Built on MagicDrive-V2, it improves generation controllability and downstream nuScenes detection without retraining for each exemplar.

Critical strengths
  1. Semantic correspondence provides a coherent mechanism for coordinating appearance transfer with layout preservation.
  2. Downstream StreamPETR improvements demonstrate utility beyond perceptual generation metrics.
  3. The method supports unseen exemplars without additional adaptation.
Critical weaknesses
  1. Target semantic maps are extracted from real camera images, making the evaluation more strongly conditioned than generation from geometry alone.
  2. RGB mean-and-variance pseudo-targets mainly supervise coloration and cannot substantiate broader claims about transferring geographic appearance or complex textures.
  3. Comparisons lack strong exemplar-conditioned alternatives given equivalent semantic inputs, leaving the advantage over existing appearance-transfer approaches unclear.
  4. Cumulative ablations do not independently isolate all components or their interactions, particularly pseudo-target supervision within the complete architecture.
  5. Downstream testing remains on nuScenes, so improved generalization to genuinely unseen cities or datasets is not demonstrated.
Questions for authors
Reviewer_qs24OpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
3
Read review
Paper summary

MirageDrive introduces an exemplar-guided framework for multi-view driving scene generation that aims to diversify scene appearance while preserving geometry and annotation alignment. The method introduces Semantic Region Control to incorporate dense semantic guidance into the generation process. It further employs Region-to-Region Attention to transfer appearance features between semantically corresponding regions of the target and exemplar images. Additionally, class-wise AdaIN is used to construct pseudo-target images that combine the target scene structure with exemplar-derived appearance statistics for training. Experiments on nuScenes evaluate generation quality, geometric controllability, appearance alignment, and downstream 3D object detection performance. The results demonstrate improvements over the MagicDrive-V2 baseline across several evaluation metrics.

Critical strengths
  1. Clear and Practically Relevant Motivation: The paper addresses a relevant problem in driving scene generation: expanding appearance diversity while preserving scene geometry and annotation alignment. This is a meaningful research direction for improving the utility of synthetic driving data.

  2. Reasonable Integration of Complementary Components: The proposed framework combines Semantic Region Control, Region-to-Region Attention, and class-wise AdaIN pseudo-target supervision. These components have distinct and complementary roles in structural preservation and exemplar-guided appearance transfer, resulting in a coherent overall design.

  3. Improved Generation Quality and Downstream Performance: The proposed method demonstrates improvements in generation quality and geometric controllability. It also achieves better downstream 3D object detection performance using generated data. The paper provides quantitative comparisons, qualitative results, human evaluations, and ablation studies to support these findings.

Critical weaknesses
  1. Insufficient Validation of the Central Motivation: The paper aims to enhance appearance diversity through exemplar-guided generation, but does not provide sufficient quantitative evaluation of appearance diversity. While improvements in generation quality and controllability are reported, it remains unclear whether these gains arise from enhanced appearance diversity or additional target semantic conditioning. Table 2 reports improved detection performance with AdaIN pseudo-target supervision, but it is unclear whether the gains stem from exemplar-derived supervision or additional fine-tuning from the pretrained MagicDrive-V2 weights. Although style distance and human evaluation are provided, the former relies on whole-image VGG feature statistics, as acknowledged by the authors, while the latter lacks comparisons with baseline methods. Furthermore, the qualitative results mainly demonstrate changes in color tones and illumination, with limited evidence of distinctive exemplar-specific appearance transfer. Overall, the central claim of enhanced appearance diversity remains insufficiently supported.

  2. Limited Discussion of Related Work: The paper does not adequately discuss closely related studies on semantic-guided style transfer, such as SEAN [1] and INADE [2]. Since these studies share key ideas of semantic-region-based appearance control, a clearer comparison is needed to distinguish MirageDrive's contributions, particularly R2R Attention, from existing approaches.

  3. Limited Methodological Details: Several implementation details of the proposed components remain unclear. In SRC (Eq. 2), the paper does not sufficiently explain how BEV-map features and perspective-view semantic features are spatially aligned before being combined through element-wise addition. Similarly, in R2R Attention (Sec. 3.3), it is unclear how exemplar features are processed and updated across successive transformer blocks, and how target tokens outside the six predefined semantic classes are handled.

[1] Zhu, Peihao, et al. "Sean: Image synthesis with semantic region-adaptive normalization." CVPR 2020

[2] Tan, Zhentao, et al. "Diverse semantic image synthesis via probability distribution modeling." CVPR 2021

Questions for authors
  1. Although MirageDrive outperforms MagicDrive-V2 in the main evaluation, it is initialized from the trained MagicDrive-V2 model and further optimized for 18,750 iterations. This makes it difficult to determine whether the improvements arise from the proposed components or additional training. Could the authors provide a comparison where both methods are trained from the same initialization under matched training budgets?

  2. Table 2 only provides cumulative ablations, with AdaIN pseudo-target supervision retained in all variants. Additional ablations, such as SRC-only and R2R without AdaIN pseudo-target supervision, would help isolate the contributions of individual components.

  3. Given that appearance diversity is the central motivation, how does the generated appearance change when different exemplars are applied to identical target conditions? A controlled qualitative comparison would help demonstrate the effectiveness of exemplar-specific appearance control.

  4. To directly validate the claimed improvement in appearance diversity, additional metrics, such as pixel diversity used in SimGen [3] or pairwise LPIPS between outputs generated with different exemplars, would be helpful.

  5. While MagicDrive-V2 supports driving video generation, MirageDrive is evaluated only on single-frame multi-view generation. It remains unclear whether the proposed appearance transfer modules preserve temporal consistency. How would MirageDrive maintain consistent exemplar appearance and scene geometry across consecutive frames?

[3] Zhou, Yunsong, et al. "Simgen: Simulator-conditioned driving scene generation." NeurIPS 2024

Reviewer_xkEpOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes MirageDrive, an exemplar-guided framework for multi-view driving-scene generation built upon MagicDrive-V2. It combines semantic region control, region-to-region attention, and class-wise RGB AdaIN pseudo-target supervision to transfer appearance from a single reference image while preserving the target layout under geometric and semantic conditions. Once trained, the model accepts unseen exemplars without additional adaptation. Experiments on nuScenes evaluate generation quality, annotation controllability, human perceptual ratings, and downstream 3D detection performance.

Critical strengths

The paper addresses the useful problem of diversifying driving-scene appearance with a single exemplar, without adaptation for each new reference. Combining semantic layout guidance with class-matched reference attention provides an intuitive way to coordinate appearance changes with the prescribed scene structure.

The evaluation goes beyond visual examples and includes the utility of generated data for 3D detection. In Table 3, after pretraining on generated data and fine-tuning on real nuScenes images, MirageDrive yields 41.27 mAP and 51.46 NDS, compared with 39.51 mAP and 50.68 NDS for MagicDrive-V2-generated data. These reported gains of 1.76 mAP and 0.78 NDS support the potential practical value of the generated data.

Critical weaknesses

Table 2 provides useful ablation results for the proposed components. It would be helpful to clarify whether the MagicDrive-V2 baseline also underwent additional fine-tuning with the same training schedule. This detail would make the improvements associated with pseudo-target supervision easier to interpret.

The qualitative results and human evaluation provide some support for the claimed capability. However, the whole-image Gram distance does not directly measure appearance correspondence between matching semantic regions, as acknowledged in Appendix A, and the human evaluation in Section 4.4 does not include competing methods. Consequently, the current evidence is insufficient to establish MirageDrive’s advantage over alternative approaches in region-level appearance alignment. Region-level metrics or a comparative human evaluation under matched conditions would strengthen this claim.

Questions for authors

Was the matched MagicDrive-V2 baseline additionally fine-tuned for the same number of iterations and on the same target training samples? Please clarify which semantic inputs and training-budget controls were used in the matched evaluation.

Can the authors provide a comparison with a baseline using the same semantic control and a matched fine-tuning budget, or explain which existing comparison isolates this factor? Such evidence would materially affect the assessment of the proposed appearance-transfer components.

Can the authors provide appearance metrics for corresponding semantic regions, or a comparative human evaluation against relevant reference-guided alternatives under matched conditions? This would help establish whether the proposed mechanism improves regional alignment beyond the alternatives.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.