acceptodds
Under review as a conference paper at ICLR 2027

Mixture of Prototype Experts with Preference-Guided Routing for Single Domain Generalized Object Detection

Abstract

Single Domain Generalized Object Detection (SDGOD) trains a detector on a single labeled source domain for generalization to unseen domains. Existing approaches learn invariant features or broaden the source distribution, but limited source diversity prevents them from modeling complex combinations of unseen visual conditions. We propose Mixture of Prototype Experts (MoPE), a framework that factorizes visual conditions into atomic prototypes across photometric, textural, and spatial dimensions, each paired with a corresponding expert. To learn input-adaptive expert composition, we formulate routing as a preference learning problem over multiple coexisting visual attributes, enabling the router to assign higher selection probabilities to scene-relevant prototype experts than to nonrelevant ones. The selected expert outputs are then sparsely aggregated with a shared expert for multi-scale feature fusion. This preference learning is grounded in a Multi-Style Augmentation (MSA) strategy that broadens illumination, weather, and appearance variations without target-domain information while preserving object content and geometry, ensuring the router observes sufficient diversity for learning generalizable routing preferences. Extensive experiments demonstrate consistent generalization gains across unseen weather conditions, artistic styles, and different backbone architectures while maintaining a favorable accuracy-efficiency trade-off. Code and data will be publicly available.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
Object Detection, Domain Generalization, Mixture of Experts, Preference Optimization

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_SusMOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes Mixture of Prototype Experts (MoPE) for single-domain generalized object detection. The method organizes visual conditions into photometric, textural, and spatial prototypes, and employs multi-label preference optimization to guide expert routing based on multimodal annotations. Multi-style augmentation further diversifies the source distribution. Experiments across weather and artistic domain shifts demonstrate improvements over existing methods.

Critical strengths
  1. The figures are well designed and clearly illustrate the motivation, overall architecture, and key components, making the proposed framework easy to understand.

  2. The manuscript is clearly written and logically structured. The abstract and introduction effectively distinguish the proposed approach from existing methods, while the methodology provides coherent explanations and smooth transitions between components.

  3. The integration of semantic prototype preferences with sparse expert routing is an interesting idea. In particular, using multi-label preference optimization to guide expert selection provides a potentially useful alternative to conventional task-driven MoE routing.

Critical weaknesses
  1. The proposed Multi-Style Augmentation (MSA) relies on FLUX.2 to generate diverse weather and illumination conditions. However, visually convincing synthetic images do not necessarily follow the distributions of real-world environments. In my previous experiments, I observed that training an object detector on rainy images generated by pix2pix resulted in worse performance on real rainy images than training with daytime images alone. Although this observation involves a different generation model, it highlights a potential limitation of synthetic weather augmentation. The manuscript does not sufficiently investigate whether the generated distributions accurately capture the visual degradation encountered in real unseen domains. While Appendix G provides qualitative checks of annotation consistency, these do not establish the distributional realism or transferability of the generated samples.

  2. The central assumption of MoPE is that experts corresponding to the semantic attributes identified by the multimodal model should be preferred during routing. However, a semantically relevant expert is not necessarily the most effective expert for object detection. MPO primarily encourages the router to select experts consistent with the multimodal annotations, but does not explicitly ensure that these experts learn more discriminative or domain-robust features. In my understanding, the framework essentially performs input-dependent feature transformations through lightweight convolutional and attention-based experts, followed by weighted aggregation with a shared expert. It remains unclear whether these experts acquire genuinely complementary detection capabilities or whether the improvements mainly arise from the increased model capacity and diversified training data. The routing visualizations demonstrate semantic consistency but provide limited evidence of actual expert specialization.

  3. The framework manually defines 28 prototypes distributed across P3, P4, and P5, with 4 photometric, 8 textural, and 16 spatial prototypes, respectively. This design is primarily motivated by assumptions about the hierarchical characteristics of YOLO backbone features. However, the manuscript does not provide sufficient experimental evidence to justify the number, semantic definitions, or stage assignments of these prototypes. Although Appendix E investigates the number of activated experts, this is different from validating the predefined prototype taxonomy itself. Comparisons with alternative prototype configurations, different stage assignments, and expert selection strategies are needed to establish whether the proposed design is necessary and effective.

Questions for authors
  1. How do the authors assess the distribution gap between FLUX.2-generated images and real-world weather conditions? Could unrealistic synthetic distributions negatively affect generalization to real unseen domains?

  2. Why should semantically relevant experts necessarily be more effective for detection?

  3. Do different prototype experts learn genuinely specialized and complementary features? Can the authors provide quantitative feature-level analysis or controlled expert-swapping experiments beyond routing activation visualizations?

  4. What motivates the specific allocation of 4, 8, and 16 prototypes to P3, P4, and P5? How sensitive is performance to the prototype taxonomy, expert counts, and semantic-to-stage assignments?

8 comments
AuthorsOpenReview ↗

Response to SusM — W1 & Q1: Distribution Realism and Transferability of MSA

We thank the reviewer for this insightful comment and for sharing the observation regarding negative transfer from pix2pix-generated rainy images. We agree that visually plausible synthetic images do not necessarily match the distribution of real-world weather conditions, and that generative augmentation may sometimes introduce unrealistic artifacts.

(1) Distribution gap assessment and motivation of MSA.

Regarding distribution gap assessment, our current evaluation focuses on annotation consistency and downstream transferability rather than explicitly measuring statistical divergence between synthetic and real weather distributions.

However, we would like to emphasize an important distinction between distribution fidelity and downstream task effectiveness. The primary objective of MSA is not to precisely reproduce the probability distribution of real weather conditions, but to introduce diverse, label-preserving appearance variations that improve the detector's robustness to unseen domains. Exact distribution matching is not a prerequisite for augmentation to be beneficial.

Under the SDGOD setting, target-domain data and statistics are unavailable during training. Therefore, MSA aims to enrich source-domain visual diversity rather than explicitly approximate any particular target distribution. Its fixed prompt bank is constructed from representative photometric and textural prototypes, without using target-domain images, annotations, or statistics. These generated variations also provide diverse observations for prototype preference supervision.

Regarding annotation quality, Appendix G provides qualitative consistency checks between original and generated images, showing that object content, scene geometry, and bounding-box alignment are generally preserved in the inspected examples. We acknowledge that these checks assess annotation consistency rather than establishing an exact match with real-world weather distributions.

(2) Experimental evidence of transferability and potential negative transfer.

Our existing ablation results provide direct evidence of such benefits. As shown in Table 4, introducing MSA alone, without MPO, improves the average [email protected] across four real-world unseen weather domains from 29.00% to 40.30% (+11.30 percentage points). Importantly, improvements are observed consistently across all four domains, including Dusk-Rainy (+15.33 points) and Night-Rainy (+16.43 points). These results demonstrate that the generated variations contribute positively to detection performance on real unseen-domain images.

Moreover, several recent studies have independently demonstrated the effectiveness of diffusion-generated training data for object detection, including InstaGen (CVPR 2024), GeoDiffusion (ICLR 2024), and ODGEN (NeurIPS 2024). These findings further support the potential of modern generative approaches to improve downstream detection performance across different settings, without requiring generated images to perfectly reproduce real-world data distributions.

We further clarify that MSA is not designed or claimed to match target-domain distributions. Instead, it introduces diverse visual variations through a fixed prototype-based prompt bank while preserving object content and geometry, without using target-domain statistics. The reviewer's pix2pix experience highlights a legitimate risk of synthetic augmentation, but our experimental results indicate that MSA does not exhibit negative transfer in the evaluated domains.

(3) Limitations and clarification.

We acknowledge that these results do not establish an exact statistical match between synthetic and real weather distributions or guarantee improvements under every possible unseen condition. Nevertheless, the consistent improvements on real unseen-domain benchmarks provide direct empirical evidence of the task-level transferability of MSA, which is the primary objective of our augmentation strategy.

We sincerely appreciate the reviewer's constructive suggestion and will explicitly clarify the distinction between distributional realism, annotation consistency, and downstream transferability in the revised manuscript.

AuthorsOpenReview ↗

Response to SusM — W2, Q2 & Q3: Expert Specialization and Sources of Performance Improvement (Part 1/2)

We sincerely thank the reviewer for these thoughtful and important concerns. We agree that semantic relevance alone does not necessarily imply superior detection capability, and that routing visualizations alone are insufficient to establish meaningful expert specialization. We would like to clarify the underlying mechanism and provide additional experimental evidence addressing these concerns.

(1) Why should semantically relevant experts be more effective for detection?

Our method does not assume semantically relevant experts are inherently more effective for detection. Instead, their specialization is encouraged and progressively learned during training.

The predefined prototypes serve as semantic anchors rather than representing pre-existing expert capabilities. Initially, the experts are independently parameterized without predefined detection specializations. During training, MPO encourages the router to assign higher selection probabilities to experts whose prototypes match the visual attributes of the input image. Since MoPE employs sparse Top-K routing, the selected experts participate in feature computation and receive task-driven gradients from the detection loss.

Consequently, different experts are exposed to different visual conditions through preference-guided routing, encouraging them to gradually develop condition-specific feature representations. For example, the daytime expert is more frequently activated for daytime images and consequently receives more detection supervision from such scenes.

Importantly, MPO guides expert selection, while the detection objective drives the learning of discriminative features. Thus, semantic specialization is encouraged through the interaction between preference-guided routing and detection optimization, rather than being an assumption imposed by the prototype labels.

This mechanism is supported by Fig. 3, which demonstrates semantically differentiated routing across unseen domains. Furthermore, Table 4 reports an improvement from 29.00% to 35.97% without MSA. With MSA, the target-domain average [email protected] improves from 40.30% to 44.21% when MPO is introduced.

(2) Controlled expert-swapping experiments: Do experts learn genuinely specialized capabilities?

Following the reviewer's suggestion, we conducted additional controlled expert-swapping experiments to quantitatively investigate the functional differentiation of the learned experts beyond routing activation visualizations.

Specifically, we conducted controlled interventions on the trained MoPE models by altering only the routing-to-expert index correspondence during inference, keeping all learned model parameters unchanged, without any retraining or fine-tuning.

At P3, we swapped the illumination expert corresponding to each evaluation domain with the unknown expert. At P4, we consistently applied the following fixed swaps across all domains: realistic ↔ fog, rain ↔ artistic, and clear ↔ snow.

We evaluated three settings: P3-only swapping, P4-only swapping, and joint P3+P4 swapping, across the source domain and four unseen target domains using three random seeds.

The expert-swapping results, their interpretation, and the capacity-matched comparison are presented in Part 2/2 below.

AuthorsOpenReview ↗

Response to SusM — W2, Q2 & Q3: Expert Specialization and Sources of Performance Improvement (Part 2/2)

Continuing our response to W2, Q2, and Q3, we present the expert-swapping results for the interventions described in Part 1/2, followed by their interpretation and the capacity-matched comparison.

The results are summarized below. Each entry reports [email protected] / mAP@[0.5:0.95] (%, mean ± standard deviation).

MethodDay Clear (Source)Night ClearDusk RainyNight RainyDay FoggyTarget Avg.
Original MoPE62.77±0.21 / 38.55±0.1651.30±0.53 / 29.70±0.2847.87±0.47 / 28.42±0.2932.70±0.66 / 19.22±0.2744.97±0.12 / 28.04±0.1144.21±0.38 / 26.34±0.21
P3 Swapping61.57±0.34 / 37.80±0.1549.56±0.43 / 28.72±0.2246.02±1.12 / 27.42±0.6829.04±0.50 / 17.17±0.2943.42±0.29 / 27.08±0.1842.01±0.32 / 25.10±0.18
P4 Swapping61.08±0.41 / 37.11±0.3748.71±0.72 / 27.98±0.5346.03±0.92 / 27.15±0.6430.46±1.20 / 17.72±0.6743.64±0.25 / 26.86±0.3042.21±0.73 / 24.93±0.52
P3+P4 Swapping60.30±0.42 / 36.76±0.4247.82±1.01 / 27.55±0.6044.66±0.81 / 26.51±0.3727.95±0.69 / 16.46±0.3442.94±0.23 / 26.55±0.3840.84±0.60 / 24.27±0.39

Consistent performance degradation. All three swapping strategies reduce mean detection performance across the source domain and all four unseen target domains. Specifically, P3-only and P4-only swapping reduce the average target-domain [email protected] by 2.20 and 2.00 percentage points, respectively, while joint swapping results in a larger decrease of 3.37 points (44.21% → 40.84%).

Consistent degradation under stricter evaluation metrics. The corresponding reductions in mAP@[0.5:0.95] are 1.24, 1.41, and 2.07 points, respectively. Under joint swapping, the average mAP@[0.5:0.95] decreases from 26.34% to 24.27%. Notably, Night-Rainy exhibits the largest [email protected] degradation under joint swapping, decreasing from 32.70% to 27.95% (−4.75 points).

(3) Evidence of functional expert differentiation.

The key observation is that only the routing-to-expert assignments are modified, while all learned model parameters remain unchanged. If different experts had learned nearly identical feature transformations, exchanging their routing assignments would be expected to have little effect on detection performance. However, the consistent performance degradation observed across different domains suggests that the learned experts have developed functionally differentiated behaviors and that their original routing correspondences play a meaningful role in detection.

Together with the semantically differentiated routing patterns in Fig. 3, these results provide further quantitative support for the effectiveness of preference-guided training in encouraging condition-dependent expert specialization. The observed performance differences also suggest that semantic routing alignment is accompanied by meaningful functional differences among the learned experts, rather than merely reflecting the predefined prototype labels.

(4) Disentangling expert design from augmentation and model capacity.

To further address the concern that improvements may primarily result from additional model capacity or diversified training data, we conducted a capacity-matched comparison with an expanded YOLO-ESMoE baseline using the same MSA-augmented data, 4/8/16 experts, and Top-K settings of 1/2/3.

The expanded ESMoE contains 55.704M parameters, compared with 55.645M for MoPE. Their inference costs are 112.9 GFLOPs and 112.1 GFLOPs, respectively. Despite this closely matched capacity, MoPE achieves 44.21% vs. 41.97% [email protected] and 26.34% vs. 24.96% mAP@[0.5:0.95], consistently outperforming the expanded baseline across all four target domains.

These results demonstrate that MoPE's improvements cannot be explained solely by data augmentation or increased model capacity.

Overall, the additional experiments strengthen the evidence that preference-guided expert routing contributes to meaningful, condition-dependent functional differentiation and improved detection generalization. While these results do not fully establish the independence or optimality of individual expert representations, they provide quantitative support beyond semantic routing consistency.

We sincerely appreciate the reviewer's constructive feedback and will incorporate these additional experimental results into the revised manuscript.

Reviewer_hTHCOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
2
Soundness
3
Read review
Paper summary

The authors propose MoPE, a prototype-expert framework designed to address domain shifts in single-domain generalized object detection (SDGOD) pipelines. The method operates in two main phases: first, broadening the source visual distribution via offline Multi-Style Augmentation (MSA) based on a fixed attribute taxonomy; second, routing input features to semantically matched prototype experts via Multi-Label Preference Optimization (MPO). To produce interpretable, input-adaptive representations, the authors leverage a multi-granularity taxonomy (photometric, textural, and spatial) to guide sparse routing and shared-expert aggregation. Additionally, the paper evaluates MoPE across diverse weather and real-to-artistic benchmarks, demonstrating consistent generalization improvements over recent baselines while maintaining a favorable accuracy-efficiency trade-off.

Critical strengths
  1. The paper addresses a challenging and practical problem: Single Domain Generalized Object Detection (SDGOD). It introduces a prototype-expert factorization framework (MoPE) where each expert is semantically anchored to a predefined prototype, offering interpretability and controllable specialization. By formulating expert routing as a preference-guided decision process, the method effectively handles coexisting visual attributes and mitigates routing collapse.
  2. The proposed MoPE framework is modular and well-organized, with a clear formulation. The paper designs a three-level, coarse-to-fine prototype taxonomy (photometric P3, textural P4, spatial P5) aligned with CNN feature hierarchies, utilizing stage-specific Top-K activation with shared-expert aggregation. The inclusion of Multi-Label Preference Optimization (MPO) to rank preferred experts above rejected ones, combined with Multi-Style Augmentation (MSA) and a clear training schedule (cold start to MPO), makes the overall workflow easy to follow.
  3. The paper is well-written and easy to read. The introduction provides a clear problem setup, and the experimental section demonstrates strong cross-domain generalization across the Diverse Weather and Real-to-Artistic datasets using multiple backbones (ResNet-101 and CSPNet-L). Comprehensive ablation studies isolate the synergistic contributions of MPO and MSA, while the analysis of model complexity (parameters and GFLOPs) and routing visualizations across unseen domains confirms a favorable accuracy-efficiency trade-off under sparse routing.
Critical weaknesses
  1. MoPE fixes the prototype taxonomy and the number of experts at each stage, but its scalability to new visual conditions has not been evaluated. If new visual prototypes are introduced, it is unclear whether the model must be retrained from scratch on the full dataset or whether the new prototypes can be incorporated through incremental training based on the existing model. Moreover, expanding the prototype set enlarges the routing space and may cause performance degradation because of increased competition among experts or insufficient supervision. Experiments comparing full retraining with incremental prototype expansion would help evaluate the scalability of the proposed method.
  2. The test datasets may not provide balanced coverage of the 16 spatial prototypes at P5. Low activation rates for some experts may result either from the absence of the corresponding scenes in the test data or from insufficient training of those experts. Additional test data covering the different P5 prototypes would therefore be useful for verifying whether the corresponding experts are selected correctly and for distinguishing missing-scene coverage from routing failure.
  3. The paper reports [email protected] but does not report stricter localization metrics. COCO-style mAP@[0.5:0.95] would provide additional information about bounding-box quality under illumination, weather, and style shifts. Per-domain results for these metrics would help determine whether the improvements extend beyond relatively loose IoU matching.
Questions for authors

Please see the weaknesses.

6 comments
AuthorsOpenReview ↗

Response to hTHC — Concern 1: Scalability and Incremental Prototype Expansion

We sincerely thank the reviewer for this insightful and constructive suggestion. Incremental prototype expansion is indeed a promising and underexplored research direction in the context of SDGOD, and we appreciate the reviewer for highlighting this possibility.

In the current MoPE framework, we adopt a predefined prototype taxonomy with a fixed prototype-to-expert mapping. Introducing new visual prototypes would require updating the semantic taxonomy, expanding the corresponding expert and router structures, and adapting the preference annotations, followed by additional training. While full retraining is a straightforward solution, whether new prototypes can be effectively incorporated through incremental learning without retraining the entire model remains an interesting open question.

We also fully agree with the reviewer's concern that expanding the prototype set may increase routing competition and lead to insufficient supervision for infrequent experts, potentially degrading detection performance. This highlights the importance of carefully designing the prototype taxonomy and balancing semantic coverage, expert specialization, and routing complexity.

Our current work focuses on validating preference-guided expert composition under a fixed, interpretable prototype taxonomy. As discussed in Appendix I, automatic taxonomy expansion is an important direction for future work. The reviewer's suggestion further motivates us to investigate incremental prototype learning and scalable expert organization in future research.

We sincerely appreciate this valuable suggestion, which provides a meaningful direction for extending the proposed framework.

AuthorsOpenReview ↗

Response to hTHC — Concern 2: Evaluation Coverage and Design of P5 Spatial Prototypes (Part 1/2)

We sincerely thank the reviewer for this thoughtful and constructive observation. We agree that the scene distribution of the evaluation datasets may affect the activation frequencies of different P5 experts, and that insufficient scene coverage or training supervision could contribute to low expert utilization. We would like to clarify our evaluation protocol and the scope of the current prototype design.

(1) Evaluation under established SDGOD benchmarks.

Our experiments follow established SDGOD evaluation protocols adopted by representative works, including S-DGOD (CVPR 2022), CLIPGap (CVPR 2023), SECoT (CVPR 2025), and SDG-DiffDet (ICCV 2025).

Specifically, we evaluate MoPE on two widely used benchmark settings: Diverse Weather, comprising four unseen target domains with different illumination and weather conditions, and Real-to-Artistic, comprising three unseen artistic domains (Clipart1k, Watercolor2k, and Comic2k).

These benchmarks are primarily designed to evaluate cross-domain detection generalization rather than to ensure balanced coverage of all predefined spatial prototypes. Following these established protocols enables fair comparisons with existing SDGOD methods.

To further investigate the reviewer's concern, we additionally analyzed the distribution of P5 spatial prototype labels in the training data. The results reveal a highly imbalanced distribution. Frequently occurring prototypes include general_object_centric (79.37%), urban_street (64.96%), residential_street (34.92%), highway (27.52%), and intersection (25.56%). In contrast, several spatial categories are rarely represented, such as tunnel_underpass (2.45%), waterfront_open (1.80%), mixed_complex_scene (1.69%), and indoor_public (0.07%), while illustrated_scene and graphic_synthetic_scene have no positive annotations in the analyzed training data.

Since P5 supports multiple coexisting semantic attributes, these percentages represent individual prototype-label frequencies rather than mutually exclusive class proportions.

The observed long-tailed distribution also helps explain the P5 routing patterns in Fig. 3, where road-scene domains predominantly activate experts associated with frequently occurring road-related and object-centric attributes.

We agree with the reviewer that insufficient training supervision for long-tail spatial categories may affect the reliability of their corresponding expert routing. Since these scene types appear infrequently in the source training data, their associated experts receive relatively limited positive preference supervision, which may constrain their ability to learn reliable routing preferences and condition-specific representations. Consequently, when such rare scenes occur during inference, the corresponding experts may not be selected as reliably as those associated with frequently observed conditions.

However, MoPE is not designed to enforce uniformly balanced expert activation, but rather to encourage input-adaptive expert selection according to the visual attributes of each image. The observed imbalance therefore does not necessarily indicate routing failure, although it highlights a limitation of the current prototype taxonomy under long-tailed training distributions.

The scope of the prototype taxonomy and future directions are discussed in Part 2/2 below.

AuthorsOpenReview ↗

Response to hTHC — Concern 2: Evaluation Coverage and Design of P5 Spatial Prototypes (Part 2/2)

Continuing our response to Concern 2, we discuss the scope of the prototype taxonomy and directions for further improvement.

(2) Scope and extensibility of the prototype taxonomy.

We also appreciate the reviewer's observation regarding the coverage of P5 prototypes. Our current work represents an initial investigation into structured semantic prototype-based expert composition for SDGOD.

The predefined taxonomy, particularly the 16 spatial prototypes at P5, is one interpretable design choice rather than a uniquely optimal or exhaustive configuration. We acknowledge that the prototype definitions could be further refined to improve semantic coverage, reduce redundancy, and encourage more complementary expert specialization.

Furthermore, as described in Sec. 3.2.2, MSA primarily introduces photometric and textural variations while preserving the original scene geometry and spatial layout. Therefore, it does not explicitly expand the coverage of rare spatial prototypes, which remains constrained by the source-domain scene distribution.

The central objective of this work is to investigate whether explicitly defined semantic prototypes, combined with preference-guided routing, can improve generalization to unseen domains. The consistent performance gains across the two established benchmark settings provide encouraging empirical support for the feasibility and effectiveness of this design, without implying that the current prototype taxonomy is uniquely optimal.

(3) Future directions.

We agree that broader evaluation with more diverse spatial scenes would help assess individual prototype coverage and distinguish underrepresented scene types from potential routing limitations.

More importantly, designing a more compact and semantically complementary prototype taxonomy, with more sufficient and balanced preference supervision across categories, represents a promising direction for improving expert specialization and routing reliability, particularly for underrepresented scene types.

As discussed in Appendix I, automatically discovering and organizing semantic prototypes, rather than relying entirely on manually predefined categories, could help reduce semantic redundancy and better align prototype definitions with the available training data.

We sincerely appreciate the reviewer's valuable suggestion, which highlights meaningful opportunities to further refine the prototype taxonomy and strengthen its evaluation.

Reviewer_MeVrOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes MoPE for single-domain generalized object detection. It combines prototype-based experts, preference-guided routing, and diffusion-based data augmentation to improve detection under unseen domain shifts. The method reports strong results on weather and artistic benchmarks.

Critical strengths

The paper tackles an important problem in domain generalization, and the motivation is clear.

The proposed framework is well organized, and the idea of combining semantic experts is reasonable.

The experiments show solid improvements across different domains and detector architectures.

Critical weaknesses

The technical novelty is limited. MPO is essentially an adaptation of existing DPO-style preference learning to expert routing. The paper does not compare against simpler alternatives, such as multi-label BCE or a pairwise ranking loss, so it is not clear why MPO is necessary.

The evidence for the core contribution is not strong enough. In Table 4, MSA alone improves mAP from 29.00 to 40.30, and adding MPO raises it further to 44.21. This suggests that a large part of the gain comes from data augmentation. To justify the proposed expert design, the paper should compare with standard MoE models trained with the same augmented data and the same model capacity.

The evidence for generalization to truly unseen conditions is limited. The augmentation prompts explicitly cover rainy nights, foggy days, and artistic styles, which closely match the evaluation domains. Although no target images are used, the paper does not evaluate conditions beyond those covered by the prompt bank. This weakens the claim of generalization to unknown domains.

Questions for authors

See Weaknesses

7 comments
AuthorsOpenReview ↗

Response to MeVr — Concern 1: Technical Novelty and Comparison with Simpler Routing Objectives (Part 1/2)

We sincerely thank the reviewer for this important comment and valuable suggestion. We agree that MPO builds upon the established DPO formulation, and we do not claim that its underlying preference optimization objective is entirely new. However, we would like to clarify that the central contribution of MoPE is not merely adapting DPO to expert routing, but introducing a prototype-based, semantically supervised expert composition framework for single-domain generalized object detection (SDGOD).

(1) Motivation: A complementary perspective on SDGOD.

As discussed in Sections 1 and 2, existing SDGOD approaches primarily focus on invariant feature learning or semantic and distribution modeling. While effective, these approaches still face the challenge of modeling diverse combinations of visual conditions from a single source domain.

Motivated by the ability of Mixture-of-Experts (MoE) architectures to dynamically combine specialized representations, we investigate their potential for modeling heterogeneous visual conditions in SDGOD. However, we observed that conventional detection-driven MoE routing may repeatedly activate similar expert combinations across different domains, limiting input adaptivity (Appendix E.3, Fig. 5).

(2) Core idea: Explicit semantic supervision for expert routing.

To address this limitation, we propose organizing visual conditions into photometric, textural, and spatial prototypes, each associated with a dedicated expert. We then employ multimodal annotation to construct image-specific, multi-label prototype preferences, transforming semantic attributes into explicit supervision for expert selection.

Unlike conventional task-driven routing, which primarily learns expert selection through detection loss, our approach explicitly guides the router to associate different visual conditions with corresponding experts. Through sparse routing and joint detection optimization, the selected experts receive condition-dependent training signals, encouraging the development of specialized representations.

(3) Why preference optimization? Is MPO the only possible solution?

Our starting point was not to introduce a DPO-style loss, but to address the insufficient semantic guidance in conventional MoE routing. To this end, we explicitly construct prototype preference annotations to guide expert selection.

Since an image may simultaneously exhibit multiple visual attributes, we formulate routing supervision using multiple preferred and rejected prototypes. This naturally motivates a preference-based optimization approach. Specifically, MPO optimizes the relative routing preferences between relevant and irrelevant experts while incorporating a frozen reference router to regularize preference changes.

We fully agree that multi-label BCE and conventional pairwise ranking are also reasonable alternatives. Unlike BCE, which independently supervises prototype labels, MPO directly optimizes relative preferences between prototype groups with reference-based regularization.

Our primary goal is to investigate whether explicitly supervising expert selection can encourage meaningful specialization and improve generalization in SDGOD.

The multi-label BCE comparison and further experimental evidence are presented in Part 2/2 below.

AuthorsOpenReview ↗

Response to MeVr — Concern 1: Technical Novelty and Comparison with Simpler Routing Objectives (Part 2/2)

Continuing our response to Concern 1, we present the additional multi-label BCE comparison and supporting experimental evidence.

(4) Additional comparison with multi-label BCE.

Following the reviewer's suggestion, we conducted an additional experiment by replacing MPO with multi-label BCE using the same model architecture, MSA training data, and prototype annotations. BCE supervision starts from the first epoch and uses no reference policy.

The results are summarized below. Each entry denotes [email protected] / mAP@[0.5:0.95] (%, mean ± sample standard deviation over seeds 42, 43, and 44).

MethodDay Clear (Source)Night ClearDusk RainyNight RainyDay FoggyTarget Avg.
Multi-label BCE62.86±0.11 / 38.59±0.0250.76±0.41 / 29.45±0.2147.42±0.94 / 27.97±0.4031.88±0.68 / 18.56±0.2844.69±0.31 / 27.75±0.2843.69±0.57 / 25.93±0.28
MoPE (MPO)62.77±0.21 / 38.55±0.1651.30±0.53 / 29.70±0.2847.87±0.47 / 28.42±0.2932.70±0.66 / 19.22±0.2744.97±0.12 / 28.04±0.1144.21±0.38 / 26.34±0.21

Across three random seeds (42, 43, and 44), MPO achieves slightly higher mean [email protected] and mAP@[0.5:0.95] than multi-label BCE on all four unseen target domains. The target-domain average [email protected] increases from 43.69% with BCE to 44.21% with MPO (+0.52 percentage points), and the average mAP@[0.5:0.95] increases from 25.93% to 26.34% (+0.41 percentage points).

The target-domain performance obtained with BCE supports the feasibility of translating semantic prototype annotations into explicit routing supervision, showing that a multi-label classification objective can also be used to learn input-adaptive expert selection. The additional improvements obtained with MPO provide empirical support for adopting preference optimization under the evaluated experimental setting.

These findings further clarify the role of MPO within our framework. MPO is one concrete implementation of the proposed semantic preference-guided routing framework, and BCE also provides a viable realization. The central idea is to explicitly supervise expert selection using image-level semantic annotations and learn condition-dependent expert representations through joint detection optimization. Different routing supervision objectives can be further explored within this framework.

(5) Further evidence for the overall framework.

Appendix E.3 illustrates the limited routing diversity of the conventional MoE baseline, while Fig. 3 demonstrates more semantically differentiated expert selection across unseen domains. Table 4 reports an improvement from 29.00% to 35.97% without MSA. With MSA, the target-domain average [email protected] improves from 40.30% to 44.21% when MPO is introduced.

Furthermore, following the reviewer's suggestion, our additional capacity-matched comparison shows that MoPE still outperforms the expanded ESMoE trained with the same MSA data by 2.24 [email protected] points, despite nearly identical parameter counts (55.645M vs. 55.704M). The inference costs are 112.1 GFLOPs for MoPE and 112.9 GFLOPs for the expanded ESMoE.

In summary, our contribution lies primarily in formulating and validating an integrated framework that connects visual diversification, semantic prototype construction, preference-guided expert routing, and dynamic expert composition for SDGOD, rather than claiming a fundamentally new preference loss.

We sincerely appreciate the reviewer's constructive feedback. The additional BCE comparison helps clarify the motivation for MPO and further strengthens the empirical evaluation of our framework. We will revise the Introduction and experimental analysis accordingly.

AuthorsOpenReview ↗

Response to MeVr — Concern 2: Effectiveness of the Proposed Expert Design Beyond Data Augmentation and Model Capacity (Part 1/2)

We sincerely thank the reviewer for this constructive suggestion. We agree that MSA contributes substantially to the overall performance improvement. However, we would like to further clarify the motivation behind MSA and its relationship with preference-guided expert routing, and provide additional experimental evidence.

(1) Motivation: MSA and MPO are designed as complementary components.

A fundamental challenge in SDGOD is the limited visual diversity available from a single source domain, which restricts both the learning of generalizable detection features and the specialization of expert networks. Our motivation is therefore not simply to improve performance through data augmentation.

Specifically, MSA serves two complementary purposes: (i) expanding the visual diversity of training data to improve detection robustness, and (ii) providing diverse visual conditions and corresponding prototype preference annotations for learning meaningful expert routing. Through MPO, these semantic preferences explicitly guide the router to favor experts associated with the input's visual attributes. The selected experts then receive detection supervision through sparse routing, encouraging condition-dependent specialization.

(2) Additional experiments: Capacity-matched comparison with standard MoE.

Following the reviewer's suggestion, we conducted an additional controlled comparison by expanding YOLO-ESMoE to match MoPE's 4/8/16 experts at P3/P4/P5 and Top-K settings of 1/2/3, using the same MSA-augmented training data.

The expanded ESMoE contains 55.704M parameters, compared with 55.645M for MoPE, a difference of only approximately 0.11%. Its inference cost is 112.9 GFLOPs, compared with 112.1 GFLOPs for MoPE.

The results on the Diverse Weather benchmark are summarized below. Each entry reports [email protected] / mAP@[0.5:0.95] (%, mean ± sample standard deviation over seeds 42, 43, and 44).

MethodNight ClearDusk RainyNight RainyDay FoggyTarget Avg.
ESMoE + MSA47.63±0.32 / 27.11±0.2042.60±0.46 / 25.09±0.3228.60±0.53 / 16.64±0.2742.37±0.06 / 25.64±0.0440.30±0.04 / 23.62±0.05
Expanded ESMoE + MSA49.17±0.24 / 28.36±0.1145.30±0.23 / 26.89±0.1129.78±0.03 / 17.76±0.1743.61±0.05 / 26.85±0.1241.97±0.11 / 24.96±0.11
MoPE (MSA + MPO)51.30±0.53 / 29.70±0.2847.87±0.47 / 28.42±0.2932.70±0.66 / 19.22±0.2744.97±0.12 / 28.04±0.1144.21±0.38 / 26.34±0.21

Increasing the number of experts in ESMoE improves the average [email protected] from 40.30% to 41.97% (+1.67 points). Nevertheless, MoPE consistently outperforms the capacity-matched ESMoE across all four unseen domains, achieving an additional +2.24 points in [email protected] and +1.38 points in mAP@[0.5:0.95].

These results demonstrate that MoPE's improvements cannot be explained solely by data augmentation or increased model capacity. Instead, they provide further evidence for the effectiveness of our proposed prototype-based, preference-guided expert framework.

The additional COCO evaluation and the overall conclusions are presented in Part 2/2 below.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.