Neural Architecture Search with LLM-Discovered Fusion Operators for Multimodal Classification

Fusion operators play an important role in multimodal classification (MMC) by determining how representations from different modalities interact and are fused, thereby affecting the achievable performance. However, the fusion operators used in existing MMC methods are largely human-designed, limiting the diversity of fusion mechanisms that can be explored.We propose LLMDFO-NAS, a neural architecture search framework that automatically constructs and explores the fusion operator space using Large Language Models (LLMs) and evolutionary search.Specifically, it leverages LLMs to generate executable and validated fusion primitives and employs evolutionary search to discover effective primitive compositions as candidate fusion operators.Candidate fusion operators are efficiently evaluated across multiple architecture contexts using an architecture-aggregated zero-cost proxy with Pareto selection, and the high quality ones among them are selected as the discovered operators.These discovered operators then define the operator library for downstream multimodal architecture search, forming a two-level automated design process. Experiments on five representative benchmarks show that the discovered operator spaces consistently improve the resulting fusion architectures across different modality configurations.Our work extends automated design from architecture optimization within a fixed functional space to the joint design of functional and structural spaces.