acceptodds
Under review as a conference paper at ICLR 2027

Elax: LLM-Guided Evolutionary Neural Architecture Search for Heterogeneous Edge Systems

Abstract

Deploying neural networks across heterogeneous edge platforms is challenging because architecture accuracy-latency trade-offs vary across hardware, while independently training and profiling large numbers of candidate architectures is expensive. We present Elax, an LLM-assisted evolutionary neural architecture search framework that combines elastic weight sharing with hardware-aware multi-objective optimization. Elax converts a pretrained backbone into an elastic supernet and trains its shared weights using feature-level transfer and mixed random and LLM-guided subnet sampling. The trained weights are then frozen and reused during profile-specific evolutionary search. Rather than serving as a fitness predictor, the LLM proposes legal subnet configurations and local architectural mutations conditioned on the current architecture, observed performance, and hardware constraints. We evaluate Elax on elastic CNN and transformer backbones across CIFAR-10, CIFAR-100, and ImageNet-1K, and perform hardware-aware benchmark comparisons on NAS-Bench-201 and HW-NAS-Bench across multiple device profiles. The results show that Elax finds subnetworks with varying accuracy and latency trade-offs across devices and achieves competitive Pareto-front quality under matched architecture-evaluation budgets compared with adapted multi-objective search baselines.

Primary area:
optimization
Keywords:
Neural architecture search, hardware-aware NAS, large language models, supernets, multi-objective optimization, evolutionary search, edge AI

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_Dpw1OpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper presents ELAX, an LLM-guided evolutionary NAS framework targeting heterogeneous edge deployment. The pipeline consists of two primary phases:

  • Elastic Supernet Construction: A pretrained backbone (e.g., ResNet, ViT, Swin, DeiT) is converted into a weight-sharing supernet using a two-stage training scheme involving feature-level knowledge transfer and task-level fine-tuning across sampled subnets.
  • Hardware-Aware Subnet Extraction: The supernet parameters are frozen, and a multi-objective evolutionary search (NSGA-II) extracts subnets that optimize validation accuracy against device-specific latency constraints.

Rather than generating complete architectures from scratch, ELAX uses a local LLM (Llama 3.1 8B) as a constrained mutation controller. The LLM proposes local structural edits conditioned on parent encodings, search history, target hardware profiles, and latency bounds. Every proposal passes through a deterministic validator to ensure search-space compliance, falling back to random mutation if malformed.

The authors evaluate ELAX on elastic CNNs and Transformers across CIFAR-10, CIFAR-100, and ImageNet-1K, alongside NAS-Bench-201 and HW-NAS-Bench across five hardware profiles (edgegpu, raspi4, pixel3, eyeriss, fpga).

Critical strengths
  • Restricting the LLM to local mutation proposals validated by a deterministic schema solver solves the hallucination and search-space violation issues common in LLM-NAS. Combining this with an elastic supernet allows LLM-guided candidate edits to be evaluated instantly without retraining.

  • Decoupling the one-time supernet training from the hardware-specific Pareto search allows a single trained supernet to be reused across multiple edge targets and latency limits without repeating full training cycles.

Critical weaknesses
  • The hardware-aware evaluation relies on static lookup tables (HW-NAS-Bench) that sum isolated operator latencies. Real edge execution is heavily influenced by compiler graph fusions, SRAM/DRAM transfer costs, cache behavior, and memory bandwidth bottlenecks. Static LUTs fail to capture these dynamic harware behaviors.

  • The evaluation focuses entirely on static vision backbones (CNNs/ViTs) on image classification datasets. Modern edge AI has pivoted heavily toward small language models and generative decoders where KV-cache management and autoregressive memory bandwidth dominate execution. Also, omitting low-precision quantization choices (INT8/INT4) leaves out a primary dimension of real-world edge hardware co-design.

  • The paper omits total search wall-clock time and LLM inference overhead. While candidate scoring on the frozen supernet is fast, querying a local Llama 3.1 8B model via Ollama for 20% of the evolutionary offspring introduces sequential generation latency. Without reporting end-to-end wall-clock search time and the exact fraction spent waiting on LLM calls, it is unproven whether ELAX provides a net time savings over standard multi-objective Bayesian optimization baselines (e.g., qLogEHVI).

Questions for authors
  • How does ELAX perform when evaluated against direct on-device profiling or compiler-in-the-loop latency measurements rather than static lookup tables?

  • Can the LLM mutation controller and template menu be expanded to co-design layer-wise quantization precision (e.g., INT8 vs INT4), and how does that impact the validator rejection rate?

  • What is the total wall-clock runtime for a complete 500-evaluation search run, and what fraction of that time is spent strictly on LLM inference latency?

2 comments

We thank the reviewer for recognizing the value of validated local edits and reusable elastic weights. The comments help us distinguish the hardware evidence available in the present study from deployment measurements that remain to be made. We have clarified the use of benchmark latency in §5.1 and Appendix A.8, stated the deployment and model-scope limitations in §6, and reported search-loop wall time in Appendix A.5.

Q1. How does ELAX perform with direct on-device or compiler-in-the-loop profiling?

We have not run the five-profile search with online compilation and profiling, so we cannot claim that its Pareto rankings or latency feasibility would be preserved under that protocol. The hardware comparison queries stored HW-NAS-Bench latency for NAS-Bench-201 architectures; it does not compile and measure each candidate during ELAX search (§5.1 and Appendix A.8). A distinction about those records is useful. For NAS-Bench-201, HW-NAS-Bench provides architecture-level latency values. Its Edge GPU, Raspberry Pi 4, and Pixel 3 latency data come from measurements of compiled networks on devices; its Eyeriss and FPGA values are estimates. The benchmark’s approximation formed by summing block costs is described for its FBNet search space, rather than the NAS-Bench-201 records used here. These distinctions follow the HW-NAS-Bench paper. The stored records support a reproducible, device-specific comparison of search methods under the same 500-evaluation budget. They do not establish the latency of newly compiled ELAX elastic subnets under different compilers, workloads, or memory conditions. The separate ResNet-50 study measures subnet latency on an RTX 4060 Ti (§5.2), but is not a deployment study across the five edge targets. We identify search and deployment of elastic subnets on multiple physical devices as future work (§6).

Q2. Can the controller include INT8/INT4 choices, and how would they affect validator rejection?

The constrained-edit approach could be extended to include device-supported precision choices in its legal menu. The validator would then need to check the proposed precision assignments against operator support, tensor and residual-path compatibility, and the target compilation constraints. Evaluating accepted candidates would also require quantized-accuracy and compiled-device latency measurements. This is a proposed extension, not an experiment reported here. The present edit menu does not include layer-wise bit-width choices, so we cannot report a quantization-specific rejection rate or an INT8/INT4 accuracy–latency result. The rate could change with the device and legal menu; it cannot be inferred from the current controller audit. Section 6 now identifies mixed-precision quantization and validation of expanded edits as future work. The current experiments also concern image-classification CNNs and vision transformers. Applying the approach to autoregressive decoders would require a suitable decoder search space and measurements of token-generation latency and KV-cache memory, including the effects of memory bandwidth. Section 6 identifies this extension without treating the vision results as evidence for decoder workloads.

Q3. What is the 500-evaluation wall time, and how much is spent on LLM calls?

We have added search-loop wall-time accounting in Appendix A.5. Across the 50 saved NAS-Bench-201 hardware-search runs, a complete 500-unique-evaluation ELAX run averaged 578.7 s, including 557.1 s in local LLM requests, or approximately 96.3% of the measured loop time. The matched random-ranking control averaged 22.4 s. For additional context, the saved qLogEHVI baseline runs averaged 28.6 s under the same evaluation budget. The present implementation therefore does not demonstrate a wall-time saving over qLogEHVI. The LLM request timing includes model execution and service and communication overhead; our logs do not isolate model computation. The matched 500-evaluation comparison remains useful for assessing the Pareto quality obtained from the same number of distinct architecture evaluations and the same benchmark records across methods. We now report that result separately from elapsed time, so the paper’s efficiency claim does not imply that the LLM controller makes the search faster. These timings cover the NAS-Bench-201 search loop, which queries stored accuracy and latency records. They do not include elastic-supernet training or deployment. The trained supernet can be reused across target profiles, but the present experiments do not measure the total cost of that reuse. We state this boundary in the limitations and identify full training-plus-search measurement and physical-device profiling as future work (section 6). Reducing controller overhead, for example through fewer or batched requests, is also a useful direction for future evaluation.

Thank you for the responses and updates. They clarified a couple of my questions.

That said, two key limitations remain: (1) the newly extracted elastic subnets were not compiled or profiled across the physical edge targets, and major edge considerations like mixed-precision quantization and autoregressive decoders remain unaddressed, and (2) spending 557.1s out of 578.7s on local LLM calls makes the search ~20x slower than qLogEHVI.

Balancing the strengths and weaknesses of this paper, I will maintain my score of weak acceptance.

Reviewer_rxKEOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

ELAX searches for neural network designs that balance accuracy and speed on different devices. It first trains one shared network whose parts can form smaller networks, then keeps those weights fixed during search. An LLM helps choose some training paths and some changes to candidate designs. The paper tests elastic CNN and transformer models, and separately compares search methods using NAS-Bench-201 accuracy records and HW-NAS-Bench latency tables. The latter tests do not use the trained shared network.

Critical strengths
  1. The search results are competitive under the same limit of 500 different designs across five hardware profiles, with ten paired runs. The paper also shows where other methods do better, including pixel3. This supports a useful comparison without claiming a win on every device.

  2. The shared-network approach is tested with both CNN and transformer models. The smaller networks offer different choices between accuracy and speed. Checks on LLM output and a fallback procedure are sensible safeguards for the search.

Critical weaknesses
  1. The extra benefit from the LLM is still uncertain. In Table 4, LLM ranking helps some averages but hurts others, and none of the paired differences is statistically significant after the reported correction. Table 3 gives the full method the highest selected accuracy, but a slightly lower HV, a measure of the overall accuracy-speed trade-off. The authors acknowledge both limits. These results do not show that the LLM is useless; they leave its added value unclear.

  2. The exact method is not fully specified. The main text describes NSGA-II, while the appendix adds ranking, prediction and candidate-screening steps. These could form one consistent method, but their order and connections are unclear. The training-time LLM sampler also needs more detail. This makes it hard to tell which steps produced each result. A single workflow and a table linking experiments to their settings could resolve much of this concern.

Questions for authors
  1. Please show the full search workflow and state which version was used in each experiment. Where do NSGA-II, the random-forest ranker and qLogEHVI enter, and how are their outputs used? Also explain how the LLM chooses training paths. This would help establish that the reported results match a clearly defined method.

  2. Please provide the paired results behind Table 4 and the number of LLM calls, retries and fallbacks, together with elapsed search time. The aim is to judge the size and consistency of any gain and its cost. A useful gain under clearly stated conditions would strengthen the contribution.

  3. For Table 3, please give the number of training runs, the device and speed limit used to choose the reported network, and the HV calculation. Explain how checkpoints and training paths were selected. This would clarify whether the accuracy gains come from a fair comparison under the same conditions.

2 comments

We thank the reviewer for the careful assessment and for recognizing the matched evaluation budgets, the CNN and transformer experiments, and the safeguards on LLM proposals. We agree that the original presentation did not clearly separate the procedures used in the different experiments. We have clarified the search workflows in Section 4.4, the training and search settings in Section 5.1, and the Table 3 protocol in Section 5.2. We also provide paired run-level results and controller audit counts for the Table 4 ablation. These revisions distinguish the performance of the complete search system from the incremental effect of LLM ranking.

Q1. What are the full workflows, and where do NSGA-II, the random-forest ranker, qLogEHVI, and training-time LLM sampling enter?

Section 4.4 now identifies the workflow behind each result. The ResNet-50/CIFAR-100 component study in Table 3 uses constraint-ranked tournament evolution with a population of 20 and 100 distinct subnet evaluations. Offspring are produced by crossover or legal local mutation. In the search-time LLM condition, the controller selects from the offered legal edits; in the corresponding control, a legal edit is selected uniformly. The separate NAS-Bench-201 hardware study uses region-aware candidate generation. A pairwise random-forest accuracy ranker shortlists proposals, after which qLogEHVI selects architectures for evaluation using an accuracy model and device-specific lookup-table latency. One candidate in each five-evaluation batch is reserved for the LLM-guided source and undergoes the same acquisition screening. NSGA-II is a comparator in this study, not a selection step in ELAX’s reported hardware-search workflow. Appendix A.8 provides the hardware-search settings, and Appendix A.9 describes its controller and validation procedure. Section 5.1 now specifies the controlled ResNet-50 Stage-B sampling procedure. The deterministic main subnet is used on 25% of minibatches. On the remaining minibatches, LLM sampling is activated with probability 0.1. Given a sampled parent, legal edits, and subnet-coverage counts, the controller selects five offered edits, uses the first immediately, and queues the others. An invalid selection retains the parent subnet; minibatches without LLM guidance use random sampling.

Q2. Can we provide paired results, controller calls and fallbacks, and elapsed time?

We provide the run-level results underlying Appendix Table 4 in the supplementary file Table4_Paired_Seed_Results.csv. It contains all 50 matched runs; seeds 500–509 on each of five hardware profiles, with both methods’ final HV, IGD, feasible Top-1 accuracy, and search time, together with their paired differences. The random-ranking control uses the same parent pool, legal edits, proposal quota, acquisition screening, and 500-evaluation budget. ELAX has higher mean final HV on two profiles and lower mean IGD on four, but no paired Table 4 difference survives Holm correction. We therefore do not claim a statistically reliable improvement from LLM ranking in this ablation. The controller audit is reported in Appendix A.4. Across the 50 LLM-guided runs, it records 1,136 logical requests and 1,227 backend calls. Semantic validation rejected 91 responses, and corrective calls recovered 88 requests. No transport retries or LLM-reserved fallback batches were recorded. Appendix A.5 reports mean search-loop time of 578.7 s for ELAX, including 557.1 s of LLM inference; approximately 96.3% of its measured search time; versus 22.4 s for random ranking. These timings exclude supernet training, which is separate from the NAS-Bench-201 benchmark experiment. We now report wall time alongside the matched evaluation budget: the complete system is competitive under that budget, while the incremental quality benefit and time cost of LLM ranking are stated separately.

Q3. How were the Table 3 training runs, checkpoints, paths, device constraint, and HV determined?

Section 5.2 now clarifies that Table 3 uses three independent Stage-A training seeds. Each Stage-A checkpoint initializes one random-sampling and one LLM-sampling Stage-B run, yielding six frozen checkpoints. Within each seed, the two Stage-B conditions use the same training settings and select checkpoints by the same criterion: the highest validation Top-1 accuracy on the deterministic main subnet. Each checkpoint is then searched both with and without search-time LLM guidance, using search seed 10, a population of 20, and 100 distinct subnet evaluations. Thus, there are 12 searches, while the training seed is the unit underlying Table 3’s reported variation. The reported network is the evaluated subnet with the highest validation Top-1 accuracy that satisfies a 4.4 ms latency limit. Latency is measured on an RTX 4060 Ti using FP16, batch size one, and p50 latency. Table 3 HV instead summarizes the non-dominated set of all 100 evaluated subnets: validation Top-1 is converted to error rate, latency in milliseconds is divided by 10, and the reference point is (1.1,1.1)(1.1,1.1). The 4.4 ms bound selects the reported feasible subnet; it does not restrict the HV calculation. This normalization differs from the full-search-space normalization used in the NAS-Bench-201 hardware study, so the two studies’ HV values should not be compared directly. The full Table 3 configuration has the highest mean selected feasible accuracy, whereas some ablations have slightly higher mean HV. With three training seeds and one search seed per checkpoint, we present these differences as descriptive.

Reviewer_GKQTOpenReview ↗
Rating
2
Confidence
3
Informativeness
1
Presentation
3
Soundness
2
Read review
Paper summary

ELAX is presented in this paper, which is an LLM-guided evolutionary NAS framework for heterogeneous edge systems. ELAX combines elastic supernet training with hardware-aware multi-objective optimization to identify subnetworks with favorable accuracy-latency trade-offs. An LLM is used to guide subnet sampling during training and propose architectural mutations during evolutionary search. The authors evaluate ELAX on CNN and transformer backbones and compare it with existing NAS methods across five hardware profiles using NAS-Bench-201 and HW-NAS-Bench.

Critical strengths
  • Hardware-aware NAS is a timely topic.
  • The authors evaluate multiple CNN and transformer backbones and compare against several multi-objective NAS baselines across five hardware profiles.
Critical weaknesses
  • I'm not fully convinced of the value of using LLMs to guide hardware-aware NAS. It is unclear to me what distinctive architectural or hardware-specific knowledge LLMs contribute that conventional search methods cannot exploit with lower overhead. It might be helpful if authors can provide more analysis of whether LLM-guided mutations actually exploit hardware-specific operator behavior or architectural characteristics or whether the LLM learns meaningful hardware-specific architecture preferences. The different Pareto fronts across HW-NAS-Bench profiles establish that architecture performance is device-dependent, but this heterogeneity is already captured by the latency lookup tables used by all compared search methods. It does not demonstrate that the LLM itself provides additional hardware-aware guidance.
  • The technical innovation of this work seems incremental. ELAX combines elastic weight sharing, feature-level knowledge transfer, evolutionary search, and LLM-guided architecture mutation. But most of these components are not new and have been well-studied.
  • Why not including the overhead of LLM inference in the search-efficiency evaluation? Without considering the overall search time, LLM inference overhead, and supernet training cost, it is hard to say whether ELAX is actually more efficient than conventional hardware-aware NAS methods.
Questions for authors
  1. What is the main advantage of using an LLM for hardware-aware NAS? Could a simpler heuristic or learned ranking model achieve similar results using the same architecture and hardware feedback? The random-ranking ablation in Appendix Table 4 shows no statistically significant improvement from LLM guidance.
  2. How well would ELAX work on real heterogeneous edge devices? The hardware-aware evaluation relies on HW-NAS-Bench latency lookup tables, while the elastic CNN and transformer experiments are evaluated separately. Could you demonstrate the complete framework by searching and deploying elastic subnets across multiple physical devices?
  3. How does the LLM actually learn or exploit hardware-specific characteristics?
2 comments

We thank the reviewer for pointing out that device-dependent Pareto fronts do not, by themselves, demonstrate an LLM-specific hardware benefit. We now make this distinction explicit in Section 5.2 and have added a matched ranking control and a device-conditioned diagnostic in Appendix A.5.

Q1. What advantage does the LLM provide over a simpler rule or learned ranker?

We compared LLM edit ranking with a history-based rule using the same paired initial populations, legal edits, parent-selection policy, proposal quota, surrogate screening, and budget of 500 unique architecture evaluations (Appendix A.5, Table 5). The rule ranks edits using observed accuracy and target-device latency changes, archive entry, and the target Pareto region. This tests whether LLM ranking adds value when an inexpensive rule receives the same search feedback. Across ten paired seeds on five profiles, ELAX has higher mean final HV on three profiles, whereas the rule has lower mean IGD on four. No LLM advantage survives Holm correction across the 20 profile–metric comparisons. Together with the random-ranking ablation in Appendix A.4, Table 4, these results do not establish that LLM edit ranking outperforms inexpensive alternatives. We have not evaluated a learned edit ranker as a replacement for the LLM; the random-forest accuracy ranker used for candidate screening serves a different role. We also report search-loop wall time in Appendix A.5. The saved ELAX runs averaged 578.7 s, including 557.1 s of LLM inference, versus 22.4 s for random ranking. In a separate current-environment comparison, the history-based rule and random ranking averaged 25.4 s and 24.9 s. These measurements exclude elastic-supernet training. We therefore report architecture-evaluation budgets and wall time separately and make no claim of a wall-time advantage.

Q2. Does the evaluation demonstrate the complete framework on physical devices?

It does not. Section 4.4 (“Candidate evaluation”) and Section 5.1 clarify that the hardware comparison uses NAS-Bench-201 accuracy records and HW-NAS-Bench latency lookup tables, while the elastic-backbone experiments are separate. The complete NAS-Bench-201 search space allows us to construct exact device-specific reference Pareto fronts, so HV and IGD can be compared under the same 500-query budget across methods. This provides a controlled search comparison, but does not establish physical-device performance for the complete pipeline. Section 6 identifies elastic-subnet search and deployment across multiple physical devices, together with full training-plus-search cost measurement, as future work.

Q3. How does the LLM use hardware-specific characteristics?

The controller receives the target device profile, observed candidate latencies, parent architectures, legal edits, and prior search outcomes through its prompt (Appendix A.9). It ranks a finite menu of legal edits. It receives neither reference-front information nor accuracy for unevaluated architectures, and its proposals undergo deterministic validation and the same acquisition screening as other candidates. In the fixed-candidate diagnostic in Appendix A.5, we held the observed architectures and parent candidates fixed while changing device-specific latency evidence. The LLM selected different edits across profiles, but device-matched edits exceeded the mean post hoc one-edit HV potential of off-device edits in only 13 of 30 contexts. Their mean potential was also below that of the history-based rule (0.001113 versus 0.001241). These are diagnostic potential gains, not realized search gains. We therefore describe sensitivity to device context, without claiming that the LLM has learned beneficial hardware-specific operator preferences. Finally, we frame the contribution as a framework combining elastic training, constrained edit ranking, and hardware-aware search, with controlled evaluations of its components. We do not claim that the individual training or acquisition methods are new.

Thanks for the response. My primary concerns on substantial overhead and lack of end-to-end evaluation on physical devices still remain. Therefore, I'll maintain my original score.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.