acceptodds
Under review as a conference paper at ICLR 2027

ZAPS: Zero-Cost Active Proxy Search for Neural Architecture Search

Abstract

Neural Architecture Search (NAS) automates network design, but evaluating a single candidate requires training it to convergence, making exhaustive search intractable. Zero-cost proxies estimate architecture quality at initialization in seconds, yet a single proxy is noisy, and combining several does not straightforwardly help: proxies are strongly correlated, so naive aggregation compounds their shared errors instead of averaging them out. Existing methods exploit either proxy signals or architectural topology—never both within a single active-learning framework. We introduce ZAPS (Zero-cost Active Proxy Search), a four-stage pipeline that closes this gap. ZAPS (i) selects a compact, non-redundant proxy subset offline via PROXYFIT, a greedy anti-redundancy criterion; (ii) seeds the search with a hybrid K-means strategy that balances exploitation and exploration; (iii) re-selects proxies at every iteration by a bootstrapped vote as the labeled set grows; and (iv) ranks candidates with an XGBoost ensemble trained jointly on proxy ranks and one-hot topological encodings, queried through an Upper Confidence Bound (UCB) acquisition function. On NAS-Bench-201 under a budget of B = 200 evaluations, ZAPS recovers 52.3 % of the true top-100 architectures on CIFAR-10 and 65.8 % on CIFAR-100, ahead of every baseline we consider—Random Search, Local Search, REA, BANANAS and TPE—and, on CIFAR-10, with less than half the run-to-run standard deviation of the strongest of them. The advantage is largest where evaluations are scarce: on NAS-Bench-201 it narrows as the budget grows, whereas on the harder NAS-Bench-101, which no method comes close to saturating, it widens instead. All methods are scored by a single criterion: how much of the true top-100 lies among the architectures they actually evaluated.

Primary area:
optimization
Keywords:
neural architecture search, zero-cost proxies, active learning, surrogate model, NAS-Bench-201, XGBoost, UCB acquisition

Reviews

Among:
Shape:
1234this paper, Bottom 2%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.33 over 3 reviews, above 2% of the others.
Reviewer_zgwpOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

ZAPS combines zero-cost architecture proxies with active surrogate-based search. Its pipeline selects a non-redundant proxy subset with ProxyFit, uses a hybrid K-means initialization, reselects proxies online with bootstrap votes, and fits an XGBoost ensemble to proxy ranks and architecture encodings for UCB acquisition. On NAS-Bench-201, it reports improved top-100 recovery under a 200-evaluation budget across CIFAR-10 and CIFAR-100. It also evaluates proxy transfer, component ablations, and NAS-Bench-101.

Critical strengths
  1. The empirical study is unusually thorough for a low-budget NAS method. It uses many random seeds, reports both recovery and regret, discusses practical effect sizes, and separates the contributions of the main search components.
  2. The analysis of proxy redundancy, initialization, and online proxy selection is useful beyond the specific method. The paper also acknowledges cases where the gains diminish or do not improve the strongest baseline, which makes the presentation more credible and appropriately cautious.
Critical weaknesses
  1. The novelty framing needs clarification. The paper presents combining proxy scores and architecture encodings in one surrogate as a distinguishing contribution. However, NAS-Bench-Suite-Zero already evaluates XGBoost predictors using proxy values together with architecture encodings. The authors should distinguish ZAPS from this prior setup and compare directly. The stronger novelty may lie in ProxyFit, initialization, and active selection.
  2. The NAS-Bench-101 comparison omits Local Search and BANANAS because of difficulties defining mutations or neighborhoods for variable-topology graphs. This leaves an incomplete comparison on the harder space. ProxyFit also uses ground-truth accuracies across the full space for offline calibration, so transferred-proxy results do not fully replace a target-label-free comparison. A baseline that concatenates all proxy features with the architecture encoding would help isolate whether ProxyFit and the active loop provide gains beyond straightforward feature aggregation.
Questions for authors
  1. How does ZAPS compare against an XGBoost surrogate that concatenates all proxy features with the architecture encoding, as in earlier proxy-augmented predictor work? Does ProxyFit and the active loop still provide a clear gain?
  2. Can the authors provide a principled NAS-Bench-101 comparison for BANANAS or Local Search, or additional evidence that the ordering is not driven by baseline and projection choices?
  3. How does ProxyFit's full-space ground-truth calibration affect the interpretation of the transferred-proxy and target-label-free experiments?
Reviewer_kCbbOpenReview ↗
Rating
1
Confidence
3
Informativeness
2
Presentation
1
Soundness
1
Read review
Paper summary

The method proposed in this paper is ZAPS, a zero-cost proxy score algorithm for neural architecture search (NAS). It selects proxies, constructs a structured initial sample, and uses an ensemble with uncertainty-guided acquisition to choose further architectures for evaluation. Experiments on tabular benchmarks report improved recovery of high-performing architectures and greater stability than the selected baselines, while improvements in finding the single best architecture are less pronounced.

Critical strengths
  • The paper addresses an important research direction.
  • The ablations are somewhat sound for the given approach.
Critical weaknesses
  • The presentation of this paper is catastrophic and heavily AI-written and needs serious rewrite before re-submission.
  • The novelty of the paper is very limited. Section 5 of NAS-Bench-Suite-Zero already combines zero-cost proxies with architecture encodings in an XGBoost predictor and integrates these features into BANANAS and NPENAS, including experiments with a budget of 200 architecture evaluations. This directly contradicts the claim that prior work has not combined both signal types within a surrogate-based search framework.
  • Related work is heavily outdated, most papers saturate around 2019-2021.
  • The headline results do not consistently respect the claimed target-evaluation budget.
  • Large-scale datasets like ImageNet are missing.
  • The only comparison with other methods is Table 1, the rest is essentially only ablation study without deeper, novel insights.
Questions for authors
  • How does your approach compare with newer methods (>2021)?
  • How would you defend novelty against NAS-Bench-Suite-Zero ?
  • How does your approach compare with other methods on larger datasets like ImageNet?
Reviewer_MgkWOpenReview ↗
Rating
1
Confidence
3
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

In this paper, authors propose ZAPS, it combines supervised proxy selection, PROXYFIT, hybrid K-means initialization, bootstrapped online proxy re-selection, and an XGBoost ensemble using proxy ranks and architecture encodings, with UCB-based acquisition. Its objective is to recover many of the true top-100 architectures within a limited evaluation budget. Experiments cover NAS-Bench-201 and a 9781architecture subset of NAS-Bench-101. The reported advantage primarily concerns top-set recovery rather than identifying a better single architecture.

Critical strengths
  1. The reported top-100 recovery gains over the included baselines are substantial at (B=200), i.e., 52.3% versus 35.4% on CIFAR-10 and 65.8% versus 50.5% on CIFAR-100. Evaluation over 200 seeds, with variability and effect sizes reported, provides useful evidence about consistency within the tested setting.

  2. The ablations help distinguish the effects of initialization and feature fusion. Cross-dataset transfer is also encouraging, it transferring the CIFAR-10 proxy subset retains 62.0% recovery on CIFAR-100 at (B=200), compared with 65.8% under target calibration. This supports further investigation of reusable proxy selection.

Critical weaknesses
  1. The central novelty claim is contradicted by directly relevant prior work, e.g., NAS-Bench-Suite-Zero, the paper supplying the benchmark data already combined zero-cost proxies and architecture encodings in XGBoost predictors, integrated them into BANANAS/NPENAS, and evaluated search with 200 architecture evaluations. Thus, the claimed first fusion of these signals in a budget-constrained surrogate is incorrect.

  2. The headline comparison does not establish the claimed strict-budget advantage. PROXYFIT accesses accuracies across the entire target space before the counted search begins. Returning only six proxy names does not eliminate that label dependence. Transfer experiments mitigate this concern, but do not validate the target-calibrated CIFAR-10 headline result. The main comparison should use source-only calibration or charge calibration labels to the budget.

  3. The “without PROXYFIT” ablation changes only initialization while retaining online selection, so it does not establish the necessity of the full selection mechanism. The NAS-Bench-101 evidence is additionally limited by the restricted candidate pool, projection-modified REA, and omission of BANANAS. The size of the full benchmark does not establish the difficulty of this restricted evaluation.

Questions for authors
  1. Can authors explain how does ZAPS compare with the proxy-augmented XGBoost/BANANAS approach from NAS-Bench-Suite-Zero and with ProxyBO under matched evaluation budgets and proxy access?

  2. What is the CIFAR-10 result when proxy selection uses no target accuracy labels? Can authors specify the calibration and hyperparameter-selection data, and report the cost of computing proxies over the candidate space rather than only the cached-benchmark runtime?

  3. Algorithm 2 returns the top-k proxies by vote count without rechecking redundancy or requiring a positive vote count. Can this reintroduce rejected or mutually redundant proxies?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.