NICE-CV: Neural Inference- and Cost-Efficient architecture search for Computer Vision models with online proxy calibration

Efficient neural architecture search (NAS) has two coupled objectives: the discovered network must maintain strong performance while achieving low inference latency, and the search must cost training time selectively. Zero-cost proxies reduce search cost, but their reliability varies across tasks and training fidelities, while fixed proxy combinations cannot adapt to new validation evidence. Therefore, we introduce NICE-CV, a multi-fidelity NAS method for inference- and search-efficient architecture discovery. To the best of our knowledge, NICE-CV is the first multi-fidelity NAS method to treat zero-cost proxy reliability as an online search state, continuously updating it from purchased validation evidence to guide architecture–fidelity decisions that jointly account for measured inference latency and wall-clock search cost. Concretely, NICE-CV converts heterogeneous proxy scores into comparable ranks, updates proxy reliability from pairwise validation comparisons, and combines the calibrated proxy evidence with cross-fidelity observations to guide subsequent search decisions. Each legal start or promotion is scored by posterior expected hypervolume improvement per predicted search second, while measured inference latency is incorporated directly into the search objective. The resulting search is invariant to monotone transformations of proxy scores, and its online reliability learner enjoys a no-regret guarantee relative to the best fixed proxy in hindsight. Across five complete tabular NAS benchmark families, NICE-CV achieves the highest mean normalized hypervolume ratio (HVR) among executable search methods once online feedback becomes available. On recorded NAS-Bench-201 fidelities, it achieves the highest HVR among the evaluated multi-fidelity methods at 0.9360 while using substantially less search time than full-fidelity baselines. Training-based searches across classification, segmentation, and detection further show that NICE-CV achieves the highest mean HVR on all four tasks; compared with the fastest baseline on each task, it requires only marginal additional search time while consistently recovering higher-quality architecture sets. These results show that continuously adapting inexpensive proxy guidance to task-specific feedback can improve both inference-efficient architecture discovery and wall-clock-efficient search, yielding stronger quality–latency tradeoffs under limited search budgets.