acceptodds
Under review as a conference paper at ICLR 2027

A Verifiable Environment for Agents That Design Neural Architectures

Abstract

Reinforcement learning and agent evaluation both need a verifier: a function that scores a candidate cheaply, with no human in the loop. While software development relies on compilers and mathematics on automated answer checkers, neural architecture design has lacked a deterministic, public verifier. We present an environment in which an agent designs a neural network by editing a typed graph, and a deterministic verifier grades the result in under a millisecond against structural, budget, and serving-cost constraints. No human and no LLM takes part in scoring. The environment includes a procedural task generator (ten families, with satisfiability and non-vacuity checks and red-teamed anti-gaming rules), a difficulty-calibration harness, and a grounding study. In that study, across architectures, the verifier blocks every corrupted graph (), no blocked graph survives a real PyTorch forward pass (), and every clean graph runs (). The to health score is a validity margin, partly a size proxy, and not a quality ranking. We report four findings. As a repair signal, verifier feedback lifts claude-sonnet-4-6 from to , grok-4 from to , and deepseek-chat from to ; the lift concentrates on arithmetic defects, while cross-branch topology is largely unfixed. As a correction to how design agents are scored, a rubric that does not propagate shapes rates the same models and points higher than they are. As a training-data generator, fine-tuning a 1.5B model on verified pairs lifts pass@1 from to on held-out tasks from the same ten families, and GRPO steps on top reach (). Under a stricter rubric the same recipe runs , and a 7B policy starts lower than the 1.5B (, because it is more verbose) and ends at . Transfer is not general: four families held out of training collapse to . A contamination audit we ran on ourselves shows the naive protocol would have reported . As ground truth, eleven LLM reward models have a false-positive rate on blatantly broken designs (ten of eleven when re-measured), yet approve to of near-miss designs that are one edit from correct. The verifier is on both tiers. The same function serves as a leaderboard, an RL gym, and a verified-data generator. We release all of it.

Primary area:
datasets and benchmarks
Keywords:
verifiable rewards, benchmark, LLM agents, neural architecture design, reinforcement learning environment, reward models, LLM-as-a-judge, synthetic data generation, procedural task generation, evaluation methodology

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_3CtXOpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
3
Soundness
2
Read review
Paper summary

This paper presents a deterministic verification environment for LLM agents that design and modify neural architectures. The framework combines typed graph representations, structural validation, procedural task generation, and verifier-guided training. The authors evaluate the environment through iterative repair, SFT/GRPO training, and LLM reward-model auditing.

Critical strengths

The paper addresses a practically relevant problem and provides a relatively comprehensive experimental evaluation. The deterministic verifier is efficient, and the authors report some useful findings regarding structural validation, feedback-based repair, and the limitations of LLM reward models. The reproducibility efforts are also appreciated.

Critical weaknesses

My main concern is the limited technical novelty of this work. Most components, including graph-based architecture editing, structural validation, and verifier-guided training, are based on existing techniques. Related studies such as NADER and GraphIR have already explored similar ideas, but the paper does not clearly explain its technical differences or provide direct comparisons. I also find the experimental evidence insufficient to establish the unique value of the proposed framework. The evaluation mainly focuses on satisfying predefined structural constraints rather than demonstrating meaningful architecture-design capabilities. The improvements from verifier feedback are not compared against equal-budget self-correction baselines, making it difficult to isolate the verifier's contribution. Furthermore, the reward-model evaluation relies on negative examples defined by the same verifier, which limits the strength of the claimed verification advantage. More importantly, performance on unseen task families drops from 22.5% to 0% after fine-tuning, raising serious concerns about generalization. Overall, while the framework is practically useful, I am not convinced that it offers sufficient technical novelty or demonstrates clear advantages over existing approaches.

Questions for authors
  1. What is the key technical contribution beyond integrating existing graph validation and agent training techniques?
  2. How does verifier feedback compare with self-correction under the same inference budget?
  3. Can the authors provide stronger comparisons with related methods and explain the severe generalization failure on unseen task families?
Reviewer_ZyGkOpenReview ↗
Rating
1
Confidence
2
Informativeness
1
Presentation
2
Soundness
2
Read review
Paper summary

This paper introduces a verifiable environment for training and evaluating agents that design neural network architectures. Agents construct or modify typed architecture graphs, while a deterministic verifier checks structural validity, shape compatibility, and resource constraints without executing the networks. The environment includes ten procedurally generated task families and supports benchmarking, verifier-guided repair, supervised fine-tuning, and reinforcement learning.

Experiments show that verifier feedback substantially improves architecture repair rates across three LLMs. Training on verified examples, followed by GRPO, also improves pass@1 for a 1.5B model on held-out tasks from the same families. The authors further demonstrate that shape-blind evaluation overestimates success rates and that LLM reward models frequently accept subtly invalid architectures.

Critical strengths
  1. The paper provides a complete environment for architecture editing, with deterministic verification, procedural task generation, and interfaces for agent training and evaluation. The experiments demonstrate its usefulness for verifier-guided repair and generating training data.

  2. The evaluation is relatively thorough. In particular, the authors investigate verifier correctness against actual PyTorch execution, identify contamination in their original train/test split, and report negative results on generalization to unseen task families. These analyses help establish the scope of the reported improvements.

Critical weaknesses
  1. The advantage of the proposed verifier over existing verification methods is not established. The main motivation is to provide cheap and reliable verification of neural architectures. However, it is unclear why a custom static verifier is necessary instead of lightweight PyTorch smoke tests or framework-native shape checking using meta tensors or FakeTensor. These alternatives use the underlying framework's operator semantics and could avoid maintaining a separate, incomplete implementation of shape constraints. Indeed, Appendix C reports that all 12 wrong-axis normalization graphs pass the proposed verifier but fail during a PyTorch forward pass. The paper reports sub-millisecond verification time, but provides no comparison of latency, throughput, or correctness against these alternatives. Without such a comparison, it is difficult to assess the benefit of the proposed verification approach.

  2. The benchmark captures a limited aspect of neural architecture design, and the learned capabilities do not generalize beyond the generated task families. The verifier checks whether an architecture satisfies structural and resource constraints, but not whether it is effective for a learning task. The authors also show that the health score does not correlate with actual training performance. More importantly, the training results are largely restricted to the procedural families used to generate demonstrations. When four families are held out, fine-tuning reduces performance from 22.5% to 0%. This suggests that the models primarily learn family-specific editing patterns rather than general architecture design capabilities. These limitations weaken the significance of the reported training improvements.

Questions for authors
  1. Comparison with existing verification methods: Could the authors compare the proposed verifier against lightweight PyTorch smoke tests and framework-native shape checking (e.g., meta tensors or FakeTensor), in terms of runtime, throughput, and correctness? What practical advantage does the custom verifier offer over these alternatives, especially given the missed runtime failures reported in Appendix C?

  2. Verification coverage: How does the verifier handle the full vocabulary of 182 layer types? Are shape and execution constraints implemented for every supported type? How frequently do invalid architectures pass verification when tested using actual PyTorch execution, beyond the manually constructed corruption families?

  3. Generalization: The performance collapse on unseen task families suggests substantial reliance on family-specific editing patterns. Can the authors further investigate whether the trained policy learns reusable architecture-editing skills, for example by evaluating unseen combinations of familiar architectural components and constraints?

  4. Benchmark objective: Since the verifier does not assess predictive performance, to what extent does success on the benchmark reflect neural architecture design rather than satisfying predefined graph constraints? Could the authors provide evidence that improvements on the benchmark translate to better performance on more realistic architecture design tasks?

Reviewer_vGKVOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper introduces a verifiable environment for agents that design neural network architectures. Instead of asking an agent to produce unconstrained code, the environment represents an architecture as a typed directed graph and restricts the agent to structured graph-edit actions. A deterministic verifier checks connectivity, shape propagation, attention divisibility, parameter budgets, KV-cache and latency constraints, and other task-specific requirements. The same verifier is used for evaluation, repair feedback, reinforcement-learning reward, and generation of verified supervised-training pairs. Experiments include grounding the verifier against real PyTorch execution, measuring verifier-feedback repair, comparing shape-aware and shape-blind rubrics, training Qwen2.5 policies with verified SFT data and GRPO, and auditing LLM reward models on near-miss architectures. The results suggest that deterministic verification improves repair and training within the generated task distribution, while LLM judges frequently accept subtle near-miss designs.

Critical strengths
  1. Reliable deterministic verification. The typed-graph representation enables fast, reproducible checks that are difficult to obtain from free-form code or LLM judging. In the grounding study, the released verifier blocked all 184 corrupted graphs while accepting all 80 clean graphs, providing useful evidence that the checks correspond to several real execution failures.

  2. A useful unified environment for evaluation and training. The same verifier supplies task scores, failure messages, RL rewards, and verified SFT pairs. Verifier feedback substantially improves several frontier-model repair rates, and verified training data improves in-distribution performance, showing a practical use case beyond offline architecture checking.

Critical weaknesses
  1. Agents edit a fixed typed graph using a constrained action vocabulary rather than writing, debugging, and executing ordinary PyTorch code. The grounding study checks whether selected graphs can be translated into PyTorch and run, but it does not establish that the learned design behavior transfers to code-level architecture development.

  2. The training experiments rely mainly on Qwen2.5-1.5B and 7B, while the strongest claims concern agent training and general architecture-design ability. The held-out-family experiment shows that performance can collapse from 22.5% to 0% when four task families are excluded from training.

Questions for authors
  1. How does the environment translate an agent’s final typed graph into ordinary PyTorch code, and what guarantees or experiments show that the translation preserves the intended architecture and remains executable?

  2. Why are the main training experiments centered on Qwen2.5-1.5B and 7B, and how do the results change with newer and stronger open-weight models? In particular, can the authors provide evidence that the observed gains are not primarily learning task-family-specific edit formats?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.