- Rating
- 2
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper introduces equiNAS, a typed DSL for architecture search over equivariant neural networks. Architectures are represented as computation graphs with semantic types encoding representation-theoretic information, and proposed modifications are expressed as typed patches that undergo static verification before training. The framework supports both deterministic and LLM-based mutation proposers.
The authors evaluate equiNAS primarily on EquiformerV3. They report improved proposal validity over untyped editing, a 1.87× training throughput improvement from graph-based compiler optimizations, and lower mean validation and test MAE on QM9. Additional experiments examine long-horizon optimization and transfer to other molecular properties.
Critical strengths
-
The typed DSL provides a useful interface for enforcing representation and dimension constraints during architecture search. The matched-intention experiment shows a clear improvement in proposal validity.
-
The implementation is carefully documented, including compilation checks, architecture and protocol hashes, and mutation histories. The reported 1.87× steady-state training speedup also demonstrates a practical benefit of the explicit graph representation.
Critical weaknesses
-
Limited methodological novelty and generality. The NAS procedure uses standard evolutionary sampling over nine predefined mutation families designed for EquiformerV3. The main architecture results come from the deterministic chooser, while the LLM selects from an existing catalogue without a demonstrated advantage over conventional search. Type-based NAS and equivariance-aware NAS have both been studied previously (e.g., NeuralArTS; Maile et al., 2023). It is unclear what new search capability the typed DSL provides beyond verifying predefined mutations. The proposal-validity comparison also does not isolate type checking from restricting the action space, as acknowledged in Appendix J.
-
Insufficient evidence for architecture improvements. The main comparison does not cleanly separate architectural changes from implementation differences. At 20k steps on seed 201, the unmodified DSL-compiled EquiformerV3 achieves 0.186 validation MAE, outperforming TypeEvo at 0.189 (Table 5). However, Table 3 compares TypeEvo against the upstream implementation at 0.191. The other seeds and long-horizon experiments lack corresponding DSL-compiled baseline controls. Moreover, the long-horizon improvement is based on a single seed, and the three-seed test improvement is small relative to the reported variance. These results do not yet convincingly establish an advantage from the discovered architecture.
Questions for authors
-
Can the authors provide comparisons against the unmodified DSL-compiled EquiformerV3 under the same protocol across all seeds and at longer training horizons? The existing comparison in Table 5 reverses the improvement reported for seed 201 in Table 3.
-
How are the primitive type contracts and mutation families constructed and verified? Can the type system certify architecture modifications outside the nine predefined families, and what would be required to extend it to another equivariant architecture?
-
What evidence supports the benefit of the LLM proposer over conventional evolutionary search? In particular, how would GLM-5.2 compare against deterministic or random action selection under the same 88-action catalogue and evaluation budget?