- Rating
- 3
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
ZERO represents mathematical research procedures as executable workflows of agents, shell commands, and researcher interactions. A meta-agent constructs or modifies these workflows using the task description, researcher input, and retrieved examples. The paper studies packing, coloring, Lagrangian relaxation, and extensions of existing research systems. It reports new constructions and counterexamples, supported by exact checks and, for some results, Lean formalization. The main contribution is a framework for organizing and reusing research procedures across tools and agents.
Critical strengths
-
The mathematical results give concrete evidence that the system can support useful research. These include the optimal packing for the previously unresolved n = 15 instance, improved coloring constructions, and counterexamples answering two questions on Lagrangian relaxation (Sections 3.1–3.3). The explicit certificates and reported Lean checks strengthen the evidence for these results. These outcomes are a substantial part of my positive assessment.
-
The framework supports several stages of research within one executable procedure, including search, proof development, deterministic checks, and human decisions. Section 3.4 gives concrete examples of adapting existing systems to add formal verification and replace commercial solvers. This demonstrates practical value for researchers who need to combine tools with different interfaces and validation requirements.
Critical weaknesses
-
The methodological contribution is not sufficiently clear. Sections 2.1–2.3 explain the workflow primitives, skills, and retrieval component, but give limited detail on how the meta-agent chooses and revises a research procedure. For example, Section 2.3 describes the case corpus and BM25 retrieval without showing how retrieved cases affect important workflow decisions. The paper also gives little analysis of which design choices help across the studies. This makes it difficult to identify the new, reusable knowledge contributed by the framework beyond the implementation and individual results.
-
The evaluation does not clearly isolate the benefit of ZERO’s workflow construction. Table 1 provides a useful comparison on one packing instance, but does not report repeated-run success rates or separate the effects of orchestration and task-specific search improvements. In Section 3.2, ZERO receives GPT Pro guidance, while it is unclear whether the comparison methods have equivalent resources. The remaining studies mainly demonstrate capabilities and extensions. This limits the evidence that the reported gains will carry over to new tasks under comparable conditions.
Questions for authors
-
Could you trace one reported case from the initial workflow to the successful result, identifying the decisions made by the meta-agent, changes prompted by execution feedback or retrieved cases, and decisions supplied by the researcher? Which workflow change was important for making progress? This would clarify the contribution of automated workflow design.
-
For Table 1 and the planar coloring comparison, please clarify which models, tools, starting artifacts, researcher input, and budgets were shared across methods. Is there evidence from the existing runs or a focused comparison that separates the benefit of workflow construction from the underlying coding agents and task-specific implementation changes? This would affect my assessment of how broadly the reported benefits apply.
2 comments
Thank you for the helpful response. The packing example clarified how retrieved cases informed the initial workflow and which decisions came from the meta-agent versus the researcher. I also appreciate the additional ablations examining the role of the workflow skills.
I maintain my rating of 3. With more concrete implementation details, I believe this work could serve as a useful tech report and contribute valuable practical knowledge to the community.
We thank the reviewer for their positive assessment and constructive questions.
Weakness 1: Methodological contribution
Our contribution is making the research procedure itself programmable, i.e., choosing and combining models, tools, verification methods, and reusable workflows, rather than fixing a particular agent harness or strategy.
We deliberately use simple BM25 retrieval with the
source-observatoryskill to identify relevant methods. ZERO can incorporate existing systems or modify their internal stages, representing procedures from evolutionary search to prover–verifier loops and FirstProof multi-agent systems (Appendix F).Workflows can also be reused: in Section 3.1.2, ZERO incorporated its successful n=15 packing workflow into a higher-level research procedure, discovering constructions for rectangles, cuboids, triangles, and tetrahedra without prescribing the instances.
Weakness 2: Evaluation and ablations
We conducted additional ablations keeping the underlying coding agent, runtime, tools, and task inputs fixed:
/design-workflow: The agent failed to construct an executable packing workflow or improve the published planar coloring result./augment-system: The agent only partially reproduced AutoOPT, failing to adapt SCIP's branch-and-bound procedure to achieve the required numerical accuracy.We believe these ablations provide more direct evidence for the contribution of ZERO's workflow mechanisms compared to the end-to-end comparisons from before.
Question 1: Packing workflow trace
For n=15, the researcher provided only the objective and existing Java search code, without prescribing a procedure.
Retrieval suggested SCIP-based geometric optimization [1] and verification-and-refinement [2]. The SCIP case study was particularly useful in suggesting that an initial local search could identify promising candidates before finer optimization. ZERO also considered AlphaEvolve-style approaches but deemed them less suitable for the discontinuous objective.
The meta-agent constructed a multi-round workflow with agents exploring pairings, optimizing solver formulations, improving efficiency, and synthesizing findings. A key choice was pursuing independent searches in separate contexts (two hours per agent using
/goal) before synthesizing results.During execution, one agent discovered promising rectangle pairings, which another passed as fixed inputs to CP-SAT. This yielded the optimal side length 50, illustrating the value of separating exploratory search from finer optimization.
The workflow itself remained unchanged during the run, the important methodological choice was its initial design, while adaptation occurred within the agents' search strategies.
Subsequently, the researcher specified a higher-level procedure, select problems, prove bounds, and search for constructions, without specifying the instances. Using
/implement-workflow, ZERO implemented this procedure, reusing and adapting the original packing workflow to new geometries (Section 3.1.2).Question 2: Comparison conditions
The comparisons shared task information and starting artifacts, while allowing different methodological choices.
Packing: All methods had 24-hour wall-clock budgets. ZERO used GPT-5.5 and achieved side length 50, versus 51 for the agent-based baselines. Token usage differed as reported in Table 1.
Planar coloring: ZERO and both Codex baselines used GPT-6 Astra as their underlying coding model, with the same task information and coding tools. ZERO achieved 3.49% removed area versus 3.57% for both baselines.
[1] Berthold et al., Global Optimization for Combinatorial Geometry Problems Revisited in the Era of LLMs. https://arxiv.org/abs/2601.05943
[2] Huang and Yang, Winning Gold at IMO 2025 with a Model-Agnostic Verification-and-Refinement Pipeline. https://arxiv.org/abs/2507.15855