acceptodds
Under review as a conference paper at ICLR 2027

Evolving Agent Workflows through Regular Expression Edits

Abstract

Agent workflows combine language-model calls with verification, retries, and tools, but existing design methods often change control flow and module implementations together. Their search results therefore give limited evidence about which structural edits improve performance. We introduce reGROW, which fixes module implementations and searches a regular grammar over typed module calls. A compiler checks each candidate’s validity and resource bounds before execution. Two indices computed from the expression describe processing opportunity and control exposure. Starting from one reasoning call, reGROW applies explicit structural edits, uses a pretrained TabPFN predictor to prioritize evaluations, and uses paired measurements to decide which edits survive. An independent confirmation set selects the lowest-exposure finalist within a quality tolerance and empirical deployment-cost budget. Across seven benchmarks, reGROW improves average performance by 0.8% relative to the strongest reported baseline. Every retained edit carries a measured parent-child comparison.

Primary area:
foundation or frontier models, including LLMs
Keywords:
LLM agents, agentic workflows, workflow optimization, syntax-guided search, structural complexity, value-based candidate selection

Reviews

Among:
Shape:
1234this paper, Bottom 2%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.33 over 3 reviews, above 2% of the others.
Reviewer_vVw2OpenReview ↗
Rating
1
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes reGROW, which optimizes agentic workflows through regular-expression edits while keeping module prompts, tools, and implementations fixed. A compiler checks candidate interfaces and execution bounds, while TabPFN predicts gains from historical edit outcomes to prioritize evaluations. Workflows before and after an edit are compared using the same tasks and random seeds. Finally, candidates are evaluated on an independent confirmation set, and the workflow with the lowest control exposure is selected from those satisfying quality-tolerance and cost constraints. Results across seven benchmarks are comparable to those of strong existing methods, and the ablations provide support for paired-gain prediction and composite edits.

Critical strengths
  1. With module interfaces fixed, routing, retries, and recovery have explicit execution rules. Field dependencies and call limits can be checked before any model invocation. This representation makes local edits easier to inspect and is useful for workflows with strict execution budgets.
  2. The reported results show that searching over workflows with fixed modules can still achieve competitive performance on reasoning, planning, and interactive tasks. Paired comparisons before and after edits are also appropriate, making the effects of individual modifications easier to assess.
Critical weaknesses
  1. MASS already reuses optimized module prompts during topology optimization, while MermaidFlow uses typed graph representations and constrained edits with candidate validation. reGROW goes further by fixing module implementations and using bounded control structures expressed through regular expressions. However, the two central ideas—separating module content from structure and checking candidates before execution—have clear precedents. Paired evaluation and TabPFN-based scheduling make the search procedure more complete, but I still view the overall method as an incremental extension of existing approaches. The additional research value of the regular-expression representation is not sufficiently clear.
  2. The default selection rule permits a preference for lower control exposure within a confirmation-score tolerance of 0.05. Control exposure sums the counts of processing actions, control decisions, and recovery sites with equal weights. I do not see why reducing this count warrants accepting a lower score. Actual execution cost is already measured separately, and the paper does not use the selected workflows to demonstrate concrete benefits from this preference. It remains unclear what useful guidance the criterion provides for choosing among verification, retry, and routing structures.
Questions for authors
  1. What practical benefit is reducing control exposure intended to provide? Could you use a final selection from an existing search run to illustrate how it differs from selecting by highest quality or lowest actual execution cost?
  2. In the ablations, does “keeping the candidate archive fixed” mean that each method starts from the same initial archive, or that all methods are compared using the same precollected archive? This distinction affects whether the results should be interpreted as evidence of search effectiveness or of the ability to select among existing candidates.
Reviewer_96QnOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper introduces reGROW, which searches bounded regular expressions over fixed typed agent modules to isolate changes in workflow structure. It combines static checking, structural opportunity/exposure indices, TabPFN-guided edit selection, paired evaluations, and independent confirmation that favors lower-exposure finalists within a quality tolerance. Experiments on seven benchmarks report an average score of 0.623 versus an externally reported AgentSwift baseline of 0.618.

Critical strengths
  1. A controlled representation for structural comparisons. Fixing module implementations while editing typed, bounded control structures makes parent-child comparisons interpretable and permits validity checks before expensive execution.

  2. Careful separation of prediction, selection, and evaluation. Predictions only prioritize evaluations, while paired measurements and independent confirmation govern subsequent decisions; the explicit acknowledgment of heuristic uncertainty and empirical cost feasibility makes the claims more credible.

Critical weaknesses
  1. The quantitative evidence is insufficient to establish the reported advantage. Table 1 compares against external scores without reporting concrete per-benchmark call budgets, split sizes/identities, or model snapshots, leaving comparability and the robustness of the 0.005 average improvement unresolved; Table 2 supplies only aggregate point estimates without uncertainty or per-benchmark results.

  2. The central structural contribution is not empirically validated. The submission presents no actual selected workflow expressions, measured edit ancestry, selected exposure values, or quality-exposure tradeoffs, and lacks comparisons against uniform search or ablations of structural stratification and minimum-exposure selection, so the benefit of these choices over simpler bounded search remains unclear.

Questions for authors

Can you provide representative selected expressions and their measured parent-child gains, together with selected exposure and execution costs? What evidence shows that structural stratification and minimum-exposure selection improve the achieved quality-complexity tradeoff relative to simpler selection rules?

Reviewer_fnnpOpenReview ↗
Rating
1
Confidence
4
Informativeness
1
Presentation
1
Soundness
1
Read review
Paper summary

The paper proposes reGROW, which fixes the implementation of each module (reasoning G, tool use T, verification V, revision R, routing S, candidate generation/aggregation Pn, decomposition Dn) and searches only over control structure. A workflow is a regular expression over typed module calls (concatenation, union, optional, bounded repetition), compiled into a finite-state controller whose validity and resource bounds are checked before execution. Two expression-derived indices, processing opportunity (A) and control exposure (E), describe each candidate. Search starts from a single G, ranks candidate edits with a pretrained TabPFN predictor of the paired parent→child gain, keeps edits based on paired measurements and racing, and finally selects the lowest-exposure finalist within a quality tolerance on a separate confirmation set. With GPT-4o-mini, reGROW reports an average of 0.623 over seven benchmarks versus 0.618 for AgentSwift.

Critical strengths
  1. Clean problem setting. Separating control structure from module implementation is a well-motivated way to make structural edits attributable, and the compiler check rejects illegal candidates before any model call.
Critical weaknesses
  1. The empirical results do not support a performance claim. The average gain over AgentSwift is 0.005 absolute (0.623 vs. 0.618), reported in the abstract as "0.8% relative". reGROW leads only on ALFWorld (0.831 vs. 0.806) and M3Tool (0.670 vs. 0.634) and is below AgentSwift on the other five benchmarks. Every ablation variant in Table 2 (0.606–0.615) also falls below AgentSwift. Moreover, all baseline rows are copied from AgentSwift's Table 1 (Table 1 caption: "Baseline rows are external reference values reported in AgentSwift Table 1"), so splits, prompts, evaluation scripts, and model snapshots are not guaranteed to match. The paper states that "For a head-to-head rerun, we set reGROW's model-call cap to the baseline's call allocation and report both counts", and Table 7 says "Each benchmark assigns the same total search budget to all compared methods", yet no rerun result and no call count appears anywhere in the paper or appendix. There is therefore no controlled comparison supporting the method.
  2. The claimed interpretability benefit is not demonstrated. The motivation is to show which structural edits improve performance, and the abstract states that "Every retained edit carries a measured parent-child comparison". However, no selected workflow expression or growth trace is reported for any benchmark; the only expressions in the appendix are the regular patterns of classic workflows (Table 8) and 65 expressions used to illustrate A–E coordinates (Figure 3). The A and E indices are presented as a contribution, but the paper states that "Neither index estimates quality, execution cost, or failure probability", and no experiment shows that selecting the lowest-E finalist yields any measurable benefit. The propositions (a Hoeffding-plus-union-bound confirmation bound, termination on a finite set, and a remark that the confirmation bounds "do not justify adaptive pruning or prediction-based stopping") do not provide guarantees about the search itself.
Questions for authors
  1. (Additional experiment) Can you report a head-to-head comparison with AgentSwift and AFlow rerun under the same split, GPT-4o-mini snapshot, and call budget?
  2. (Clarification) How many model calls and tokens did the search consume per benchmark?
  3. (Clarification) What regular expression was finally selected for each benchmark, and what paired gain did each retained edit produce?
  4. (Additional experiment) On the test set, how does the lowest-E selection compare with simply selecting the highest-quality confirmed finalist?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.