- Rating
- 2
- Confidence
- 3
- Informativeness
- 1
- Presentation
- 2
- Soundness
- 1
Read review
Paper summary
This paper proposes CORE-Flow, which extends AFlow-style automated agentic workflow optimization using experience from evaluated edits. Each evaluated parent-child transition is stored as a "state-diagnosis-action-effect" experience, with the effect defined as the child's validation score minus the parent's. A prompt and a validator restrict each candidate edit to one workflow component, allowing the measured effect to be attributed to that edit. An LLM writes the diagnosis after observing the effect.
Each round has three steps:
- Sample a parent from a scored candidate pool using a softmax over validation scores.
- Retrieve the top-K experiences by embedding similarity. Retrieval uses a one-sentence description of the workflow's information flow generated by an LLM.
- Propose B edits to the same parent and retain the best child.
The method is evaluated on established benchmarks with GPT-4.1-mini as the optimizer and GPT-5-mini as the executor. A single tracked MATH run shows lower token cost and wall-clock time than both baselines.
Critical strengths
- The paper addresses a practical problem: LLM-driven workflow optimization is costly and repeatedly diagnoses similar workflow states. Reusing measured edit outcomes is an intuitive response to this repeated work.
- The method is simple and training-free, and it can be integrated into existing AFlow-style pipelines.
- Under the reported setup, CORE-Flow performs better than the compared baselines on average.
Critical weaknesses
-
The contribution is incremental. CORE-Flow largely follows AFlow, which already supplies past edits and their outcomes, and adopts JudgeFlow-style actions. The additions are modest, and related work such as GEPA, MASS, and ERM is not discussed.
-
The claims exceed the evidence. The “counterfactual” effect is a validation score difference, without reported subset size, evaluation count, or variance. The optimizer still uses current and LLM-written diagnoses, so the method supplements model-based diagnosis rather than moving beyond it.
-
The experiments are limited. They cover six established benchmarks, one optimizer-executor pair, and no agentic or tool-use tasks. Three benchmarks already reach scores of 94 to 98. All analyses beyond Table 1 use only MATH; ablations do not isolate single-component edits or post-hoc diagnosis. Cross-run or cross-task reuse and standard deviations are not reported.
-
The efficiency evidence rests on one MATH run, with unclear accounting for optimizer, diagnosis, and embedding calls.
References:
Questions for authors
-
How does the experience pool differ from AFlow's tree-structured experience? Is “Optimization only” in Table 3 equivalent to AFlow, and does “+Experience retrieval” retain AFlow's native experience?
-
What is ? Is each candidate evaluated once? How often does re-evaluating the same edit reverse the sign of its effect (Eq. 13)?
-
Which calls do the costs in Tables 2 and 4 include? Are reasoning tokens counted, and what prices are assumed? How many executor calls does each method make per round?
-
Does experience transfer across runs, tasks, or executors? Does top-k retrieval outperform giving the optimizer all past experiences?
-
Do the gains hold on harder agentic benchmarks, such as tool use or software engineering?