- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
INFLOWOP combines cost-guided workflow construction with local repair during execution. A shared estimator scores agent suitability and checks subtask conditions. The paper also introduces BRAID, which combines existing questions with distractor sources, and reports improvements across several backbones.
Critical strengths
Jointly considering task granularity and agent suitability is a useful direction. The consistent improvements over single-agent baselines suggest practical value in the resulting workflow construction.
Critical weaknesses
The methodological advance over existing workflow systems appears limited. AFlow (Zhang et al., 2025) already automates workflow search, while DyFlow (Wang et al., 2025) adapts workflows using intermediate feedback. The proposed cost-guided construction and local repair offer a limited extension, whose central rubric estimator is insufficiently specified. It remains unclear how scores are computed from task requirements, agent descriptions, and execution records, or why they should represent success probabilities without calibration. Using the same estimator to accept repairs also does not establish that those repairs improve final correctness. References
The empirical gains do not convincingly justify the overhead or establish an advantage over existing workflow methods, which are not directly compared. In-flow repair adds only 1.59 percentage points for Qwen3.5-9B, while Figure 9 shows roughly 20–30 times the single-agent token consumption for GPT-5-mini and GPT-5.4-mini. Matching turns does not resolve this resource imbalance. Furthermore, BRAID bundles questions and distractor sources without introducing dependencies between the questions. This tests source retrieval and multi-question processing, but does not substantiate the claimed need for interdependent multi-agent reasoning beyond single-agent capability.
Zhang et al. (2025). AFlow: Automating Agentic Workflow Generation. Wang et al. (2025). DyFlow: Dynamic Workflow Framework for Agentic Reasoning.
Questions for authors
How exactly is the rubric estimator implemented, and what supports interpreting its scores as success probabilities and using them to validate repairs without additional training?
What substantive capability does INFLOWOP add beyond existing workflow search and feedback-driven replanning, particularly given its token overhead?
What in BRAID requires interdependent reasoning rather than locating the relevant sources and answering each question separately?
4 comments
Author Response (2/2): Thank you for your valuable review and insightful feedback
3. Repairs and final correctness (W1 "does not establish that those repairs improve final correctness", Q1 "using them to validate repairs without additional training"). A repair is kept only if it scores strictly higher than the original on the same detector scale, so a tie or an unscoreable repair keeps the original. On final answers over the 16 arms of Tab. 1 with GPT-5.6-luna, a repair is accepted on 826 tasks, and on them the final score improves on 85.23% (Δ ≥ 0, Δ: the change from COALESCE alone) and worsens on 14.77% (Δ < 0). Accepted repairs thus improve the final score on the large majority of their tasks, without any training.
W2+Q2: Gains and token overhead
Thank you for your thoughtful comments.
1. The in-flow gain (W2 "In-flow repair adds only 1.59 percentage points for Qwen3.5-9B"). Pairing each task across COALESCE and COALESCE with in-flow optimization on Qwen3.5-9B, the +1.59 has a bootstrap 95% confidence interval of [+1.24, +1.94] and is significant on every benchmark (p < 0.01). It improves 93.72% of tasks (Δ ≥ 0) and worsens 6.28% (Δ < 0), at matched turns and without reference answers or re-runs. Best-of-N instead repeats the whole workflow four times and keeps the outcome with the highest graded score, yet it stays below INFLOWOP on every backbone (§E, Fig. 9).
2. Token overhead (W2 "roughly 20–30 times the single-agent token consumption", W2 "Matching turns does not resolve this resource imbalance", Q2 "particularly given its token overhead"). Across the five backbones of Fig. 9, INFLOWOP's tokens per task span 1.8× (339k to 608k), while the Single Agent's span 27× (12.7k to 347k). The ratio thus follows how the Single Agent's spending varies across backbones. We further give the Single Agent of Tab. 1 INFLOWOP's token budget on each task instead of its turn budget, on Qwen3.5-9B and on GPT-5.4-mini. Accuracy (%) over the 16 arms below shows that INFLOWOP stays far higher at the same token budget:
| Backbone | Single Agent, turn budget | Single Agent, INFLOWOP's token budget | INFLOWOP |
|---|---|---|---|
| Qwen3.5-9B | 17.38 | 15.24 | 25.13 |
| GPT-5.4-mini | 17.49 | 15.72 | 29.46 |
On GPT-5.4-mini, the Single Agent spends only 39.0k tokens per task against a budget of 339.0k, so a larger budget does not close the gap. With the same tools, AFlow and DyFlow also spend more tokens per task than INFLOWOP (W1+Q2). On Qwen3.5-9B, correction, including profile updates, takes 17.7% of the tokens and buys the +2.92 of INFLOWOP over COALESCE alone (Tab. 2).
W2+Q3: Interdependence in BRAID
Thank you for your insightful question.
1. What BRAID requires (W2 "without introducing dependencies between the questions", Q3 "What in BRAID requires interdependent reasoning"). Each BRAID task provides more sources than its questions need, without stating which source serves which question (§C.2). In a Distributed task, each of the questions has its own gold source among sources. So sources have to be assigned to questions jointly, since the source that answers one question answers no other. In an Anchored task, all questions are answered from one gold source among sources, so every part depends on first locating that source. Fig. 7 (§C.1) shows the effect: on the original datasets, a single agent is more accurate than a workflow an LLM constructs (54.37% vs. 42.35%), while after BRAID composition the single agent drops to 10.82% and the workflow reaches 48.99%.
2. Answering each question separately (Q3 "rather than locating the relevant sources and answering each question separately"). We add a single agent that answers each question of a task in its own run, with the same tools, skills, and sandbox, under matched token budgets. With Qwen3.5-9B on the 16 arms (Tab. 1), it reaches 19.39%, stronger than the Single Agent's 17.38% in Tab. 2, yet INFLOWOP's 25.13% is +5.74 higher, and higher in each of the 16 arms. Such a split still leaves each question to locate its own gold source among all the sources of its task, which is the routing that BRAID composition introduces.
References
[1] J. Zhang et al. AFlow: Automating Agentic Workflow Generation. 2025.
[2] Y. Wang et al. DyFlow: Dynamic Workflow Framework for Agentic Reasoning. 2025.
Thank you for your valuable review and constructive feedback. We sincerely hope our response and clarifications above can meaningfully address your remaining concerns, and we will make sure to apply them to our revisions to further enhance our paper quality.
Thank you for the thoughtful response and the effort put into the additional experiments. I appreciate these improvements, though my main concerns remain. I will keep my score of 2.
Author Response: Thank you for your valuable review and follow-up feedback
Thank you very much for your thoughtful follow-up and for taking the time to review our additional experiments and clarifications. We sincerely appreciate your recognition of our efforts and continued engagement with our work. We have sought to address the concerns raised in your review through additional experiments, analyses, and clarifications. If there are any specific points or concerns that you feel remain insufficiently addressed, we would be very grateful for the opportunity to provide further clarification. We sincerely appreciate your time and consideration.
Author Response (1/2): Thank you for your valuable review and insightful feedback
Thank you for your careful review and constructive feedback, and for recognizing that our work is useful and our consistent improvements suggest practical value. We hope to further address your remaining concerns below.
W1+Q2: Advance over existing workflow methods
Thank you for your insightful comments.
1. Direct comparison (W2 "which are not directly compared", W1 "AFlow (Zhang et al., 2025) already automates workflow search"). We add AFlow [1] and DyFlow [2] from their official code. As released, both are text-in/text-out and cannot open BRAID's documents and images, so we run both with Qwen3.5-9B and the same tools, skills, sandbox, budget, and run limits as INFLOWOP, on five datasets of Tab. 1. Mean accuracy (%) per dataset and tokens per task below show that COALESCE alone is already higher than both methods on every dataset with fewer tokens, and INFLOWOP is the highest:
2. What INFLOWOP adds (Q2 "What substantive capability does INFLOWOP add beyond existing workflow search and feedback-driven replanning"). AFlow improves a workflow offline over graded tasks, and DyFlow re-plans its next steps from intermediate feedback. INFLOWOP adds two workflow-level capabilities beyond these. Before execution, one label-free cost decides the granularity of the workflow, the agent of each subtask, where a new specialist is needed, and creating the specialist that a subtask demands (§3.1). During execution, a fault is repaired at the subtask where it arises, every verified result is kept, and no reference answer, re-run of the workflow, or training is needed (§3.2). Results above show this substantive capability gap empirically: COALESCE alone already exceeds both methods on every dataset, and INFLOWOP exceeds the stronger of the two by up to +12.67. We will add this comparison and discussion to our revision.
W1+Q1: Estimator specification and repair validation
Thank you for your detailed questions.
1. Implementation (W1 "how scores are computed from task requirements, agent descriptions, and execution records", Q1 "How exactly is the rubric estimator implemented"). Our rubric estimator implements match without any LLM call (§3.1). A frozen sentence encoder (all-MiniLM-L6-v2) embeds a demanded text and a supplied text. An increasing function then maps their cosine similarity to p^∈[0,1]. In construction, the demanded text is what a subtask requires, and the supplied text is the agent's card (Agent Card box, p.19). Where a subtask declares several conditions, the weakest one sets the score. In the full INFLOWOP, each card is combined with the agent's self-evolving profile (Fig. 5), a kernel-weighted average over its past execution records, whose weight grows as records accumulate. In detection, the estimator scores each declared condition against the realized input or output (Eq. 3).
2. Scores without calibration (W1 "why they should represent success probabilities without calibration", Q1 "what supports interpreting its scores as success probabilities"). Every decision INFLOWOP makes from the scores depends only on how they compare with one another or with a fixed bar: each subtask is bound to its lowest-cost agent (Alg. 3), agent creation is triggered when no agent clears the competence bar Cˉ (Eq. 1), a condition is unmet once it reaches the unmet bar uˉ (Eq. 3), and re-assignment takes the next-best agent (Eq. 20). Rescaling the scores and bars in any way that preserves their order therefore leaves every decision unchanged, so none requires calibrated probabilities. We further validate the scores on a small random sample from the Qwen3.5-9B run, with human labeling. At the configured bars, the reliability estimates predict subtask success with 90.48% precision and 86.36% recall over 50 subtasks, and violation detection finds unmet conditions with 87.80% precision and 85.71% recall over 50 conditions.