acceptodds
Under review as a conference paper at ICLR 2027

CoRefine: Joint Reconfiguration of Concurrent Agent Workflows

Abstract

Multi-agent systems powered by large language models (LLMs) have demonstrated strong capabilities in solving complex tasks through coordinated reasoning and execution. However, existing approaches largely optimize individual workflows or coordinate multiple workflows through scheduling and model routing within prescribed structures, leaving runtime joint reconfiguration under shared resource constraints underexplored. We propose CoRefine, a runtime orchestration system that jointly optimizes the structure of concurrent agentic workflows to balance quality, latency, and token cost under shared resource constraints. CoRefine first narrows the search space by evaluating each workflow’s candidate configurations according to their expected quality and execution cost. It then jointly evaluates the remaining combinations, accounting for the additional waiting time that workflows impose on one another through shared inference resources, and selects the combination with the highest estimated overall utility. In the evaluation run across 1,024 BigCodeBench episode contexts, CoRefine achieves the highest average utility among the evaluated methods. Compared with the fixed-workflow baseline, CoRefine reduces token usage by 68.8% and flow time by 35.3%, while achieving an accuracy of 81.0% versus 80.0%. These findings highlight workflow structure as an important control for allocating inference computation efficiently under shared resource constraints.

Primary area:
foundation or frontier models, including LLMs
Keywords:
agentic workflows, LLM serving, workflow orchestration, model routing, online resource allocation, cost-quality trade-off, workflow reconfiguration, code generation

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_mr9tOpenReview ↗
Rating
2
Confidence
4
Informativeness
1
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes CoRefine, which jointly reconfigures the pending computation of two concurrent agent workflows upon a new task’s arrival to balance task quality, flow time, and token cost.

Critical strengths
  1. The paper studies a practical problem in shared inference systems. Extra computation can improve one workflow’s output but delay others.
  2. The reconfiguration rules are clear. CoRefine only changes pending operations and leaves completed outputs and dispatched calls unchanged.
Critical weaknesses
  1. The Task 2 success proxy assigns identical predicted quality to three versus five samples and one versus two repair rounds (Eq. 10), excluding potential marginal gains from additional computation without sufficient validation of the resulting configuration rankings.

  2. As written, Eqs. (7)–(8) admit exact full-catalog optimization of the surrogate by grouping Task 2 plans according to binary strong-tier usage, with linear scoring complexity once local estimates are available, leaving the need for potentially lossy top-K pruning unclear.

  3. Although token savings and the single-replica results provide positive evidence, the overall conditional intervals for utility and mean-flow improvements over Independent include zero (Tables 5–6), so the additional benefit of joint coordination is only partially established.

  4. The evaluation covers only two foreground workflows, one fixed arrival checkpoint with no unfinished dispatched calls, and a single reconfiguration within a predefined sample-and-repair grammar, limiting the breadth of the demonstrated coordination and structural adaptation.

  5. The choice of α\alpha and β\beta is insufficiently motivated, and the sensitivity analysis only rescores fixed outcomes rather than assessing whether the planner makes effective decisions under different preferences (Section 5.6 and Appendix D.4).

Questions for authors
  1. How are the profiling data separated from evaluation tasks, and what held-out calibration or ranking evidence supports the success and cost estimates across sampling budgets, repair depths, and execution states, particularly after an observed failure?

  2. Does the implementation admit the exact two-group decomposition implied by Eqs. (7)–(8), and how does full-catalog optimization compare with K=7K=7 in selected configurations, surrogate utility, and planning overhead?

  3. With catalogs, profiles, shortlists, anchors, and tie-breaking held fixed, is Independent decision-equivalent to joint selection with the contention term removed, and, if not, can the authors identify and control the differences to isolate the term’s effect?

  4. MCPP [1] has studied closed-loop model and parallel-sampling allocation under budget and deadline constraints. The author should clarify clarify what innovative aspects this article has in Problem formulation, as compared to this previous work.

  5. What motivates the nominal α\alpha and β\beta, and do the comparisons with Independent remain favorable when the planner is rerun under different justified time–token preferences?

[1] On Time, Within Budget: Constraint-Driven Online Resource Allocation for Agentic Workflows, NeurIPS, 2026.

4 comments

(Part 1/2)

We thank the reviewer for the detailed technical assessment, and for recognizing the practical problem and the clarity of our reconfiguration rules. The reviewer's questions led us to find and fix an implementation error and to several other concrete changes, which we describe below in order.

W1 & Q1. Data separation, calibration and the Task-2 prior

Separation. The profiles are estimated on the 350-task training split, which is disjoint from the 128 evaluation tasks. The frozen Task-1 table uses 348 of these tasks, because two records were completed after the table was frozen. On LiveCodeBench, the 50 profiling problems are likewise disjoint from the 104 evaluation problems.

Calibration on the evaluation tasks. We compared the predicted success of candidate configurations with their measured hidden-test outcomes on the evaluation tasks.

StatePredicted (%)Measured (%) [95% CI]
Task 1, visible tests passed85.690.9 [81.8, 97.7]
Task 1, visible tests failed, no extra samples72.7–75.020.0–25.0
Task 1, visible tests failed, 4 extra samples per writer79.9–80.940.0–50.0
Task 2, by plan (20 plans)63.0–82.059.4–89.8

Predictions are reasonable when the visible tests pass, but after an observed failure they are far too high, as the reviewer suspected. We traced this to an implementation error: after a failure, the Task-1 estimate used the unconditional screening-failure rate instead of conditioning on the observed failure. Conditioning correctly on the training split gives 21–33%, close to the measured 20–25%. We fixed this and reran all affected settings on live serving, adding 184 new executions. With the corrected profiler, CoRefine still reduces tokens by 62.2% and flow time by 32.2% relative to Fixed, with accuracy unchanged (+0.3 points, CI [−1.8, 2.4]). Utility is slightly below the submitted version (1.564 vs. 1.578) but still above Fixed (1.528). We will report the corrected version in the revision.

Ranking. Within each setting, the Spearman correlation between predicted and measured candidate quality has a median of 0.45 for Task-2 candidates (0.40 for joint configurations): positive but moderate. This agrees with our oracle analysis (see our response to Reviewer raZv, W2/Q2), which locates the main remaining loss in the coarse prior for the arriving task.

Costs. Task-1 token predictions are within about 22% on average. Task-2 costs are overestimated, mainly because budgeted repairs are skipped once the visible tests pass; together with the flat bonuses, this biases the selection toward smaller budgets.

Identical estimates for different budgets. We agree with the reviewer. We are rerunning the experiment with a revised Task-2 prior whose bonuses grow with the number of samples and repair rounds, and we will post the results in this thread before the end of the discussion period.

W2 & Q2. Exact two-group decomposition

Yes. Since the only cross-workflow term depends on whether the Task-2 plan uses the strong tier, the joint objective decomposes exactly into two groups, and the implementation admits this decomposition. On all 4,096 decision points of the main protocol (submitted profiler), the two-group solver matches brute-force enumeration of the full catalog exactly.

SolverDecisions differing from full-catalog optimumMax surrogate-utility lossMedian planning time
Top-7 (submitted)12.5%2.4 × 10⁻⁴22.5 ms
Full-catalog enumeration—032.6 ms
Exact two-group0%00.79 ms

At the nominal weights the loss of top-7 is small; all differences come from the Task-1 shortlist, and the Task-2 shortlist never excluded the optimum. Across a grid of 27 time–token weights, however, the loss reaches 0.0077, and top-7 occasionally responds non-monotonically to β. We therefore adopt the exact solver for the two-workflow case. Shortlisting remains useful when the coupling is not binary, e.g., with more than two adjustable workflows.

(Continued in Part 2/2.)

(Part 2/2)

W3 & Q3. Isolating the contention term

Q3. Yes. Although the implementations differ (Independent takes local best responses under shared resource rents, without joint reranking or anchors), all rents are zero at the nominal weights and Independent's choices always lie within the joint candidate set. Consequently, on all 4,096 BigCodeBench and 400 LiveCodeBench decision points, Independent selects exactly the same configuration as joint selection with only the contention term removed, with catalogs, profiles, shortlists, anchors and tie-breaking held fixed. The comparison with Independent therefore isolates the contention term.

W3. The joint reranker is designed for the case where concurrent workflows compete for the strong tier. When their individually optimal choices do not conflict, the joint optimum coincides with the independent one by construction, and the planner incurs no cost where coordination is unnecessary. In the two-workflow protocol with the corrected profiler (W1), the contention term never changes the selected configuration, so CoRefine and Independent select the same configurations. The contention term is meant for settings where many workflows compete: in our continuous-arrival experiment (W4), Independent has 2.8× CoRefine's flow time under high contention. The gains in the two-workflow protocol come from the arrival-triggered joint reconfiguration of the in-flight and arriving workflows, which clearly outperforms the restricted variants that reconfigure only one workflow or only the model tier. We will make this distinction explicit in the revised paper.

W4. Narrow setting

We ran a live continuous-arrival experiment with many coexisting workflows, decisions at every arrival, and dispatched calls still running at most decision points; all coefficients are frozen (full setup in our response to Reviewer raZv, W1/Q1).

ConditionMethodAcc. (%)Tokens / task ↓Mean flow (s) ↓
HighFixed87.869,215.5285.84
Task2-only86.712,856.849.95
Independent87.282,471.2137.55
CoRefine87.864,123.248.59
LowFixed87.289,226.933.00
Task2-only88.442,405.419.18
Independent87.862,389.119.40
CoRefine88.442,721.219.68

Under high contention, CoRefine matches Fixed's accuracy with 55% fewer tokens and 83% lower flow time. Under low contention, all adaptive methods perform similarly, consistent with W3. p95 planning latency is 26.45 ms with up to 11 concurrent workflows.

W5 & Q5. Motivation of α and β, and rerunning the planner

The weights encode an accuracy-first preference: they are set so that the time and token cost of a canonical workflow (about 40 s and 18k tokens) amounts to roughly 3 points of accuracy, so costs mainly decide between configurations of similar predicted quality. They express a preference rather than a measured price, which is why we evaluate a wide range of alternatives.

We agree that rescoring fixed outcomes does not test the planner. We therefore reran the planner under 27 preferences spanning a 16× range in each weight (plus α = 0 and β = 0). With the exact solver, the decisions respond monotonically: a larger α never increases predicted flow time, and a larger β never increases predicted tokens. With the submitted profiler, for the 10 weight settings whose selected configurations were all executed, CoRefine is never worse than Independent: it is identical in 6 settings and better in 4 (significantly so in 1).

Q4. Relation to MCPP

We thank the reviewer for the pointer and will cite and discuss MCPP. MCPP allocates models and parallel samples within a single workflow under hard budget and deadline constraints, closing the loop over that workflow's own execution. CoRefine addresses a different decision problem. (i) Reconfiguration is triggered by the arrival of a new task and jointly revises multiple concurrent workflows: the pending computation of in-flight workflows together with the plan of the arriving one. (ii) It operates non-preemptively on in-flight workflows: completed outputs and dispatched calls are preserved, and only not-yet-dispatched sampling and repair can change. (iii) Because the workflows share serving capacity, the objective trades quality against flow time, including the queueing delay imposed by other workflows, and token cost, rather than maximizing success under fixed constraints for one workflow.

All new results will be added to the appendix of the revised PDF, which we will upload before the end of the discussion period. We thank the reviewer again for the careful reading, which led to concrete improvements of both the method and the evaluation.

Thank you for your clarification. I agree with the other reviewers' comments that the paper has too many assumptions regarding the validity of Profiler (HuJ6 W1, raZv W3), and that the setting is too narrow to fully demonstrate the generalizability of the for general dynamic workflow (HuJ6 W2). I also believe that the reply did not adequately address my concerns. Therefore, I will maintain my score.

Thank you for your reply and for engaging in the discussion. We respect your assessment and would like to briefly summarize, for the record, how we addressed the two points you raise.

Profiler assumptions. We did not take the profiler’s validity for granted; we tested it directly. The calibration analysis revealed an implementation error after observed failures, which we fixed, and with the corrected profiler the gains remain (tokens −62.2%, flow time −32.2%, accuracy unchanged). The Oracle-Q analysis locates the remaining loss in the cold-start prior for newly arrived tasks. The gains also hold on LiveCodeBench with the Task-2 prior and utility weights frozen from BigCodeBench. Because the profiler is a separate module, a better predictor can replace it without changing the utility or the solver. The revised PDF states these assumptions and the limitation of the Task-2 prior explicitly.

Narrow setting. Beyond the two-workflow protocol, we evaluated a live continuous-arrival setting with up to 11 coexisting workflows and replanning at every arrival, as well as a second benchmark. We agree that all evaluated workflows are sample-and-repair code generation, and the revised paper states this scope explicitly.

The rerun with a graded Task-2 prior could not be completed within the discussion period; the revised PDF lists it as a limitation.

Thank you again for the detailed review, which led to concrete corrections of both the method and the evaluation.

Reviewer_HuJ6OpenReview ↗
Rating
2
Confidence
4(+1)
Informativeness
1
Presentation
2
Soundness
1
Read review
Paper summary

CoRefine addresses runtime configuration of concurrent agent workflows sharing limited inference capacity. At a new task's arrival, it jointly selects the pending continuation of an existing task and the complete plan for the new task, within a predefined sample-and-repair grammar. A Workflow Profiler estimates success probability and execution costs using offline statistics and limited execution feedback. A local selector shortlists configurations, and a joint reranker adds a heuristic penalty for contention at the strong-model backend. Completed outputs and dispatched calls are preserved. The main evaluation uses 128 BigCodeBench tasks across 1,024 task-pair and serving-condition settings, comparing several restricted adaptation baselines. Relative to Fixed, the paper reports 68.8% lower token use and 35.3% lower mean aggregate flow time, with observed success rates of 81.0% and 80.0%, respectively.

Critical strengths
  1. The paper addresses a relevant resource-allocation problem. Additional sampling or repair for one task can delay others sharing the inference backend. Reconsidering pending computation when a new task arrives is a meaningful systems question.

  2. The evaluation includes actual serving measurements. Experiments use model-serving backends under different hardware and load conditions. These measurements support the reported efficiency gains in the evaluated setting.

Critical weaknesses
  1. The narrow setting and unvalidated quality estimates limit the contribution. The evaluation involves only two foreground tasks and a single decision at a fixed arrival checkpoint, when no dispatched calls remain unfinished. The existing task retains its writers and can only adjust further sampling and a small set of repair sequences. This setting is very narrow and leaves the main challenges of more general workflow orchestration largely untested. The success prior for new tasks uses preset model-tier scores of 0.55/0.70/0.85, a one-half scaling factor, and separate 0.02 bonuses for multiple sampling and repair. It assigns the same predicted quality to three and five samples, and to one and two repair rounds. Larger configurations therefore incur additional costs without receiving additional quality credit, favoring smaller budgets by construction. In such a restricted setting, these manually specified coefficients effectively encode workflow preferences in advance, making the method resemble a hand-designed workflow policy and further weakening its methodological contribution. These assumptions require empirical validation because they are intended to estimate success probability. Unlike the time/token utility weights, they do not simply express design preferences. The current evidence does not establish whether the method provides reliable adaptation beyond reducing a generous initial budget.
  2. The experiments provide insufficient evidence for robustness, the benefit of joint reranking, and reproducibility. The current system is evaluated only on BigCodeBench. The earlier MBPP diagnostic uses a smaller search space and a different objective, so it does not provide a second benchmark evaluation of the current method. Expanding 128 tasks into 1,024 settings does not address the limited task and workflow diversity. The coefficient analysis also rescales the utility of fixed outcomes without rerunning the planner, leaving the robustness of its decisions unresolved. The manuscript mentions an experimental archive and regeneration scripts, but the archive was not accessible in the reviewed materials. The manuscript alone does not sufficiently explain how to reconstruct the frozen profiles or reproduce the full decision process. This prevents independent verification of implementation choices that directly affect the results. Access to the stated archive and reproduction instructions could resolve this concern.
Questions for authors
  1. Validation of the quality estimates: How were the model-tier scores, one-half scaling factor, and 0.02 bonuses selected, and were they fixed before evaluation? Please clarify whether held-out evidence supports the predicted benefits of sampling and repair, particularly the identical quality estimates assigned to different budgets.

  2. Access to reproducibility materials: Where can reviewers access the experimental archive mentioned in the manuscript?

  3. Evidence beyond the evaluated setting: Is there existing evidence for the current method, with its coefficients fixed, on another benchmark or under a different arrival setting?

5 comments

(Part 1/2)

We thank the reviewer for the detailed and critical assessment, and for recognizing the practical relevance of reconsidering pending computation when a new task arrives, as well as the value of evaluating with real serving measurements. We take the concerns about the narrow setting and the Task-2 prior seriously and address them below.

W1 (narrow setting) & Q3 (different arrival setting). Continuous multi-task arrivals

We agree that a single decision at a fixed checkpoint, with no unfinished dispatched calls, leaves the main challenges of general orchestration untested. We therefore ran a new live experiment on real vLLM serving, without simulation or offline replay, in which tasks arrive continuously as a Poisson process and many workflows coexist (full setup in our response to Reviewer raZv, W1/Q1). The planner is invoked at every arrival. Most decisions are made while dispatched calls of other workflows are still running, which is exactly the condition the reviewer notes the original evaluation excluded. In-flight workflows are reconfigured only in their not-yet-dispatched computation, and completed outputs and dispatched calls are always preserved.

Regarding Q3, all coefficients are kept fixed in this experiment: the profiles, the plan catalog, the Task-2 prior and the utility weights are frozen from the main experiment. We evaluate a high-contention condition (one strong-model replica, near-saturated background load) and a low-contention condition (two replicas, light load), comparing CoRefine with Fixed, Task2-only and Independent (without the contention term) on identical arrival traces.

ConditionMethodAcc. %Tokens / task ↓Mean flow (s) ↓
HighFixed87.869,215.5285.84
Task2-only86.712,856.849.95
Independent87.282,471.2137.55
CoRefine87.864,123.248.59
LowFixed87.289,226.933.00
Task2-only88.442,405.419.18
Independent87.862,389.119.40
CoRefine88.442,721.219.68

With the prior frozen, the main findings carry over to this setting. Under high contention, CoRefine matches Fixed's accuracy with 55% fewer tokens and 83% lower flow time, while Independent, which ignores contention, has 2.8× CoRefine's flow time. Under low contention, all adaptive methods perform similarly (within one solved task and 0.5 s). Planning latency stays at the millisecond level (p95 26.45 ms) with up to 11 concurrent workflows.

W1 (Task-2 prior) & Q1. How the coefficients were chosen

We thank the reviewer for this precise observation and agree with it. The model-tier scores (0.55/0.70/0.85), the one-half scaling factor and the 0.02 bonuses form a simple heuristic carried over from an earlier version of our pipeline. There, it served only to rank candidates before a measured selection stage, and its values were not fitted to data. As the reviewer points out, the binary bonuses assign the same predicted quality to three and five samples and to one and two repair rounds, so larger budgets incur additional cost without receiving additional quality credit. These coefficients are intended to estimate success probability, not to express a design preference, and therefore require empirical validation, as the reviewer correctly notes.

The coefficients were frozen before the main experiment and were not tuned on the evaluation tasks; they are also kept unchanged in the LiveCodeBench and continuous-arrival experiments.

To test whether our conclusions depend on this construction, we are rerunning the main experiment with a revised Task-2 prior whose bonuses increase with the number of samples and repair rounds, so that different budgets receive different predicted quality. All other components stay unchanged. We will post these results in this thread before the end of the discussion period.

(Continued in Part 2/2.)

(Part 2/2)

W2 (single benchmark) & Q3 (another benchmark). LiveCodeBench with frozen coefficients

We agree that a single benchmark limits the evidence. We therefore evaluated CoRefine on LiveCodeBench (LCB): algorithmic problems from AtCoder and LeetCode with stdin/stdout and function-based interfaces, which differ substantially from BigCodeBench's library-oriented tasks. We use 104 problems released after all three model checkpoints were published, all disjoint from the 50 problems used for profiling. Regarding Q3, the Task-2 prior coefficients, the plan grammar and the utility weights are kept fixed from BigCodeBench; following the standard profiling procedure, only the Task-1 statistics and the cost profile are re-estimated on the 50 disjoint profiling problems.

MethodAcc. (%)Tokens / episode ↓Flow (s) ↓Utility ↑
Fixed57.018,92273.91.047
Model-only58.317,89578.31.071
Task1-only57.514,47056.31.079
Task2-only58.712,68463.71.098
CoRefine59.27,50542.61.134

Relative to Fixed, CoRefine reduces tokens by 60.3% (95% CI [57.2, 63.3]) and flow time by 42.4% (CI [38.8, 45.9]) and raises utility by 8.3% (CI [2.6, 16.1]), while maintaining accuracy (+2.2 points, CI [−0.7, 6.0]). We agree that the earlier MBPP study is only a diagnostic and does not count as a second benchmark.

W2 (1,024 settings). We agree that the 1,024 settings are not independent samples, since each task pair appears under eight serving contexts. All confidence intervals in our responses are therefore computed by a paired bootstrap over task pairs, keeping all contexts of a pair together, and we additionally checked robustness with clusters of tasks shared across pairs.

W2 (benefit of joint reranking). We ran the controlled ablation requested by the reviewers. With catalogs, profiles, shortlists, anchors and tie-breaking held fixed, the Independent baseline selects exactly the same configuration as joint selection with only the contention term removed at all 4,096 BigCodeBench decision points, so the comparison with Independent isolates the contention term. The term changes the decision in every setting where the in-flight task has failed its visible tests, and in no other setting. On the 320 executed settings where it changes the configuration, it reduces tokens by 3,821 per episode (CI [3,371, 4,307]) and flow time by 5.5 s (CI [1.4, 9.8]), with non-negative point estimates for accuracy (+1.9 points, CI [−1.1, 5.9]) and utility (+0.047, CI [−0.012, 0.129]). Averaged over all settings, including those where the term changes nothing, the effect is diluted, which explains the overall intervals reported in the paper.

W2 (coefficient analysis). We agree that rescoring fixed outcomes does not test the planner. We therefore reran the planner under 27 time–token preferences spanning a 16× range in each weight. The decisions respond monotonically: a larger time weight never increases predicted flow time, and a larger token weight never increases predicted tokens. For the 10 settings whose selected configurations were all executed, CoRefine is never worse than Independent: it is identical in 6 settings and better in 4 (significantly so in 1).

W2 & Q2. Reproducibility materials

We apologize that the archive was not accessible. We have now uploaded it as the updated supplementary material of this submission on OpenReview. It contains the raw records of all 1,024 main-experiment episodes, the decision freezes, task probes, calibration and load-sweep data, the seed-replication and calibrated-settlement campaigns, and CPU-only scripts that:

  • rebuild the frozen profiles from the calibration data, producing files whose SHA-256 hashes match the frozen ones exactly;
  • replay the full decision process, recomputing all 24,576 method decisions (4,096 states × six methods) and matching every decision freeze and all 6,144 executed decisions with zero mismatches;
  • regenerate the result tables from the raw records.

A quick verification takes about five minutes on a CPU; no GPU, model weights or network access is needed (see the README for commands). The README also documents that the frozen Task-1 table uses 348 of the 350 training tasks, because two records were completed after the table was frozen. Rebuttal-period experiments will be added in a later update of the supplementary material.


All new results will be added to the appendix of the revised PDF, which we will upload before the end of the discussion period. We thank the reviewer again for the critical and constructive feedback.

Dear Reviewer HuJ6,

As the discussion period ends today, we would like to briefly check whether our responses have addressed your concerns. In summary, we added:

a live continuous-arrival experiment with many coexisting workflows and all coefficients frozen (W1/Q3); an evaluation on LiveCodeBench with the plan grammar, the Task-2 prior and the utility weights frozen (W2/Q3); a controlled ablation isolating the contention term, and a planner rerun under 27 time–token preferences (W2); the full reproducibility archive in the updated supplementary material (Q2).

Regarding the rerun with a graded Task-2 prior: unfortunately it could not be completed within the discussion period, so we cannot report its results here. In the revised PDF, which we are uploading today, we describe the current prior and its limitation explicitly, namely that it assigns the same predicted quality to different sampling and repair budgets, and we identify a graded or learned prior as a direct next step.

We would be grateful to know whether any concern remains, and we are happy to clarify further. If our responses have resolved your concerns, we would appreciate it if you could take them into account in your final assessment.

Reviewer_raZvOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes COREFINE, a runtime orchestration system that jointly reconfigures pending computations of concurrent agentic workflows sharing inference capacity, balancing the value of extra sampling/repair against delays imposed on competing tasks. Triggered by new task arrivals, it adjusts sampling budgets, repair operations, and model assignments for both in-flight and arriving workflows while preserving completed outputs and dispatched calls, using a profiler to predict success probability and cost, a local candidate selector to build bounded shortlists, and a contention-aware joint reranker to optimize aggregate utility. Evaluated across 1,024 BigCodeBench episode settings, COREFINE reduces token usage by 68.8% and mean aggregate flow time by 35.3% relative to a fixed-workflow baseline while retaining comparable success rates, and further identifies that retaining more computation is preferable under light load and higher serving capacity.

Critical strengths
  1. The paper formulates the novel problem of arrival-triggered joint reconfiguration of pending computations across concurrent agentic workflows under shared inference capacity, unifying structural workflow adaptation and resource contention in a single decision framework and advancing agent orchestration beyond fixed-structure scheduling or isolated single-task adaptation.
  2. The three-stage COREFINE design—combining offline-calibrated profiling, bounded local candidate shortlisting, and contention-aware joint reranking—delivers near-optimal configuration selection with only ~20 ms of online planning overhead and no additional LLM inference, making it lightweight and suitable for real-time production serving.
  3. The empirical evaluation is comprehensive and rigorously controlled, spanning 1,024 episode settings across task difficulties, background loads, and serving capacities with multiple ablated baselines, and it honestly identifies regimes where retaining full computation outperforms reconfiguration, yielding nuanced, credible conclusions rather than one-sided gains.
Critical weaknesses
  1. The experimental evaluation (Section 5) is restricted to a simplified two-workflow, single-arrival reconfiguration setting without testing under realistic continuous multi-task arrival scenarios with repeated decision points, which is a fixable scope limitation that can be addressed by extending simulations to longer horizons with more concurrent workflows.
  2. The Workflow Profiler (Section 4.2) uses overly coarse success probability and cost estimates—Task 2 relies on a simple heuristic prior and Task 1 uses pooled aggregate statistics—without task-specific features or online adaptation, which is a fixable methodological gap that may degrade configuration selection optimality and can be improved with more expressive predictive models.
  3. The proposed framework is exclusively instantiated and evaluated on sample-and-repair code generation workflows from BigCodeBench, with no validation of generalizability to other common agentic workflow types such as tool-use, retrieval-augmented reasoning, or multi-step planning, which is a fixable generalizability limitation that weakens the broader applicability of the core idea.
Questions for authors
  1. Since the current evaluation is limited to a two-workflow single-arrival setting, have you tested COREFINE under continuous multi-task arrival scenarios with repeated reconfiguration points, and does the method retain its efficiency advantages in more realistic concurrent serving environments?
  2. The workflow profiler uses relatively coarse heuristic estimates for success probability and execution cost; have you quantified how much prediction error in the profiler degrades the final configuration selection quality compared to an oracle with perfect information?
  3. All experiments are conducted exclusively on sample-and-repair code generation workflows; have you verified whether COREFINE can deliver consistent efficiency gains on other common agentic workflow types such as tool-augmented reasoning or multi-step planning tasks?
3 comments

(Part 1/2)

We sincerely thank the reviewer for recognizing our novel problem formulation, the lightweight three-stage design requiring no additional LLM inference, and the comprehensive, controlled evaluation. We address the comments below.

W1 & Q1. Continuous multi-task arrivals with repeated reconfiguration

We agree. The original evaluation isolates one decision triggered by one arrival with two workflows. We therefore ran a live experiment on vLLM serving where many workflows arrive and coexist.

Setup. Tasks from an 80-task BigCodeBench development subset arrive as a Poisson process in 8 streams per condition, with the original background-load driver. High contention: one strong-model replica, near-saturated background load. Low contention: two replicas, light load. All methods share identical arrivals, task order, background traces and seeds (paired comparison); profiles, plan catalog, quality prior and utility weights are frozen. Up to 11 workflows coexist, so decisions are often made while other workflows' calls are still running, the in-flight condition the original setting excluded.

Planner. CoRefine runs at every arrival, jointly choosing the new task's plan and the not-yet-dispatched computation (remaining samples, repair suffix) of in-flight workflows whose visible-test outcome is already known; dispatched calls and completed outputs are kept. It retains the original resource pricing and top-7 selection, enumerating shortlists for at most two adjustable workflows and using coordinate refinement over shortlists for more. It reproduces the submitted decisions at all 4,096 original two-workflow decision points.

Baselines. Fixed: the paper's fixed workflow. Task2-only: only the arriving task is planned. Independent: each workflow is optimized without the contention term.

ConditionMethodAcc. %Tokens / task ↓Mean flow (s) ↓
HighFixed87.869,215.5285.84
Task2-only86.712,856.849.95
Independent87.282,471.2137.55
CoRefine87.864,123.248.59
LowFixed87.289,226.933.00
Task2-only88.442,405.419.18
Independent87.862,389.119.40
CoRefine88.442,721.219.68

Under high contention, CoRefine matches Fixed's accuracy with 55% fewer tokens and 83% lower flow time; Independent, which ignores contention, has 2.8× CoRefine's flow time. Under low contention, all adaptive methods perform similarly (within one solved task and 0.5 s).

Planning overhead. Across all arrival decisions with up to 11 concurrent workflows, planning latency has a median of 14.24 ms, p95 of 26.45 ms and maximum of 38.27 ms, negligible relative to flow time.

W2 & Q2. Effect of profiler errors

We quantified this with Oracle-Q, which is identical to CoRefine as submitted (same candidate set, cost predictions and utility) except that the predicted success of each candidate is replaced by its measured hidden-test outcome. Both were evaluated on all 1,024 settings of the main BigCodeBench protocol with live execution.

MethodAcc. (%)Acc. Task 1 (%)Acc. Task 2 (%)Tokens / episodeFlow (s)Utility
Fixed80.075.085.118,19454.61.528
CoRefine81.076.285.75,68335.31.578
Oracle-Q85.175.594.64,15721.41.676
Oracle-Q − CoRefine+4.1 [1.2, 7.0]−0.7 [−4.1, 2.6]+8.9 [3.9, 14.2]−1,526−13.9+0.098 [0.040, 0.156]

Flow is summed over the two tasks of an episode. Brackets give 95% CIs from a paired bootstrap over task pairs.

CoRefine already obtains 89% of Oracle-Q's token savings and 58% of its flow-time reduction relative to Fixed. The profiler's remaining loss is in accuracy and comes almost entirely from the newly arrived task (Task 2): for the in-flight task (Task 1), where visible-test feedback is available, the pooled statistics already match the oracle.

We therefore agree that the profiler is not yet optimal, and its weakness is specifically the cold-start prior for arriving tasks, which have no execution feedback. Because the profiler is a separate module, a stronger predictor for new arrivals can replace it without changing the utility or the solver. We will state this limitation explicitly in the paper and leave it to future work. Since Oracle-Q uses outcomes from a finite number of generations within this candidate set, it is a reference point rather than a strict upper bound.

(Continued in Part 2/2.)

(Part 2/2)

W3 & Q3. Generalization beyond BigCodeBench

We agree that a single benchmark limits the evidence. We therefore evaluated CoRefine on a second benchmark, LiveCodeBench (LCB): algorithmic problems from AtCoder and LeetCode with stdin/stdout and function-based interfaces, which differ substantially from BigCodeBench's library-oriented tasks. To limit contamination, we only use problems released after all three model checkpoints were published (after 2024-12-12).

Setup. The evaluation covers 104 distinct problems: the 32 problems of our initial LCB run plus 72 new problems, each used in a single task pair (60 AtCoder / 44 LeetCode; 26 easy / 33 medium / 45 hard), all disjoint from the 50 problems used for profiling. The plan grammar, the Task-2 prior coefficients and the utility weights are frozen from BigCodeBench. Following the standard profiling procedure, only the Task-1 statistics and the cost profile are re-estimated on the 50 disjoint calibration problems.

MethodAcc. (%)Tokens / episode ↓Flow (s) ↓Utility ↑
Fixed57.018,92273.91.047
Model-only58.317,89578.31.071
Task1-only57.514,47056.31.079
Task2-only58.712,68463.71.098
CoRefine59.27,50542.61.134

272 episodes over four serving contexts (1 or 2 strong replicas × low or near-saturated background load). Flow is summed over the two tasks of an episode.

The results mirror the main experiment. Relative to Fixed, CoRefine reduces tokens by 60.3% (95% CI [57.2, 63.3]) and flow time by 42.4% (CI [38.8, 45.9]) and raises utility by 8.3% (CI [2.6, 16.1]), while maintaining accuracy (+2.2 points, CI [−0.7, 6.0]). As on BigCodeBench, reconfiguring only one of the two tasks recovers only part of the gain. The savings are consistent across platforms and difficulty levels: on the 45 hard problems, for example, tokens drop by 47% and flow time by 40%, with no loss in accuracy.

Limitation. We acknowledge that both benchmarks use sample-and-repair code-generation workflows, and that tool-augmented reasoning, retrieval-augmented reasoning and multi-step planning remain untested. The framework itself is not tied to code. The profiler only needs an intermediate feedback signal and per-configuration estimates of success and cost; examples of such signals are tool-call errors or answer verification. The utility and the solver do not change. We will state this scope limitation explicitly and leave other workflow types to future work.


All new experiments in this response, namely the continuous-arrival experiment (W1/Q1), the Oracle-Q analysis (W2/Q2) and the LiveCodeBench evaluation (W3/Q3), will be added to the appendix of the revised PDF, together with full protocols, confidence intervals and all baselines. We will upload the revision before the end of the discussion period.

Thank you again for the constructive feedback. We hope these new experiments address your concerns, and we would be glad to clarify anything further.

Dear Reviewer raZv,

As the discussion period ends today, we would like to briefly check whether our responses have addressed your concerns. In summary, we added:

a live continuous-arrival experiment with many coexisting workflows and replanning at every arrival (W1/Q1); an Oracle-Q analysis, which locates the profiler’s remaining loss in the cold-start prior for arriving tasks (W2/Q2); a second benchmark, LiveCodeBench, with all coefficients frozen from BigCodeBench (W3/Q3).

All of these are included in the appendix of the revised PDF. We would be grateful to know whether any concern remains, and we are happy to clarify further. If our responses have resolved your concerns, we would appreciate it if you could take them into account in your final assessment.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.