acceptodds
Under review as a conference paper at ICLR 2027

ACCORDION: Breaking Dependency Barriers in Multi-Agent LLM Workflows via Context Speculation

Abstract

Hierarchical multi-agent workflows typically enforce strict parent–child dependency barriers: a child starts decoding only after its parent finishes, making end-to-end latency dominated by the longest dependency chain. We identify a key source of slack: children can begin decoding under partial parent context, and revise only when new parent content arrives. The core challenge lies in controlling speculative waste from context updates. To address this, we propose ACCORDION, a context-level speculative decoding framework for dependency-constrained workflows. The parent streams its output as incremental prompt commits, while each child decodes in parallel under its current partial context. After each commit, the child verifies drafted tokens against the updated context, preserves the longest accepted prefix, and rolls back from the first rejection. We further give an information-theoretic view of prefix reuse: under an ideal maximal-reuse verifier, expected acceptance after an update is lower-bounded by a decreasing function of the update's conditional entropy, while the parent's self-surprisal provides an online proxy under a stated calibration assumption, yielding a surprisal-triggered commit rule that trades off reuse and rollback. Across diverse workloads, ACCORDION reduces end-to-end latency by up to 46.6% relative to sequential execution and achieves a more stable speed–accuracy trade-off than heuristic streaming baselines while maintaining comparable task performance. It further composes with model-level speculative decoding.

Primary area:
infrastructure, software libraries, hardware, systems, etc.
Keywords:
Multi-agent LLM systems, context speculation, surprisal

Reviews

Among:
Shape:
1234this paper, Top 32%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.50 over 4 reviews, above 68% of the others.
Reviewer_GWP7OpenReview ↗
Rating
3
Confidence
4
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper presents ACCORDION, a runtime optimization for multi-agent LLM workflows with parent-child dependencies. Instead of waiting for a complete parent message, a child starts generating from partial context and checks its draft when the parent supplies more information. The two main design choices are a surprisal-based rule for sending context updates and prefix verification with rollback to retain useful child computation. The paper also analyzes the relationship between information in a context update and prefix reuse. Experiments on reasoning and code-generation tasks, several dependency graphs, and a dynamic plotting workflow report up to 46.6% lower latency than sequential execution. ACCORDION also provides additional acceleration when combined with EAGLE-3.

Critical strengths
  1. Overlapping parent and child decoding is a useful idea. Batching and prefix caching alone cannot eliminate the wait for a complete parent message. Verification and rollback give a concrete way to retain useful child computation as the context changes.

  2. The evaluation covers different graph structures and model sizes, streaming-prefill overlap, and composition with EAGLE-3. I particularly appreciate the OpenManus experiment, where tools and revision rounds change the workflow during execution. The discussion of synchronization and cache-recovery costs also helps explain where the method is useful.

Critical weaknesses
  1. The practical benefit depends on spare serving capacity. Speculation creates additional verification and discarded computation, which can become costly when many workflows share the GPUs. Appendix E.4 examines this issue, but uses fixed 16-token commits and controlled acceptance rates. The operating range of the default policy on actual task workloads therefore remains unclear.

  2. The evidence for the adaptive commit rule is less strong than the evidence for early child execution. Fixed-size commits recover much of the latency benefit and are sometimes equally good or faster. For example, Chunk-16 and ACCORDION have the same accuracy and almost the same latency on six-agent HumanEval. The adaptive rule may reduce the need to choose different chunk sizes for different tasks, but this advantage deserves a clearer explanation.

Questions for authors
  1. How do latency and throughput change under increasing concurrent load on the benchmark tasks, using the default ACCORDION policy? At what load does the extra speculative work outweigh the benefit of overlapping generation?

  2. How does the surprisal rule compare with one fixed chunk size chosen on validation data? Does it give a more consistent latency-quality trade-off across tasks, or mainly reduce the need for task-specific tuning?

  3. Does the same commit threshold work when parent and child agents use different model families or have very different capabilities? The OpenManus result is a useful heterogeneous example. Are there results for other mixed parent-child model pairs?

4 comments

Reviewer GWP7 (1/2)

Thank you for your positive assessment and precise systems questions. We are glad you found verification and rollback a concrete way to preserve useful child work and valued the dynamic OpenManus experiment. We answer each question below with new measurements.

Color-coded revision guide: Main-text updates for your review appear in blue across the Abstract, Sections 1, 3.4, 5.2, and 6, and the Ethics statement, supported by Appendices E.4, E.10, F, and G.

W1 & Q1: behavior under concurrent load, and where speculative work outweighs the benefit.

Across real four-agent workflows on one engine, ACCORDION delivers substantial throughput and latency gains throughout the low-to-moderate load regime, raising throughput by 1.43–1.47× and cutting P50 latency by 30–34% at c=1c=1, before reaching break-even at about 4 workflows in flight on MMLU and between 4 and 8 on GSM8K. Moving beyond the synthetic study in our original Appendix E.4, this closed-loop sweep runs the default policy on four-agent MMLU and GSM8K with cc workflows in flight on one engine:

Table R1. Workflow throughput of ACCORDION relative to Sequential.

cc12481632
MMLU1.43×1.22×1.00×0.78×0.58×0.41×
GSM8K1.47×1.32×1.14×0.94×0.79×0.61×

P50 / P95 workflow latency including queueing, Sequential →\rightarrow ACCORDION:

  • MMLU: at c=1c=1, 24.0/39.4 →\rightarrow 16.9/29.8 s; at c=2c=2, 24.5/38.6 →\rightarrow 19.6/32.5 s; at c=8c=8, 26.8/47.0 →\rightarrow 35.3/63.1 s.
  • GSM8K: at c=1c=1, 13.8/23.0 →\rightarrow 9.1/14.6 s; at c=4c=4, 15.0/24.7 →\rightarrow 13.3/19.6 s; at c=8c=8, 16.9/28.7 →\rightarrow 17.7/27.2 s.

Accuracy shows no trend with load, staying within ±4 pp across concurrency levels. Past the crossover, ACCORDION's extra work from prompt rebuilds, verification, and discarded drafts saturates the engine. Sequential, which runs only one agent per workflow at a time, is far less affected: from c=1c=1 to c=8c=8 its P50 latency rises 12–22%, whereas ACCORDION's roughly doubles. MMLU crosses over earlier because its parents revise more heavily, discarding 30–41% of child tokens versus 8–16% on GSM8K in Table R7.

Because the server directly tracks how many workflows are in flight, a calibrated admission threshold that falls back to Sequential above the crossover is never worse than Sequential across this sweep while retaining 1.14–1.47× at low load. SmartSpec by Liu et al. (2024) applies the same principle to model-level speculative decoding. The revision adds this sweep with P50 and P95 latencies to Appendix E.4 and deployment guidance to Appendix G.

Q3: the same threshold for parent and child models of different families or capabilities.

Yes: with τ=10\tau=10 and Mmax⁡=64M_{\max}=64 unchanged, mixing Qwen2.5-14B-Instruct and Llama-3.1-8B-Instruct across model families and sizes yields 1.33–1.58× speedups over Sequential in both directions while preserving accuracy. On four-agent MMLU and GSM8K, each model in the mixed runs occupies its own engine and A100 GPU; commits cross engines as plain text, each child tokenizes updates with its own tokenizer, and Sequential uses the exact same placement.

Table R2. Cross-family runs: speedup with paired 95% CI, and accuracy from Sequential →\rightarrow ACCORDION.

Root / middle / final agentGSM8KMMLU
Qwen / Qwen / Llama1.58× (1.51–1.65); 87.0 →\rightarrow 89.5%1.43× (1.38–1.48); 67.5 →\rightarrow 66.5%
Llama / Qwen / Qwen1.33× (1.28–1.38); 89.0 →\rightarrow 89.5%1.44× (1.38–1.50); 74.0 →\rightarrow 74.0%
all Qwen, 1-GPU reference1.44× (1.39–1.49); 93.0 →\rightarrow 92.5%1.42× (1.37–1.47); 75.5 →\rightarrow 74.5%
all Llama, 1-GPU reference1.37× (1.29–1.46); 84.5 →\rightarrow 87.0%1.54× (1.49–1.61); 57.0 →\rightarrow 59.0%

The single-family reference rows run on one GPU, so each row should be compared only with its own Sequential baseline; every accuracy change has a paired 95% CI that includes zero. Crucially, the mechanism is agnostic to model pairing: the commit signal uses only the parent's own token NLL, and retention is decided only by the child's own Top-5 check, so neither assumes a shared tokenizer or model family.

While the best τ\tau naturally scales with the parent's NLL, ACCORDION-online in Section 3.4 selects τ\tau and Mmax⁡M_{\max} directly from the parents' own surprisal streams, so a new model pair requires no sweep. OpenManus in Appendix E.3 provides an additional multimodal heterogeneous case with Qwen2.5-VL-7B and Qwen2.5-14B, reaching 1.18× at τ=10\tau=10. Table R2 is added as Appendix E.10.

Reviewer GWP7 (2/2)

W2 & Q2: the evidence for the adaptive commit rule, and the rule vs. one fixed chunk chosen on validation data.

It achieves both: with τ\tau and Mmax⁡M_{\max} adapted online across requests, our surprisal rule is more consistent across tasks than a validation-selected fixed chunk and eliminates task-specific tuning altogether. Early child execution with verification and rollback is the foundation every streaming policy in Table 1 shares; the commit rule decides when children receive new parent text, and therefore how well that foundation is used. Because optimal granularity depends on the task, ACCORDION-online in Section 3.4 updates τ\tau and Mmax⁡M_{\max} online between requests from completed parent surprisal streams, needing no validation set.

For the direct comparison you requested, we selected the fastest single chunk size on 200 held-out questions across four- and six-agent MMLU and GSM8K in Appendix F.6, which chose Chunk-64:

Table R3. Speedup in Table 1 averaged over three runs, for 3 / 4 / 5 / 6 agents.

PolicyMMLUGSM8KHumanEval
Chunk-64, validation-selected1.23 / 1.49 / 1.61 / 1.541.41 / 1.54 / 1.50 / 1.571.11 / 1.34 / 1.20 / 1.40
ACCORDION-online1.20 / 1.46 / 1.55 / 1.551.38 / 1.50 / 1.49 / 1.511.29 / 1.61 / 1.59 / 1.79

Across all 12 settings, ACCORDION-online achieves the highest overall mean speedup at 1.49×, beating both the best single chunk in hindsight, Chunk-48 at 1.43×, and validation-selected Chunk-64 at 1.41×. More importantly, it stays within 4.0% of the best fixed chunk size in every setting, whereas every fixed chunk falls 15–44% behind in at least one, with Chunk-64 trailing by up to 26% on HumanEval in Appendix F.6.

Addressing your three sub-questions:

  • A more consistent latency trade-off? Yes, because the cost of a commit is task-dependent and ACCORDION-online measures it directly, as analyzed in Appendix F.4. On MMLU, one commit rebuilds 2.3–3.4 running children and discards 24–47 drafted tokens per rebuild, so sparser commits win and it selects τ\tau around 28–30. On HumanEval, one commit rebuilds only 1.2–1.5 children and discards under one token, so frequent commits win and it selects τ\tau around 1.75–3.5. Because larger deliveries discard more drafts, Mmax⁡M_{\max} stays at most 64.
  • A more consistent quality? Accuracy is reliably preserved: ACCORDION-online's mean accuracy change across the 12 settings is −0.11 pp, and every schedule in Table 1 remains within run-to-run variation.
  • Mainly less task-specific tuning? Yes, it removes per-task tuning entirely. Its warm-start point, τ=10\tau=10 and Mmax⁡=64M_{\max}=64, was chosen once during development on the 200 four-agent MMLU questions of Table 1, generalizes at 1.42–1.45× on unseen four-agent MMLU sets, and is used unchanged for every other task, model, and topology. Because code generation favors frequent commits whereas MMLU and GSM8K favor coarser ones in Appendix F.3, every fixed chunk size lags on some workload. ACCORDION-online avoids manual sweeps altogether, pricing every candidate between requests in 21–39 ms and settling to adjacent grid values in the second half of each run, as shown in Appendix F.5.

In the revised manuscript, Section 3.4 and Appendix F present ACCORDION-online, the introduction notes that τ\tau and Mmax⁡M_{\max} are adapted online across requests, and the abstract, contributions, and Section 5.2 highlight that ACCORDION-online achieves the highest mean speedup in Table 1 while staying within 4.0% of the best fixed chunk in every setting.

These results answer all three questions with direct measurements: where ACCORDION pays off under load, how its online commit rule outperforms any single fixed chunk across tasks, and why one threshold carries across model families. We would be grateful if you would reflect this in your score, and we are happy to provide any further detail.

Thank you for providing the detailed response to my questions. I think this addresses most of my concerns. Good luck!

Thank you for the quick reply! We're glad the response addressed most of your concerns, and your review genuinely made the paper better.

If anything still holds you back, just let us know and we'll address it before the discussion closes. Otherwise, if you feel the revision has resolved your concerns, we'd be grateful if you could consider raising your score.

Reviewer_g6krOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper presents ACCORDION, a context-level speculative decoding framework form multi-agent LLM workflows. The key insight is that in multi-agent LLM workflows, children can begin decoding under partial parent context. The parent streams its output as incremental prompt commits, and each child uses the partial context to decode. After each commit, the child verifies drafted token based on the updated context. The paper provides an information-theoretical view of prefix reuse. The evaluation compares ACCORDION with the baselines using a set of benchmarks.

Critical strengths
  • The paper presents a speculative decoding framework for multi-agent LLM workflows. The idea of speculative decoding under partial parent context is interesting.
  • The evaluation compares ACCORDION with a couple of baselines. It includes different benchmarks and models.
Critical weaknesses
  • The benefits are not significant. According to the main results in Table 1, ACCORDION does not always significantly outperform the baselines. In some cases, ACCORDION performs worse than the baselines.
Questions for authors
  1. In Section 3.2 "Commit rule", how do you choose the values for \tau and M_max? Are these hyperparameters chosen empirically?

  2. The evaluation uses NVIDIA A100 GPUs. How many GPUs do you use? Do you change the number of GPUs when there are different number of agents?

  3. In Table 1, why do ACCORDION and the baselines sometimes have higher and sometimes have lower accuracy than Sequential?

3 comments

Reviewer g6kr (1/2)

Thank you for your constructive review, and for finding partial-context speculative decoding interesting and broadly evaluated across benchmarks and models. Below we establish statistical significance across three runs of every policy with paired tests, clarify the schedules in Table 1, and answer each question.

Color-coded revision guide: Main-text updates for your review appear in green in Section 1, the three-run Table 1, Section 5.1 on repeated runs and single-GPU setup, and Section 5.2, alongside Section 3.4 on ACCORDION-online in blue, with supporting details in Appendices C.5, E.8, E.9, E.11, and F.

W1: significance of the gains.

(1) Against Sequential execution, speedups are large and statistically significant across all 12 settings. Table 1 now reports mean ± std over three independent runs of every policy across all three tasks with ACCORDION-online, introduced in Section 3.4 and our General response:

Table R4. Speedup over Sequential in Table 1 as three-run mean ± std, for 3 / 4 / 5 / 6 agents.

MethodMMLUGSM8KHumanEval
ACCORDION-online1.20±0.03 / 1.46±0.03 / 1.55±0.04 / 1.55±0.011.38±0.01 / 1.50±0.03 / 1.49±0.03 / 1.51±0.011.29±0.02 / 1.61±0.01 / 1.59±0.03 / 1.79±0.04
  • Paired tests. ACCORDION-online cuts latency by 16.7–44.0%, and the 95% CI excludes zero in all 12 settings, with a lowest lower bound of 14.3% and speedup std of at most 0.04×.
  • Appendix E.8. Four runs on the Table 1 questions plus two runs on unseen, disjoint question sets give 1.20–1.52×, and every 95% CI across the clean reruns excludes zero.
  • Accuracy. The mean change across all 12 settings is −0.11 pp, well within run-to-run variation.

(2) Other streaming policies run inside our ACCORDION runtime, and ACCORDION-online is the fastest overall. Sequential is the external non-speculative baseline, whereas Chunk-kk in Table 1 and the entropy and delimiter policies in Appendix E.9 execute inside our runtime and share early child execution and Top-5 verification; only the commit schedule differs. Across all 12 settings, ACCORDION-online achieves the highest mean speedup at 1.49× versus 1.43× for the best single chunk, and stays within 4.0% of the best fixed chunk size in every setting, whereas every fixed chunk falls 15–44% behind in at least one, as shown in Table R3 of our response to Reviewer GWP7.

(3) Against external baselines, ACCORDION maintains a clear lead. Within the same engine, FlashAgents-style streaming prefill stays within 1.03× of Sequential while ACCORDION reaches 1.08–1.37× in Appendix E.2, and stacked on top of EAGLE-3, ACCORDION adds a further 1.18× speedup.

Reviewer g6kr (2/2)

Q1: how τ\tau and Mmax⁡M_{\max} were chosen.

The warm-start defaults, τ=10\tau=10 and Mmax⁡=64M_{\max}=64, were set empirically once during development, and ACCORDION-online in Section 3.4 now removes manual tuning altogether. Chosen once on the 200 four-agent MMLU questions of Table 1, those defaults were applied unchanged to every other task, model, and topology, generalizing at 1.42–1.45× on unseen four-agent MMLU sets in Appendix E.8. Starting from these values, ACCORDION-online adapts τ\tau and Mmax⁡M_{\max} online between requests from completed parent surprisal streams.

Optimal τ\tau depends on the task, as analyzed in Appendix F.3: code generation favors small τ\tau, with four-agent HumanEval reaching 1.53× at τ=2\tau=2, whereas MMLU and GSM8K favor larger τ\tau. On four-agent MMLU in Table R6, τ=5/10/15/20\tau = 5 / 10 / 15 / 20 gives 1.35 / 1.42 / 1.45 / 1.45×, and Mmax⁡=16/32/48/64M_{\max} = 16 / 32 / 48 / 64 gives 1.26 / 1.34 / 1.43 / 1.42×. Physically, frequent commits pay off when a commit is cheap on HumanEval and cost time on MMLU where one commit rebuilds 2.3–3.4 active children, while Mmax⁡M_{\max} caps uncommitted low-surprisal spans. ACCORDION-online prices these measured costs between requests with no manual sweep, as detailed in Appendix F.

Q2: number of GPUs.

Strictly one GPU per run, invariant to the number of agents: each run uses a single A100-SXM4-80GB with one vLLM engine at TP=1, and all agents as well as EAGLE-3's draft model share one set of weights and one KV-cache pool.

Other studies documented in Appendix C.5 use H100 NVL for MATH-500 and ResearchBench, and H200 NVL for PaperBananaBench, CriticBench in B.6, synthetic throughput in E.4, and threshold adaptation in E.5. All new rebuttal runs use one A100-SXM4-80GB per policy and run; only the cross-family runs in GWP7 Q3 give each model its own A100.

Q3: why accuracy is sometimes above and sometimes below Sequential.

Because these small differences reflect symmetric trajectory variance rather than systematic bias. Across three runs totaling 600 paired evaluations per setting, flips in both directions are nearly equal:

MMLU, agents3456
Sequential wrong →\rightarrow ACCORDION right41575951
Sequential right →\rightarrow ACCORDION wrong43535048

On GSM8K, only 2–4 questions flip each way. Two sources account for these flips:

  • Top-5 acceptance. Keeping a plausible non-argmax token under partial context can alter downstream phrasing.
  • Run-to-run variation. Even under greedy decoding, outputs vary with batch composition because inference kernels are not batch-invariant, as noted by Thinking Machines (2025), and with live Wiki Searcher results on MMLU, where two Sequential runs agree on only 78–79% of questions.

All schedules in Table 1 share both sources of variance, and Table 1 now reports mean ± std over three runs.

With three runs and a paired CI that excludes zero in every setting, the gains are significant. We would appreciate it if you would update your score accordingly, and we are glad to answer any further questions.

Thank you again for your review. Before the discussion closes, here's where your points stand in the revised paper:

  • Significance. Table 1 now reports three runs of every policy. ACCORDION-online cuts latency by 16.7–44.0%, and the 95% confidence interval excludes zero in all 12 settings, with a run-to-run std of at most 0.04× (Appendix F.2). Six more paired runs of the fixed default, including two on unseen question sets, give 1.20–1.52× (Appendix E.8).
  • Choosing τ and M_max. ACCORDION-online now selects both online between requests, so nothing is hand-tuned per task (Section 3.4).
  • GPUs. Every run uses one A100, regardless of the number of agents (Section 5.1).
  • Accuracy above or below Sequential. The flips go both ways in nearly equal numbers, and the mean change across the 12 settings is −0.11 pp, within run-to-run variation.

If you'd like us to clarify anything else, just let us know. If these address your concerns, we'd be grateful if you'd consider updating your score.

Reviewer_VQXtOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
3
Soundness
3(+2)
Read review
Paper summary

This paper addresses a common issue in hierarchical multi-agent workflows: a child starts decoding only after its parent finishes, making end-to-end latency dominated by the longest dependency chain. The authors propose ACCORDION, a speculative decoding framework using self-surprisal to determine when the child can decode in parallel under the parent's partial context. ACCORDION reduces latency by 46.6%+ while maintaining accuracy.

However, I think this paper has some limitations in terms of applicability and novelty. Moreover, I have several concerns about the experimental results, especially the fairness of the comparisons, the reliability of the results and reproducibility.

Critical strengths
  1. The parent–child dependency barrier is a common issue in multi-agent workflows. Letting the child decode in parallel under the parent's partial context is a good insight.
  2. The paper provides comprehensive results and theoretical analysis to support its method (lots of charts/graphs, 18-page appendix).
Critical weaknesses
  1. I don't think all tasks are suitable for ACCORDION. For example, a parent agent may change its initial direction as it generates more context, forcing the child to roll back and regenerate. In such cases, the time savings from speculative execution may be offset by the additional computational overhead. I think ACCORDION is better suited to tasks where subsequent changes in upstream information are unlikely to affect the content already generated by downstream agents. In this sense, one simple possible solution is to add a task router deciding whether to use ACCORDION or sequential execution.
  2. Surprisal measures how unexpected a token is to the parent model, but this is not equivalent to determining whether the current context is stable enough to pass to the child. A high surprisal token may have nothing to do with the downstream task. A low surprisal token may change the direction of the downstream agent.
  3. ACCORDION does not modify the reasoning strategies, so it's unclear why it would improve the accuracy. The authors report the mean and std over 6 runs in Appendix E.8, while the main results cherry-pick a single run. A possible reason is that the accuracy improvements are due to LLM randomness.
  4. ACCORDION reduces the E2E latency by sacrificing token consumption. However, the experimental results lack important metrics related to token consumption, such as rejected tokens after regeneration and total generated tokens including rejected tokens.
  5. For MMLU and GSM8K, the paper only states that a fixed set of 200 questions is used for each benchmark. However, it does not explain how these questions were selected. What's more, in Appendix E.2, the EAGLE-3 comparison uses 153 MMLU questions and 1,319 GSM8K problems. Why different?
  6. There is another inconsistency in the results: Tables 1, 2, and 4: ACCORDION achieves an accuracy of 76.0% with a speedup of 1.47×. Table 3: ACCORDION achieves the same accuracy, but the reported speedup is 1.38×. And both M_max=32 and 48 outperform the default setting of 64 in terms of both accuracy and speedup. I think 1.38x is a typo (should be 1.47x).
  7. Fig. 5 compares ACCORDION with Ayo and Parrot on MMLU, GSM8K and HumanEval. However, these baselines were not evaluated on these benchmarks in their original papers, which means they are not optimized on these benchmarks. As demonstrated in Fig. 5, these baselines are even slower than the Sequential baseline. The selection of baselines and benchmarks is questionable.
  8. Appendix D only provides prompts for MMLU, GSM8K, and HumanEval. It does not include other prompts used in ResearchBench, PaperBananaBench, and CriticBench. Also, the source code is not provided.
Questions for authors
  1. Can the authors clarify the scenarios in which ACCORDION is applicable or the failure conditions according to the experimental results?
  2. Why is parent self-surprisal a reliable signal for deciding when to pass context to the child?
  3. Can the authors report token metrics such as rejected tokens after regeneration and total generated tokens including rejected tokens?
  4. How were the 200 questions for MMLU and GSM8K selected? Why does EAGLE-3 use 153 MMLU and 1319 GSM8K?
  5. Why didn't the authors use the benchmarks from the original Ayo and Parrot papers to ensure fairness? Did the authors use the original implementations of these baselines? Are there any stronger baselines to compare with?
9 comments

Reviewer VQXt (1/3)

Thank you for your detailed review, and for recognizing both the prevalence of the parent–child barrier and the insight behind partial-context decoding. Your questions regarding fairness, statistical reliability, and reproducibility directly shaped most of our new experiments and our code release. Because Q1–Q5 pair with W1, W2, W4, W5, and W7, we address each pair together below.

Color-coded revision guide: Main-text updates for your review appear in vermillion across Section 5.1 on question selection and HumanEval scoring, Section 5.2 on Ayo, Parrot, and EAGLE-3, Section 5.3, and Tables 2, 3, and 4, alongside three-run Table 1 in green and Section 3.4 in blue, with full details in Appendices D.3–D.5, E.6, E.8, F, and G.

W3: single run vs. Appendix E.8, and accuracy.

Table 1 now reports mean ± std over three independent runs of every policy on MMLU, GSM8K, and HumanEval across 3–6 agents, and the text quotes means throughout. Every speedup std is at most 0.05×, and ACCORDION-online, introduced in Section 3.4 and our General response, achieves 1.20–1.79× speedups, as summarized in Table R4 of our response to Reviewer g6kr.

While re-benchmarking under our unified vLLM 0.11 implementation and standardizing HumanEval to score strictly the final agent's program throughout Appendix D.3, we noted that our earlier HumanEval numbers differed under the standardized setup. To ensure complete end-to-end consistency across the paper, we reran all HumanEval settings in Table 1 over three runs per policy and refreshed Tables 2 and 4 in paired runs of ACCORDION-online against Sequential, reaching 1.29–1.81× from 7B to 32B and 1.04–1.74× across all five topologies.

Appendix E.8 now documents all six paired runs of the fixed default policy transparently:

  • Four runs on the Table 1 questions, including three fresh reruns with all diagnostics disabled; earlier repeats had logging and tracing enabled, which penalized ACCORDION more than Sequential, and Appendix E.8 describes both.
  • Two runs on new, disjoint sets of 200 questions, independent of Table 1 and of each other, neither of which was used to set τ\tau.

Table R5. Appendix E.8: speedup of the default policy over Sequential.

DatasetAgentsTable 1 questions, 4-run mean ± stdTwo new question sets
MMLU31.21 ± 0.031.23, 1.20
MMLU41.41 ± 0.051.45, 1.42
MMLU51.52 ± 0.061.51, 1.47
MMLU61.46 ± 0.091.42, 1.37
GSM8K31.40 ± 0.061.35, 1.39
GSM8K41.49 ± 0.031.49, 1.45
GSM8K51.47 ± 0.091.44, 1.42
GSM8K61.50 ± 0.041.49, 1.52

Across the three clean reruns alone, latency drops by 16–33% and every 95% CI excludes zero.

On accuracy, we agree: ACCORDION does not alter the reasoning strategy, so it preserves Sequential's accuracy rather than claiming accuracy improvements. Across the 12 settings of Table 1, ACCORDION-online's mean accuracy change is −0.11 pp. Across the four E.8 runs, mean changes span −0.5 to +1.4 pp, well below MMLU's own std of up to 4.5 pp. Per-question flips are symmetric, as detailed in our response to g6kr Q3, arising from Top-5 acceptance and run-to-run variation under greedy decoding due to BF16 batch numerics and live Wiki search on MMLU. Section 5.2 now describes accuracy as preserved alongside mean ± std in Table 1.

W6: Table 3 vs. Table 1.

Those two default entries came from separate runs, so we reran every Table 3 configuration on the exact same 200 four-agent MMLU questions sharing one default run. Under this unified protocol, our default setting lies directly on the speed–accuracy frontier: no configuration beats it on both speed and accuracy.

Table R6. Table 3 rerun under one unified protocol on four-agent MMLU.

ConfigurationSpeedupAccuracy
τ\tau = 5 / 10 / 15 / 201.35 / 1.42 / 1.45 / 1.45×73.0 / 76.0 / 73.5 / 75.5%
Mmax⁡M_{\max} = 16 / 32 / 48 / 641.26 / 1.34 / 1.43 / 1.42×77.0 / 76.5 / 75.5 / 76.0%
Top-1 / Top-5 / Top-∞\infty1.00 / 1.42 / 1.75×73.0 / 76.0 / 73.5%

The slightly faster settings, Mmax⁡=48M_{\max}=48 and τ=15\tau=15 or 20, gain only 0.01–0.03×, well within the ±0.05× run-to-run std. Conversely, small Mmax⁡M_{\max} values slow execution by forcing frequent commits, each requiring a prompt rebuild and verification pass; ACCORDION-online prices this exact cost in Section 3.4. The revision reports the verification ablation in Table 3 and the τ\tau and Mmax⁡M_{\max} sweep in Appendix F.3.

Reviewer VQXt (2/3)

W4 & Q3: token metrics.

Table R7 in Appendix E.6 reports full child-token accounting per workflow averaged over three runs, where Generated counts all child tokens produced across rounds including regenerations and Discarded counts tokens dropped upon re-verification. ACCORDION trades bounded extra token generation for large critical-path latency reductions: 8–23% more child tokens yield 1.40–1.50× speedups on GSM8K, while 49–69% extra tokens on revision-heavy MMLU still deliver 1.21–1.52×.

Table R7. Child tokens per workflow averaged over three runs.

WorkloadGeneratedDiscardedDiscard %Total generation / Sequential
MMLU, 3 / 4 / 5 / 6 agents1041 / 1952 / 2449 / 2706316 / 795 / 1013 / 110030 / 41 / 41 / 41%1.49 / 1.68 / 1.69 / 1.64×
GSM8K, 3 / 4 / 5 / 6 agents754 / 779 / 1059 / 1269122 / 66 / 143 / 9716 / 8 / 14 / 8%1.23 / 1.08 / 1.20 / 1.12×

The tokens a child ultimately keeps are 0.97–1.04× of Sequential's output, confirming that final responses do not grow and extra tokens come solely from discarded drafts. Together with prompt rebuilds and verification passes, this extra work explains why speedups taper at high load in Table R1, and provides the exact cost signal ACCORDION-online measures to set τ\tau and Mmax⁡M_{\max} in Appendix F.4. Model-level speculative decoding makes the same compute-for-latency trade-off, as shown by Leviathan et al. (2023) and SmartSpec by Liu et al. (2024).

W5 & Q4: question selection, and the EAGLE-3 subsets.

Both follow standard data loaders from prior multi-agent literature:

  • MMLU: Following GPTSwarm and AgentPrune, the 1,531-question validation split is shuffled with fixed seed 888, taking the first 200 questions.
  • GSM8K: The first 200 test problems in file order.
  • EAGLE-3: GPTSwarm evaluates the first 10% of that same shuffled MMLU order, 153 questions that match the exact first 153 of our 200, while AgentPrune, G-Designer, and KVCOMM use those 153 MMLU questions and the full 1,319-problem GSM8K test set. Our EAGLE-3 study kept these defaults.

Section 5.1 now states this explicitly alongside the sampling code in our repository. To align the subsets directly, we reran MMLU composition on all 200 questions: EAGLE-3 + ACCORDION is 1.18× faster than EAGLE-3 alone, with paired 95% CI [1.14, 1.23]. On GSM8K, restricting the full-set run to our 200 problems gives 1.28× [1.22, 1.35] versus 1.10× on the full 1,319-problem set in Fig. 5; the revision reports both while keeping 1.10× as the conservative headline figure.

W1 & Q1: when ACCORDION applies and when it fails.

Our new data establish clear, checkable operating conditions governed by three factors:

  1. Load. Gains require spare capacity, reaching break-even at about 4 workflows in flight on MMLU and 4–8 on GSM8K, as shown in our response to Reviewer GWP7.
  2. Parents that change direction. This matches your example and registers directly as discarded child tokens. MMLU parents revise heavily, discarding 30–41% of child tokens across 9.5–15.2 context restarts per agent at 4–6 agents, versus 8–16% and roughly 4 rounds on GSM8K. Even under 30–41% upstream-induced rollbacks, ACCORDION remains 1.2–1.5× faster at low load because verification preserves the surviving prefix; under heavy load, this waste sets the crossover.
  3. Fan-in. Very high fan-in becomes rebuild-dominated: ResearchBench workflows with 17 or more agents run slower than Sequential in Fig. 6b, and GSM8K star yields 1.06× in Table 4.

We agree that a router is the natural way to deploy ACCORDION. Using a calibrated fan-in rule on ResearchBench that runs ACCORDION for workflows with at most 15 agents and Sequential otherwise raises overall speedup across all 100 workflows from 1.06× to 1.14×, reaches 1.20× on the 86 routed workflows, and avoids regressions on all 14 high-fan-in graphs. The cut-off is a calibrated threshold, set on these 100 workflows. Combined with online load and runtime discard rates, these signals form our new Appendix G on when to use ACCORDION.

Reviewer VQXt (3/3)

W2 & Q2: why parent self-surprisal.

Parent self-surprisal provides a zero-cost statistical signal of when a parent update is likely to shift the child, and it need not be exact because child-side verification then decides what draft tokens are kept:

  1. Zero overhead. Token NLL is an immediate by-product of parent decoding, and ACCORDION-online reuses the exact same stream off the critical path to select τ\tau and Mmax⁡M_{\max} in Section 3.4.
  2. Tracks downstream distribution shift. Across 800 fixed-length 16-token CriticBench commits with matched redundant-prompt controls in Fig. 4b and Appendix B.6, the shift in the child's next-token distribution rises by 9.1 percentile points from the lowest to highest parent-surprisal quintile, with 95% CI [2.1, 15.3]. Because verification follows every commit, the trigger only needs to place commits where updates are more likely to matter.
  3. Both failure modes you note are bounded.
    • A high-surprisal but irrelevant token triggers one extra rebuild and verification pass, losing zero child work if the draft still passes Top-5; where rebuilds are costly, ACCORDION-online raises τ\tau automatically, reaching 28–30 on MMLU.
    • A low-surprisal but pivotal token is delayed by at most Mmax⁡M_{\max} tokens, after which verification rolls back any child token falling outside Top-5 under the updated context.

W7 & Q5: Ayo and Parrot.

We evaluated the unmodified public releases of Ayo and Parrot, and Section 5.2 now presents Fig. 5 explicitly as an end-to-end comparison of released systems on dependency-constrained workflows:

  • Implementations. We kept their scheduling and serving code untouched, writing only the task workflow definitions.
  • Why they trail Sequential. Both releases are built on older vLLM versions than our vLLM 0.11 baseline, so Fig. 5 reflects end-to-end system execution including engine version rather than isolating their scheduling algorithms.
  • Why these workflows. Fig. 5 asks a focused question: on dependency-constrained workflows, do serving-level optimizations remove the parent–child barrier? Because both systems still start a child only after its parent finishes, their techniques are complementary to, not a substitute for, overlapping dependent decoding, and Section 5.2 scopes our comparison accordingly.
  • Fair same-engine baselines. Within the exact same vLLM 0.11 engine, ACCORDION outperforms Sequential and FlashAgents-style streaming prefill at 1.08–1.37× versus at most 1.03× in Appendix E.2, beats on average the four fixed-chunk schedules, including held-out Chunk-64 in Appendix F.6, and the entropy and delimiter triggers in Appendix E.9, and adds 1.18× on top of EAGLE-3.

W8: prompts and code.

Our anonymized code is live at https://anonymous.4open.science/r/accordion-7F3A/ and includes:

  • the ACCORDION runtime as an overlay on vLLM 0.11;
  • reproduction and table-printing scripts for Table 3, the Sequential and chunk runs of Table 1, the fixed-default runs of Appendix F.6, and Appendices E.4 and E.8;
  • the MMLU and GSM8K question loaders, plus runners for ResearchBench and the CriticBench study.

Appendix D now includes the full ResearchBench and CriticBench prompts, while PaperBananaBench runs on the released OpenManus setup described in Appendix E.3.

These responses address your concerns about fairness, reliability, and reproducibility: the main results now have three runs and paired CIs, the question selection is documented, and the code is public. We would appreciate it if you would update your score accordingly, and we are glad to discuss any remaining point.

Reviewer_av25OpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

In multi-agent LLM workflows, a downstream agent usually has to wait until the upstream agent finishes its whole message. This paper lets the downstream agent start generating early from the partial message, then check its draft whenever more of the message arrives and redo only the part that no longer fits. The upstream agent decides when to send new pieces based on how surprising its own output is, which the authors motivate with an information-theoretic argument. Across several reasoning and coding workflows, this reduces end-to-end latency with similar accuracy.

Critical strengths

The problem is real and the idea is simple: letting downstream agents start early is a natural way to cut waiting time. The evaluation is also broad, across tasks, workflow shapes and model sizes.

Critical weaknesses
  • The speedup comes from accepting approximate outputs, not from true speculation. Unlike standard speculative decoding (Leviathan et al., 2023), the child keeps drafted tokens that it would not have produced with the full message. When the check is made exact, the speedup disappears completely (Table 3). So the method trades some output fidelity for speed, which should be stated clearly and tested on open-ended tasks, where small differences matter more.
  • It is unclear whether the surprisal-based policy is better than simply using fixed-size chunks. The theoretical analysis makes assumptions that do not fully hold in practice, and the threshold still needs to be tuned. As shown in Table 1, using fixed-size chunks achieves similar speedups and accuracy in most settings. This makes it hard to see the benefit of the more complicated surprisal-based approach.
Questions for authors
  1. With exact (top-1) verification the speedup is 1.00x. Why does exact prefix reuse almost never happen, given the intuition in Figure 1 that the first mismatch comes late?
  2. On open-ended tasks, how close are ACCORDION's outputs to the sequential outputs (e.g. token-level or semantic agreement)?

References:

  • Leviathan, Y., Kalman, M., and Matias, Y. Fast Inference from Transformers via Speculative Decoding. ICML 2023.
3 comments

Reviewer av25 (1/2)

Thank you for your thoughtful review, and for finding the parent–child barrier real, the core idea clean, and the evaluation broad. You raise the central question of output fidelity, and our new measurements answer it directly: question by question, ACCORDION's final answers agree with Sequential about as often as two independent Sequential runs agree with each other. Relaxed Top-5 acceptance unlocks 1.20–1.79× speedups where strict greedy Top-1 gives 1.00×, changing wording rather than conclusions.

Color-coded revision guide: Main-text updates for your review appear in purple in the Abstract, Section 5.3 on Top-1 versus Top-5 verification, and the Ethics statement, alongside Section 3.4 on ACCORDION-online in blue and Table 3 in vermillion, with full analyses in Appendices C.2, E.11, and F.

W1 & Q2: approximate outputs, and how close they are to Sequential's.

ACCORDION performs verified speculation with rollback, where kk sets the trade-off between speed and fidelity explicitly. After every commit, the child re-checks its draft under the updated context, keeps tokens that stay within its Top-kk predictions, and rolls back from the first rejection. Top-1 keeps only exact greedy choices, to which the distribution-preserving verifier in Appendix C.2 also reduces under greedy decoding, whereas our default Top-5 is a relaxed acceptance rule in the family of typical acceptance in Medusa by Cai et al. (2024). The revised Abstract and Section 5.3 state this explicitly.

Table R8. Verification rule vs. speed, accuracy, and agreement with Sequential on four-agent MMLU from the unified rerun of Table R6.

VerificationSpeedupAccuracySame final answer as Sequential
Top-1, exact1.00×73.0%81.0%
Top-5, default1.42×76.0%83.5%
Top-∞\infty, no rollback1.75×73.5%81.0%

Notably, two Sequential runs produce the same final answer on 77.0% of these questions, so Top-5 agrees with Sequential at 83.5%, at least as often as Sequential agrees with itself. Whereas Top-1 discards nearly every draft and Top-∞\infty never rolls back stale text, Top-5 rolls back precisely when a parent update makes a drafted token implausible.

Across multi-run benchmarks for Q2, question-by-question answer agreement matches Sequential's own run-to-run consistency:

Table R9. Share of questions with the exact same final answer.

SettingACCORDION vs. SequentialSequential vs. another Sequential run
MMLU, 4 / 6 agents77.2 / 79.0%77.7 / 79.3%
GSM8K, 4 / 6 agents97.8 / 98.2%98.3 / 97.7%

MMLU outputs also depend on live Wiki search results. At the text level, the approximation alters surface phrasing, not final conclusions: on six-agent GSM8K, final messages are word-for-word identical to Sequential on 28% of questions versus 70% between two Sequential runs, while final answers agree on 98.2%.

On the open-ended ResearchBench task in Fig. 6c, where each final research proposal is scored by embedding similarity to the reference proposal, ACCORDION scores at least as high as Sequential in every agent-count group, reaching 0.816 versus 0.789 overall, consistent with preserved pass@1 across three runs of HumanEval code generation in Table 1:

Table R10. ResearchBench embedding similarity to the reference proposal by agent count.

Agents35791113151719+All
Workflows1722211512868100
Sequential0.8260.7840.8130.7960.7850.7820.7610.7950.7470.789
ACCORDION0.8450.8190.8240.8290.8090.8160.7800.8390.7850.816

The revision adds Tables R8–R11 and per-question flip counts as Appendix E.11.

Reviewer av25 (2/2)

Q1: why exact verification gives 1.00×, given Figure 1's "late divergence".

Exact verification gives 1.00× because per-token agreement is very high overall, yet the first lexical mismatch arrives early, and exact Top-1 must truncate the entire remaining draft at that first mismatch. We verified this directly by teacher-forcing every retained speculative segment under the child's complete final context across 2,657 interrupted child agents on four- and six-agent MMLU and GSM8K:

Table R11. Agreement of retained speculative tokens with the child's final-context predictions.

SettingTop-1 agreementTop-5 agreementLeading exact-match run, medianWhole segment equals argmaxTop-1 floor on final-context tokens
GSM8K, 4 / 6 agents97.5 / 98.1%100%6 / 7 tokens23 / 28%99.5 / 99.8%
MMLU, 4 / 6 agents94.2 / 95.1%100%22 / 14 tokens5 / 6%97.4 / 97.8%

Two results stand out. First, 100% of retained speculative tokens remain inside the child's Top-5 predictions under the complete final context, enforced by final-commit verification. Second, part of the Top-1 mismatch is purely numerical: tokens generated under the final context itself match full-prefix teacher-forced argmax only 97.4–99.8% of the time due to incremental and prefix-cached BF16 computation, so the evolving context lowers per-token Top-1 agreement by only 1.7–3.2 pp. Top-5 tolerates these benign early tie-breaks after 6–22 tokens.

W2: the surprisal policy vs. fixed-size chunks, the theory's assumptions, and threshold tuning.

  • Fixed chunks. With τ\tau and Mmax⁡M_{\max} adapted online via ACCORDION-online in Section 3.4, the surprisal rule is both the fastest schedule on average and the most consistent across tasks. Over the 12 settings of the three-run Table 1, ACCORDION-online reaches the highest mean speedup of 1.49×, beating 1.43× for the best single chunk, Chunk-48, and 1.41× for held-out Chunk-64 in Table R3. Crucially, it stays within 4.0% of the best fixed chunk size in every setting, whereas every fixed chunk falls 15–44% behind in at least one.
  • Theoretical assumptions. Section 4 states explicitly that Theorem 4.1 analyzes an idealized prefix-maximal verifier. Assumption 4.2, that parent surprisal bounds child-side surprisal up to a slack, is supported empirically in Fig. 4a alongside rising child distribution shift in Fig. 4b. Together they motivate parent surprisal as the right commit budget, while ACCORDION-online sets that budget from measured rebuild costs in Appendix F.4.
  • Threshold tuning. No per-task tuning is required. The warm-start default, τ=10\tau=10 and Mmax⁡=64M_{\max}=64, was chosen once on the 200 four-agent MMLU questions of Table 1, generalizes at 1.42–1.45× on unseen four-agent MMLU sets, and is used unchanged across models and topologies. Because optimal τ\tau varies by task in Appendix F.3, ACCORDION-online selects it automatically without sweeps, converging to τ\tau around 28–30 on MMLU, 7.25–9 on GSM8K, and 1.75–3.5 on HumanEval, while Mmax⁡M_{\max} caps uncommitted low-surprisal spans.

These answers make the fidelity trade-off explicit and measured, and they show that, with its parameters selected online, the surprisal rule is the fastest commit policy on average. We would appreciate it if you would update your score accordingly, and we are happy to provide more detail.

Thank you again for your review. Before the discussion closes, here's where your points stand in the revised paper:

  • Output fidelity. With Top-5 verification, final answers match Sequential on 83.5% of four-agent MMLU questions, more often than two Sequential runs match each other (77.0%). On GSM8K, agreement is 97.8–98.2%, the same as between two Sequential runs, and on ResearchBench ACCORDION's proposals score at least as high as Sequential's in every agent-count group (Appendix E.11).
  • Why Top-1 gives 1.00×. Every retained token stays within the child's Top-5 under the final context, and 94–98% equal its argmax; Top-1 just cuts the whole draft at the first argmax mismatch, which comes early. Since Top-5 accepts every prefix Top-1 accepts, the bound of Theorem 4.1 carries over to it (Appendix C.2).
  • Fixed chunks and tuning. ACCORDION-online now selects τ and M_max online, with no per-task tuning, and has the highest mean speedup (1.49× vs. 1.43× for the best fixed chunk) while staying within 4.0% of the fastest policy in every setting.

If anything is still unclear, we're happy to answer. If these settle your concerns, we'd appreciate it if you could take another look at your score.

Comments on the paper

1 comment

General response to all reviewers

We thank all reviewers for their careful reading and constructive feedback. We are encouraged that all four reviewers found partial-context decoding with verification and rollback a natural and useful mechanism and valued our broad evaluation across tasks, topologies, and model sizes, while also noting the prevalence of the parent–child barrier and our dynamic OpenManus study. Below we summarize our new experiments and manuscript updates; each response addresses every point in detail. Unless noted, new runs use the models and questions of Table 1.

Color-coded revision guide. Revisions in the PDF, including the appendices, are color-coded by reviewer: blue for GWP7 in the Abstract, Sections 1, 3.4, 5.2, and 6, and Ethics; green for g6kr in Section 1, Table 1, and Sections 5.1–5.2; vermillion for VQXt in Sections 5.1–5.3 and Tables 2–4; and purple for av25 in the Abstract, Section 5.3, and Ethics, with full details in Appendices C.2, D.4–D.5, E.4, E.6, E.8–E.11, F, and G.

ACCORDION-online in Section 3.4 and Appendix F. To remove manual hyperparameter tuning across workloads, the revision introduces ACCORDION-online while keeping our core commit rule unchanged. After each completed request, ACCORDION-online replays the parents' surprisal streams across candidate (τ,Mmax⁡)(\tau, M_{\max}) pairs, prices each pair by the measured cost of rebuilding running children's contexts and discarding drafts, and uses the best pair for the next request. It runs between requests and off the critical path in 21–39 ms per request, requiring no validation set. Table 1 now reports ACCORDION in this mode.

What the new experiments establish

  1. Statistically significant speedups across three full runs of Table 1. Reporting three runs of every policy across MMLU, GSM8K, and HumanEval in Table 1 and Table R4, ACCORDION-online cuts latency by 16.7–44.0%, or 1.20–1.79×, with every 95% CI excluding zero and a mean accuracy change of −0.11 pp. Appendix E.8 adds six paired runs of the fixed default policy, including two unseen, disjoint 200-question sets that achieve 1.20–1.52× speedups.
  2. Cross-task robustness with zero per-task tuning. Optimal commit granularity depends strongly on the task, and ACCORDION-online adapts to it on its own. Across all 12 settings of Table 1, it achieves the highest overall mean speedup of 1.49×, ahead of 1.43× for the best single chunk, and stays within 4.0% of the best fixed chunk size in every setting, whereas every fixed chunk falls 15–44% behind in at least one, as detailed in Section 3.4 and Appendix F.
  3. Rebuild-aware cost modeling explains when commits pay off. Appendix F.4 shows that one commit rebuilds 1.2–3.4 running children while larger deliveries discard more drafts. Pricing both costs directly, ACCORDION-online commits sparingly on MMLU with τ\tau around 28–30 and frequently on HumanEval with τ\tau around 1.75–3.5.
  4. Final answers match Sequential within normal run-to-run variation. Across Tables R8–R11 in Appendix E.11, ACCORDION agrees with Sequential about as often as two Sequential runs agree with each other. Moreover, 94–98% of retained speculative tokens equal the child's final-context greedy choice, 100% fall within final-context Top-5, and on open-ended ResearchBench ACCORDION scores at least as high as Sequential at every agent count.
  5. Measured deployment boundaries and admission guards. In Tables R1 and R7 and Appendix G, ACCORDION raises low-load throughput by 1.43–1.47×, a calibrated load switch is never worse than Sequential across our concurrency sweep, and a fan-in router raises overall ResearchBench speedup from 1.06× to 1.14×.
  6. Seamless composition with model-level speculation. Stacked on top of EAGLE-3, ACCORDION yields an additional 1.18× speedup on the Table 1 MMLU questions.

Code. Our anonymized repository is available at https://anonymous.4open.science/r/accordion-7F3A/ and documented in our response to Reviewer VQXt.

Manuscript updates

  • Tables 1, 2, and 4: Table 1 now reports mean ± std over three runs of every policy with means quoted throughout the text, and Tables 2 and 4 report ACCORDION-online paired against Sequential.
  • Section 3.4 and Appendix F: Introduce and analyze ACCORDION-online. Its warm-start setting, τ=10\tau=10 and Mmax⁡=64M_{\max}=64, was chosen once on the 200 four-agent MMLU questions of Table 1, generalizes at 1.42–1.45× on unseen four-agent MMLU sets in Table R5, and is used unchanged for every other task, model, topology, and mixed Qwen/Llama pair in Table R2.
  • Expanded appendices: We add the real-task load sweep in Appendix E.4, token accounting in E.6, entropy and delimiter schedules in E.9, cross-family runs in E.10, fidelity audits in E.11, and deployment guidance in Appendix G.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.