acceptodds
Under review as a conference paper at ICLR 2027

Coordinated Mixed-Precision KV Cache Quantization for Efficient LLM-based Multi-agent Systems

Abstract

LLM-based multi-agent systems solve complex tasks through problem decomposition, specialized tool use, and iterative verification. However, collaboration produces multiple growing key-value (KV) caches that can coexist in memory, making cache storage a bottleneck. KV-cache quantization reduces this footprint, but independently applying single-model policies ignores differences in agents' precision needs. This can over-compress sensitive computations while spending precision on more tolerant caches, potentially degrading collective reasoning. In this paper, we introduce **Co**ordinated **Mix**ed-Precision **KV** Cache Quantization (**CoMixKV**), a novel method that integrates online sensitivity estimation with joint bit-width allocation under a shared memory budget. To our knowledge, **CoMixKV** is the first mixed-precision KV-cache quantization framework to jointly allocate precision across simultaneously resident agent caches under such a budget. An online estimator derives each agent's sensitivity from a local attention-error bound using recent queries, keys, and values, without calibration data, trial quantization, or additional model passes. A shared controller combines these scores with remaining quantizable capacity and byte-level storage costs to minimize total estimated error. Allocations are updated as agents start or finish and fresh observations become available. Selected widths apply only to newly quantized blocks; recent states retain native precision and previously quantized blocks remain unchanged. Across four benchmarks and three LLM backbones at average 2-bit, 3-bit, and 4-bit KV-cache settings, **CoMixKV** achieves strong accuracy compared with the evaluated quantization baselines. At the average 3-bit setting on Gemma-4-31B-it, **CoMixKV** achieves 68.35% average accuracy, outperforming the strongest quantized baseline by 0.59%. On Qwen3-32B, **CoMixKV** further reduces peak KV-cache memory by 32.08% and achieves a 1.38 inference speedup over FullKV.

Primary area:
foundation or frontier models, including LLMs
Keywords:
LLM-based Multi-agent Systems, KV Cache Quantization

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_Upe7OpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

This paper introduces a quantization technique that applies different bit precisions token-blockwise and layerwise in LLM-based multi-agent systems. Within a fixed bitwidth budget, the tokens generated online are grouped into blocks, the layer-wise quantization sensitivity is estimated for each block, and a bit width is allocated to each layer. The experiments were conducted in a multi-agent system, and within the experimental results, it showed superiority over other baselines.

Critical strengths
  1. The process of deriving a calibration-free upper bound on the quantization error for each layer and deriving the layerwise sensitivity from it is reasonable. This derivation method can inspire other studies that aim to update the hyperparameters of quantization, pruning, compression, etc. online.
Critical weaknesses
  1. The multi-agent graph in S3.1, the method in S4, and the experimental setting in S5 are not aligned with each other. The method presents a quantization approach that assigns layer-wise bit widths online. However, the estimation method used in the method can be used regardless of the multi-agent environment, and since a fixed budget is used for each token block within each agent, there is also no budget exchange between agents at all. Unlike the multi-agent graph described in S3.1, the experimental setting in S5 has no case in which two or more subtasks are executed in parallel, and it deals with a situation in which only one agent is always operating sequentially.

  2. The experimental results are insufficient to support the claims. In Table 1, the proposed method outperforms the other baselines at 3 bits, but in Table 2, when the weighted mean is computed considering the number of questions in each benchmark, it is difficult to regard it as outperforming KIVI and PM-KVQ. In addition, in both tables, there are frequent cases in which the quantization methods exceed the accuracy of Vanilla, and considering the magnitude of the gap from Vanilla and the gap between the different methods, it is difficult to regard these as reliable experimental results.

Questions for authors
  1. To me, the introduced methodology appears to be, rather than belonging to the field of multi-agent systems, the existing KIVI method with layer-wise online bit width assignment added. While each token block uses a fixed budget, could you explain how different agents or the state of the multi-agent graph can affect the precision decision?

  2. In the ablation Table 8, is Uniform allocation the same method as the KIVI baseline? If that is the case, the accuracy difference between Online sensitivity updates and Uniform allocation on AIME25 (which is 3.33 %p) corresponds to only a single question. Since this paper deals with experiments that include the use of external tools, the accuracy variation due to changes in tool outputs can be large. (Please let me know if external tools were not used in Table 8.) Therefore, in order to claim superiority in accuracy, it would be good to report the main results, including this table, over multiple runs together with the variance.

4 comments
AuthorsOpenReview ↗

Response to Reviewer Upe7 (1/2)

We greatly appreciate your time, detailed comments, and valuable suggestions.

W1(a) The multi-agent graph in S3.1, the method in S4, and the experimental setting in S5 are not aligned with each other. The method presents a quantization approach that assigns layer-wise bit widths online. However, the estimation method used in the method can be used regardless of the multi-agent environment, and since a fixed budget is used for each token block within each agent, there is also no budget exchange between agents at all.

Thank you for pointing out that the connection between these sections needed clarification. The original reasoning experiments in Section 5 apply the allocation method in Section 4 while following the execution dependencies described in Section 3.1.

  1. Section 3.1 describes execution dependencies. The graph edges represent task delegation, the transfer of results, and verification feedback. Executing the manager, workers, and verifier sequentially is one valid execution of this graph.
  2. Section 4 specifies how precision is allocated within an invocation. CoMixKV estimates layer sensitivity online from the QKV observed in each invocation. For newly quantized cache blocks, it assigns higher widths to sensitive layers and balances them with lower widths in less sensitive layers to preserve the target mean packed-code width. This allocation can run within a single invocation.
  3. Section 5 evaluates this allocation policy. The original experiments evaluate online layer-wise allocation in sequential reasoning workflows at fixed target mean bit widths. The revision adds a parallel coding workflow to examine how precision is allocated under a shared budget when multiple request caches are resident together (Section 5.5). In this workflow, the controller constrains the total reserved KV bytes, and per-request mean widths can vary.

Q1 To me, the introduced methodology appears to be, rather than belonging to the field of multi-agent systems, the existing KIVI method with layer-wise online bit width assignment added. While each token block uses a fixed budget, could you explain how different agents or the state of the multi-agent graph can affect the precision decision?

  1. Connection to KIVI. Both methods group keys by channel and values by token. KIVI's grouping determines how to encode a cache block at a given width. CoMixKV uses QKV from the current invocation to select the width of each layer online. With concurrent requests, it also decides which layer of which request should receive extra precision under the shared budget.
  2. How agents and execution state affect allocation. Inputs and histories differ across invocations, producing different QKV observations and potentially different layer widths at the same target mean. In parallel coding, the execution graph also determines which requests run concurrently. For example, when one worker completes and releases its cache, a worker that is still running may use the freed capacity to raise the precision of future blocks in a sensitive layer. Previously quantized blocks remain charged at their stored widths, and updates affect subsequent blocks (Sections 3.2 and 5.5).

W1(b) Unlike the multi-agent graph described in S3.1, the experimental setting in S5 has no case in which two or more subtasks are executed in parallel, and it deals with a situation in which only one agent is always operating sequentially.

The graph in Section 3.1 describes task dependencies and message passing among agents. The original reasoning experiments in Section 5 use a sequential schedule. For each problem, the manager delegates subtasks to specialized workers, combines their reports, and passes a candidate answer to the verifier. One invocation generates tokens at a time, with different agents carrying out different stages of the same task. This execution order follows the dependencies in the graph in Section 3.1.

To evaluate a graph with parallel branches, the revision adds a parallel coding experiment. After the manager finishes planning, two workers concurrently generate candidate code and test assertions. The verifier then revises the code using execution feedback (Section 5.5). We traced the same 16 prespecified tasks under both budgeting policies and observed concurrent KV cache residency for the two workers in all 32 traces. Under the shared budget, we recorded 632 allocations during cache overlap. In 525 events, a worker’s reservation exceeded its independent limit, with zero recorded team-budget violations (Section 5.5).

AuthorsOpenReview ↗

Response to Reviewer Upe7 (2/2)

W2(a) The experimental results are insufficient to support the claims. In Table 1, the proposed method outperforms the other baselines at 3 bits, but in Table 2, when the weighted mean is computed considering the number of questions in each benchmark, it is difficult to regard it as outperforming KIVI and PM-KVQ.

Table 2 of the original submission compares performance across four types of reasoning tasks, so Bamboogle, GAIA, AIME25, and GPQA each receive 25% of the weight. Weighting by question count gives GPQA’s 198 questions 43.42% of the overall score and AIME25’s 30 questions 6.58%, a 6.6-fold difference. Applying these weights to our cross-benchmark comparison would make the influence of each task type depend on benchmark size. For our evaluation objective, the equal-weight macro-average in the original submission provides a fair comparison across all four task types.

W2(b) In addition, in both tables, there are frequent cases in which the quantization methods exceed the accuracy of Vanilla, and considering the magnitude of the gap from Vanilla and the gap between the different methods, it is difficult to regard these as reliable experimental results.

A model with a quantized KV cache can score higher than its full-precision counterpart on individual tasks. One possible explanation is that quantization maps KV activations in the same bin to a single value, which can suppress small variations that are irrelevant to the task. The resulting changes in attention can alter subsequent generations and lead to more accurate answers for some inputs.

Prior work on KV-cache quantization reports similar results on specific tasks.

  • KIVI, Table 3. On Llama-2-13B, 4-bit KIVI scores 23.65% on GSM8K versus 22.67% for the 16-bit reference, a gain of 0.98 percentage points.
  • PM-KVQ, Table 2. On DeepSeek-R1-Distill-Llama-8B, the mixed 4/8-bit configuration with batch size 6 achieves a mean AIME 2024 pass@1 of 47.71% versus 44.17% for the 16-bit reference, a gain of 3.54 percentage points.
  • KVQuant, Table 3. On LLaMA-2-7B-32K, the 3-bit configuration retaining 1% of values as outliers scores 13.67 F1 on Qasper in LongBench, compared with 10.51 for the FP16 reference, a gain of 3.16 points.

Q2(a) In the ablation Table 8, is Uniform allocation the same method as the KIVI baseline? If that is the case, the accuracy difference between Online sensitivity updates and Uniform allocation on AIME25 (which is 3.33 %p) corresponds to only a single question.

Uniform allocation is an ablation within CoMixKV. It uses the CoMixKV quantization implementation and fixes all quantized layers at 4 bits. Online allocation uses the same target mean bit width and assigns widths by layer sensitivity. KIVI is evaluated as a separate baseline in Section 5.5. The Uniform ablation therefore measures the effect of online allocation versus fixed allocation within the same quantization implementation.

Your interpretation of the difference is correct. AIME25 contains 30 problems, so 3.33 percentage points corresponds to one additional correct answer. In Section 5.5 of the revised manuscript, online allocation solves 11/30 problems and Uniform solves 10/30, so the observed difference in this ablation is one problem.

Q2(b) Since this paper deals with experiments that include the use of external tools, the accuracy variation due to changes in tool outputs can be large. (Please let me know if external tools were not used in Table 8.) Therefore, in order to claim superiority in accuracy, it would be good to report the main results, including this table, over multiple runs together with the variance.

  1. Tool use in Table 8. AIME25 uses a closed-book setting with web search and page visits disabled. Local Python is available for computation. In the revised ablation, online allocation and Uniform use the same tool permissions and random seed. The execution logs show Python calls in both conditions (Section 5.5).

  2. Repeated runs and variability. In the main 4-bit AIME25 comparison, we evaluated CoMixKV and KIVI with seeds 0 and 1. CoMixKV leads in both runs, with a mean difference of +5.00 percentage points and a question-paired 95% confidence interval of [−8.33, 16.67] (Section 5.5). The online-versus-Uniform ablation uses seed 0. Its observed difference is +3.33 percentage points, with a question-paired 95% confidence interval of [−13.33, 20.00] (Section 5.5). These intervals quantify uncertainty across questions.

Thank you for the additional experiments and the effort put into the revision. I note that the rebuttal was submitted less than a day before the end of the discussion period. I have only partially gone through it so far, and I have not been able to review it thoroughly enough to engage in further discussion before the deadline. I will carefully review the rebuttal and the revised manuscript during the remaining review period after the discussion closes.

Thank you for your thoughtful review and for taking the time to follow up. We recognize that our response came late in the discussion period, leaving little time for further exchange. We appreciate your willingness to read the rebuttal and revised manuscript after the discussion closes. Section 5.5 addresses your question about how one agent can affect another’s precision allocation. In the parallel coding workflow, the traces show that one worker could use KV-cache capacity left unused by the other while staying within the team’s total budget. On HumanEval+, sharing this capacity improved accuracy by 9.15 percentage points over independent allocation at the same total budget. Thank you again for raising this question and for taking the time to consider the revision.

Reviewer_XuLnOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes COMIXKV, a mixed-precision KV-cache quantization framework for LLM-based multi-agent systems. It estimates layer-wise sensitivity from recent queries, keys, and values, then uses these estimates to assign bit widths under a shared memory budget. Updated assignments apply only to newly quantized blocks, while recent states retain their native precision.

Critical strengths

S1. I find the problem practically relevant. Multiple agents can maintain KV caches simultaneously, creating memory pressure even when they are not generating tokens at the same time. Reducing this memory footprint is a useful direction for deploying multi-agent systems. S2. I also appreciate that the sensitivity estimator does not require offline calibration data or additional model forward passes. Using observations collected during ordinary inference makes the method easier to apply and allows the allocation to respond to changes during execution. S3. The cache-management design is sensible. Preserving recent states at native precision and leaving previously quantized blocks unchanged avoids repeatedly requantizing historical caches. These choices show attention to the practical costs of updating precision during inference.

Critical weaknesses

W1. My main concern is that the accuracy improvements are quite small. Across the main evaluated settings, the average gains over the strongest quantized baseline range from 0.04 to 0.59 percentage points. Without repeated-run results or confidence intervals, I am not convinced that these margins establish a reliable advantage. Reporting variability would help distinguish consistent improvements from fluctuations in generation and tool interactions. W2. I would also like to see an evaluation on coding or software-engineering tasks. Coding is an important application of LLM agents, and these tasks often involve extended tool interactions and growing execution histories. The current benchmarks cover several reasoning domains, but they leave this application area unexplored. A coding evaluation would strengthen the evidence for the method’s applicability to agent workloads. W3. I noticed an inconsistency between the main and ablation results that needs clarification. For Qwen3-32B on AIME25 with an average 4-bit KV-cache budget, Table 2 on page 15 reports 30.00% accuracy for COMIXKV, whereas Tables 7 and 8 on pages 31–32 report 40.00% for the default online configuration. I could not find an explanation of any configuration or evaluation-protocol differences accounting for this gap. The authors should reconcile these numbers, since the discrepancy makes the ablation results difficult to interpret.

Questions for authors

Pls see weaknesses

1 comment
AuthorsOpenReview ↗

Response to Reviewer XuLn

Thanks for your careful review and helpful comments.

W1 My main concern is that the accuracy improvements are quite small. Across the main evaluated settings, the average gains over the strongest quantized baseline range from 0.04 to 0.59 percentage points. Without repeated-run results or confidence intervals, I am not convinced that these margins establish a reliable advantage. Reporting variability would help distinguish consistent improvements from fluctuations in generation and tool interactions.

Thank you for pointing out that these small gains warrant repeated runs. In the original 3-bit experiments in Table 1, CoMixKV and TurboQuant are both within one percentage point of FullKV in macro-average accuracy across four benchmarks on each model. At this accuracy level, CoMixKV leads TurboQuant by 0.36 and 0.59 percentage points on Qwen3.8-27B and Gemma-4-31B-it, respectively, and PM-KVQ by 3.52 and 1.23 points. In the original 2-bit AIME25 comparison in Table 2, CoMixKV scores 40.00% and PM-KVQ scores 36.67%.

To see how the result varies with the seed, we repeated the 4-bit AIME25 comparison between CoMixKV and KIVI on Qwen3-32B. Both methods use the same evaluation settings, tool permissions, and random seed within each run.

Random seedCoMixKVKIVIDifference (percentage points)
036.67%30.00%+6.67
143.33%40.00%+3.33

The difference is positive at both seeds and averages +5.00 percentage points. We resample paired outcomes by question, keeping both seeds together for each sampled question. The resulting 95% confidence interval is [−8.33, 16.67] percentage points and includes zero, so the magnitude of the gain remains uncertain (Section 5.5 of the revised manuscript).

W2 I would also like to see an evaluation on coding or software-engineering tasks. Coding is an important application of LLM agents, and these tasks often involve extended tool interactions and growing execution histories. The current benchmarks cover several reasoning domains, but they leave this application area unexplored. A coding evaluation would strengthen the evidence for the method’s applicability to agent workloads.

Thank you for suggesting a coding evaluation. We added a function-level coding workload in which agents generate code, run tests, and use the execution feedback to revise their solutions.

  1. Code generation and execution feedback. A manager plans the solution, and two workers generate candidate code and test assertions concurrently. The candidates are checked against the generated tests, and a verifier revises and submits the final code based on the execution results. The official hidden tests are used for final scoring (Section 5.5).

  2. Cache allocation during parallel execution. Each worker retains its own KV cache while running. Across the 16 tasks selected in advance for tracing, we recorded 632 allocation events while both caches were resident. In 525 events, a worker's reservation exceeded its independent limit, with zero recorded team-budget violations (Section 5.5).

  3. HumanEval+ results. In this workflow, CoMixKV passes 109/164 tasks (66.46%), compared with 101/164 (61.59%) for KIVI. CoMixKV improves accuracy by 4.88 percentage points (Section 5.5).

W3 I noticed an inconsistency between the main and ablation results that needs clarification. For Qwen3-32B on AIME25 with an average 4-bit KV-cache budget, Table 2 on page 15 reports 30.00% accuracy for COMIXKV, whereas Tables 7 and 8 on pages 31–32 report 40.00% for the default online configuration. I could not find an explanation of any configuration or evaluation-protocol differences accounting for this gap. The authors should reconcile these numbers, since the discrepancy makes the ablation results difficult to interpret.

We checked the archived default online run for Qwen3-32B at 4 bits. It scores 40.00% (12/30) on AIME25, consistent with Tables 7 and 8 of the original submission. The revised Table 2 in Appendix A reports this result and updates the four-benchmark macro-average to 51.06%.

Reviewer_7N4COpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper presents COMIXKV, which utilizes dynamic bit-width quantization to compress KV caches differently across agent roles and layers, according to their quantization sensitivity estimated from local attention output errors. The results show that, at equivalent average bit widths, COMIXKV outperforms other baselines and comes close to FullKV in accuracy. It also demonstrates 1.38× speedup and 32.08% memory reductions compared with FullKV.

Critical strengths
  1. The online sensitivity measurement in this paper adapts precision without need of calibration data or other additional heavy data-path, which makes the estimation light-weighted.

  2. The evaluation is sound, with different aspects from accuracy to systematic metrics, such as speedup and memory efficiency.

Critical weaknesses
  1. While COMIXKV mainly applies online mixed-precision KV-cache quantization to MAS, the contribution beyond the sensitivity estimator remains unclear, which limits the novelty of this paper.

  2. The experimental results needs to be more clarified, the accuracy gains over strong quantization baselines are small, some of the results are not consistent in the paper, such as Qwen3-32B’s 4-bit accuracy in AIME in Table 2 (30%) and Table 8 (40%).

Questions for authors
  1. Under the same quantization bitwidth, what is the point to arrange the union allocation among agents, rather than independently allocate the bitwidth of each agent?

  2. Some of the experimental results are not consistent in the paper as mentioned in the "Weakness" part. Please explain it if it's under different experimental settings.

  3. The MAS workflow used in the paper is relatively constrained and naive, the real MAS workflow may appear different and more dynamic than the architecture presented in the paper, can you evaluate the method on more realistic workflows or traces?

  4. Are the small accuracy gains over the strongest quantized baselines consistent across repeated runs? Please provide variance estimates or confidence intervals.

4 comments
AuthorsOpenReview ↗

Response to Reviewer 7N4C (1/2)

We greatly appreciate your time, detailed comments, and valuable suggestions.

W1 & Q1 While COMIXKV mainly applies online mixed-precision KV-cache quantization to MAS, the contribution beyond the sensitivity estimator remains unclear, which limits the novelty of this paper. Under the same quantization bitwidth, what is the point to arrange the union allocation among agents, rather than independently allocate the bitwidth of each agent?

Thank you for pointing out the connection between CoMixKV and prior work on KV-cache quantization. We agree that CoMixKV shares the goal of reducing quantization error within a limited cache budget. We build on this principle by deciding which request and layer should receive extra precision when several agents’ caches are resident at the same time.

  1. Quantization representation. CoMixKV and prior work are similar in that they group keys and values separately. KIVI, for example, groups keys per channel and values per token and retains a native-precision tail. They differ in that KIVI's grouping addresses how to encode each cache block at a given width, whereas CoMixKV separates block encoding from precision allocation. It supports multiple candidate widths with a common codec and uses layer sensitivity and cache cost across requests to decide which layer of which request can receive higher precision.

  2. Sensitivity and bit-width updates. CoMixKV and prior mixed-precision methods are similar in that they use differences in sensitivity across layers to allocate precision. PM-KVQ, for example, uses calibrated sensitivities to allocate precision across Transformer blocks. They differ in when they update bit widths and which blocks they update. PM-KVQ progressively lowers the precision of an existing cache as a request's sequence grows. CoMixKV estimates sensitivity online from QKV observed during ordinary inference and updates widths for future blocks when a request starts, finishes, or yields new observations. Previously quantized blocks retain their stored widths and count toward the budget. Even with the same estimated layer sensitivities, changes in which requests are resident and how many bytes their historical blocks occupy affect which upgrades fit.

  3. Allocation across agents at the same total budget. CoMixKV and independent allocation are similar in that they both choose widths by layer. They differ in whether requests can draw on each other's unused reservations. With two parallel workers, independent allocation reserves half for each request, which selects per-layer widths within its own reservation. When one leaves part of its reservation unused, CoMixKV can use the spare bytes for a sensitive layer of the other, such as upgrading its future blocks from 2 to 3 bits. The solver ranks upgrades by estimated error reduction per additional byte. With the same width options and total bytes, requests can therefore receive different mean widths as needed.

    Among 632 allocations recorded with both caches resident, one worker’s reservation exceeded its independent ceiling in 525 events, with zero recorded team-budget violations (Section 5.5). Compared with independent layer allocation, CoMixKV gains 9.15 percentage points on HumanEval+ (Section 5.5). These experiments show that joint allocation across agents, as used by CoMixKV, is necessary to redirect unused capacity to requests that need higher precision under a fixed team budget in the parallel coding setting we evaluated.

AuthorsOpenReview ↗

Response to Reviewer 7N4C (2/2)

W2(a) & Q4 The experimental results needs to be more clarified, the accuracy gains over strong quantization baselines are small, some of the results are not consistent in the paper, such as Qwen3-32B’s 4-bit accuracy in AIME in Table 2 (30%) and Table 8 (40%). Are the small accuracy gains over the strongest quantized baselines consistent across repeated runs? Please provide variance estimates or confidence intervals.

Thank you for pointing out the size of the accuracy gains. In the 3-bit experiments in the original Table 1, both CoMixKV and TurboQuant, the strongest quantized baseline, are within one percentage point of FullKV in macro-average accuracy across four benchmarks. With both methods already close to unquantized accuracy, the average gains over TurboQuant are modest. CoMixKV still leads TurboQuant by 0.36 and 0.59 percentage points on Qwen3.8-27B and Gemma-4-31B-it, respectively. The gains over PM-KVQ are larger. The table below also includes the comparison with KIVI.

Source and settingCoMixKVBaselineDifference (percentage points)
Original Table 1 · Qwen3.8-27B, 3 bits, macro-average across four benchmarks80.91%PM-KVQ 77.39%+3.52
Original Table 1 · Gemma-4-31B-it, 3 bits, macro-average across four benchmarks68.35%PM-KVQ 67.12%+1.23
Original Table 2 · Qwen3-32B, 2 bits, AIME2540.00%PM-KVQ 36.67%+3.33
Revised Section 5.5 · Qwen3-32B, 4 bits, AIME25 (seeds 0 / 1)36.67% / 43.33%KIVI 30.00% / 40.00%+6.67 / +3.33, mean +5.00

In the 4-bit AIME25 comparison, CoMixKV and KIVI use the same evaluation settings and matched random seeds. CoMixKV leads KIVI in both runs. The question-paired 95% confidence interval for the mean difference across the two seeds is [−8.33, 16.67] percentage points and includes zero (Section 5.5 of the revised manuscript).

W2(b) & Q2 The experimental results needs to be more clarified, the accuracy gains over strong quantization baselines are small, some of the results are not consistent in the paper, such as Qwen3-32B’s 4-bit accuracy in AIME in Table 2 (30%) and Table 8 (40%). Some of the experimental results are not consistent in the paper as mentioned in the "Weakness" part. Please explain it if it's under different experimental settings.

Thank you for pointing out this inconsistency. The revised Table 2 reports 40.00% (12/30) for the archived default online run, consistent with Tables 7 and 8 of the original submission. We updated the four-benchmark macro-average to 51.06% and synchronized Section 5.2 and Figure 4(a).

Q3 The MAS workflow used in the paper is relatively constrained and naive, the real MAS workflow may appear different and more dynamic than the architecture presented in the paper, can you evaluate the method on more realistic workflows or traces?

Thank you for the suggestion. In the revised manuscript, we add a parallel coding workflow. Below we report execution traces and HumanEval+ results.

  1. Parallel coding workflow. A manager first prepares a plan. Two workers then generate candidate code and test assertions concurrently. A verifier uses feedback from running the generated tests to revise and submit the final code. The two workers overlap in execution, and their KV caches are resident at the same time. Official hidden tests are used only for final scoring (Section 5.5).

  2. Execution traces. Across the 16 tasks prespecified for tracing in this coding workflow, we recorded 632 allocation events with both worker caches resident. In 525 events, a worker received a reservation above its independent allocation limit, with zero recorded team-budget violations (Section 5.5). The traces show that CoMixKV uses shared capacity across requests during parallel execution.

  3. HumanEval+ comparison with KIVI. In the parallel coding workflow, CoMixKV passes 109/164 tasks (66.46%), compared with 101/164 (61.59%) for KIVI, a gain of 4.88 percentage points (Section 5.5).

Thanks for the new parallel coding experiment and for correcting the AIME25 numbers. My remaining concern is whether the small accuracy gains over the strongest quantized baselines are consistent. The repeated AIME25 comparison is against KIVI in one setting, and its confidence interval still includes zero, which does not fully resolve this concern. I will keep my current rating.

Thank you for your careful review and for following up on the revised results. Section 5.5 reports the evaluation settings, the results for each matched seed, and the paired 95% confidence interval. CoMixKV achieves strong accuracy even under tight KV-cache budgets. At 3 bits, its average accuracy across four reasoning benchmarks was within one percentage point of the full-precision KV-cache reference on both Qwen3.8-27B and Gemma-4-31B-it. It outperformed PM-KVQ by 3.52 and 1.23 percentage points on the two models, respectively. On 4-bit AIME25, CoMixKV outperformed KIVI at both matched seeds, with a mean gain of 5.00 percentage points. In the parallel HumanEval+ workflow, CoMixKV achieved 66.46% accuracy. Under the same total KV-cache budget, shared allocation improved accuracy by 9.15 percentage points over independent allocation. We will release our code as open source upon acceptance. Thank you again for taking the time to review the revision.

Comments on the paper

1 comment
AuthorsOpenReview ↗

Global Response

We thank all three reviewers and the area chair for their time and thoughtful feedback. The reviews asked us to clarify how CoMixKV differs from prior KV-cache quantization methods, when coordination across agents matters, and how to interpret the reported accuracy gains. We summarize our responses and manuscript revisions below.

Main questions addressed

  • Novelty and multi-agent coordination (7N4C and Upe7). We distinguish CoMixKV from KIVI and PM-KVQ in how it estimates sensitivity, updates widths for future blocks, and allocates a shared KV-cache budget when requests overlap. We also explain how the original sequential reasoning workflows follow the execution graph and how the new parallel coding workflow tests allocation when two KV caches are resident concurrently.
  • Reliability and interpretation of the results (all reviewers). We report matched-seed AIME25 comparisons and paired confidence intervals, clarify the role of Uniform allocation and local Python in the ablation, and explain why the four reasoning benchmarks receive equal weight in the macro-average. We also reconcile the AIME25 discrepancy using the archived default online result.

Changes in the revised manuscript

  • Added a parallel HumanEval+ evaluation and execution traces (Section 5.5). We describe how two workers generate code and tests concurrently and how the verifier revises the code from test feedback. Under the same total KV-cache budget, shared allocation improves accuracy by 9.15 percentage points over independent allocation. CoMixKV also leads KIVI by 4.88 points in the same workflow. The traces show that shared allocation can exceed an individual worker's independent reservation ceiling while respecting the team budget.
  • Added repeated 4-bit AIME25 results and the Uniform control (Section 5.5). CoMixKV leads KIVI at both matched seeds, with a mean gain of 5.00 percentage points. The paired 95% confidence interval includes zero. We also report the matched seed-0 comparison with Uniform and explain how the intervals were computed.
  • Updated the original AIME25 result (Table 2, Section 5.2, and Figure 4(a)). Table 2 now reports the archived default score of 40.00% (12/30) for 4-bit CoMixKV. The four-benchmark macro-average is updated to 51.06% in the table, main text, and figure.

The added text and updated values are marked in blue in the revised PDF.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.