- Rating
- 1
- Confidence
- 4
- Informativeness
- 1
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper proposes a cache-aware cost-efficient compressor to manage the context of LLM agent. The cost is controlled by directly appending new messages to the history, utilizing prefix cache. An additional step YAMATO is applied to prevent the loss of important tool-execution information. Experiments demonstrate that this method keeps performance on representative agentic benchmarks while reducing token cost.
Critical strengths
This paper shows promising experimental results: DMC-5 achieves performance improvement on all the three agentic benchmarks while reducing total cost.
Critical weaknesses
- The paper claims that fine-tuned or customized LLM endpoints often do not enjoy cached-input discount (in line 141). As far as I know, this is not true for most model providers. The referred document (https://help.aliyun.com/en/model-studio/context-cache) for this claim also does not contain this policy. Hence, the scenario of this paper may not hold.
- The method for optimizing cost is simply ensuring cache hit by appending new messages to the history messages, which is already a common practice, lacking novelty or contribution to real application.
Questions for authors
- Please provide more materials that support the fundamental claim that fine-tuned or customized LLM endpoints often do not enjoy cached-input discount.
- The input information of V and YAMATO all comes from the same source -- the whole trajectory for compression. It seems meaningless to design 2 steps. What about merging the two steps and getting the output via a single LLM request?
- The method and benchmarks seems specifically designed for function-calling agent, where the tools are all well-formulated API. While in practice, most current agents work as "coding agents", where the tool-calling may be arbitrary command and the tool result can be any unstructured information or logs. What about the performance of DMC-5 on these more common and complicated benchmarks, like swe-bench or tau-bench?
8 comments
Overall, it confirms that the original deployment scenario is common across the surveyed fine‑tuned endpoint offerings, while the additional cache‑enabled evaluation demonstrates that DMC‑5 also remains cost‑effective when the acting agent benefits from prefix caching. Together, these results show that the motivating setting is well represented in current serving configurations and DMC-5 is robust across different cache‑pricing configurations. We will refine the introduction to describe the deployment conditions more precisely.
W2. Contribution beyond append-only cache reuse
Thank you for raising this important question. We agree that appending new messages to history is a common practice and is not, by itself, our contribution. Rather, DMC‑5's contribution lies in jointly optimizing end‑to‑end inference cost, task performance, and execution‑state integrity though an asymmetric dual‑model allocation framework: an Append‑only Canonical View as the cache‑stable full‑history input for the compressor, while constructing a separate compact Decision View for the specialized agent, together with YAMATO's exact restoration.
(1) Optimizing actual end-to-end inference cost rather than token reduction alone.
Reducing agent input tokens does not necessarily reduce the actual inference bill. The total cost also depends on cached and uncached input prices, compressor overhead, output tokens, and additional agent interactions caused by information loss. DMC‑5 explicitly formulates these factors into a unified end-to-end cost objective (Eqs. 3‑6).
The following results from the original Table 1 and Fig. 3 demonstrate why this distinction matters:
| Benchmark | Method | Agent Input Reduction | Compressor Cache Hit | Total Cost | Task Score |
|---|---|---|---|---|---|
| AppWorld | No Compression | – | – | 13.71 | 77.0 |
| FIFO | 40.0% | – | 8.98 | 36.3 | |
| AgentDiet | 30.1% | 0% | 11.93 | 64.8 | |
| ACON | 38.0% | 0% | 11.57 | 67.3 | |
| DMC-5 | 46.6% | 80.3% | 10.31 | 78.2 | |
| OfficeBench | No Compression | – | – | 5.25 | 87.0 |
| FIFO | 30.6% | – | 3.95 | 59.3 | |
| AgentDiet | 28.0% | 0% | 4.13 | 83.8 | |
| ACON | 35.7% | 0% | 4.19 | 83.3 | |
| DMC-5 | 38.5% | 78.0% | 4.49 | 88.0 | |
| 8-objective QA | No Compression | – | – | 35.40 | 39.1 / 47.6 |
| AgentDiet | 59.9% | 0% | 25.88 | 35.5 / 43.6 | |
| ACON | 60.0% | 0% | 25.94 | 34.2 / 41.5 | |
| DMC-5 | 57.0% | 85.0% | 25.50 | 44.9 / 54.0 |
These results illustrate the distinction between token reduction and actual cost optimization. For example, on 8‑objective QA, DMC‑5 achieves a smaller agent‑input reduction than ACON (57.0% vs. 60.0%), yet incurs lower end‑to‑end cost (25.50 vs. 25.94) and achieves substantially higher EM (44.9 vs. 34.2). This directly demonstrates why agent‑side token reduction alone is an insufficient proxy for practical inference efficiency. Across all three benchmarks, DMC‑5 reduces actual cost by 14.5‑28.0% relative to No Compression, while maintaining or improving task performance.
(2) An asymmetric dual-model architecture that preserves specialized agent capability while reducing inference cost.
DMC‑5 assigns full‑history integration to a general‑purpose compressor and task‑specific action generation to a fine‑tuned agent. Table 4 directly evaluates this model-role allocation:
| Configuration | Compressor | Acting Agent | AppWorld Success | Total Cost |
|---|---|---|---|---|
| Specialized only | – | Fine-tuned Qwen3.5-27B | 77.0% | 13.71 |
| General only | – | DeepSeek-V4-flash | 70.6% | 6.93 |
| Specialized → Specialized | Fine-tuned Qwen3.5-27B | Fine-tuned Qwen3.5-27B | 77.4% | 21.49 |
| General → General | DeepSeek-V4-flash | DeepSeek-V4-flash | 67.9% | 9.07 |
| General → Specialized (DMC-5) | DeepSeek-V4-flash | Fine-tuned Qwen3.5-27B | 78.2% | 10.31 |
The results show that the general model alone does not achieve the task‑specific performance of the fine‑tuned agent (70.6% vs. 77.0%). Using the specialized model for both compression and action generation also raises the cost substantially (21.49). In contrast, the asymmetric configuration achieves 78.2% success at a cost of 10.31, combining the complementary capabilities of the two models.
(3) YAMATO preserves execution-state integrity, preventing compression errors from increasing interaction cost.
Context compression can lose or alter information required by subsequent tool operations. YAMATO addresses this problem through provenance‑aware summaries, exact restoration of selected raw records, and persistent failure tracking. Table 5 isolates its contribution:
| Metric | ACON w/o YAMATO | ACON w/ YAMATO | DMC-5 w/o YAMATO | DMC-5 w/ YAMATO |
|---|---|---|---|---|
| Task Accuracy | 67.3% | 72.0% | 69.1% | 78.2% |
| Total Cost | 11.57 | 12.11 | 13.31 | 10.31 |
| Compression Events with Errors | 4.2% | 1.0% | 3.0% | 0.04% |
| Corrupted Critical Items | 1.23% | 0.35% | 0.33% | 0.01% |
| Critical Item Recall | 63.3% | 87.1% | 77.7% | 98.3% |
| Runs with ≥5 Repeated Failures | 1.6% | 0.6% | 1.6% | 0.2% |
| Maximum Same-Action Repeats | 39 | 12 | 29 | 6 |
YAMATO increases DMC-5 task accuracy from 69.1% to 78.2% while simultaneously reducing total cost from 13.31 to 10.31. This demonstrates that information integrity directly affects end‑to‑end efficiency: preventing corrupted historical state also avoids repeated failures and unnecessary agent interactions.
(4) Append-only history enables cache-aware compression within the dual-view architecture**.**
We agree that appending messages is common in agent context management. However, preserving an append-only history as the input to a context compressor is fundamentally different from simply appending messages to an agent prompt. Conventional context compression typically modifies historical content to reduce token usage, through selective deletion, recursive summary rewriting, or repeated full‑history recompression. These operations can sacrifice prefix‑cache reuse or incur repeated uncached processing costs, while potentially losing information needed for subsequent decisions.
DMC‑5 instead optimizes actual end‑to‑end billed cost. Its Append‑only Canonical View preserves the complete, immutable trajectory for cache‑aware full‑history integration by the general compressor, while a separate compact Decision View provides the specialized agent with only the context needed for action generation. This design combines compressor‑side cache reuse with agent‑side context reduction, while YAMATO protects execution-critical information during compression.
The controlled comparison in original Table 2 isolates the effect of this design:
| Benchmark | Strategy | Cached Compressor Tokens | Uncached Compressor Tokens | Compressor Cost | End-to-End Cost | Task Score |
|---|---|---|---|---|---|---|
| AppWorld | Recursive Summary (ACON) | 0 | 17,269 | 2.17 | 11.57 | 67.3 |
| Append-only (DMC-5) | 51,837 | 12,717 | 2.50 | 10.31 | 78.2 | |
| OfficeBench | Recursive Summary (ACON) | 0 | 5,052 | 0.83 | 4.19 | 83.3 |
| Append-only (DMC-5) | 15,635 | 4,410 | 0.89 | 4.49 | 88.0 | |
| 8-objective QA | Recursive Summary (ACON) | 0 | 60,441 | 10.84 | 25.94 | 34.2 |
| Append-only (DMC-5) | 198,150 | 34,844 | 9.08 | 25.50 | 44.9 |
As further illustrated in Fig. 4, the append‑only prefix cache‑hit rate increases from 0 to approximately 96% by the final compression round, whereas recursive summary provides no reusable prefix. Across the three benchmarks, DMC‑5 reuses 15.6K‑198.2K cached compressor input tokens per task, reduces uncached compressor input by 12.7‑42.3%, and improves task scores by 4.7‑10.9 percentage points over recursive summary. Despite processing more total compressor input, its compressor cost remains comparable to recursive compression; the resulting end‑to‑end cost is lower on AppWorld and 8‑objective QA and slightly higher on OfficeBench, where the task-score improvement is 4.7 points.
Therefore, the contribution is not the append operation itself, but its use as part of an end‑to‑end compression architecture: the Canonical View preserves full‑history access and cache reuse, the Decision View reduces repeated agent input, and YAMATO protects execution‑critical information during compression. This combination turns a common history‑management mechanism into a practical cost-aware compression strategy.
cognition of DMC-5's consistent task-performance improvements across three agentic benchmarks while reducing end-to-end inference cost.
We will further address your concerns below through additional analyses on the deployment assumptions and generality of cache‑aware cost optimization, the complementary roles of the compressor and YAMATO in preserving execution‑critical information, and the applicability of DMC-5 to agents with unstructured tool interactions.
W1 / Q1. Deployment assumptions and cached-input pricing for fine-tuned endpoints
Thank you for raising this point. We conducted a systematic survey of 11 major providers offering fine‑tuned inference endpoints. Our survey shows that discounted prefix caching is unavailable for fine‑tuned endpoints at major providers. Among the 11 providers surveyed, 7 offer no cached‑input discount, while the remaining 4 support them only for a small subset of fine‑tuned models. None provides universal cache‑discount support across its fine‑tuned model catalog. Thus, the pricing scenario motivating DMC-5 applies to a broad range of practical fine-tuned deployments.
We additionally examined three deployment services (Hugging Face Inference Endpoints, Baseten, and Replicate ) that primarily charge by GPU‑hours or instance runtime. Since these services do not use token‑level billing, they are excluded from the 11‑provider comparison above.
All data in this survey were derived from official provider websites, pricing pages, and fine‑tuning, deployment, and caching documentation. The Alibaba Cloud documentation((https://help.aliyun.com/en/model-studio/context-cache) cited in the manuscript does specify this deployment condition. Specifically, under Model Inference → Dedicated Deployment → Deployment Modes → Token‑based deployment, the documentation states that token‑based deployment supports LoRA fine‑tuned models and is billed according to the number of input and output tokens. The corresponding billing rule specifies a single input‑token price, with no separate cached‑input price or cache‑hit discount for these fine‑tuned deployments. This policy is also directly verified by our actual deployment. The fine‑tuned Qwen3.5‑27B LoRA endpoint used in our experiments returned 0 cached tokens out of 130M input tokens in the API usage records. We will make the exact documentation location and this empirical verification explicit in the revised manuscript.
To better clarify the generalization ability of the proposed method, we further evaluate DMC‑5 under the alternative setting where the specialized agent itself supports discounted prefix caching. Since the original fine‑tuned endpoint does not provide this pricing option, we use an unfine‑tuned Qwen3.8‑27B base endpoint of the same parameter scale, which supports cached‑input pricing (0.3/M cached input tokens, 1.5/M uncached input tokens, and 6.0/M output tokens). The evaluated task set remains exactly the same.
The uncompressed baseline already benefits substantially from prefix caching, with 70.5% of its agent input served from cache. Nevertheless, DMC‑5 further reduces total cost by 9.4% while maintaining task performance. Our additional break‑even analysis shows that, at the actual uncached‑input price, the cached‑input price would need to fall below approximately 9.1% of the uncached price before the uncompressed baseline becomes cheaper.