- Rating
- 3(+1)
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
The paper proposes Agentic Soft-Token Compression (ASTC) for hybrid Gated Delta Network–attention models. ASTC uses native recurrent-state signals to select compressible context, protects recognized exact values, and merges selected hidden states into soft tokens. Shorter downstream caches are reused across agent steps.
ASTC uses a reader calibrated from 22 records across 12 Train/Dev tasks. Experiments mainly use Qwen3.5-9B on AppWorld, OfficeBench, and 8-Objective QA, reporting comparable task quality and up to 50.68% fewer analytical attention pairs.
Critical strengths
-
A coherent training-free mechanism ASTC combines native recurrent signals, exact-value protection, soft-token merging, and persistent caches. This offers a practical direction for reducing agent-context computation without updating model weights.
-
Useful experimental controls The paper includes selector controls, component ablations, and a fixed-continuation cache-reuse experiment. Removing protection reduces AppWorld accuracy from 33.33% to 20.83%, clearly demonstrating the importance of preserving exact information.
Critical weaknesses
- The comparisons do not establish practical efficiency or SOTA superiority
Most hard-token baselines compress full context, while ASTC uses a protected subset, making the comparison difficult to interpret. Training and adaptation details for the learned baselines are also incomplete. Moreover, ACON’s core method uses gradient-free guideline optimization, so labeling it a trained compressor needs clarification. Attention-pair counts include auxiliary calls but do not measure actual latency, memory, or total computation. Relevant internal/cache comparators such as FastKV and KVzip are missing. ACON, FastKV, KVzip
- Evidence on harder horizons and the distinctive mechanism remains limited
AppWorld improves by only two successes out of 168; hard deletion is only one success below ASTC. The largest workload reduction comes from OfficeBench, where only repeated, byte-identical interface scaffolds are compressed, making a deduplication baseline important. AppWorld Test-Challenge is not evaluated. Static long-context tests, three-attempt MBPP repair, and near-floor Kimi-Linear results provide limited additional evidence for broad long-horizon robustness.
Questions for authors
-
Can you provide the learned baselines’ training/adaptation procedures and distinguish ACON guideline optimization from parameter training?
-
What are the measured end-to-end latency, peak memory, and compression overhead under matched hardware and serving settings?
-
Does ASTC outperform baselines with identical compression targets, protection rules, and retained lengths? How does OfficeBench compare with cache-compatible deduplication?
-
How were calibration records, thresholds, and the compression layer selected? Were these fixed before testing? Please provide paired uncertainty estimates and results on AppWorld Test-Challenge or the longest trajectories.
5 comments
Response to Review KkTX(2/3)
How were calibration records, thresholds, and the compression layer selected? Were these fixed before testing? Please provide paired uncertainty estimates and results on AppWorld Test-Challenge or the longest trajectories.
Thank you for raising this question. The original manuscript used empirically set compression parameters, and the layer was determined empirically before the layer comparison in Figure 1, without using that comparison to search for parameters. This time, we conducted a small-scale search over χ and ψ on the dev set and found improvements in both quality and efficiency on OfficeBench and 8-Objective QA. Meanwhile, the parameter choices across the three benchmarks are similar, with some even being identical, demonstrating the algorithm's strong robustness and the potential to reach a higher performance ceiling through further parameter optimization in the future.
| Dataset | Method | Acc./EM% ↑ | F1% ↑ | D-KV% ↓ | Attn.% ↓ | Gen.% ↓ |
|---|---|---|---|---|---|---|
| OfficeBench | Original manuscript | 47.37 | — | 54.52 | 49.32 | 69.23 |
| OfficeBench | Revised | 54.74 | — | 42.09 | 40.16 | 61.24 |
| 8-Objective QA | Original manuscript | 30.56 | 38.46 | 78.69 | 98.51 | 86.55 |
| 8-Objective QA | Revised | 30.81 | 39.18 | 57.96 | 74.29 | 82.02 |
We also added the more challenging AppWorld Test-Challenge. This experiment follows the existing AppWorld settings, and the uncertainty estimates of Attention Pairs are [74.17, 85.12]. We have also provided the longest successful trajectories on AppWorld Test-Challenge in our anonymous GitHub repository.
| Method | Acc.% ↑ | D-KV% ↓ | Attn.% ↓ | Gen.% ↓ |
|---|---|---|---|---|
| Base | 20.86 | 100.00 | 100.00 | 100.00 |
| ASTC | 19.90 | 87.13 | 79.43 | 95.34 |
Most hard-token baselines compress full context, while ASTC uses a protected subset, making the comparison difficult to interpret.
Thank you for pointing out this factor. The baselines mainly compress the full context, but we observed that a large portion of an agent's context is generated by tools, and this context both contains compressible redundant information and is challenging to compress; we therefore first applied compression to tool outputs. For a fair comparison, we compared full-context compression in Appendix F of the original manuscript and found comparable results.
| Dataset | Method | Acc./EM% ↑ | F1% ↑ | D-KV% ↓ | Attn.% ↓ | Gen.% ↓ |
|---|---|---|---|---|---|---|
| AppWorld | Base | 32.14 | — | 100.00 | 100.00 | 100.00 |
| AppWorld | ASTC (Tool) | 33.33 | — | 81.37 | 80.56 | 94.66 |
| AppWorld | ASTC (Context) | 32.14 | — | 80.52 | 81.83 | 95.73 |
| OfficeBench | Base | 48.42 | — | 100.00 | 100.00 | 100.00 |
| OfficeBench | ASTC (Tool) | 54.74 | — | 42.09 | 40.16 | 61.24 |
| OfficeBench | ASTC (Context) | 51.58 | — | 37.11 | 40.76 | 58.91 |
| 8-Objective QA | Base | 28.91 | 37.51 | 100.00 | 100.00 | 100.00 |
| 8-Objective QA | ASTC (Tool) | 30.81 | 39.18 | 57.96 | 74.29 | 82.02 |
| 8-Objective QA | ASTC (Context) | 31.31 | 41.40 | 66.83 | 77.75 | 87.40 |
Relevant internal/cache comparators such as FastKV and KVzip are missing.
Thanks for your question. FastKV and KVzip are works on KV caching and efficient inference, while our research focuses on context compression for long-horizon agents, so FastKV and KVzip do not adequately address the problems in our setting. However, we also conducted additional experiments with FastKV and KVzip following the settings in their original papers. On real-world long-horizon agent tasks such as AppWorld, they clearly underperform ASTC.
| Method | Training-free | Acc.% ↑ | D-KV% ↓ | Attn.% ↓ | Gen.% ↓ |
|---|---|---|---|---|---|
| Base | — | 32.14 | 100.00 | 100.00 | 100.00 |
| ASTC | ✓ | 33.33 | 81.37 | 80.56 | 94.66 |
| FastKV | ✓ | 0.00 | 2.53 | 561.23 | 55.40 |
| KVzip | ✓ | 11.31 | 87.98 | 5,976.41 | 145.73 |
AppWorld improves by only two successes out of 168;
Thank you for your suggestion; research on context compression for long-horizon agent tasks primarily aims to reduce inference costs while maintaining comparable task performance relative to the uncompressed baseline, rather than directly improving the agents' inherent task-solving capabilities. Our work follows this general research objective. Even so, ASTC improves quality on all three benchmarks while providing efficient acceleration. On a real-world and challenging benchmark such as AppWorld, it achieves a +1.19 pp improvement, while Attention Pairs are 80.56% [69.36%, 93.47%] of Base; it also improves quality over the baselines, which require 12.3×–194.9× more Attention Pairs per successful task.
Response to Review KkTX(3/3)
hard deletion is only one success below ASTC.
Thank you for raising this question; hard deletion in the ablation study is actually a variant of ASTC that uses ASTC's protection and selection rules but directly deletes redundant, uninformative positions without generating Soft tokens. Therefore, it still benefits from ASTC's internal selection and protection. The direct-deletion method you refer to is the Prompting baseline in Table 1. We have clearly distinguished the two in the revised manuscript.
| Variant | Acc.% ↑ | D-KV% ↓ | Attn.% ↓ | Gen.% ↓ |
|---|---|---|---|---|
| Base | 32.14 | 100.00 | 100.00 | 100.00 |
| Prompting | 25.00 | 143.59 | 64.79 | 104.25 |
| ASTC | 33.33 | 81.55 | 80.70 | 94.83 |
| ASTC (deletion) | 32.74 | 88.83 | 86.85 | 99.58 |
We added the following experiments to demonstrate that Soft tokens aggregate information within each group under a limited length budget and preserve usable representations for subsequent reasoning.
① A control using Hard representatives of equal length tests whether aggregating information within a group is more effective than retaining a single original state under the same representation budget. With a shared input prefix, we keep the protection rules, grouping, number of representatives, and logical positions identical. The Hard version retains the hidden state of the last original token in each group, while ASTC aggregates the hidden states within the group.
② Masking subsequent reads further examines the role of representatives when they are read later. Starting from the next forward pass after the Soft representatives are generated, we block subsequent Attention from reading these representatives while retaining the physical KV, logical positions, and original GDN update rules.
| Method | Readable by subsequent Attention | Successes | Acc.% ↑ | D-KV% ↓ | Attn.% ↓ | Gen.% ↓ |
|---|---|---|---|---|---|---|
| Base | — | 54/168 | 32.14 | 100.00 | 100.00 | 100.00 |
| Hard representatives of equal length | Yes | 52/168 | 30.95 | 89.20 | 85.72 | 99.08 |
| ASTC Soft | Yes | 56/168 | 33.33 | 81.37 | 80.56 | 94.66 |
| ASTC Soft | No | 46/168 | 27.38 | 89.17 | 85.06 | 99.25 |
Static long-context tests, three-attempt MBPP repair, … provide limited additional evidence for broad long-horizon robustness.
Thanks for your question. Table 1 of the original manuscript already demonstrates ASTC's strong performance on long-horizon agent tasks; this experiment aims to test whether ASTC's effectiveness extends to different task types, and the results show that its performance remains strong. In Appendix E of the original manuscript, we use RULER, LongBench v2, and interactive MBPP repair, following the evaluation approach of previous work on general-purpose context compression, to cover capabilities such as long-context retrieval, understanding, and code repair with feedback. Compared with previous work such as LCLM, ASTC reduces both Decode KV and Attention Pairs while maintaining task performance close to Base. Together with the performance on the agent benchmarks in the main text, these results support that ASTC's quality and efficiency gains extend across multiple task types, demonstrating strong robustness.
near-floor Kimi-Linear results provide limited additional evidence for broad long-horizon robustness.
Thank you for your question. This experiment is presented in Appendix G of the original manuscript, and we have also added results on Qwen3.8-27B; ASTC performs strongly on all three models. These results demonstrate that ASTC can improve quality across different backbones or provide broad long-horizon robustness guarantees.
| Model | Method | Acc.% ↑ | D-KV% ↓ | Attn.% ↓ | Gen.% ↓ |
|---|---|---|---|---|---|
| Qwen3.5-9B | Base | 32.14 | 100.00 | 100.00 | 100.00 |
| Qwen3.5-9B | ASTC | 33.33 | 81.55 | 80.70 | 94.83 |
| Qwen3.8-27B | Base | 78.57 | 100.00 | 100.00 | 100.00 |
| Qwen3.8-27B | ASTC | 77.98 | 91.70 | 64.43 | 103.82 |
| Kimi-Linear-48B-A3B | Base | 1.47 | 100.00 | 100.00 | 100.00 |
| Kimi-Linear-48B-A3B | ASTC | 4.41 | 84.01 | 88.76 | 94.70 |
Thank you again for your careful and specific feedback. To the best of our knowledge, ASTC is among the first training-free, in-model soft-token methods for context compression in long-horizon LLM agents. We hope that these point-by-point explanations and additional experiments address your concerns; if any points require further clarification, we would be happy to continue the discussion.
Response to Review KkTX(1/3)
Thanks very much for your thorough review and insightful questions. We will reply point-by-point below and hope to address your concerns!
Thanks for pointing this out. We have provided these details in our anonymous GitHub repository; all methods follow the settings in their original papers, as described in the appendix of our paper, and the comparisons of ASTC (Agentic Soft Token Compression) with baselines such as LCLM, ACON, and LLMLingua are fully fair. To avoid confusion, we have also provided a more detailed explanation in our revised manuscript as follows: Specifically, ACON requires searching for and selecting guidelines on the training tasks; this offline optimization process incurs actual model calls, token usage, and time costs, and the original ACON paper also states that ACON participates in the training process. We generate initial guidelines and trajectories without compression on Train, then perform two stages of optimization, UT and UTCO, generating, evaluating, and selecting candidates at each stage; the final guideline is fixed at test time. The UTCO optimization model logged 69 calls and 716,220 tokens, and the guideline search and selection process took approximately 19 hours and 25 minutes. We explain ACON's training cost in Figure 2; ASTC does not search for compression guidelines or train compressor parameters, but instead reuses the model's native signals. LCLM uses the publicly available pretrained latent-context/0.6b-4b-LCLM-4x compressor, freezes the 0.6B encoder and existing 1024→2560 adapter, and trains only a newly added 2560→4096 MLP Bridge and LoRA on the Qwen decoder to adapt the representations to the same initial Qwen3.5-9B backbone used by ASTC. LLMLingua v1 uses Qwen3.5-9B for causal token scoring; LLMLingua-2 uses the publicly available pretrained microsoft/llmlingua-2-xlm-roberta-large-meetingbank. For the additionally trained In-model Attention in Figure 2, the inserted Attention–MLP compression module is trained through on-policy distillation.
Thank you for your suggestion. On NVIDIA A100-SXM4-40GB, we directly apply ASTC to AppWorld and 8-Objective QA, and the results show that ASTC reduces both mean and P95 latency, shortens total queue time, and increases overall queue throughput. We have added these results to our revised manuscript.
Thank you for your question. We added a controlled Hard-token comparison using a frozen LLMLingua-2 scorer. All protected positions are retained, and the final numbers of retained physical positions are strictly identical for the two methods. The results show that ASTC clearly outperforms LLMLingua-2 in both quality and efficiency.
Thank you for your question. We added an exact deduplication comparison on OfficeBench. We retain the initial complete interface, all modified interfaces, and tool observations, and delete only complete copies in newly appended content that are byte-for-byte identical to retained interfaces. ASTC achieves clearly higher quality than the deduplication approach.