- Rating
- 1
- Confidence
- 4
- Informativeness
- 1
- Presentation
- 1
- Soundness
- 1
Read review
Paper summary
The paper proposes SMA (Semantic Message Anchors), a training-free prompt compressor that does not access the reader model's internal states. It frames compression through the Minimum Description Length (MDL) principle as a "two-part code": verbatim evidence units (anchors) plus compact statistics that summarize the omitted background. The pipeline has four stages.
Critical strengths
None
Critical weaknesses
- First, I think the theory part is mostly decorative. For example, Eq. 1 is not MDL in the usual sense. It is a Lagrangian relaxation of a budget-constrained objective. Prop. 3 assumes most of its conclusion. Prop. 2 is a direct application of the data-processing inequality and holds for every compressor, including R2C and EHPC.
- Experiment setting has a lot of problems:
- The paper formally structure the method as prompt compression, so kv cache eviction can also be counted as baselines according to section 3.1. However, non of the SOTA KV cache eviction baselines are featured.
- Some baseline numbers look low. For example, LongLLMLingua, which is designed for question-aware long-context compression, scores below Selective Context and LLMLingua on several categories, and only 28.49 on Qwen at 2K tokens, against 49.65 uncompressed. This is far weaker than results typically reported for it on LongBench.
- Featured datasets are limited. The experiments are mostly conducted on LongBench.
- Heuristics are unjustified and there is no sensitivity analysis. Reserving 55.2% of the budget for units that match negation, hedge, or list regexes is a strong and unusual design choice. The other settings (τ_agg = 0.08, the 0.15 hedge cap, the 8-word merge threshold, and the 384-token chunk cap) are also fixed with no sensitivity study. Lemma 8 presents the derivation of 0.552 as a principled reparameterization, but the value is still a hand-chosen default.
Questions for authors
- Can we feature kv cache eviction baselines?
- Can we have more datasets for long-context? LongBench is not considered enough anymore in nowadays' agentic era, where a lot of agentic benchmarks are suitable candidates for testing the effectiveness of prompt compression methods (and if the authors are aware, prompt compression method are being used a lot in agentic models like Claude or GPT nowadays).
- A lot of hyper-param in the paper is fixed. Can we have ablation studies to solve weakness 3?
4 comments
Response to Reviewer VFQs (2/2)
Q1. KV-eviction baselines
Where to look: Section 3.1.
Answered above: the mechanisms act at different stages and compose (Section 3.1). A same-table comparison cannot hold the serving stack constant, because KV eviction modifies engine-internal cache management whereas SMA operates strictly at the text interface.
Q2. More datasets
Where to look: Section 5.1.
LongBench v2 is added. Agentic traces and InfiniteBench lie outside the measured set. Agentic traces need a multi-turn controller and InfiniteBench needs a separate long-input loader; neither is part of this revision.
Q3. Hyperparameter ablations
Where to look: Table 4 and Appendix B.
Table 4 and Appendix B cover the safeguard fraction and the aggregation reserve. The three fixed constants are documented with their scope, but they are not individually swept; we state this rather than presenting a component ablation as a single-parameter sensitivity.
SMA leads EHPC by 12.94 points on Qwen and 11.04 on Llama at 2K, reaches 46.98 on Qwen 3K and 38.65 on ZeroSCROLLS, and holds the high-accuracy end of the latency-quality frontier. MDL frames the representation, and the KV comparison sits outside the text interface.
I thank the authors for the response, yet my concerns remain:
- For (1), I'm now doubting if it's the authors write the response or if it's AI-generated, since the response seems to not answer my concern at all. Here, I state again some main points of my concern "the theory part is mostly decorative", "a Lagrangian relaxation of a budget-constrained objective", or "Prop. 3 assumes most of its conclusion". The authors reply "MDL enters as a design principle: ". If I get it correctly, are the authors saying my claim is correct that the theory part offers little theoretical contribution, and mostly serve as a design cosmetic?
- For (2), I'm clearly asking for an experimental comparison with kv cache eviction method. The authors direct me to section 3.1, and no experiment results can be found.
- For (4), the authors state " InfiniteBench and agentic benchmarks are not tested.". I don't quite get it why the authors need to state this in the rebuttal at all. Would it help clarify any of my questions or concerns?
- For (5), I clearly stated at the beginning "Heuristics are unjustified", so I'm trying to ask for justification, but no justification is provided, when I go to "Where to look: Table 4, Appendix B, and Table 3.".
I stop reading after (5). I think at least the authors need to value the effort and time both the authors and at least me, the reviewer, spend on (assumingly) researching and proposing constructive feedback to improve the work further. Please take this into consideration for your next submission to respect the effort and time of, at least, reviewers who still read your full paper and write honest reviews, not AI-generated ones.
I hereby also flag this to AC, as a reviewer who has decent experience in this field, my reply to the submission's rebuttal. I therefore recommend "clear rejection".
Response to Reviewer VFQs
We thank you for the detailed review, and I am sorry that my previous reply pointed at sections instead of answering you directly. I did use an LLM to tighten text drafted by the authors, hoping to make all words tight and uniform so each change would be easy to locate in the revised paper. That made the response read like a checklist, but the technical positions are ours.
On the theory, I should have stated our position plainly. Equation 1 is the Lagrangian form of the budget-constrained objective, with lambda never estimated, and Proposition 1 in the revised manuscript (Proposition 3 in the submitted version) is a conditional bound. The paper claims a structural decomposition and an explicit set of conditions under which the bound holds. We do not claim a new theoretical law.
On KV-cache eviction, we have not run such a baseline, so we cannot claim an empirical comparison. Eviction rewrites engine-internal state during inference, whereas our evaluation benchmarks text-interface prompt compressors (from Selective Context and LLMLingua to RECOMP, CPC, R2C, and EHPC) under a uniform reader serving stack. We should have stated this experimental scope in Section 3.1.
Likewise, listing benchmarks we did not run, InfiniteBench and agentic traces, added nothing to the reply. We did add two trained compression baselines, RECOMP and CPC, and one benchmark, LongBench v2.
On the heuristics, we swept one parameter, the safeguard fraction tau_t1. Raising it to 0.75 adds 0.91 points on the eight-task mean, lowering it to 0.35 drops PassageCount by 7.21 points (54.31 to 47.10), and keeping 0.552 reserves room for query-ranked anchors. The 384-token encoder window caps memory during batch encoding: across 94,247 prose units from 150 LongBench documents, 7 units exceed it and the 99th percentile is 87 tokens, so it is virtually non-binding in practice. The 0.15 hedge cap and the eight-word merge threshold are engineering defaults that we did not sweep individually.
Thank you for reading the full paper and holding us to a higher standard. Even if these gaps cannot be closed in this cycle, your feedback gives us a clear roadmap for the next version.
Response to Reviewer VFQs (1/2)
We thank you for the detailed review. We answer the theory, KV-eviction, LongLLMLingua, dataset, and heuristic points below, scoping each answer to what we measured.
W1. Theory
Where to look: Section 3.2 and Appendix A.2-A.4.
MDL enters as a design principle: intact anchors play the role of a model code, and a compact residual card codes the omitted background. Equation 1 is a Lagrangian allocation objective; lambda is not estimated. The data-processing inequality (Remark 4, Appendix A.3) bounds frozen-encoder selection. Proposition 1 carries its additivity, ranking, and length assumptions, which is the scope of the bound. Remark 2 predicts the observed QASPER trade-off. The budget reparameterization is stated directly.
W2. KV-cache eviction
Where to look: Section 3.1.
Text-level prompt compression and engine-internal KV eviction act at different stages and address complementary bottlenecks. KV eviction still requires prefilling the full N-token prompt inside a self-hosted serving engine, and it cannot reduce the input tokens billed by closed APIs. SMA operates strictly at the text interface before inference, so it composes with KV eviction on self-hosted stacks; Section 3.1 states this scope.
W3. LongLLMLingua
Where to look: Section 5.3 and Table 2.
LongLLMLingua's target_token is a per-chunk ratio with thresholded token dropout, not a hard cap; the library documentation notes that realized lengths generally fall below the target and vary with the tokenizer. The compressor uses Llama-2 tokenization, while Table 1 counts reader tokens including instruction, query, and chat framing. We report the realized length and do not pad it, because padding would depart from the official implementation. At 2K the realized means are 1,761 on LongBench, 1,670 on ZeroSCROLLS, and 1,464 on LongBench v2; per-task means range from 969 (MuSiQue) to 2,240 (RepoBench-P). The lower score coincides with the unused capacity, but it is not the sole cause: Selective Context uses 2,048 tokens and scores 31.75, LLMLingua 1,945 and 29.88, and LongLLMLingua 1,761 and 28.40 on Qwen LongBench. Table 1 here comes from the corrected re-run: repairing the upstream assertion fallback in LongLLMLingua changed its Qwen 2K LongBench score from 28.49 to 28.40.
W4. Datasets
Where to look: Section 5.1 and Table 1.
LongBench v2 is now a main-table benchmark: 503 questions, about 83K average reader input, 120K window. LongBench and ZeroSCROLLS are retained. InfiniteBench and agentic benchmarks are not tested.
W5. Heuristics
Where to look: Table 4, Appendix B, and Table 3.
Table 4 varies the safeguard fraction: 0.552 to 0.75 adds 0.91 to the eight-task mean, driven by PassageCount +4.40; lowering it to 0.35 reduces PassageCount from 54.31 to 47.10. The aggregation-reserve check moves HotpotQA by -0.25 and +0.30 and PassageCount by +0.02. The 0.15 hedge cap, the eight-word merge threshold, and the 384-token encoder window are documented with their scope in Appendix B; we did not sweep them individually. As noted in Section 5.4, these rules are lightweight surface heuristics rather than an intent classifier: count and existence triggers can be noisy, while the summary and query-free branches drive the measured gains, and misrouting only alters the appended card without affecting anchor selection or the budget guard.