- Rating
- 3
- Confidence
- 3
- Informativeness
- 3
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
MANTA introduces information-adaptive context compression using nested Matryoshka token prefixes and a tolerance-conditioned Elastic Allocator. A per-chunk allocation target is defined by the smallest prefix whose excess loss stays below a tolerance, allowing one model to operate at multiple compression levels. The paper evaluates MRQA in- and out-of-domain splits, several decoder scales and backbone families, and needle-QA at longer context lengths.
Critical strengths
The central design is coherent: nested prefixes make arbitrary-length allocation possible, while conditioning the allocator on a tolerance gives the model a direct inference-time control. The uniform-versus-adaptive comparison and the no-allocator/naive-allocator ablations isolate the value of differentiated allocation better than many compression papers do. Results are reported on both in-domain and out-of-domain MRQA data, across multiple model scales, another backbone family, and contexts longer than those used in training. The gains over the listed compression baselines are consistent enough to make the idea promising.
Critical weaknesses
The comparison should be made more carefully before the method can support a stronger acceptance recommendation. MANTA is reported at nominal ratios such as 65x while baselines are shown at 64x or nearby separately trained ratios; realized retained-token counts and total encoder/decoder cost are not reported. The per-chunk oracle target is computed through repeated loss evaluations while other chunks use the allocator’s own predictions, so the paper should quantify the resulting training/inference mismatch and compare against a strong adaptive baseline under the same protocol. The absolute out-of-domain performance remains substantially below the uncompressed reference, and the long-context needle-QA experiment is not enough by itself to establish robustness to natural long documents.
The related-work coverage is incomplete. The paper should also discuss the following directly relevant works:
- Perception Compressor: A Training-Free Prompt Compression Framework in Long Context Scenarios — Findings of NAACL 2025
- GMSA: Enhancing Context Compression via Group Merging and Layer Semantic Alignment — ACL 2026
- RECOMP: Improving Retrieval-Augmented Language Models with Compression and Selective Augmentation — EMNLP 2023
- Prompt Compression with Context-Aware Sentence Encoding for Fast and Improved LLM Inference — AAAI 2025
- AttentionRAG: Attention-Guided Context Pruning in Retrieval-Augmented Generation — arXiv, 2025
These missing references would help position MANTA against recent adaptive, sentence-level, training-free, and retrieval-oriented compression methods.
Questions for authors
Please add comparisons at identical realized token budgets and report wall-clock latency, peak memory, and encoder cost. How much does performance change when the allocation target is constructed with a fixed background allocation rather than the allocator’s own predictions? A question-disjoint additional dataset and a stronger adaptive baseline would help establish whether the gains reflect the proposed allocation mechanism rather than the particular MRQA training recipe.
4 comments
Response Part II
Q2: Offline oracle labels, deployment-time allocation, and fixed background.
We thank the reviewer for drawing attention to the consistency between oracle-target construction during training and allocation at deployment.
1. Offline oracle supervision and deployment-time allocation.
The oracle is used only as offline supervision. At deployment, the Elastic Allocator uses only the context and and produces the allocation in a single run.
The oracle uses the gold answer only during training or diagnosis to construct a supervision signal specifying how many Matryoshka tokens each chunk should retain. At deployment, the Elastic Allocator reads only the context and ; it does not read the question or the answer, nor does it scan candidate allocations. We verify the consistency between deployment-time prediction and the offline oracle with :
| Data distribution | Pearson | k-MAE |
|---|---|---|
| IID | 0.96 | 1.32 |
| OOD | 0.88 | 1.94 |
The Pearson and k-MAE values support the consistency of the deployment prediction with the offline oracle on both IID and OOD data. Therefore, the oracle is an offline supervision tool rather than a deployment-time input.
2. Predicted-background and fixed-background oracle targets.
Fixed-background training causes only the reported limited changes relative to predicted-background training, so the conclusion does not depend on using the allocator's own predictions as the background target.
We directly compare the two oracle-target constructions with the same for both models. The predicted-background target uses the allocator's own prediction for the other chunks. The fixed-background target instead sets those chunks to the maximum during oracle construction. The table reports the absolute evaluation of both trained models:
| Target construction | IID F1 | IID EM | OOD F1 | OOD EM |
|---|---|---|---|---|
| Predicted-background | 55.88 | 45.43 | 39.87 | 31.29 |
| Fixed-background | 54.97 | 44.56 | 39.78 | 30.89 |
At the same , our predicted-background training outperforms fixed-background training by +0.91/+0.87 in IID F1/EM and by +0.09/+0.40 in OOD F1/EM. This matched ablation supports the conclusion that the result does not depend on using the allocator's own predictions as the background target.
3. Absolute OOD performance across compression ratios.
MANTA's absolute OOD performance degrades smoothly with compression rather than remaining far below the uncompressed reference, and closely approaches it at low compression ratios.
The statement that absolute OOD performance is below the uncompressed reference is most relevant at aggressive compression ratios, not throughout the operating curve. The 1B dense Full-FT reference has OOD token-F1 66.59. On the same full-MRQA protocol, MANTA gives:
| Method | Compression ratio | IID F1 | OOD F1 | OOD recovery |
|---|---|---|---|---|
| Full-FT | 1.00 | 81.73 | 66.59 | 100.0% |
| MANTA, | 10.30 | 70.77 | 55.11 | 82.8% |
| MANTA, | 13.07 | 69.19 | 53.66 | 80.6% |
| MANTA, | 15.09 | 67.22 | 52.03 | 78.1% |
At the least-compressed evaluated adaptive point, MANTA reaches 55.11 absolute OOD token-F1 and recovers 82.8% of the uncompressed reference. As allocation becomes more aggressive, the expected gap grows; this is a compression-quality trade-off rather than evidence that MANTA has intrinsically low absolute OOD performance.
Response Part III
Q3: Question-disjoint transfer to natural long documents with a strong adaptive baseline.
We thank the reviewer for the valuable suggestion to evaluate question-disjoint data and natural long documents.
MANTA transfers to the audited question-disjoint LongBench suite and outperforms both the fixed-allocation SAC baseline and the strong question-aware adaptive Perception Compressor.
To evaluate transfer to natural long documents without conflating it with the MRQA training pool, we evaluated an 8B LongBench suite containing Qasper, 2WikiMQA, and MuSiQue. We audited the selected IDs against all MRQA fine-tuning examples and found zero exact overlap in both questions and contexts after lowercasing and whitespace normalization. This establishes normalized exact-match disjointness only, and does not rule out semantic overlap or pretraining contamination. To address the request for a stronger adaptive comparison, we added Perception Compressor (PC), a training-free, question-aware adaptive baseline:
| Method | Compression ratio | Qasper | 2WikiMQA | MuSiQue | Average F1 |
|---|---|---|---|---|---|
| Full-FT | 1.00× | 22.99 | 43.26 | 36.99 | 34.41 |
| SAC | 63.97× | 14.24 | 23.45 | 10.48 | 16.06 |
| MANTA | 68.14× | 18.44 | 27.41 | 16.62 | 20.82 |
| Perception Compressor | 59.08× | 6.98 | 15.71 | 13.24 | 11.98 |
On this audited suite, MANTA reaches a three-task macro F1 of 20.82, exceeding SAC by 4.76 points (29.6% relative) and PC by 8.84 points (73.8% relative). MANTA also operates at a higher realized compression ratio than both SAC and PC.
Q4: Related work—positioning of the representation contract and comparison with related work.
We sincerely thank the reviewer for providing these important references and note that we have added all of them to the Related Work section of the revised manuscript and discuss them in depth. Although these state-of-the-art methods all aim to reduce inference-time context overhead, MANTA differs from existing work in its underlying Representation Contract. Perception Compressor and AttentionRAG depend on a specific question input and perform query-aware text pruning; RECOMP and CPC focus on filtering retrieval results or on sentence-level hard-text compression; and GMSA focuses on organizing the internal compression flow through group merging and layer alignment. MANTA does not rely on any prior question or specific retrieval path. With only a context-only Allocator, it produces a high-information-density soft compressed representation in a single pass. This representation can be used directly as a general-purpose key-value state and reused by the downstream model across arbitrarily many subsequent questions. Thus, MANTA shifts the design point from "question-bound text pruning" to a "multi-question reusable soft-representation contract."
Dear Reviewer, The authors have provided their responses to the raised concerns and revisions. Please review their responses and join the discussion before the end of the author-review discussion phase (Nov 18, AoE). Thanks.
Best, AC
Response Part I
Dear Reviewer pFqG,
We thank the reviewer for the careful and constructive comments. We address each concern below, and the additional experiments and discussions have been incorporated into the revised paper.
Q1: Nominal compression ratio, actual allocation, and end-to-end cost.
We thank the reviewer for highlighting the need to distinguish nominal compression ratios from realized allocations and to report end-to-end costs.
1. Realized token allocation and accuracy.
MANTA retains fewer tokens than SAC and ICAE while achieving higher IID F1 under the reported realized allocation.
MANTA compresses adaptively according to the content of the text, the compression ratio cannot be fixed in advance. Instead, MANTA's Elastic Allocator assigns Matryoshka tokens to each chunk based on its local informativeness and ϵ. The table below reports the actual allocation and results on MRQA:
MANTA retains only 11.84 tokens on average. Despite compressing more tightly than SAC and ICAE (46.53× vs. 43.18×), it improves IID F1 by +3.03 (5.8% relative) over SAC and +9.34 (20.4% relative) over ICAE. Thus, the realized-allocation results support the claim that adaptive allocation identifies high-information-density regions and avoids wasting budget on redundant background content.
2. End-to-end latency, encoder cost, and memory.
MANTA's encode cost is comparable to the baselines, and its wall-clock latency is lower than full context and ICAE, although SAC is faster in this measurement.
We also directly measured request-level latency and memory on the same pool under identical hardware:
These request-level measurements confirm that MANTA achieves its accuracy advantage with a tighter retained-token budget and low end-to-end overhead, while the latency comparison also makes clear that SAC remains faster in this measurement.