- Rating
- 4
- Confidence
- 3
- Informativeness
- 3
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
DEX-Comp is a two-stage training recipe for query-independent soft context compression in RAG. A compressor maps each retrieved document into a shorter sequence of continuous embeddings, which a compression-aware decoder consumes. The key idea is to allocate different learning signals according to teacher correctness. Stage I, Pure Distillation, trains only on teacher-correct examples, aligning the compressed student’s response distribution with a frozen full-context RAG teacher via KL divergence. Stage II, Hard Exploration, initializes from the PD checkpoint and trains on teacher-failed examples with outcome-based GRPO, directly rewarding answer correctness rather than teacher agreement.
The paper evaluates on five open-domain QA datasets at retrieval depths top-5, top-15, and top-30, using Mistral-7B and several query-independent compression baselines. At 16× compression, DEX-Comp surpasses uncompressed RAG on average CEM and LLM-judged accuracy across depths, and substantially improves TTFT, GFLOPs, and peak memory. Ablations support the PD/HE split, and additional experiments show generalization across backbones and OOD datasets.
Critical strengths
1.The teacher-correctness-based allocation of distillation and exploration is well motivated and empirically supported. PD trains only on , avoiding imitation of teacher-failed responses, while HE uses with outcome-based GRPO. The ablation shows PD improves average LLM-judged accuracy from 64.79 to 67.75 before RL, and HE further improves it to 72.02. Exploration on teacher-failed examples also outperforms exploration on teacher-correct examples (70.60) or their mixture (71.47) from the same PD checkpoint. This is consequential because it directly validates the paper’s central design choice, not just the overall pipeline.
2.DEX-Comp closes the compression gap while delivering large efficiency gains. At 16× compression, it surpasses uncompressed RAG on every dataset and depth in the main table, with average CEM/LLM gains above two points. It also reduces TTFT by 4.4×–23.7×, GFLOPs by 6.67×–13.61×, and peak memory, with larger gains at greater retrieval depths. This is consequential because it shows compression is not merely an efficiency trade-off; it can match or exceed full-context RAG under realistic retrieval depths.
Critical weaknesses
The following are minor concerns rather than material weaknesses:
1.Teacher correctness is determined by Gemini 3 Flash. Judge errors could misallocate examples between (\mathcal{S}^{+}) and (\mathcal{S}^{-}), though the ablation and main results suggest the signal is sufficiently reliable.
2.OOD evaluation is mixed: DEX-Comp improves on CovidQA and FEVER but is lower on BioASQ. This is a limitation, not a contradiction of the main claims.
3.The “first query-independent soft compression method shown to surpass uncompressed RAG” claim is strong; the paper could phrase it more cautiously, but the evidence supports the narrower claim under its evaluated settings.
Questions for authors
1.Teacher-judge sensitivity: How sensitive are PD/HE partition and final accuracy to the choice of teacher-correctness judge? Can you report the fraction of examples whose correctness label changes under a different judge, and the downstream effect on DEX-Comp?
2.OOD BioASQ: Why does DEX-Comp underperform uncompressed RAG on BioASQ? Is this due to domain shift, retrieval depth, or compression rate? A short characterization would clarify when compression should be avoided.
3.Offline compression and storage cost: The paper notes that offline compression and embedding storage are amortized across queries. Can you quantify storage and recompression cost at 16× on the Wikipedia-KILT corpus, and how updated documents are handled?
2 comments
Response to Reviewer BVw3 (Part 2/2)
W3. The “first query-independent soft compression method shown to surpass uncompressed RAG” claim is strong; the paper could phrase it more cautiously, but the evidence supports the narrower claim under its evaluated settings.
We thank you for this helpful reminder. This “first” claim appears only in Section 5.1 (Main Results), where we discuss the experimental results. We will revise it to a more cautious statement limited to our evaluated settings.
Q3. Offline compression and storage cost: The paper notes that offline compression and embedding storage are amortized across queries. Can you quantify storage and recompression cost at 16× on the Wikipedia-KILT corpus, and how updated documents are handled?
We thank you for this practical question. At 16× compression, caching the BF16 embeddings for approximately 23.85M Wikipedia-KILT documents requires about 1.56 TB. A full-corpus compression pass will take only 6.9 hours on eight RTX PRO 6000 Blackwell GPUs. For subsequent updates, we compress only new or modified documents. With a single-GPU compression throughput of 119.8 documents/s, this keeps small updates inexpensive. We then insert or replace the corresponding embeddings in the document store. When downstream queries retrieve these documents, the generator uses the updated cached embeddings in place of the original document text.
Overall, offline compression trades storage for inference time: for a fixed compressor, the full corpus is processed once, while later updates are incremental. The more often documents are reused across queries, the greater the cumulative inference time saved and the more the initial compression cost is amortized.
Response to Reviewer BVw3 (Part 1/2)
We sincerely thank you for your careful reading and positive assessment of our work. We also appreciate the valuable questions you raised, which motivated the further analyses presented below.
We thank you for this helpful suggestion. To investigate the effect of judge choice, we replace the original Gemini 3 Flash judge with Gemini 3.8 Flash. As a result, both the PD/HE partition and final accuracy remain stable under this change.
Each judge partitions the training questions into teacher-correct examples for Pure Distillation (PD) and teacher-failed examples for Hard Exploration (HE). Label changes are measured against the original Gemini 3 Flash partition. All final accuracies are evaluated by Gemini 3 Flash, reporting the five-dataset mean at top-30 retrieval and 16× compression.
Changing the judge alters only 2.1% of labels, with a final accuracy difference of just 0.12 points. These results support the robustness of both the data partition and final performance to the choice of judge model.
We thank you for this helpful question. To investigate the BioASQ gap, we inspected 50 questions that DEX-Comp answered incorrectly. The observed failure pattern points primarily to domain shift, particularly on factoid questions requiring precise biomedical terminology. The errors include substitutions between similar names or related concepts, as illustrated below.
This pattern is consistent with a mismatch between BioASQ's specialized vocabulary and our general-domain, Wikipedia-based training, which may make precise biomedical distinctions harder to preserve under compression. We therefore recommend caution with soft compression in domains rich in rare terminology, especially when exact names, numerical values, or sequence motifs determine correctness.