- Rating
- 3
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
ReValue is a query-aware hybrid context compressor that retains token-level representations for selected segments and compact vectors for the remainder. Its central idea is to allocate the limited high-fidelity budget according to a segment's marginal value given the segments already selected. A greedy DPP log-determinant objective combines query relevance with multi-scale similarity, while an auxiliary diversity regularizer acts on compact representations alongside next-token prediction. Experiments evaluate downstream quality across backbones and compression ratios, compare allocation rules with other components fixed, measure supporting-fact coverage, and examine end-to-end inference efficiency.
Critical strengths
-
Controlled allocation and evidence coverage (PDF pp. 8–9, Figures 4–5, Table 3 and Section 4.4). Table 3 changes the inference-time selector while keeping the remaining components fixed. ReValue obtains 61.51 NaturalQA EM at 4× compression, versus 56.85 for query relevance and 57.09 for static relevance–redundancy. Its advantage over the static rule extends across all five evaluated budgets, ranging from 4.42 to 8.03 EM points. This directly supports the implemented allocation mechanism within the proposed compression framework, beyond the evidence supplied by whole-system comparisons. The coverage analysis on 1,000 held-out 2WikiMQA examples provides a complementary diagnostic: annotated supporting facts identify relevant segments, and ReValue retains a larger fraction at high fidelity than independent query-relevance ranking. Coverage does not establish that every retained fact is subsequently used correctly, but its alignment with downstream gains makes the allocation result more informative than an improvement in answer scores alone.
-
Gains over retrained compressors and supporting ablations (PDF pp. 6–9, Tables 1–6; pp. 15–17, Appendices B–D). Appendix B.1 reports training RAM, COMI, and ICAE under the same experimental settings as ReValue. This supports the relevance of the compressor comparisons when assessing the overall method. ReValue improves average performance across tested backbones and budgets, including LongBench 28.48 versus RAM's 23.20, and the additional Qwen3-8B results extend the evidence beyond a single backbone. The regularization-weight, stop-gradient and segment-size studies also help explain concrete implementation choices. In particular, the segment-size analysis connects the unit of allocation to the quality of compression, while the stop-gradient study examines how the auxiliary objective interacts with learned quality weights. The paper additionally measures latency including compression and LLM inference, with a separate allocation-overhead breakdown. Together, these experiments provide a useful empirical assessment of the proposed framework rather than relying solely on its mathematical motivation.
Critical weaknesses
-
What does the DPP construction add beyond dynamic redundancy discounting? (PDF pp. 2 and 9, Tables 3/6; pp. 17–19, Appendix E.) Table 3 provides meaningful controlled evidence, but none of its alternative selectors updates redundancy against the currently selected set. Its static relevance–redundancy rule instead penalizes similarity to other context segments regardless of whether they have been retained. Thus, two explanations remain compatible with the results: the determinant structure provides an important advantage, or a simpler set-dependent rule could capture much of the benefit. Appendix E explains the DPP mechanism through its marginal gain and projection-residual interpretation; the concern is the empirical distinction between these alternatives, not an absence of theoretical explanation. Since the allocation criterion is central to the methodological contribution, a representative non-DPP greedy comparator that updates redundancy against the selected set would be particularly informative. Keeping representations, quality scores, high-fidelity budget and the remaining pipeline fixed would make this a focused test. A related, narrower issue concerns Table 6: the comparisons between Gaussian mixtures are informative, but without a single-bandwidth Gaussian, the improvement over direct similarity does not isolate the benefit of multiple scales from changing the kernel transformation. Neither concern invalidates the existing allocation results; both limit how specifically those results support the proposed formulation.
-
The serving-cost comparison is incomplete for the closest compression baselines (PDF p. 8, Figure 3; pp. 15–17, Appendices B.1/D). The reported latency includes compression, prefill and decoding, which is appropriate, and Appendix D separately quantifies kernel construction and greedy-selection overhead. However, Figure 3 does not identify its “Baseline”, and corresponding RAM/COMI serving costs are not reported. Equal compression ratios do not ensure equal compute or latency, so the quality improvements at those ratios do not establish a better quality–cost trade-off over these compressors. Identifying the reference and measuring these methods under the existing end-to-end protocol would make the practical comparison interpretable without requiring a different evaluation framework. This matters when deciding whether the measured quality gain justifies the resources needed to obtain it. Appendix B.1 already identifies the retrained compression baselines and distinguishes the zero-shot Original Prompt; the latter remains useful as a descriptive reference, but does not isolate compression from task-specific adaptation.
I lean toward weak acceptance. The fixed-component selector gains, supporting-fact coverage and improvements over retrained compressors across backbones provide a meaningful empirical contribution. My principal reservation concerns how much of that contribution is specific to the DPP allocation criterion; the serving-cost question chiefly limits the deployment conclusion. A consistent advantage over a matched dynamic selector would strengthen the methodological case. If that selector matched ReValue across the tested budgets with lower overhead, I would place less weight on the distinctive allocation contribution and reconsider this borderline recommendation. The single-Gaussian comparison would refine the multi-scale claim. These are unresolved outcomes, and I do not assume they will favor the alternatives; the evidence currently available places the paper just above the acceptance boundary.
Questions for authors
- For W1 — main priority: In Table 3's fixed-component setup, how does ReValue compare with a non-DPP greedy rule that updates redundancy against the selected set at 4× and 8×? Please retain the same representations, quality scores and budget, and tune any relevance–redundancy weight without using evaluation outcomes. This would distinguish the criterion from the general benefit of updating redundancy.
- For W1: What happens with a single-bandwidth Gaussian under the same setup as Table 6? This would clarify which part of the multi-scale interpretation is supported.
- For W2: Please identify Figure 3's Baseline and its hardware/inference precision. Can RAM/COMI be measured under the same end-to-end protocol, with quality reported at the corresponding operating points?
2 comments
Thank you for the detailed response and revisions. I have checked the revised manuscript against the rebuttal. The three main requests in my review have been substantially addressed, and the corresponding experiments and protocol clarifications are now present in the paper. I have no remaining major concern on the three issues raised in my review.
Point-to-Point Response to Reviewer XbAW
We're truly grateful that you found our allocation mechanism effective and our empirical evaluations comprehensive! Thank you for your thoughtful and detailed review. We provide point-by-point responses below.
W1&Q1: Comparison with dynamic redundancy-aware selection.
A1: Insightful suggestion! We have added Greedy Maximal Marginal Relevance (MMR) as an additional baseline in Table 3. At each step, MMR selects the candidate maximizing MMR(i∣Y)=λri−(1−λ)maxj∈YSij, where ri and Sij follow Eqs. 2-3, respectively, and Y denotes the current high-fidelity set. We select λ=0.5 from {0.3,0.5,0.7} based on performance on 2,000 validation instances from 2WikiMQA, without using evaluation results. For a controlled comparison, we use the same model checkpoint, learned representations, query relevance estimates, similarity matrix, and high-fidelity budgets as ReValue, replacing only the allocation criterion.
The results on NaturalQA are summarized below:
First, Greedy MMR consistently outperforms static relevance–redundancy scoring, demonstrating the effectiveness of a dynamic redundancy-aware alternative. Second, ReValue further improves EM by 3.06 and 5.21 points over dynamic MMR at 4× and 8× compression, respectively. While MMR penalizes the maximum pairwise similarity to selected segments, our DPP objective (Eq. 7) accounts for their joint similarity structure, enabling it to capture redundancy collectively explained by multiple segments. These results provide empirical evidence for the additional benefit of DPP-based set-level modeling beyond dynamic redundancy discounting alone.
We have incorporated these results into Table 3 and the corresponding discussion in Sec. 4.4 of the revised manuscript.
W1&Q2: Effect of single-bandwidth Gaussian similarity.
A2: Good comment! We have added a single-bandwidth Gaussian baseline (σ=1.0) to Table 6, keeping all other settings unchanged. The updated results are shown below:
The single-bandwidth Gaussian with σ=1.0 improves the average EM by only 0.36 points over direct similarity, whereas our multi-scale formulation further improves it by 1.77 points. These results indicate that the improvement is not solely attributable to adopting a Gaussian kernel. Combining multiple bandwidths further improves performance over the tested single-bandwidth baseline, consistent with our motivation to capture semantic overlap at different scales. We have incorporated this additional baseline and the corresponding analysis into Table 6 and Sec. 4.3 of the revised manuscript.
W2&Q3: End-to-end serving costs and accuracy-efficiency trade-off.
A3: Valuable suggestion! First, we clarify that the "Baseline" in Figure 3 refers to Original Prompt (uncompressed inference). Following your suggestion, we have conducted additional comparisons with RAM and COMI on the same NaturalQA evaluation samples, using Qwen3-4B-Instruct as the backbone, an NVIDIA H20 GPU, BF16 inference precision, batch size 1, and identical generation settings. All methods are evaluated under 4× compression, with end-to-end (E2E) latency covering context compression, prefill, and decoding. Speedups are measured relative to the same uncompressed Original Prompt.
The results are summarized below:
The results show that ReValue achieves a favorable accuracy–efficiency trade-off compared with RAM and COMI. Specifically, ReValue improves EM by 5.28 points over RAM while maintaining nearly identical end-to-end speedup (2.24× vs. 2.26×). Compared with COMI, ReValue achieves 18.35 points higher EM with a greater speedup (2.24× vs. 1.97×). These results demonstrate that ReValue achieves substantial accuracy improvements while maintaining inference costs comparable to RAM.
We have clarified the baseline in the caption of Fig. 3, specified the evaluation protocol in Sec. 4.2 and Appendix B.1, and added the matched accuracy–efficiency comparisons in Tables 7.
Thank you again for your thoughtful review! We hope our responses have addressed your concerns and would be happy to provide further clarification if needed.