acceptodds
Under review as a conference paper at ICLR 2027

What Still Matters: Compressing Long Context via Set-Conditioned Marginal Value

Abstract

Long-context inference is essential to many applications of large language models (LLMs), yet processing lengthy inputs remains computationally expensive and often suffers from substantial information redundancy. Existing context compression methods alleviate this burden by reducing input length, and recent approaches further assign non-uniform compression rates according to segment-wise importance. In this work, we argue that the value of preserving a segment at high fidelity is inherently set-dependent: a segment that appears highly relevant in isolation may contribute little once its evidence is already covered, while a less relevant but complementary segment can provide greater marginal utility. To capture this dependency, we introduce ReValue a set-conditioned context compression framework that continually reassesses what still matters as the high-fidelity set evolves. ReValue first derives compact query-aware representations for context segments, and then constructs a Determinantal Point Process (DPP)-based set objective that jointly captures query-conditioned quality and multi-scale semantic overlap. Rather than relying on fixed segment-wise scores, each candidate segment is evaluated by its marginal contribution relative to the current high-fidelity set, allowing redundant evidence to be progressively discounted while complementary information remains valuable. The resulting hybrid context preserves full token-level representations for high-value segments and compact representations for the rest, further supported by a representation diversity regularizer. Experiments demonstrate consistent gains across backbones, compression ratios, and benchmarks, including up to 11.04 points in average F1 and 5.01 faster first-token latency.

Primary area:
foundation or frontier models, including LLMs
Keywords:
Long-Context Compression, Large Language Models, Query-Aware Compression, Set-Conditioned Allocation

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_XbAWOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

ReValue is a query-aware hybrid context compressor that retains token-level representations for selected segments and compact vectors for the remainder. Its central idea is to allocate the limited high-fidelity budget according to a segment's marginal value given the segments already selected. A greedy DPP log-determinant objective combines query relevance with multi-scale similarity, while an auxiliary diversity regularizer acts on compact representations alongside next-token prediction. Experiments evaluate downstream quality across backbones and compression ratios, compare allocation rules with other components fixed, measure supporting-fact coverage, and examine end-to-end inference efficiency.

Critical strengths
  1. Controlled allocation and evidence coverage (PDF pp. 8–9, Figures 4–5, Table 3 and Section 4.4). Table 3 changes the inference-time selector while keeping the remaining components fixed. ReValue obtains 61.51 NaturalQA EM at 4× compression, versus 56.85 for query relevance and 57.09 for static relevance–redundancy. Its advantage over the static rule extends across all five evaluated budgets, ranging from 4.42 to 8.03 EM points. This directly supports the implemented allocation mechanism within the proposed compression framework, beyond the evidence supplied by whole-system comparisons. The coverage analysis on 1,000 held-out 2WikiMQA examples provides a complementary diagnostic: annotated supporting facts identify relevant segments, and ReValue retains a larger fraction at high fidelity than independent query-relevance ranking. Coverage does not establish that every retained fact is subsequently used correctly, but its alignment with downstream gains makes the allocation result more informative than an improvement in answer scores alone.

  2. Gains over retrained compressors and supporting ablations (PDF pp. 6–9, Tables 1–6; pp. 15–17, Appendices B–D). Appendix B.1 reports training RAM, COMI, and ICAE under the same experimental settings as ReValue. This supports the relevance of the compressor comparisons when assessing the overall method. ReValue improves average performance across tested backbones and budgets, including LongBench 28.48 versus RAM's 23.20, and the additional Qwen3-8B results extend the evidence beyond a single backbone. The regularization-weight, stop-gradient and segment-size studies also help explain concrete implementation choices. In particular, the segment-size analysis connects the unit of allocation to the quality of compression, while the stop-gradient study examines how the auxiliary objective interacts with learned quality weights. The paper additionally measures latency including compression and LLM inference, with a separate allocation-overhead breakdown. Together, these experiments provide a useful empirical assessment of the proposed framework rather than relying solely on its mathematical motivation.

Critical weaknesses
  1. What does the DPP construction add beyond dynamic redundancy discounting? (PDF pp. 2 and 9, Tables 3/6; pp. 17–19, Appendix E.) Table 3 provides meaningful controlled evidence, but none of its alternative selectors updates redundancy against the currently selected set. Its static relevance–redundancy rule instead penalizes similarity to other context segments regardless of whether they have been retained. Thus, two explanations remain compatible with the results: the determinant structure provides an important advantage, or a simpler set-dependent rule could capture much of the benefit. Appendix E explains the DPP mechanism through its marginal gain and projection-residual interpretation; the concern is the empirical distinction between these alternatives, not an absence of theoretical explanation. Since the allocation criterion is central to the methodological contribution, a representative non-DPP greedy comparator that updates redundancy against the selected set would be particularly informative. Keeping representations, quality scores, high-fidelity budget and the remaining pipeline fixed would make this a focused test. A related, narrower issue concerns Table 6: the comparisons between Gaussian mixtures are informative, but without a single-bandwidth Gaussian, the improvement over direct similarity does not isolate the benefit of multiple scales from changing the kernel transformation. Neither concern invalidates the existing allocation results; both limit how specifically those results support the proposed formulation.

  2. The serving-cost comparison is incomplete for the closest compression baselines (PDF p. 8, Figure 3; pp. 15–17, Appendices B.1/D). The reported latency includes compression, prefill and decoding, which is appropriate, and Appendix D separately quantifies kernel construction and greedy-selection overhead. However, Figure 3 does not identify its “Baseline”, and corresponding RAM/COMI serving costs are not reported. Equal compression ratios do not ensure equal compute or latency, so the quality improvements at those ratios do not establish a better quality–cost trade-off over these compressors. Identifying the reference and measuring these methods under the existing end-to-end protocol would make the practical comparison interpretable without requiring a different evaluation framework. This matters when deciding whether the measured quality gain justifies the resources needed to obtain it. Appendix B.1 already identifies the retrained compression baselines and distinguishes the zero-shot Original Prompt; the latter remains useful as a descriptive reference, but does not isolate compression from task-specific adaptation.

I lean toward weak acceptance. The fixed-component selector gains, supporting-fact coverage and improvements over retrained compressors across backbones provide a meaningful empirical contribution. My principal reservation concerns how much of that contribution is specific to the DPP allocation criterion; the serving-cost question chiefly limits the deployment conclusion. A consistent advantage over a matched dynamic selector would strengthen the methodological case. If that selector matched ReValue across the tested budgets with lower overhead, I would place less weight on the distinctive allocation contribution and reconsider this borderline recommendation. The single-Gaussian comparison would refine the multi-scale claim. These are unresolved outcomes, and I do not assume they will favor the alternatives; the evidence currently available places the paper just above the acceptance boundary.

Questions for authors
  1. For W1 — main priority: In Table 3's fixed-component setup, how does ReValue compare with a non-DPP greedy rule that updates redundancy against the selected set at 4× and 8×? Please retain the same representations, quality scores and budget, and tune any relevance–redundancy weight without using evaluation outcomes. This would distinguish the criterion from the general benefit of updating redundancy.
  2. For W1: What happens with a single-bandwidth Gaussian under the same setup as Table 6? This would clarify which part of the multi-scale interpretation is supported.
  3. For W2: Please identify Figure 3's Baseline and its hardware/inference precision. Can RAM/COMI be measured under the same end-to-end protocol, with quality reported at the corresponding operating points?
2 comments
AuthorsOpenReview ↗

Point-to-Point Response to Reviewer XbAW

We're truly grateful that you found our allocation mechanism effective and our empirical evaluations comprehensive! Thank you for your thoughtful and detailed review. We provide point-by-point responses below.

W1&Q1: Comparison with dynamic redundancy-aware selection.

A1: Insightful suggestion! We have added Greedy Maximal Marginal Relevance (MMR) as an additional baseline in Table 3. At each step, MMR selects the candidate maximizing MMR⁡(i∣Y)=λri−(1−λ)max⁡j∈YSij,\operatorname{MMR}(i\mid Y)=\lambda r_i-(1-\lambda)\max_{j\in Y}S_{ij}, where rir_i and SijS_{ij} follow Eqs. 2-3, respectively, and Y denotes the current high-fidelity set. We select λ=0.5\lambda=0.5 from {0.3,0.5,0.7} based on performance on 2,000 validation instances from 2WikiMQA, without using evaluation results. For a controlled comparison, we use the same model checkpoint, learned representations, query relevance estimates, similarity matrix, and high-fidelity budgets as ReValue, replacing only the allocation criterion.

The results on NaturalQA are summarized below:

Allocation Strategy2×4×8×16×32×
Static Relevance-Redundancy59.9657.0951.3748.6845.95
Greedy MMR63.8358.4554.1950.7647.49
ReValue66.8561.5159.4056.0153.86
Improvement+3.02+3.06+5.21+5.25+6.37

First, Greedy MMR consistently outperforms static relevance–redundancy scoring, demonstrating the effectiveness of a dynamic redundancy-aware alternative. Second, ReValue further improves EM by 3.06 and 5.21 points over dynamic MMR at 4× and 8× compression, respectively. While MMR penalizes the maximum pairwise similarity to selected segments, our DPP objective (Eq. 7) accounts for their joint similarity structure, enabling it to capture redundancy collectively explained by multiple segments. These results provide empirical evidence for the additional benefit of DPP-based set-level modeling beyond dynamic redundancy discounting alone.

We have incorporated these results into Table 3 and the corresponding discussion in Sec. 4.4 of the revised manuscript.

W1&Q2: Effect of single-bandwidth Gaussian similarity.

A2: Good comment! We have added a single-bandwidth Gaussian baseline (σ=1.0\sigma=1.0) to Table 6, keeping all other settings unchanged. The updated results are shown below:

Kernel ScalesNQHQAMQAAvg.
Direct Similarity60.0444.3448.5050.96
{1.0}61.1644.4848.3251.32
{0.5, 1.0}63.1644.9248.3952.16
{0.25, 0.5, 1.0}61.5147.4950.2653.09

The single-bandwidth Gaussian with σ=1.0\sigma=1.0 improves the average EM by only 0.36 points over direct similarity, whereas our multi-scale formulation further improves it by 1.77 points. These results indicate that the improvement is not solely attributable to adopting a Gaussian kernel. Combining multiple bandwidths further improves performance over the tested single-bandwidth baseline, consistent with our motivation to capture semantic overlap at different scales. We have incorporated this additional baseline and the corresponding analysis into Table 6 and Sec. 4.3 of the revised manuscript.

W2&Q3: End-to-end serving costs and accuracy-efficiency trade-off.

A3: Valuable suggestion! First, we clarify that the "Baseline" in Figure 3 refers to Original Prompt (uncompressed inference). Following your suggestion, we have conducted additional comparisons with RAM and COMI on the same NaturalQA evaluation samples, using Qwen3-4B-Instruct as the backbone, an NVIDIA H20 GPU, BF16 inference precision, batch size 1, and identical generation settings. All methods are evaluated under 4× compression, with end-to-end (E2E) latency covering context compression, prefill, and decoding. Speedups are measured relative to the same uncompressed Original Prompt.

The results are summarized below:

MethodRatioEM ↑E2E Speedup ↑
Original Prompt1×32.771.00×
COMI4×43.161.97×
RAM4×56.232.26×
ReValue4×61.512.24×

The results show that ReValue achieves a favorable accuracy–efficiency trade-off compared with RAM and COMI. Specifically, ReValue improves EM by 5.28 points over RAM while maintaining nearly identical end-to-end speedup (2.24× vs. 2.26×). Compared with COMI, ReValue achieves 18.35 points higher EM with a greater speedup (2.24× vs. 1.97×). These results demonstrate that ReValue achieves substantial accuracy improvements while maintaining inference costs comparable to RAM.

We have clarified the baseline in the caption of Fig. 3, specified the evaluation protocol in Sec. 4.2 and Appendix B.1, and added the matched accuracy–efficiency comparisons in Tables 7.


Thank you again for your thoughtful review! We hope our responses have addressed your concerns and would be happy to provide further clarification if needed.

Thank you for the detailed response and revisions. I have checked the revised manuscript against the rebuttal. The three main requests in my review have been substantially addressed, and the corresponding experiments and protocol clarifications are now present in the paper. I have no remaining major concern on the three issues raised in my review.

Reviewer_Pc1fOpenReview ↗
Rating
2
Confidence
3
Informativeness
1
Presentation
3
Soundness
2
Read review
Paper summary

The paper proposes ReValue for query-aware long-context compression. It uses a quality-weighted Determinantal Point Process (DPP) to select segments for high-fidelity preservation under a fixed budget. Selected segments retain token-level representations, while the remaining segments use compact vectors. A multi-scale similarity kernel and a representation diversity regularizer support the selection process. Experiments report improvements across several backbones, compression ratios, and downstream tasks.

Critical strengths
  1. The reported empirical improvements are substantial and extend across multiple settings. The evaluation includes two main backbones, another backbone in supplementary experiments, different compression ratios, and out-of-domain LongBench tasks. Table 3 also isolates inference-time allocation strategies while holding other components fixed, providing useful evidence that the proposed selection rule contributes to downstream performance.
  2. The allocation objective is coherent and interpretable. Its decomposition into query-conditioned quality and set-dependent redundancy makes the selection behavior transparent. The supporting-fact coverage analysis provides additional evidence that the method can allocate its limited high-fidelity budget more effectively than independent relevance ranking.
Critical weaknesses
  1. The overall architecture closely follows RAM’s parallel encoding and hybrid representations. Quality-weighted DPP selection, multi-scale kernels, and diversity regularization are established techniques. The manuscript does not sufficiently establish an original contribution beyond adapting and combining these components for high-fidelity segment allocation.
  2. Reassessing relevance and redundancy against previously selected content is central to methods such as MMR. However, Table 3 only compares against static redundancy scoring. Without a dynamic MMR baseline using identical representations and budgets, the experiments do not establish the necessity of the DPP formulation.
  3. Every segment already remains available through its compact representation. The relevant decision is therefore whether upgrading a segment to token-level representations provides sufficient task benefit. Query relevance and semantic diversity do not directly quantify this benefit, and the paper provides limited evidence connecting its DPP score to fidelity-upgrade utility.
  4. The contributions of representation learning and selection remain insufficiently separated. Table 3 substitutes selectors at inference time on ReValue-trained representations. Although the regularizer is separately ablated, these experiments do not establish whether DPP remains advantageous when alternative selectors receive comparably adapted representations. The reported gains therefore provide incomplete evidence for the specific selection mechanism.
Questions for authors
  1. How does ReValue compare with dynamic MMR under the same representations, model checkpoint, and high-fidelity budget? Can the authors distinguish the benefit of selected-set-dependent redundancy reduction from the benefit of the specific DPP objective?
  2. What exact EM and F1 definitions and evaluation implementations are used for NaturalQA? Please explain the EM > F1 results, including normalization, reference handling, and aggregation.
  3. Is an accompanying implementation or supplementary configuration available? Please provide the missing training and decoding settings, DPP numerical stabilization, GPU configuration, and timing protocol needed to reproduce the results.
3 comments
AuthorsOpenReview ↗

Point-to-Point Response to Reviewer Pc1f [1/3]

We sincerely appreciate your positive feedback on our empirical results, clear formulation, and effective fidelity allocation. We address your thoughtful concerns below.

W1: Contributions.

A1: Sorry for the confusion. We respectfully argue that our work identifies a previously underexplored principle of adaptive long-context compression: the value of preserving a segment at high fidelity is not fixed, but evolves with the high-fidelity set. Motivated by this insight, we develop a principled, set-conditioned compression framework. Below, we clarify contributions in terms of motivation and technical design.

(1) Motivation.

Existing query-aware adaptive compression methods typically assign representation fidelity based on fixed scores: RAM relies on query relevance, and COMI considers static redundancy. However, these overlook a fundamental dependency in fidelity allocation: a segment's preservation value changes with the evidence already retained at high fidelity. Even a highly relevant segment may contribute little additional information once its evidence is covered.

In contrast, we are the first to formulate adaptive compression as a set-conditioned fidelity allocation problem and introduce DPP-based Set-Conditioned High-Fidelity Allocation, which continually re-evaluates each segment's marginal preservation value as the high-fidelity set evolves. This represents a task-specific shift from fixed segment-wise scoring to set-conditioned valuation, rather than a generic application of DPP.

(2) Technical Design

(i) Differences from RAM.

  • Problem Formulation: RAM allocates fidelity based on query relevance, whereas ReValue introduces set-conditioned fidelity allocation.
  • High-Fidelity Representations: RAM preserves token embeddings from the decoder's embedding layer, whereas ReValue retains contextualized token-level hidden states from encoder.
  • Training Objective: RAM requires segment-level supervision for its contrastive loss, whereas ReValue relies on label-free diversity regularization.

(ii) Task-Motivated Technical Designs. Our formulation raises two closely connected challenges: dynamically evaluating marginal preservation value and learning discriminative compact representations to support reliable valuation. We address them through two complementary, task-specific designs:

Set-Conditioned Hybrid Context Compression.

  • Motivation: Effective high-fidelity allocation requires identifying which segments remain valuable given the current allocation, motivating set-conditioned marginal valuation.

  • Technical Design: We explore DPP-based representation fidelity allocation, a largely unexplored direction in adaptive long-context compression. Different from conventional DPP formulations aiming to select high-quality, diverse subsets, we adapt DPP's set-conditioned marginal valuation to allocate a limited high-fidelity budget in a hybrid compressed context: selected segments retain their original token-level representations, while the remaining segments remain available in compact form. Our DPP kernel combines query-conditioned segment quality with multi-scale semantic overlap, allowing each segment’s preservation value to be reassessed as the high-fidelity set evolves. Unlike a single-scale kernel, which may overlook partial redundancy or over-penalize complementary segments, our multi-scale kernel averages Gaussian similarities across bandwidths to capture both close semantic matches and broader semantic overlap between segments. Ablations validate the benefits of DPP-based allocation over dynamic MMR (Tab.3) and multi-scale modeling (Tab.6).

Representation Diversity Regularization.

  • Motivation: While DPP enables set-conditioned valuation, reliable set-conditioned fidelity allocation requires compact representations that preserve meaningful distinctions among segments.

  • Technical Design: Unlike generic diversity regularization that merely encourages dissimilar representations, our regularizer explicitly links representation learning with set-conditioned fidelity allocation through a shared multi-scale kernel. It encourages more discriminative compact representations for estimating set-dependent redundancy. Crucially, our tailored stop-gradient design preserves query-quality weighting while restricting regularization gradients to the similarity branch, preventing direct optimization of relevance scores. This establishes a direct connection between the training objective and the downstream allocation mechanism, rather than simply adding a standalone diversity loss. Ablations confirm the necessity of the regularizer (Tab.4) and stop-gradient design (Tab.9).

Thus, we believe our work offers valuable new insights and technical contributions to the field of long-context compression.

AuthorsOpenReview ↗

Point-to-Point Response to Reviewer Pc1f [2/3]

W2&Q1: Comparison with dynamic MMR and the necessity of the DPP formulation.

A2: Insightful suggestion! We have added Greedy Maximal Marginal Relevance (MMR) as an additional baseline in Table 3. At each selection step, Greedy MMR selects the candidate maximizing MMR⁡(i∣Y)=λri−(1−λ)max⁡j∈YSij,\operatorname{MMR}(i\mid Y)=\lambda r_i-(1-\lambda)\max_{j\in Y}S_{ij}, where rir_i and SijS_{ij} are defined in Eqs. 2-3, respectively, and Y denotes the current high-fidelity set. We tune λ\lambda over {0.3, 0.5, 0.7} on 2,000 validation samples from 2WikiMQA and select λ=0.5\lambda=0.5 based on validation performance. For a controlled comparison, we use identical learned representations, model checkpoint, similarity matrix, and high-fidelity budgets, replacing only the allocation objective.

Allocation Strategy2×4×8×16×32×
Static Relevance-Redundancy59.9657.0951.3748.6845.95
Greedy MMR63.8358.4554.1950.7647.49
ReValue66.8561.5159.4056.0153.86
Improvement+3.02+3.06+5.21+5.25+6.37

First, Greedy MMR consistently outperforms static relevance–redundancy scoring, demonstrating the effectiveness of a dynamic redundancy-aware alternative.

Second, ReValue further improves EM by 3.02-6.37 points over dynamic MMR, with larger gains under tighter compression budgets. While MMR measures redundancy through the maximum pairwise similarity to selected segments, our DPP objective (Eq. 7) additionally accounts for the joint similarity structure of the selected set, allowing it to assess redundancy collectively captured by multiple segments. These controlled results support the additional value of the DPP objective beyond selected-set-dependent redundancy reduction alone.

We have incorporated these results into Table 3 and the corresponding discussion in Sec. 4.4 of the revised manuscript.

W3: Relationship Between DPP Scoring and Fidelity-Upgrade Utility.

A3: Thoughtful comment! We clarify that our DPP score is not a direct estimate of fidelity-upgrade utility, but an efficient, set-conditioned surrogate for guiding high-fidelity allocation. To empirically examine this relationship, we conduct an additional experiment measuring how well the DPP score correlates with the actual benefit of upgrading individual segments.

(1) Direct Evaluation of Fidelity-Upgrade Utility. We evaluate 1000 held-out examples from 2WikiMQA using Qwen3-4B-Instruct under 4× compression. For each example, the high-fidelity budget K is determined following Sec. 3.3 (segment length = 50). We construct a common high-fidelity set of K-1 segments using ReValue and individually upgrade each remaining candidate from compact to token-level representations, keeping all other representations unchanged. We measure the upgrade utility by the change in ground-truth answer log-likelihood and compute the Spearman rank correlation between the allocation scores and measured utilities across candidates, averaged over all examples.

Scoring MethodSpearman ρ ↑
Query Relevance0.18
Greedy MMR0.27
ReValue0.52

Our DPP score achieves a correlation of 0.52, substantially outperforming Query Relevance (0.18) and Greedy MMR (0.27). This provides direct empirical evidence that DPP-based scoring better aligns with the measured benefits of upgrading individual segments to high-fidelity representations.

(2) Supporting Evidence. Fig. 4 further shows that ReValue allocates the limited high-fidelity budget to a greater proportion of ground-truth supporting segments than query-relevance-based allocation. More importantly, Table 3 confirms that this allocation strategy translates into higher downstream QA accuracy. Under identical representations and compression budgets, ReValue improves NaturalQA EM over Query Relevance by 9.08/9.32 points and over Greedy MMR by 3.06/6.37 points at 4×/32× compression, respectively.

Together, these results show that DPP-based scoring aligns more closely with the measured likelihood-based utility of fidelity upgrades, without requiring repeated downstream evaluations during allocation. We have incorporated this additional fidelity-upgrade utility analysis into Sec. 4.4 of the revised manuscript.

AuthorsOpenReview ↗

Point-to-Point Response to Reviewer Pc1f [3/3]

W4: Disentangling representation learning and selection.

A4: We appreciate your thoughtful suggestion! We additionally train a Greedy MMR variant from the same initialization, replacing DPP-based allocation with Greedy MMR (λ=0.5\lambda=0.5) throughout both training and inference. We keep the backbone, training data, training schedule, compression budgets, and other components unchanged. Importantly, we remove the representation diversity regularizer from both methods to eliminate its potential influence on the comparison. This allows each selection strategy to learn representations adapted to its own allocation objective under matched training conditions.

Allocation StrategyNaturalQAHotpotQA2WikiMQAAvg.
Greedy MMR w/o REG55.4036.0839.7843.75
ReValue w/o REG59.7444.0848.6750.83

ReValue consistently outperforms the independently trained Greedy MMR variant across all three benchmarks, with an average improvement of 7.08 EM points. These results show that the advantage of DPP-based allocation persists even when MMR receives its own representation adaptation and neither method uses the DPP-inspired regularizer. Together with Table 3, which compares allocation strategies under identical learned representations, these experiments provide complementary evidence that the advantage of DPP-based allocation is not solely attributable to its learned representations.

We have incorporated this additional comparison into Appendix C of the revised manuscript.

Q2: Clarification of NaturalQA evaluation metrics.

A5: Thanks for your careful review! We use SQuAD-style answer normalization, including lowercasing, punctuation and article removal, and whitespace normalization. Specifically, EM uses normalized substring matching, counting a prediction as correct if it contains any normalized reference answer, while F1 measures token-level precision and recall over the full generated response. For multiple reference answers, we take the maximum score for each metric across references and average the results over all evaluation examples. This explains why EM can exceed F1: a generated response may contain an exact answer span while including additional tokens, resulting in EM = 1 but a lower F1. We have clarified these evaluation details in Appendix B.2.

Q3: Implementation details.

A6: Good suggestion! We have provided additional implementation details and experimental configurations to facilitate reproducibility, as summarized below.

Training and decoding. We use AdamW with a learning rate of 1×10−51 \times 10^{-5}, a linear scheduler, and zero weight decay. Training is conducted on 24 NVIDIA H800 GPUs with BF16, DeepSpeed ZeRO-2, and a global batch size of 24. Inference uses greedy decoding, a batch size of 1, and a maximum of 64 generated tokens.

DPP numerical stability. We perform DPP kernel computations in FP32, with a quality floor of 0.010.01. The regularizer uses Cholesky-based log-determinant computation with 10−410^{-4} diagonal regularization. Greedy selection applies a 10−610^{-6} numerical floor and non-negative residual clipping.

GPU configuration and timing. Accuracy evaluation uses 8 NVIDIA H20 GPUs, whereas efficiency is measured on a single H20. End-to-end latency includes compression, prefill, and decoding. For the revised timing evaluation, we adopt 5 warm-up runs and 10 measured runs, with CUDA synchronization at timing boundaries, and report the mean latency.


We sincerely appreciate your constructive feedback and hope we addressed all your concerns. Please let us know if you'd like any further information.

Reviewer_2X2WOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes ReValue, a long-context compression framework that dynamically allocates high-fidelity representations based on each segment's marginal value relative to the selected set. It employs a DPP-based objective combining query relevance and semantic diversity, along with a representation diversity regularizer. Experiments across multiple LLM backbones and benchmarks demonstrate improved compression performance and inference efficiency.

Critical strengths
  1. The paper introduces a well-motivated DPP-based framework that dynamically balances query relevance and redundancy for high-fidelity allocation.

  2. ReValue consistently outperforms strong compression baselines across multiple backbones, compression ratios, and benchmarks.

  3. Extensive ablation studies and supporting-fact coverage analysis validate the effectiveness of the proposed approach.

Critical weaknesses
  1. Although ReValue is compared against adaptive compression methods such as RAM and COMI, the allocation ablation lacks comparisons with simpler set-conditioned selection strategies (e.g., greedy MMR). Thus, it remains unclear whether the DPP formulation provides additional benefits beyond dynamically accounting for redundancy during selection.

  2. Efficiency gains are primarily evaluated against uncompressed inference rather than competing compression methods. Without direct latency and memory comparisons under matched settings, it remains unclear whether ReValue offers a better accuracy–efficiency trade-off than existing methods.

Questions for authors
  1. How does ReValue compare with simpler set-conditioned selection methods, such as greedy MMR, under the same representations and compression budgets? This comparison would help clarify the additional benefits of the DPP-based formulation over dynamic redundancy-aware selection.

  2. Could you provide comparisons of TTFT, total inference latency, TFLOPs, and peak GPU memory with RAM and COMI under matched backbone, compression ratio, and hardware settings?

1 comment
AuthorsOpenReview ↗

Point-to-Point Response to Reviewer 2X2W

Glad to see that our motivation, method, and experimental results resonated with you! Thank you for your insightful review. We provide point-by-point responses below.

W1&Q1: Comparison with simpler set-conditioned selection strategy (Greedy MMR).

A1: Insightful suggestion! Following your suggestion, we have added Greedy MMR as an additional baseline in Table 3. Specifically, at each step, Greedy MMR selects the segment maximizing MMR⁡(i∣Y)=λri−(1−λ)max⁡j∈YSij,\operatorname{MMR}(i\mid Y)=\lambda r_i-(1-\lambda)\max_{j\in Y}S_{ij}, where rir_i and SijS_{ij} are the query relevance and multi-scale similarity defined in Eqs. 2-3, respectively, and Y denotes the current high-fidelity set. We tune λ\lambda over {0.3,0.5,0.7} on 2,000 validation samples from 2WikiMQA and select λ=0.5\lambda=0.5 based on validation performance. For a fair comparison, we use the same learned representations and compression budgets as ReValue, replacing only the allocation strategy.

The results on NaturalQA are summarized below:

Allocation Strategy2×4×8×16×32×
Greedy MMR63.8358.4554.1950.7647.49
ReValue66.8561.5159.4056.0153.86
Improvement+3.02+3.06+5.21+5.25+6.37

ReValue improves EM by 3.02–6.37 points over Greedy MMR, and the advantage becomes more pronounced under tighter compression budgets. Although both methods dynamically re-evaluate candidates, Greedy MMR penalizes redundancy based on the maximum pairwise similarity to the selected segments. In contrast, our DPP formulation evaluates each candidate against the joint similarity structure of the entire selected set, allowing it to account for redundancy collectively captured by multiple segments. These results support the additional benefit of DPP-based set-level modeling beyond dynamic redundancy-aware selection alone.

We have incorporated these results into Table 3 and the corresponding discussion in Sec. 4.4 of the revised manuscript.

W2&Q2: Efficiency comparison with RAM and COMI.

A2: Valuable comment! We have conducted additional efficiency comparisons with RAM and COMI under the same Qwen3-4B-Instruct backbone, 4x compression ratio, hardware, and inference settings. Following our original evaluation protocol, all speedups and reductions are measured relative to Original Prompt (uncompressed inference), including context compression, prefill, and decoding in the end-to-end latency measurements.

MethodContext LengthTTFT Speedup ↑Total Latency Speedup ↑FLOPs Reduction ↑Peak Memory Reduction ↑
ReValue32K3.59×2.38×77.49%40.6%
RAM32K3.60×2.35×77.50%37.4%
COMI32K3.05×1.97×66.36%33.35%
ReValue64K5.01×4.05×83.49%50.0%
RAM64K5.10×4.11×83.50%48.1%
COMI64K4.37×3.79×76.26%33.38%

The results show that ReValue consistently outperforms COMI across all four efficiency metrics while achieving efficiency comparable to RAM. Combined with the accuracy improvements over RAM and COMI reported in Table 1, these results demonstrate that ReValue provides a favorable accuracy-efficiency trade-off, achieving higher task performance without substantial additional inference costs. We have incorporated these comparisons and the corresponding discussion into the revised manuscript's efficiency analysis.


We appreciate again your thoughtful review and hope we addressed all your concerns. Please let us know if you'd like any further information.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.