- Rating
- 1
- Confidence
- 3
- Informativeness
- 1
- Presentation
- 2
- Soundness
- 1
Read review
Paper summary
In general, this paper want to address some problems of long-context distillation. Specifically, it frames context compression as a subset selection problem—constrained by a token budget and aligned with a solver—and introduces TaCo, a method that employs REINFORCE to train the selection policy while incorporating structural and geometric priors as features within the state representation. Experiments conducted across multiple benchmarks demonstrate some performance improvements, and ablation studies illustrate the impact of the various components.
Critical strengths
The problem addressed in this paper have some practical significance, as long-context tasks are indeed common in real-world scenarios (and thus may attract some interest from the community). Furthermore, the paper candidly discusses certain limitations.
Critical weaknesses
In fact, I am a bit confused or seriously concerned about some of the experimental results and data presented in this paper.
For instance, the "Task Score" for the "w/o Symbolic" ablation in Table 4 appears identical to that of the "Dense Top-K" baseline in Table 2 across multiple benchmarks. However, there is no mention of their equivalence (that's why I am confused); furthermore, the results for the two differ on the Financial-250k dataset. I thus suggest that the authors should provide a thorough explanation and analysis regarding these results.
Additionally, regarding the latency comparison (the "Com" column) for TaCo in Table 11—specifically the values 0.12, 0.24, 0.36, 0.48 (i.e., 0.12*n), etc.—there is a consistent increment of 0.12 between each interval. Is there a specific pattern behind this? The authors should also clarify this point.
Questions for authors
See Critical Weaknesses and,
-
How were the results in Table 11 obtained?
-
The values for "w/o Symbolic" in Table 4 and "Dense Top-K" in Table 2 appear to be identical; do they represent the same method?