- Rating
- 1
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 1
- Soundness
- 1
Read review
Paper summary
This paper addresses topic-focused news timeline summarization, emphasizing the preservation of important milestones that may be overlooked by article-level summarization. It proposes MILES, a training-free framework consisting of topic-aware multi-event extraction, frequency-based date-cluster shortlisting, and multiview LLM milestone selection. Experiments on Entities, Crisis, and T17 report improvements of 35.6%, 33.3%, and 64.1% over LLM-TLS in semantic-alignment F1, respectively. The authors further provide component ablations, human preference studies, and LLM-based evaluation analyses to support their conclusions.
Critical strengths
S1. Meaningful problem formulation
The paper identifies an important limitation of existing timeline summarization methods. Article-level salience does not necessarily reflect the importance of individual events to a topic's development. Preserving secondary but consequential events is a worthwhile research direction.
S2. Extensive empirical analysis
The authors conduct experiments across three benchmarks and provide multiple component ablations and evaluation diagnostics. These experiments offer useful observations about candidate retention and LLM-based semantic selection.
Critical weaknesses
W1. Limited methodological novelty and insufficient justification of the multiview design
The proposed framework primarily combines existing techniques, including LLM-based event extraction, frequency-based candidate filtering, and prompt-based selection. Its contribution appears largely incremental and pipeline-oriented, without a sufficiently novel technical mechanism or theoretical insight. Moreover, the effectiveness of multiview union-compress is not convincingly established. In Table 3, its SA score of 0.2116 only marginally exceeds single-pass selection at 0.2066, while a separately tuned single-pass preference prompt achieves a higher score of 0.2149. Although the latter receives additional tuning, these results raise questions about whether the added multiview complexity is justified.
W2. Fundamental limitations of the candidate shortlisting strategy
The paper emphasizes preserving critical milestones that might otherwise be irrecoverably discarded. However, its shortlisting mechanism prioritizes dates by event frequency, potentially excluding infrequently reported but important milestones, particularly terminal events or long-term outcomes. Furthermore, retaining only one representative per date cluster can discard distinct milestones occurring on the same date. Such information losses cannot be recovered by downstream LLM selection. This creates a fundamental tension between the paper's motivation of preserving critical evidence and its frequency-driven shortlisting mechanism. The current experiments do not sufficiently address this concern.
W3. Insufficient justification of the shortlist multiplier as a methodological design choice
The shortlist multiplier, such as 8L or 12L, is not merely a computational hyperparameter. It directly determines which milestones remain accessible to the downstream selector. Adjusting the multiplier cannot fundamentally eliminate the structural bias introduced by frequency-based filtering. Furthermore, selecting the multiplier based on observed evaluation performance, without a principled selection criterion or an independent tuning protocol, raises concerns about robustness and generalization. The current pool-size ablations mainly compare aggregate performance and do not adequately explain how different multiplier choices affect the preservation of low-frequency but important milestones.
W4. Limited presentation quality (Minor)
The paper contains only one explanatory figure, which is presented as a raster image rather than a vector graphic. There are also visible inconsistencies between the text background and the surrounding figure background. Additional explanatory figures and improved visual consistency would help communicate the framework and its methodological details more clearly.
Questions for authors
Q1. How does the frequency-based shortlisting mechanism ensure the preservation of low-frequency but semantically important milestones, particularly terminal events? Can the authors provide recall analyses stratified by event frequency and temporal position?
Q2. How does the framework handle multiple distinct milestones occurring on the same date? What information is lost by retaining only one representative per date cluster, and how does this affect downstream selection?
Q3. Can the authors demonstrate that multiview union-compress provides a statistically significant advantage over well-designed single-pass selection under comparable model and inference budgets?
Q4. What is the principled justification for choosing the shortlist multiplier? Beyond aggregate SA scores, can the authors demonstrate that this choice is robust across datasets and does not systematically exclude critical low-frequency milestones?
Thank you for addressing the weaknesses and questions I raised. However, the responses have not substantially changed my overall assessment of the paper, so I will maintain my original score.