acceptodds
Under review as a conference paper at ICLR 2027

MILES: A Milestone-Centric Framework for LLM-Driven News Timeline Summarization

Abstract

A news timeline must preserve the events that explain how a topic develops, even when those events are buried in articles about something else. This makes timeline summarization a problem of retaining relevant events and selecting the final milestones. We introduce MILES (MIlestone selection with LLM-guided Evidence Structuring), a training-free framework for topic-focused timeline summarization (Topic-TLS) from a fixed collection of news articles. Topic-aware multi-event extraction retains both article-main events and embedded topic milestones; date-cluster shortlisting exposes a bounded candidate set; and an LLM selects the final milestones by their semantic importance. On Entities, Crisis, and T17, MILES achieves the highest semantic-alignment F1 among the evaluated systems, improving over LLM-TLS by 35.6%, 33.3%, and 64.1%, respectively. Controlled experiments locate the benefit: on identical candidate pools, every tested LLM selector outperforms the mechanical alternatives, while extraction and shortlisting determine which milestones remain available to select. Human preference studies and a diagnostic with four LLM judges support the semantic evaluation. These results support retaining topic events and using an LLM to select milestones from a bounded candidate pool.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
timeline summarization, large language models, milestone selection

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_Cag4OpenReview ↗
Rating
1
Confidence
3
Informativeness
2
Presentation
1
Soundness
1
Read review
Paper summary

This paper addresses topic-focused news timeline summarization, emphasizing the preservation of important milestones that may be overlooked by article-level summarization. It proposes MILES, a training-free framework consisting of topic-aware multi-event extraction, frequency-based date-cluster shortlisting, and multiview LLM milestone selection. Experiments on Entities, Crisis, and T17 report improvements of 35.6%, 33.3%, and 64.1% over LLM-TLS in semantic-alignment F1, respectively. The authors further provide component ablations, human preference studies, and LLM-based evaluation analyses to support their conclusions.

Critical strengths

S1. Meaningful problem formulation

The paper identifies an important limitation of existing timeline summarization methods. Article-level salience does not necessarily reflect the importance of individual events to a topic's development. Preserving secondary but consequential events is a worthwhile research direction.

S2. Extensive empirical analysis

The authors conduct experiments across three benchmarks and provide multiple component ablations and evaluation diagnostics. These experiments offer useful observations about candidate retention and LLM-based semantic selection.

Critical weaknesses

W1. Limited methodological novelty and insufficient justification of the multiview design

The proposed framework primarily combines existing techniques, including LLM-based event extraction, frequency-based candidate filtering, and prompt-based selection. Its contribution appears largely incremental and pipeline-oriented, without a sufficiently novel technical mechanism or theoretical insight. Moreover, the effectiveness of multiview union-compress is not convincingly established. In Table 3, its SA score of 0.2116 only marginally exceeds single-pass selection at 0.2066, while a separately tuned single-pass preference prompt achieves a higher score of 0.2149. Although the latter receives additional tuning, these results raise questions about whether the added multiview complexity is justified.

W2. Fundamental limitations of the candidate shortlisting strategy

The paper emphasizes preserving critical milestones that might otherwise be irrecoverably discarded. However, its shortlisting mechanism prioritizes dates by event frequency, potentially excluding infrequently reported but important milestones, particularly terminal events or long-term outcomes. Furthermore, retaining only one representative per date cluster can discard distinct milestones occurring on the same date. Such information losses cannot be recovered by downstream LLM selection. This creates a fundamental tension between the paper's motivation of preserving critical evidence and its frequency-driven shortlisting mechanism. The current experiments do not sufficiently address this concern.

W3. Insufficient justification of the shortlist multiplier as a methodological design choice

The shortlist multiplier, such as 8L or 12L, is not merely a computational hyperparameter. It directly determines which milestones remain accessible to the downstream selector. Adjusting the multiplier cannot fundamentally eliminate the structural bias introduced by frequency-based filtering. Furthermore, selecting the multiplier based on observed evaluation performance, without a principled selection criterion or an independent tuning protocol, raises concerns about robustness and generalization. The current pool-size ablations mainly compare aggregate performance and do not adequately explain how different multiplier choices affect the preservation of low-frequency but important milestones.

W4. Limited presentation quality (Minor)

The paper contains only one explanatory figure, which is presented as a raster image rather than a vector graphic. There are also visible inconsistencies between the text background and the surrounding figure background. Additional explanatory figures and improved visual consistency would help communicate the framework and its methodological details more clearly.

Questions for authors

Q1. How does the frequency-based shortlisting mechanism ensure the preservation of low-frequency but semantically important milestones, particularly terminal events? Can the authors provide recall analyses stratified by event frequency and temporal position?

Q2. How does the framework handle multiple distinct milestones occurring on the same date? What information is lost by retaining only one representative per date cluster, and how does this affect downstream selection?

Q3. Can the authors demonstrate that multiview union-compress provides a statistically significant advantage over well-designed single-pass selection under comparable model and inference budgets?

Q4. What is the principled justification for choosing the shortlist multiplier? Beyond aggregate SA scores, can the authors demonstrate that this choice is robust across datasets and does not systematically exclude critical low-frequency milestones?

Reviewer_b1iJOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper introduces MILES, a framework for topic-focused timeline summarization from a fixed news corpus. The pipeline extracts both main and additional topic-relevant events from each article, retains one representative per date, and shortlists candidates according to date-cluster frequency. An LLM selects milestones from three complementary perspectives, followed by a consolidation step that produces a timeline of the requested length. The authors report improvements over baseline methods on Entities, Crisis, and T17 using an LLM-based semantic-alignment metric. They further analyze the contributions of extraction, shortlisting, and final selection and assess the evaluation protocol through human studies and cross-judge diagnostics.

Critical strengths
  • The approach is clearly motivated by two limitations: article-level main-event extraction can miss important developments, and frequently reported events are not always the most important. Retaining multiple events per article and using an LLM to select meaningful milestones is a reasonable design that directly addresses these limitations.
  • The proposed method outperforms the evaluated baselines on the primary semantic-alignment F1 metric across Entities, Crisis, and T17, providing consistent empirical support for its effectiveness. The effects of individual components are also analyzed through ablation studies and other experiments.
Critical weaknesses
  • Although the proposed pipeline is well motivated, its substantive novelty relative to existing LLM-based timeline summarization approaches is not sufficiently clear. For example, R2A-TLS, which is cited in the paper, already incorporates LLM-based selection of important events as part of its final timeline-merging process.
  • Table 3 demonstrates the effectiveness of LLM-based selection over mechanical selection rules. However, the proposed Multiview union-compress achieves an SA score of 0.2116, only a small improvement over the single-pass approach at 0.2066, and falls below the tuned single-pass approach at 0.2149. Although the latter comparison requires caution because the tuning conditions differ, the advantage of the proposed method over simpler LLM-based selection approaches is not sufficiently established.
Questions for authors
  • Based on the description around line 173, is it correct that at most one candidate event is retained per date? If so, do the reference timelines contain cases where multiple distinct, important events occur on the same date, and to what extent does this restriction limit the system’s ability to recover those events?
1 comment

Thank you for addressing the weaknesses and questions I raised. However, the responses have not substantially changed my overall assessment of the paper, so I will maintain my original score.

Reviewer_d5EcOpenReview ↗
Rating
3
Confidence
4
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper proposes MILES, a training-free framework for news timeline summarization. It combines topic-aware multi-event extraction, date-cluster shortlisting, and multiview LLM-based milestone selection. Experiments on three benchmarks show consistent improvements over LLM-TLS under semantic-alignment F1.

Critical strengths
  • Well-motivated problem. Preserving important events beyond article-level summaries is meaningful for timeline summarization. The improvements in candidate coverage support this motivation.

  • Solid empirical results. The method consistently outperforms LLM-TLS across three benchmarks. The component-level ablations also help explain where the improvements come from.

Critical weaknesses
  • Limited methodological novelty. The framework mainly combines existing extraction, heuristic ranking, and LLM prompting techniques. In particular, the multiview selector provides only marginal improvements over a simple single-pass approach (0.2116 vs. 0.2066), making its additional complexity difficult to justify.

  • Evaluation concerns. The primary semantic-alignment metric does not directly account for date correctness, and MILES underperforms DATEWISE on some temporal metrics for T17. Also, the use of reference temporal scope in Crisis candidate filtering needs clarification.

Questions for authors
  • How is the temporal scope for Crisis determined? Is it available at inference time without referring to the ground-truth timeline?

  • Is the improvement of multiview selection over single-pass selection statistically significant, considering its additional inference cost?

  • Would the main conclusions remain consistent under alternative semantic judges and evaluation metrics that account for temporal correctness?

1 comment
Area_Chair_Q7HeOpenReview ↗

Reminder: Discussion deadline approaching

Dear Reviewer,

Thank you for your time and effort in reviewing Submission 58355. As the discussion deadline is approaching, could you please read the authors’ response and any additional evidence, reply in the discussion to indicate whether your concerns have been addressed, and update your evaluation accordingly as soon as possible? Thank you for your contribution to the review process.

Best regards, Area Chair

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.