acceptodds
Under review as a conference paper at ICLR 2027

Beyond Topics: From Weak Signals to Persistent Issues in Evolving Public Discourse

Abstract

Important collective problems may begin as sparse and heterogeneous observations long before they become obvious topics. Existing discourse-analysis methods can identify recurring semantic patterns and track their prevalence over time, but they do not explicitly determine when distributed evidence is sufficient to form a collective issue, or whether subsequent observations reflect the same evolving problem or a genuinely distinct one. We formulate this challenge as Continual Emerging Issue Discovery (CEID), an open-world task that models how collective issues emerge from fragmented evidence, persist as their manifestations change, and differentiate over time. We instantiate CEID with an issue-centric continual GraphRAG that separates Discourse, Evidence, and Issue representations, retains weak signals under insufficient support, and preserves issue identities across temporal updates. We construct a controlled temporal benchmark grounded in real-world public-service feedback and evaluate issue formation, discourse grounding, temporal identity preservation, and emergence latency. Our method improves issue discovery and discourse grounding, substantially strengthens temporal identity preservation, and detects emerging issues earlier than topic-, LLM-, retrieval-, and graph-based baselines. These results suggest that evolving public feedback is better modeled as an evidence-accumulation process over persistent collective issues than as a sequence of independently discovered topics.

Primary area:
learning on graphs and other geometries & topologies
Keywords:
Emerging Issue Discovery, Topic Modeling, Public Discourse, Issue Evolution

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_uzp7OpenReview ↗
Rating
3
Confidence
3
Informativeness
3
Presentation
3
Soundness
2
Read review
Paper summary
  • The study introduces Continual Emerging Issue Discovery (CEID) and an issue-centric GraphRAG framework that identifies emerging collective issues from fragmented, weak signals while preserving their identities and tracking their evolution over time.

  • The study develops a temporal evaluation benchmark grounded in real-world public feedback, demonstrating improved issue discovery, evidence grounding, temporal identity preservation, and earlier detection of emerging issues compared with topic-modeling, LLM-based, and GraphRAG baselines.

Critical strengths

The paper moves beyond conventional topic modeling by introducing evidence-driven, continual issue discovery that captures how collective problems emerge, persist, and evolve. This is significant because it addresses a fundamental limitation of existing approaches: their inability to reliably preserve issue identity across changing discourse.

Critical weaknesses

The framework is evaluated exclusively on a synthetic benchmark rather than real-world longitudinal discourse. This limits confidence in its generalizability, particularly its ability to detect weak signals and track evolving issues in noisy, unpredictable real-world environments.

Additionally, the authors have accidentally disclosed their identities in the following statement on page 6, line numbers 321-322: "The CEID benchmark is grounded in a private corpus of NHS citizen feedback previously used in our earlier work (Shi et al., 2025)."

Questions for authors

Why didn't the authors consider real-world datasets as opposed to using a synthetic benchmark (seeded with real-world data)?

Reviewer_68X7OpenReview ↗
Rating
3
Confidence
1
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper introduces Continual Emerging Issue Discovery (CEID), a task that aims to identify collective issues from fragmented public feedback, maintain issue identities as evidence and manifestations evolve over time, and distinguish newly emerging issues from existing but related ones. The authors propose an issue-centric continual GraphRAG framework that represents discourse, evidence, issues, and communities separately. Incoming feedback is decomposed into evidence units, associated with historical evidence and issues, and processed through creation, addition, or differentiation decisions. The framework is evaluated on a controlled temporal benchmark constructed from NHS citizen feedback. Compared with topic-modeling, LLM-based, retrieval-based, and graph-based baselines, the method improves issue discovery, discourse grounding, temporal identity preservation, and emergence detection delay. However, the evaluation relies primarily on synthetic temporal data, and the results provide less convincing evidence for accurate issue-state prediction and generalization to real-world feedback streams.

Critical strengths
  1. A well-motivated task formulation. The paper clearly argues that topic prevalence differs from the formation and persistence of collective issues, and this distinction is practically relevant, for example for monitoring public-service feedback. Treating weak signals as deferred candidates rather than noise, and separating identity-preserving updates from issue differentiation, is a sensible conceptual framing.

  2. Persistent state shows a clear benefit within the benchmark. On the authors' benchmark, maintaining the graph across waves substantially improves temporal identity preservation (TIP 0.50 vs. at most 0.21) and reduces emergence delay compared with independently rebuilt graphs. This suggests the persistent design is useful, subject to the validation concerns below.

Critical weaknesses
  1. There is no independent ground truth, so the method's ability to discover real issues is not validated.

    • The "ground-truth issue bank" is itself produced by an LLM that extracts, merges, and filters issues from a private corpus. No human annotation, expert review, or inter-annotator agreement is described.
    • The test discourse is then generated by the same LLM from this bank, and evaluation matching is also LLM-verified.
    • The benchmark therefore measures how well the method recovers an LLM-generated abstraction of the data, not whether it identifies real collective problems.
    • No result is reported on the original feedback, and there is no comparison against externally documented issues.

    This undermines the central claim of earlier emergence detection, which cannot be meaningfully assessed without real temporal ground truth. Using the same LLM throughout also introduces a risk of circularity.

  2. The experiments are insufficient to support the claims.

    • Each result comes from a single run, with no variance or significance testing, and several absolute scores are low (e.g., ARI 0.08).
    • The method has the lowest State F1 of all methods (0.23, below LDA at 0.24), yet the paper does not acknowledge this.
    • There is no component-wise ablation of the core mechanisms (candidate pool, Differentiate, historical association).
    • Key elements are undefined: the criterion for "sufficient collective support", the formulas for TIP and EDD, and basic benchmark statistics.
    • All retrieval uses hashing-based bag-of-words vectors, which may handicap embedding-based baselines.
    • Directly relevant baselines discussed in related work, such as dynamic topic models and event detection methods, are not compared.
Questions for authors
  1. How is "sufficient collective support" determined for issue formation? Is it a fixed threshold, or is it decided entirely by the LLM?
  2. Please provide formal definitions of TIP and EDD, and report basic benchmark statistics (number of ground-truth issues, discourse items, and waves).
  3. The proposed method has the lowest State F1 of all methods (0.23 vs. 0.24–0.29 for the baselines). What causes this, given that lifecycle tracking is part of the CEID task?
  4. Was the ground-truth issue bank validated by human experts? If so, please report the annotation protocol and inter-annotator agreement. Can the method be evaluated against independently documented issues, for example service problems recorded in official reports for the same period?
  5. The same LLM is used for benchmark generation, the method, and evaluation matching. Can you repeat the experiments with a different LLM for generation and evaluation to rule out circularity?
  6. Can you provide component-wise ablations (candidate pool, Differentiate, historical association), results over multiple runs with variance, and comparisons against dynamic topic models and event-detection baselines using standard sentence embeddings?
Reviewer_Xvu3OpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper introduces Continual Emerging Issue Discovery to identify issues emerging from public discourse over time rather than just identifying recurring topics. The authors propose a GraphRAG framework that collects evidence from feedback, maintains historical information and decides whether to create a new issue or update an existing one. They evaluate on a synthetic dataset generated from NHS public feedback and show improvements over topic modeling, LLM and RAG baselines.

Critical strengths
  1. Interesting problem formulation - I like the idea of tracking how issues emerge over time from weak signals instead of just identifying common topics. This seems useful for real-world public feedback analysis.
  2. Persistent graph representation - Maintaining issue identities and historical evidence across different time periods is a useful approach and can be used in downstream applications.
Critical weaknesses
  1. Evaluation is entirely synthetic- This is my biggest concern. Although the data is based on real NHS feedback, all experiments use LLM-generated data. The same LLM is also used in the proposed framework. It is unclear how well the results would generalize to real public discourse, which can be much noisier and less structured.
  2. Limited novelty and weak experimental validation- Most of the proposed framework combines existing techniques like LLM-based extraction, retrieval, and graph updates. I don't see a clear new algorithmic contribution. There are also no proper component-wise ablations to understand what actually contributes to the improvements.
Questions for authors
  1. Why are the experiments limited to synthetic data? Is there any evaluation or independent human validation on real-world feedback?
  2. Can the authors provide ablations to show how much each component contributes to the overall method?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.