- Rating
- 1
- Confidence
- 3
- Informativeness
- 1
- Presentation
- 2
- Soundness
- 1
Read review
Paper summary
GraphTalk is a training-free pipeline for generating per-slide narration from an existing slide deck and supporting documents. It extracts slide-anchored topics, enriches them with document evidence, and uses DivRank and capacity-constrained matching to allocate topics before drafting and polishing narrations. Experiments on M3AV and SEC EDGAR with two generators report improved coverage and relevance under LLM judges. A small human study supports coverage and document usage but gives lower scores on several other quality dimensions.
Critical strengths
- The task setting is relevant and the pipeline is modular. Existing slide decks do need grounded narration, and exposing topics and evidence could be useful for inspection. However, this limited strength does not compensate for the weak experimental comparison and evaluation problems below.
Critical weaknesses
- Weak comparisons and limited evidence for the planner. Experiments use Pixtral-12B and Gemma-3-27B with only Direct, Summ, and simple RAG baselines, leaving stronger multimodal and structured-generation comparisons untested. Tables 10 and 14 show that most gains come from slide anchoring, while DivRank and ILP contribute little. Missing prompts and retrieval details also prevent checking comparison fairness.
- Unconvincing evaluation. SEC EDGAR relies on LLM judges, while human evaluation covers only six presentations and two team annotators. Table 5 rates GraphTalk substantially below Direct/RAG on relevance, faithfulness, and several other dimensions despite higher coverage. This conflict remains unexplained, and incomplete failure/cost reporting further weakens the conclusions.
Questions for authors
- Clarification: Are prompts, visual inputs, context limits, and polishing matched across methods, and what evidence isolates the planner's benefit beyond slide anchoring?
- Clarification: How do you explain the lower human-rated relevance and faithfulness in Table 5, and how were evaluation subsets and failed judge calls handled?