- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper presents SciSlide, a system that turns scientific papers into presentation slides. It first builds an argument graph from the paper, then plans slides around individual claims and their supporting evidence, and finally generates the layout with a fine-tuned language model. It also introduces a dataset of 7,500 paper–slide pairs and 16 automatic evaluation metrics. The results are encouraging, including the human evaluation. My main concern is that the paper makes stronger claims about verification than its checks support. I also have questions about the evaluation setup and how much each component contributes to the final improvement.
Critical strengths
- The problem is useful, and the overall approach makes sense. Separating argument extraction, slide planning, and layout generation gives the system a clear structure.
- The evaluation goes beyond visual appearance. I appreciate the attempt to measure content coverage, slide order, and figure placement. The paper also discusses cases where a metric gives a good score to a poor presentation. This is useful.
- Several experimental checks strengthen the results. The paper includes component ablations, a Gemini baseline explicitly prompted with assertion–evidence rules, and an additional evaluation using a graph extracted by GPT-4o. The human study also provides evidence beyond the automatic metrics.
Critical weaknesses
- The verification checks do not establish that the scientific claims are correct. This is my main concern. In Section 3.3, a node passes the grounding check if its source span appears in the paper. That establishes where the text came from, but it does not establish that the assigned role, inferred relation, or rewritten slide headline is correct. Similarly, placing a figure on the slide associated with its assigned node does not establish that the figure supports the headline. For example, a headline could turn a result observed under one experimental setting into a general claim, while still pointing to a valid source passage. A figure could also be attached to the wrong claim during graph construction and then pass the later placement check. I would like the authors to narrow this statement and directly check a sample of graph relations, rewritten headlines, and figure–claim links against the source papers. The agreement with hand-computed metric values in Appendix B checks the implementation of the rules; it does not resolve this question.
- The argument metrics still make it difficult to separate presentations from closer agreement with the system’s representation. SciSlide plans its content using an argument graph, while NC@budget and CER evaluate outputs against that graph. The GPT-4o graph experiment is a useful check, and I appreciate its inclusion. However, it uses the same extraction procedure and still measures node realization through verbatim eight-word matches. A correct paraphrase can therefore miss credit, while copied text can receive credit without explaining the argument clearly. I also do not see a common slide-count or total content budget across systems. Limiting each slide to five content items does not control how many slides a system produces or how long each item is. This matters when comparing coverage.
- The human results are promising, but the reported preference rate needs more context. Section 5.3 reports an average score of 3.8 for SciSlide and 3.5 for PPTAgent. The headline result of 65% preference combines comparisons against all five baselines. I would like to see the preference rate against each baseline, especially PPTAgent. The pairwise preferences are derived from five-point ratings rather than collected as direct pairwise choices. Please explain how the two raters’scores are combined and how ties are handled. There is also a dependency issue to clarify: the 150 pairs come from only 30 source papers, and each SciSlide deck is reused in five comparisons. Please explain whether the confidence interval accounts for this grouping.
- The ablations do not yet show which components improve the final presentation quality. Appendix E shows that removing components worsens their corresponding metrics. These results are useful, but they mainly show that the components affect the properties they were designed to target. For example, removing assertion labels lowers the assertion-headline score, while removing graph conditioning lowers graph-based coverage. I would like to see a small human comparison of the full system against versions without graph conditioning and without correction.The learned layout component also needs a clearer connection to presentation quality. Matching the coordinates of one reference layout is a reasonable prediction task, but several different layouts may work equally well. It remains unclear how much the exact-match improvement translates into better readability or visual organization.
- Please reconcile the BLR results. Table 1 reports a BLR of 0.011 for SciSlide, while the full-system value in the A3 ablation is 0.09. If these refer to different configurations, that needs to be stated.
- The explanation for the geometry-loss result is too strong. The reference layout can minimize both the coordinate prediction loss and the geometry penalty. Having additional valid layouts does not itself establish that the objectives conflict.
Questions for authors
NA