- Rating
- 3(+1)
- Confidence
- 2
- Informativeness
- 1
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This work presents AudienceDeck, a semantic transduction system that converts scientific papers into audience-aware presentation slides while preserving scientific fidelity.
Critical strengths
The experiments are comprehensive, covering a wide range of baseline systems and evaluation metrics.
Critical weaknesses
-
Insufficient motivation. The pipeline design is underjustified and appears closely aligned with Academic50. It is unclear why jointly targeting Experts, Novices, and Adopters is preferable to generating audience-specific presentations, while in practice a presentation is usually adapted for a specific audience. Moreover, the optimization objectives closely match the benchmark metrics, raising concerns about generalizability beyond these settings.
-
Unmatched baseline settings. As acknowledged in Lines 348–349, audience briefs, slide counts, and talk durations are not matched across systems. Comparisons under consistent settings would strengthen the results.
-
Lack of human evaluation. The evaluation relies primarily on automated metrics and LLM judges. Human evaluation of presentation quality and audience comprehension would provide more convincing evidence.
-
Limited ablation studies. The contributions of core components, such as the transduction graph and PrefixEval, are not isolated, making their effectiveness difficult to assess.
-
Missing efficiency analysis. API costs, token consumption, and generation latency should be reported and compared with the baselines.
Questions for authors
None
2 comments
Thanks for providing the additional experimental results and clarifications. I will raise my score to 3.
Author Response: Motivation, Evaluation and Ablations
Thank you for the constructive feedback. We clarify the intended setting and report additional baseline comparisons and component ablations.
Q1. Motivation and generalizability
Audience-specific decks are useful when listeners receive separate presentations. We address a single conference talk or seminar attended by people with different backgrounds and goals. Because everyone follows one sequence within a fixed time budget, the deck must explain prerequisites while retaining technical evidence and practical implications. Expert, Novice, and Adopter capture these complementary information needs, rather than fixed expertise levels.
The contract specifies source constraints and audience priorities, while the graph ties content transformations to their evidence. PrefixEval checks for potentially misleading inferences from the information visible so far. These choices follow from the demands of sequential communication. We also evaluate on ArcBench, whose narrative, visual, and quiz criteria provide a test beyond the Academic50 protocol.
Q2. Comparable baseline settings
The original runs used a 20-minute researcher brief on ArcBench and default settings on Academic50. These settings did not fully align audience conditioning across systems. We therefore added ArcDeck-Joint with the same Expert/Novice/Adopter prompt and a 20-slide, 20-minute target on all 100 ArcBench and 50 Academic50 papers. The table reports mean slide counts and Academic50 duration estimates, following X+Slides:
Estimated duration = 0.25 × slide count + visible word count / 130.
On ArcBench, the joint specification increases ArcDeck's Judge/Quiz scores from 60.9/72.0 to 67.6/82.2, compared with 86.6/83.8 for AudienceDeck. Their actual output lengths still differ, as shown above.
Q3. Human evaluation
Our metrics assess information availability, source support, and question-based recovery as reproducible proxies for conditions supporting comprehension. X+Slides includes human and agent evaluation of probe fidelity, audience-relevant importance, and duplication (Fig. 2, Secs. 4.3, 6.1). ArcDeck/ArcBench includes a human evaluation in which 25 participants ranked slide decks by content quality and narrative coherence (Appendix A.1).
Q4. Transduction graph and PrefixEval
In a separate ablation of updated variants on 28 ArcBench papers, we held the source inventory, audience contract, narrative strategies, feasibility gates, backbone (GPT-5.5), and renderer constant.
O-T replaces explicit source–decision–slide bindings with a structured outline. G-T and G-P use the same candidate graphs: G-T maximizes final utility, while G-P also considers prefix risk.
G-T versus O-T isolates the graph: Judge improves by 1.28 and Quiz by 8.29.
Process evidence supports gate-guided revision. Of 236 initial drafts, 133 were flagged for conclusions preceding necessary conditions; 94 passed all gates after one revision. A blinded model evaluation covered 38 initial–revised pairs from 23 papers. The judge saw only visible text from the first several slides, without version labels or gate decisions. The proportion of judgments indicating conclusions preceding necessary conditions showed a paper-paired reduction of 2.18 percentage points (95% interval [0.94, 3.46]).