- Rating
- 3(+1)
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
VASP is a framework for generating scientific posters with two components: SemParser and VisComposer. SemParser combines language-model judgments about document semantics with deterministic PDF geometry to extract complete figures. VisComposer organizes content around visual evidence using explicit claim–evidence associations, geometric checks, and iterative refinement. The evaluation compares VASP with six existing methods on 100 papers and examines figure extraction, visual presentation, information coverage, and runtime costs.
Critical strengths
- The design addresses practical problems, and the figure extraction results are well supported.
- The experiments demonstrate a useful trade-off between quality and cost.
Critical weaknesses
- The ablations do not fully isolate the proposed design choices, and the gains in scientific content organization need a more careful interpretation. In Table 6, replacing visually anchored composition with a fixed template changes both visual prioritization and layout flexibility. The drop from 91.4 to 80.8 therefore cannot be attributed specifically to the visual-first strategy. A stronger control would retain the same assets, flexible layout, and refinement budget while changing only the planning priority. Removing semantic verification reduces the overall score from 91.4 to 81.1, but Assoc. remains at 100, with most of the decline coming from Space and Density. This supports the usefulness of programmatic layout feedback, but does not isolate the contribution of claim–evidence checks from ordinary geometric checks. Since the ablation covers only 20 papers, possible saturation of the association metric also deserves discussion. Content gains under the same backbone are modest. PaperQuiz improves from the strongest baseline’s 69.25 to 69.96. Overall coverage reaches 82.0 versus PosterGen’s 79.7, but Results coverage is lower, at 71.7 versus 74.7, as is Full coverage, at 20% versus 22%. These results support some average improvement, while broader claims about scientific communication need qualification. The stronger content scores with Opus-5 should also be interpreted alongside baselines using that backbone.
- The new evaluation has a specific scoring concern and lacks independent calibration of poster quality. Appendix D.2 assigns N/A to EN05 when a poster has no substantive branding or decoration. However, the formula in D.3 retains a fixed denominator and awards credit only for PASS. Under this definition, a poster without these elements loses approximately 6.67 overall points. This may penalize a reasonable design choice. The authors should report N/A frequencies by method and examine whether rankings and margins remain stable when genuinely inapplicable criteria are excluded. The new evaluations rely primarily on GPT-5.6-Sol. The manual crop checks do not calibrate judgments of poster quality or information coverage. Extracting seven reference units from the author poster, auditing them against the paper, and freezing them before evaluation is a useful procedure. Still, this measures retention of selected information. Independent human assessment on a subset would help establish whether the visual scores and coverage judgments agree with readers’ assessments of readability and content.
Questions for authors
- Can visual prioritization, claim–evidence checks, and ordinary geometric checks be isolated while keeping assets, layout flexibility, and refinement budgets fixed?
- How frequently is N/A assigned to EN05 and other criteria for each method? Are the main rankings and margins stable under appropriate N/A handling?
- Can the authors provide human assessment on a subset, or agreement results using an independent evaluation model?
- Can the smaller PaperQuiz and coverage gains be accompanied by confidence intervals paired by paper? If the Opus-5 results are central to the claims, can a representative baseline also be evaluated with that backbone?
9 comments
Response to Reviewer qPYK (continued)
W.1.2 Interpreting content gains and qualifying claims on scientific communication
Trade-off on Table 4. On a fixed single-page canvas, packing more text trades off directly against visual legibility. Under GPT-5.6-Sol, PosterGen scores slightly higher on Results coverage (74.7% vs. 71.7%) and Full coverage (22% vs. 20%), but its Visual Presentation drops to 65.6% compared with 94.3% for VASP (Table 3). Conversely, ResearchStudio keeps Visual Presentation high (88.1%) by omitting content, dropping Overall Coverage to 73.0% compared with 82.0% for VASP. VASP is the only fixed-canvas method that remains strong on both visual presentation and content coverage.
Modest margins on PaperQuiz. A single-page poster has limited space and is meant to help readers grasp the core work quickly, rather than retain every detail of the paper. PaperQuiz instead samples questions across the entire paper. As a result, even author-designed official posters score only 67.55% on PaperQuiz (despite reaching 96.4% on Table 4), below text-heavy Paper2Poster (69.25%) and unbounded-height P2P (85.20%). Paper-paired confidence intervals for PaperQuiz and Table 4 are provided under Q.4.
Qualifying claims on scientific communication. We agree that our benchmarks do not directly test human reader comprehension. Instead, they measure visual legibility, figure preservation, claim-evidence layout, and retention of key takeaways from the author poster. We will update the manuscript as follows:
- Abstract and Introduction. Scope "communicating central claims" to visual legibility, claim-evidence layout, and key-information retention.
- Results (§ 5, Table 4). Explicitly note PosterGen's higher Results (74.7% vs. 71.7%) and Full coverage (22% vs. 20%) alongside its 28.7-point drop in Visual Presentation.
- Limitations (Appendix). Note that direct reader-comprehension user studies are not evaluated.
W.1.3 Baselines under Claude Opus-5 and Qwen3.8-Flash-Next
To interpret the Opus-5 scores alongside baselines using the same backbone, we evaluated VASP with ResearchStudio (strongest visual baseline), PosterGen (strongest coverage baseline), and Paper2Poster across all 100 papers under both Claude Opus-5 and Qwen3.8-Flash-Next, keeping the evaluation protocol unchanged:
\hline \text{Method} & \text{VLM-as-Judge} & \text{PaperQuiz} & \text{Visual Presentation} & \text{Key-Information} \\\\ & \text{(Table 1)}\uparrow & \text{(\\%)}\uparrow & \text{(\\%)}\uparrow & \text{Coverage (\\%)}\uparrow \\\\ \hline \text{Claude Opus-5 backbone} & & & & \\\\ \mathbf{VASP} & \mathbf{3.70} & \mathbf{78.08} & \mathbf{92.5} & \mathbf{88.7} \\\\ \text{ResearchStudio} & \underline{3.58} & 71.77 & \underline{88.7} & 77.9 \\\\ \text{PosterGen} & 3.00 & \underline{76.05} & 65.9 & \underline{86.6} \\\\ \text{Paper2Poster} & 2.76 & 74.55 & 44.0 & 78.0 \\\\ \hline \text{Qwen3.8-Flash-Next backbone} & & & & \\\\ \mathbf{VASP} & \mathbf{3.58} & \mathbf{81.35} & \mathbf{89.2} & \mathbf{89.3} \\\\ \text{ResearchStudio} & \underline{3.54} & \underline{73.68} & \mathbf{89.2} & 80.4 \\\\ \text{PosterGen} & 3.04 & 73.28 & 65.7 & \underline{82.6} \\\\ \text{Paper2Poster} & 2.42 & 73.16 & 46.7 & 74.7 \\\\ \hline \text{Human reference} & & & & \\\\ \text{Official posters} & 3.12 & 67.55 & 83.0 & 96.4 \\\\ \hline \end{array}$$ When baselines use Claude Opus-5 as well, VASP leads on all four benchmarks at once: it exceeds PosterGen on both PaperQuiz (78.08% vs. 76.05%) and Coverage (88.7% vs. 86.6%) with a 26.6-point margin on Visual Presentation (92.5% vs. 65.9%), and exceeds ResearchStudio by 6.31 points on PaperQuiz and 10.8 points on Coverage. The same ordering holds under Qwen3.8-Flash-Next.Response to Reviewer qPYK
W.2 Scoring of N/A on EN05 under a fixed denominator, and lack of independent calibration of poster quality.
W.2.1 N/A frequency by method and ranking stability excluding N/A
Across all 900 posters (9 methods × 100 papers × 12 criteria = 10,800 judgments), 7 of the 12 criteria have zero N/A verdicts on every poster. Almost all N/A verdicts occur on EN05 (subordinate branding/decoration in Visual Guidance, triggered on minimalist posters without decorative elements), with the remaining 0%–0.8% occurring on the three Figure Presentation criteria when a poster displays zero figures (plus 2 official posters without a conclusion block on EN07). Under Excl. N/A, for each poster p and group g, we replace the fixed group size in Eq. 6 (Appendix D.3) with the number of applicable (non-N/A) criteria on that poster displays no figures at all), and average the three groups equally:
\hline & \text{N/A Rate (\\%)} & & & \text{Visual Guidance} & & & \text{Overall} & \\\\ \hline \text{Method} & \text{EN05} & \text{Total} & \text{Figure Pres.}\uparrow & \text{Paper}\uparrow & \text{Excl. N/A}\uparrow & \text{Layout}\uparrow & \text{Paper}\uparrow & \text{Excl. N/A}\uparrow \\\\ \hline \text{Ours} & & & & & & & & \\\\ \mathbf{VASP}\\ \text{(Opus-5)} & 5.0 & 0.7 & \underline{81.3} & \underline{98.4} & \underline{99.4} & \mathbf{97.8} & \underline{92.5} & \underline{92.8} \\\\ \mathbf{VASP}\\ \text{(GPT-5.6-Sol)} & 0.0 & 0.0 & \mathbf{88.0} & \mathbf{100.0} & \mathbf{100.0} & \underline{95.0} & \mathbf{94.3} & \mathbf{94.3} \\\\ \hline \text{Baseline methods} & & & & & & & & \\\\ \text{Paper2Poster} & 84.0 & 7.8 & 35.0 & 61.6 & 73.9 & 43.0 & 46.5 & 50.6 \\\\ \text{PosterMELD} & 40.0 & 3.8 & 71.7 & 80.6 & 87.6 & 46.5 & 66.3 & 68.6 \\\\ \text{PosterForest} & 91.0 & 8.3 & 38.0 & 56.4 & 68.8 & 42.0 & 45.5 & 49.6 \\\\ \text{PosterGen} & 28.0 & 2.8 & 53.0 & 75.4 & 79.7 & 68.3 & 65.6 & 67.0 \\\\ \text{P2P} & 2.0 & 0.9 & 11.0 & 76.4 & 76.7 & 61.3 & 49.6 & 49.6 \\\\ \text{ResearchStudio} & 0.0 & 0.0 & 73.7 & 97.0 & 97.0 & 93.5 & 88.1 & 88.1 \\\\ \hline \text{Human reference} & & & & & & & & \\\\ \text{Official posters} & 3.0 & 0.8 & 72.7 & 86.4 & 87.4 & 90.0 & 83.0 & 83.5 \\\\ \hline \end{array}$$ Excluding *N/A* raises minimalist baselines such as Paper2Poster ($+4.1$) and PosterForest ($+4.1$) as expected, while leaving the top-6 ranking unchanged and VASP ahead of ResearchStudio (94.3 and 92.8 vs. 88.1) and PosterMELD/PosterGen ($>24$ points). Dropping *EN05* altogether gives virtually identical Overall scores (94.3, 92.9, 88.3, 82.6, 68.3, 65.9). We will adopt the *Excl. N/A* denominator in Eq. 6 in the revision.
Response to Reviewer qPYK
We sincerely thank Reviewer qPYK for the detailed and technically demanding review. We are glad the reviewer finds that our design addresses practical problems with well-supported figure extraction results, and that our experiments demonstrate a useful trade-off between quality and cost. We address every weakness and question below.
W.1 The ablations do not fully isolate the proposed design choices, and the gains in scientific content organization need a more careful interpretation.
W.1.1 Isolating visual-first priority and claim-evidence vs. geometric checks
In Table 6, replacing visually anchored composition with a fixed template changed both planning priority and layout flexibility. On the 20-paper subset, we built the exact control the reviewer suggests: a text-first variant that keeps the same extracted assets, template-free flexible CSS grid, and two-round refinement budget as VASP (full), changing only the planning priority so section prose is drafted and budgeted first and figures are placed into the remaining space.
For verification in Table 6, two factors obscured the separate roles of semantic and geometric checks. First, Assoc. (AL05) is a layout-proximity check, not a semantic one: on the rendered poster, it tests whether labels such as headings, captions, and legends sit visually next to the elements they belong to, rather than floating apart. Removing SV changes what a section's text says, not where headings or captions are placed, so Assoc. stays at 95%–100% even when the body text never mentions the figure next to it. To directly measure text-to-figure semantic alignment, we evaluate Claim-Evidence Grounding (CEG), a blind VLM check on the rendered poster testing whether each displayed figure or table is substantively explained by its section's body text. Second, in the submitted Table 6, the w/o semantic verification row disabled the revision controller—turning off both iterative SV and ordinary geometric/whitespace verification (GV), while retaining initial (round-0) SV planning. Below we vary Composition Priority, Initial SV, Iterative SV, and GV separately on the 20-paper subset:
\hline & & \text{Figure (\\%)} & & \text{Layout (\\%)} & & & \text{Area (\\%)} & \text{Semantics (\\%)} & \\\\ \hline \text{Variant} & \text{Overall}\uparrow & \text{Fit}\uparrow & \text{Text}\uparrow & \text{Assoc.}\uparrow & \text{Space}\uparrow & \text{Density}\uparrow & \text{Largest}\uparrow & \text{CEG}\uparrow & \text{Coverage}\uparrow \\\\ \hline \text{Composition priority (with full SV and GV enabled)} & & & & & & & & & \\\\ \mathbf{VASP}\\ \text{(full: visual-first)} & \mathbf{91.4} & \mathbf{80} & \mathbf{65} & \mathbf{100} & \underline{80} & \mathbf{90} & \underline{7.3} & 82.3 & 75.7 \\\\ \text{w/o anchored comp. (template)} & 80.8 & 30 & 30 & \mathbf{100} & \mathbf{95} & 65 & 4.5 & 75.8 & \underline{81.4} \\\\ \text{w/o anchored comp. (text-first)} & 72.8 & 25 & 15 & \mathbf{100} & 75 & 30 & 2.7 & \underline{87.0} & \mathbf{89.3} \\\\ \hline \text{Verification decoupling (with visual-first composition fixed)} & & & & & & & & & \\\\ \text{w/o iterative SV (init. SV + GV)} & \underline{90.6} & \mathbf{80} & \mathbf{65} & \mathbf{100} & 75 & \underline{85} & 6.9 & 78.2 & 77.1 \\\\ \text{w/o all SV (GV only)} & 89.0 & 60 & \underline{60} & \mathbf{100} & \mathbf{95} & \mathbf{90} & \mathbf{8.9} & 25.0 & 67.9 \\\\ \text{w/o GV (init. + iter. SV)} & 80.1 & 65 & 50 & 95 & 15 & 65 & 7.0 & \mathbf{88.1} & 80.0 \\\\ \text{w/o iter. SV \\& GV (Table 6)}^{\dagger} & 81.1 & \underline{70} & \underline{60} & \mathbf{100} & 10 & 65 & 7.0 & 72.1 & 77.1 \\\\ \hline \end{array}$$ *$^{\dagger}$Submitted Table 6 row, which retained initial SV planning while disabling iterative SV and GV during revisions.* Every factor separates cleanly. **Visual-first priority governs figure scale and layout balance**: with assets, flexible layout, and refinement budget fixed, switching from visual-first to text-first drops Overall from 91.4 to 72.8, as unconstrained prose crowds figures into leftover space (*Largest* figure area 7.3% → 2.7%, *Fit* 80% → 25%, in-figure *Text* 65% → 15%, *Density* 90% → 30%). **GV governs whitespace**: disabling GV drops *Space* to 15% and 10%, whereas keeping GV holds *Space* at 75%–95%, explaining the 80% to 10% drop in Table 6. **Initial SV establishes claim-evidence grounding**: with GV fixed, removing initial SV as well collapses CEG from 78.2% to 25.0% and Coverage from 77.1% to 67.9%. **Iterative SV preserves grounding across revisions**: with GV enabled, removing iterative SV lets geometry-only edits trim figure-explaining prose (82.3% to 78.2% CEG), whereas without GV, adding iterative SV lifts CEG from 72.1% to 88.1%. We will expand these decoupled ablations to all 100 papers in the revision.