- Rating
- 1
- Confidence
- 4
- Informativeness
- 1
- Presentation
- 1
- Soundness
- 1
Read review
Paper summary
this paper focuses on the evaluation of the paper-to-poster from a logical path coverage perspective.
the experiments on 134 NeurIPS papers, and the findings suggest that logical connections should be evaluated separately from factual recovery.
Critical strengths
the logic bench treat each paper as a logic graph, and frame the poster evaluation as the assessment of logical path coverage. despite this might be novel but it still incremental and it cannot support a ICLR submission.
Critical weaknesses
-
the contribution of this paper is incremental. This paper do not introduce new poster generation method and also the idea of logicquiz also no significant new compare with the past evaluation such as paperquiz. the assumption should be inspected carefully, as human posters usually have subjectives, such as visual appeal for attention, rather than objective information presentation. Thus, the logical path might be just part of a rubric rather than a comprehensive framework.
-
bad evaluation: select 134 paper-poster pairs randomly, this ignore the diversity in terms of paper topics, and only source form nips 25. Which might introduce huge bias. The paper do not disclose the human study details, such as how many people, distributions. There is no ablation about the choices the paper readers and poster readers.
Questions for authors
please refer to the question is weaknesses.
the writing and presentation of this paper look significantly rely on AI. such as the fig2 and fig3 size makes no sense, it almost occupy the half of the paper... also come across the full paper, i do not find any examples, and how different examples perform scores.
5 comments
Response to Revewier M71Y
3 AI Overuse
This paper is not AI-generated. AI was only used as an assistive tool for manuscript drafting and editing. Below are our responses to the specific concerns:
-
Large figure size: We deliberately highlight Figures 2 and 3 as they present the core empirical discoveries of the paper. In fact, the current frontier LLMs prefer generating diagrams with small size. Furthermore, we created our figures in PowerPoint and Python, and we can provide the source files as proof.
-
No examples: We have included detailed examples in Appendix C. Each one displays the multi-hop reasoning path alongside its verified question and targeted distractors.
Follow up for Reviewer M71Y
Dear Reviewer M71Y,
We appreciate your candid feedback and have carefully considered your concerns about novelty, evaluation diversity, and presentation.
We respectfully offer the following clarifications and revisions:
-
Novelty (Response 1): LogicBench identifies a blind spot across the field—generators optimize for aesthetics while breaking scientific reasoning. PaperQuiz scores of 90.8%– 95.6% contrast sharply with LogicBench scores of 24.4%–61.8%, demonstrating that existing benchmarks miss this dimension entirely. This is a benchmark contribution, not a generation method, and we believe it provides actionable guidance for future work.
-
Data diversity (Response 2.1): Papers span five subfields (Deep Architectures: 34, Optimization: 31, Language/Multimodal: 28, CV: 22, RL: 19). Cross-venue validation (ICML, ACL, CVPR) shows consistent trends.
-
Human study (Response 2.2): We have added three human studies—expert audit (93.8% pass rate), reader perception (0.3% human-VLM gap), and filter validation.
-
Presentation (Response 3): Figures 2 and 3 are intentionally large to highlight core findings. Detailed examples are in Appendix C.
We understand your concerns about incremental contribution. However, we believe exposing a systematic failure mode across the field is a meaningful contribution to the ICLR community.
If you find the revisions address your concerns, we would appreciate your reconsideration.
Respectfully,
The Authors
Response to Revewier M71Y
1 Novelty and the Purpose of Poster Evaluation
We respectfully disagree that the paper’s contribution is incremental. The novelty of our work lies in identifying and formalizing a critical blind spot across the entire paper-to-poster field: current generative models optimize heavily for visual appearance while systematically breaking the scientific reasoning of the paper.
This conceptual novelty was explicitly recognized by all three other reviewers:
Below, we address the reviewer’s specific concerns with concrete data from the paper:
Scientific posters must be "Content and Logic First: The reviewer argues that posters are subjective tools aimed at "visual appeal for attention, rather than objective information presentation." We agree that visual appeal is necessary to draw readers in, but a scientific poster is ultimately an academic communication medium. A poster that catches the eye but drops the core reasoning steps behind a paper’s claims fails at basic scientific communication.
Our contribution is a new benchmark, not a new generation method: Our paper is in Benchmarks track, introducing a benchmark that exposes where existing generation methods fail is an essential contribution. And LogicBench addresses this by providing guidance for future models toward preserving scientific logic integrity alongside visual design.
LogicBench is fundamentally different from PaperQuiz: PaperQuiz and P2PEval only test surface-level facts (e.g., retrieving an individual number, dataset name). On PaperQuiz, the poster generation problem looks almost solved: generation methods achieve 90.8%–95.6% on Understanding and 75.4% on Detail. However, on LogicBench, these methods drop to 24.4%–61.8% because the generated posters fail to present the logic (e.g., tracing how theoretical assumptions justify a final regret bound) in the original paper.
2 Insufficient Evaluation
2.1 Data Bias
Topic bias: Our 134 randomly sampled papers represent five distinct areas:
Conference bias: To evaluate conference bias, we conducted evaluations on 30 additional papers with 300 questions across three venues. The results show similar trends across all conferences.
Sample scale bias: Our dataset size of 134 papers and 1,354 questions exceeds prior poster evaluation sets, including PaperQuiz with 100 papers and P2PEval with 121 papers.
We have added a discussion on this in Appendix B of the revision.
2.2 Clarification on "Human Reference"
Our benchmark includes no humen study. As stated in Section 4.1, "Human Reference" designates the official conference posters created by the original paper authors. Evaluated under the identical scoring pipeline, these author-designed posters establish an objective ceiling, confirming that human posters preserve full reasoning chains. Furthermore, to thoroughly validate human perception and benchmark quality, we have incorporated extensive human evaluations into the revised paper: domain experts confirmed a 93.8% pass rate across audited logic graphs and questions (Fleiss’ =0.85; see 96em-3.1), a direct human perception study closely matched the VLM reader with an average gap of only +0.3% (53.4% vs. 53.1%; see 96em-3.2), and expert reviews verified the soundness of our question filtering criteria (see 96em-1.2).