- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
EvoWriter learns reusable writing workflows and skills by comparing generated manuscripts with expert papers and validating proposed updates on separate tasks. Experiments report improvements over writing baselines, transfer across writer models, and favorable human preferences on 45 held-out papers.
Critical strengths
Separating update proposal from validation provides a useful framework for learning writing procedures across tasks. Cross-model transfer and human preference over PaperOrchestra support the practical value of the final system.
Critical weaknesses
-
The experiments do not isolate the benefit of expert-paper comparisons from iterative optimization. Reference-augmented baselines use frozen guidance, while Figure 4 compares evolving alternatives only on training papers. The held-out evaluation compares complete systems with different workflows.
-
The benchmark may pre-solve part of the argument construction that motivates the method. Dossiers extracted from finished papers already specify motivation, main findings, and author interpretations, allowing writers to inherit decisions about which results matter and how they support the paper’s argument.
-
Support weights atomic claims equally, regardless of their role in the argument. A hypothetical manuscript containing 99 supported claims and one unsupported central conclusion would score 99%, even if that conclusion determines the paper’s main scientific message.
Questions for authors
Are evaluators and update budgets matched across Figure 4 conditions, and do the improvements persist on its held-out split?
How much of the original papers’ interpretation and argument structure remains in the input dossiers?
Can an expert audit establish whether generated central conclusions remain within the supplied evidence?