- Rating
- 3
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 3
Read review
Paper summary
This paper proposes TRACE, a structured reasoning framework for scientific editing. It represents the editing process using Goal, Local Edit, and Safeguard, allowing the stated edits to be checked against the final outputs. The method uses SFT with structured traces followed by outcome-based GRPO. Experiments on molecular editing and other scientific tasks show consistent improvements over answer-only and free-form reasoning baselines.
Critical strengths
- The paper addresses an important problem: the gap between what a model says and what it actually does.
- The structured edit representation is simple and useful. It also allows automatic verification without reference answers at inference time.
- The experiments are comprehensive, covering multiple tasks, model sizes, and training settings. The ablation and intervention studies provide useful evidence for the role of Local Edit.
Critical weaknesses
- The novelty could be better justified. The framework combines structured supervision, domain-specific verification, and existing training methods, but the distinction from prior structured reasoning approaches is not fully clear.
- TRACE provides additional edit information during SFT compared with the answer-only baseline. It is unclear how much of the improvement comes from the extra supervision and how much comes from the structured reasoning format itself.
- The verification relies on task-specific tools and predefined edit rules. Its applicability to more open-ended scientific reasoning tasks remains uncertain.
Questions for authors
- Can the authors further separate the benefits of additional edit supervision from those of the structured TRACE format?
- How well would TRACE work on tasks where edit operations cannot be easily extracted or verified?
- Does higher say-do consistency always lead to better task performance, especially on more difficult editing tasks?
- How does the additional training and inference cost of TRACE compare with the answer-only baseline?