Do What You Say: Towards Verifiable Reasoning for Scientific Editing
Abstract
Language models increasingly produce plans or rationales before their final outputs to guide generation and support supervision. When these traces are used for auditing or process supervision, a mismatch between their stated actions and the final output creates a say–do gap. Scientific editing provides a concrete testbed for this problem because domain tools can check both the requested outcome and the realized input–output transformation. Yet free-form traces make such comparison difficult because the claimed edit must first be interpreted from prose. We introduce TRACE, a structured reasoning-supervision framework that makes the declared edit explicit and directly checkable. TRACE represents each edit as a typed, replayable Goal–Local Edit–Safeguard contract constructed from task-computable facts. Through supervised fine-tuning, models learn to generate this contract before answering, while outcome-based reinforcement learning further improves task performance. Across three molecular editing benchmarks, TRACE consistently outperforms matched answer-only and free-form reasoning baselines, with gains that persist across model scales, backbones, and RL optimizers. On TOMG with Qwen3-1.7B, TRACE improves the average success–similarity score by over matched answer-only training. TRACE also makes agreement between stated and realized edits directly auditable. Controlled interventions provide evidence that the input-specific Local Edit carries most of the action-relevant signal, while the full contract performs best. The resulting verifier further improves say–do consistency through process supervision and enables reference-answer-free Best-of- selection that improves both editing performance and trace–answer consistency. Beyond molecular editing, TRACE extends to RNA codon optimization and reaction repair, demonstrating broader applicability to scientific editing.
est. 32% chance this paper gets accepted at ICLR 2027.
What do you think this paper will get?
All positions stay anonymous.