Evidence Implementation Alignment for Paper-to-Code Generation
Abstract
Faithfully translating scientific papers into complete code repositories remains challenging even when the relevant algorithmic requirements have already been correctly identified during paper analysis. During coding, these requirements may still be omitted, partially realized, or implemented inconsistently. We refer to this mismatch between identified requirements and their code realizations as the Analysis-to-Code Grounding Gap. To address it, we introduce Evidence Implementation Alignment (EIA), which maintains a unit-level correspondence between paper-grounded implementation requirements and their concrete repository realizations. EIA organizes these requirements into Algorithm Units and verifies their paper support before code generation, then traces the same units into the generated repository and diagnoses their alignment to guide targeted repair. We evaluate EIA on PaperBench CodeDev and Paper2CodeBench using DeepSeek-v4-Flash and Qwen2.5-Coder-32B-Instruct-AWQ. On PaperBench CodeDev, EIA improves over PaperCoder from 52.40% to 63.75% and from 35.98% to 43.48%, respectively. On Paper2CodeBench, EIA achieves the highest average score among the evaluated methods under DeepSeek-v4-Flash and remains close to PaperCoder under Qwen2.5-Coder-32B-Instruct-AWQ. Stage-level ablations and unit-level diagnostics further show incremental gains from both stages and lower measured evidence-supported implementation misalignment after targeted repair.
est. 32% chance this paper gets accepted at ICLR 2027.
What do you think this paper will get?
All positions stay anonymous.