When the Verifier Is Silent: Process Advantages for Agentic Coding
Abstract
Reinforcement learning for agentic software engineering benefits from strict executable verifiers, but their binary feedback is sparse. In group-relative policy optimization, a task provides no outcome-based learning signal when all sampled rollouts receive the same verifier outcome, despite substantial differences in their process quality. Adding dense process rewards creates a structural tension: small coefficients may vanish under outcome normalization, whereas large coefficients can overturn the verifier-induced success or failure ordering. We introduce KAT, a verifier-preserving process-supervision method that makes process signals subordinate by construction. KAT partitions rollouts by executable outcome, standardizes process signals within each outcome stratum, and combines bounded process advantages with margin-scaled outcome advantages. This guarantees that no judge output or nonnegative process weight can rank an unverified trajectory above a verified one, while outcome-homogeneous groups still provide learning signals whenever process quality varies. KAT derives process signals from a frozen software-engineering lifecycle rubric covering reproduction, root-cause diagnosis, post-fix and regression validation, and tool use. An LLM judge performs evidence-linked structured matching, while deterministic components validate evidence and aggregate scores using reliability-based weights. Starting from Qwen3.6-35B-A3B, KAT improves pass@1 from 64.40% to 69.40% on SWE-bench Verified, 57.00% to 63.00% on SWE-bench Multilingual, 40.63% to 45.96% on SWE-bench Pro, and 32.02% to 41.02% on Terminal-Bench 2.1. It also transfers to SciCode, improving pass@1 from 37.53% to 44.20%. Controlled experiments show that KAT provides usable supervision and outperforms outcome-only and additive-reward alternatives. Models will be publicly available.
est. 32% chance this paper gets accepted at ICLR 2027.
What do you think this paper will get?
All positions stay anonymous.