Beyond Chain Selection: Compiling Proposal Archives into Frozen Agent Harnesses
Abstract
Agent harness evolution couples two distinct operations: proposing useful edits and selecting edits that generalize. We study whether useful proposal content can be recovered even when chain-level selection fails. We introduce cross-chain proposal compilation, which constructs a single frozen harness from logged edits with no additional model calls at compile time. We instantiate this idea in two forms: prose compilation assembles proposal text into instructions, and typed compilation selects recurrent, executable mechanisms with development-ranked implementations. On a primary pool of 1,007 BFCL v4 tasks, union prose, recurrent prose, and typed consensus improve paired held-out accuracy by +2.29, +2.02, and +1.65 percentage points. All three paired intervals exclude zero. A fresh five-arm follow-up on 400 additional tasks finds smaller effects whose intervals include zero, indicating limited category transfer. In a separate held-out comparison, single-chain finalists underperform the baseline despite positive development gains. These results support proposal-level reuse beyond chain-level selection, with bounded evidence for transfer and inference cost isolated as a separate outcome.
est. 32% chance this paper gets accepted at ICLR 2027.
What do you think this paper will get?
All positions stay anonymous.