Understanding Coding Agents for Patch Presence Testing

Patch presence testing determines whether a binary incorporates the security fix for a known vulnerability. Existing systems generally require symbols and cannot operate end to end on stripped binaries, limiting their practical applicability. Recent advances in large language models and coding agents make it possible to use agents directly for patch presence testing. We conduct a systematic empirical study of agent systems for end-to-end patch presence testing. Our findings show that: (1) with symbols available, a coding agent using only binutils tools achieves performance comparable to the state-of-the-art specialized system; (2) stripping and varying the build configurations have limited impact, whereas patch evolution remains challenging; (3) exposing additional binary-analysis interfaces introduces regressions and higher analysis cost, while enriched patch metadata raises performance at lower cost. Finally, we evaluate 12 combinations of four model configurations and three agent implementations, and derive a practical deployment recommendation that balances decision success and cost.