- Rating
- 1
- Confidence
- 1
- Informativeness
- 1
- Presentation
- 1
- Soundness
- 1
Read review
Paper summary
This paper presents KS Gov, a coding-agent framework designed for long-running software development tasks. Authors argue that the system combines context management, persistent state, and prompt-based procedures for discovery and adversarial testing. The authors demonstrate the framework through two case studies involving a key-value store and a query engine, along with experiments on Terminal-Bench 2.0
Critical strengths
- Practical motivation. Supporting long-running coding tasks while maintaining progress and correctness is considered as an important problem in LLM agent research.
Critical weaknesses
- Writing clarity. The presentation is almost impossible to follow, particularly the relationships between the system components, proposed procedures, and experimental evaluations.
- Unclear evaluation. The two case studies lack comparable baselines and ablations, making it difficult to attribute the reported improvements to the proposed procedures. Also, the terminal-bench experiments seems to omit the discovery and adversarial-testing procedures, so the results don't seem directly validate the paper's main ideas.
Questions for authors
-
Can the authors provide ablations to isolate the contributions of the proposed procedures and individual system components?
-
Can the authors clarify which findings from the Terminal-Bench experiments support the proposed approach to long-running discovery and adversarial testing?