- Rating
- 1
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 1
- Soundness
- 1
Read review
Paper summary
This paper studies an interesting problem, when an agent completes a task, is that success the correct metric? did the agent check the relevant evidence and execute accordingly, or does it just happen that action is the correct one?
Authors took some agentic trajectories and made two versions, where one fact changed, and they evaluated if the agent checks that fact before acting. The experimental results shed some lights, and not too surprisingly, agents show significant performance drop when considering if they checked the current or changed state.
Critical strengths
- The research questions raised in the study are valid. Focusing just on the task success doesn't tell the full story of agent performance.
- The way authors created the data to study this problem is simple and clever. They keep the history and with one changed fact, two different worlds are obtained to assess if the agent check that relevant fact.
- They have some interesting findings. In many cases, the agent don't check the state, and though often harmless, in risky cases agent performs real state-changing write before checking.
Critical weaknesses
Writing: The paper's language is too complex and dense, it's very hard to follow (I can't tell how much this is due to AI writing). There are a lot of jargons and dense terminology. The paper can be much simplified to convey the same message.
All the appendix provided seem to be just agent logs. This is not the right content for a paper.
Authors introduced many metrics, NSR, PCR, SQP, SFT, JointATIS, etc, 7 primary metrics and a half dozen others (some are in Appendix). Too many results are undermining the paper's real message, that is, outcome success looks high, but evidence grounded success is low. Readers can't follow all the results. And not all of those are necessary.
The experimental setup and results left some open questions. The absence of human validation of the contracts is a big gap. AI audits are not enough, and even the paper own AI auditors showed the risky cases having major issues. The validity of the results is thin, partly due to the limited environment used.
Questions for authors
About the experimental setup, the agents used are relatively small models, do we know if this is a real agent problem? With the changed agent trajectories, is there a way to measure using frontier models to help us understand whether it's an agent or a small model problem?