- Rating
- 1
- Confidence
- 3
- Informativeness
- 1
- Presentation
- 1
- Soundness
- 1
Read review
Paper summary
The paper studies tool-using agents that pass intermediate objects between tool calls as serialized text. It separates two sources of failure. Across 3 domains, 7 models, local properties are answered at 51.3–99.5% accuracy while global properties reach only 20.8–49.0%. In multi-step pipelines, raw re-emission aborts 16.2–73.5% of graph pipelines and up to 99.7% of tabular pipelines.
Critical strengths
- A decomposition with a controlled design. Separating "cannot derive" from "cannot re-emit", and crossing decision exposure with state preservation in a factorial design, isolates the two failure sources more cleanly than end-to-end agent benchmarks do.
Critical weaknesses
- The writing raises concerns, including suspected excessive AI involvement.
- The 9-page main text contains only one table but more than 60 pointers to the appendix, and it cites about 20 appendix tables (Tables 2–43) for its key numbers. Most claims cannot be checked from the main text.
- The prose is dominated by coined notation (Π-Oracle, Π-Agnostic-Hit/Miss, , Casc(All)/Casc(Clean), OHG, AG, , ArtifactRef) and by long strings of numeric ranges, with little explanation of why each analysis is needed (e.g., the five RQ findings on p.2).
Questions for authors
- Which deployed agent frameworks or APIs actually require full-object re-emission as tool arguments? Can you provide evidence, such as framework analysis or real traces, that this interface is common?
- What completion cap was used in the Experiment 2 pipelines? Why do non-semantic aborts increase from about 35 to about 130 under constrained decoding in the tabular cell?
- How were the ArtifactRef cohorts selected?