- Rating
- 2
- Confidence
- 2
- Informativeness
- 1
- Presentation
- 1
- Soundness
- 1
Read review
Paper summary
The authors develop a framework called FormatGain to assist in selecting tool-call formats for an incoming model. Concepts such as interaction share, oracle gain, and decision margins are introduced to guide format selection.
Critical strengths
Provides a theoretical contribution to interaction analysis.
Critical weaknesses
-
The motivation for studying the interaction effect is not clearly articulated.
-
It is unclear why analyzing historical data is beneficial.
-
The paper insufficiently situates its contribution within existing literature. For instance, Section 3.1 should be connected to two-way ANOVA. Is a newly introduced quantity?
Questions for authors
-
Who should care? Is the target audience a general agent R&D team, tool developers, or model providers? What is the key takeaway for each?
-
Why is this study needed? Table (a) already appears informative on its own. Why are newly developed concepts such as oracle gain and selection error necessary? Why not using benchmark for the selection?
-
How are computed? Is in Equation (1) the same as in Equation (3)?
-
What is the format selection algorithm? How is the proposed method practically useful?
Other Comments
An important format, TOON, could be studied, as it is designed to reduce token consumption.