- Rating
- 3(+1)
- Confidence
- 3
- Informativeness
- 3
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper presents ARBOR, a memory framework for metrics. The intuition is straightforward: trajectories with identical outcomes (e.g. correctness) can have other differences, which could be further fine-grained using this framework.
Critical strengths
- ARBOR shows consistent performance improvement in 3 sizes and across many tasks.
- The idea of having an online metrics framework is quite unique.
Critical weaknesses
- The necessity of having an online metric framework is not strictly proven. A high percentage of groups are outcome-homogeneous groups, for which the induction stage will be skipped. Does this mean that, without memory (in Figure 3), there will be no rubrics? In Appendix E.1 the author write Ideal Search Agent Vision, can the author use that as metrics and run the experiment to see if online update are necessary. Some of the top-5 metrics created by ARBOR is quite similar to (overlapped with) the Ideal Search Agent Vision.
- The training only goes for 60 steps, and each rollout only contains 8 rollouts. Does ARBOR work better with low rollout counts, as there will be a much low chances of a group of trajectories are either all correct/wrong if the rollout counts are higher. Also, does having more training step address this issue? as ARBOR may provide some additional gradients in the early steps, but may not give better results when the training is sufficient, or it could also give misleading rubrics.
Questions for authors
See Weakness.
Thanks for the reply. All my concerns have been resolved. I have increased my score to 3-weak accept.