- Rating
- 1
- Confidence
- 4
- Informativeness
- 1
- Presentation
- 1
- Soundness
- 2
Read review
Paper summary
This work introduces a method to estimate the benefit of allowing more token budget. Basing on this estimate, the method conducts test-time scaling to most promising queries.
Critical strengths
1, The proposed method seems to achieve strong results on their proposed metrics. 2, The work targets on an important problem: test-time token budge allocation.
Critical weaknesses
1, The writing is hard to follow. Here are some examples:
- "instance-compute alignment" is introduced as one of the major contributions. However, it is not discussed in the method section.
- There are several uncommon terminologies and metrics, including: FEP, FEP reliability, oracle-action accuracy, macro score.
- Eq (14) is defined without defining the notation or any further explanation; the 84.0 was referring to the result in Table 1, without any pointer. 2, The work seem to introduces several new metrics, without justifying. This would make it harder to judge the comparison.
Questions for authors
It is mentioned that macro score is an unweighted average across several metrics. Can you explain how similarity and proof score are definited? Would it be the case that they are not in the same scale?