- Rating
- 2(+1)
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 1
Read review
Paper summary
This paper demonstrates that fixed-budget latent reasoning models can get improved accuracy with an early termination signal. The first stage of latent reasoning often promotes the gold solution, but the later stage can reverse it and promote an incorrect answer. This paper presents a method that can estimate the benefit of continuing latent reasoning and how many more latent tokens should be generated. Their method largely outperforms fixed-budgets of latent tokens.
Critical strengths
S1) Clearly demonstrates that additional latent reasoning is not always beneficial, and that accuracy can be improved by estimating the realized constructive progress and early termination of reasoning. This work sits well amongst concurrent work in making more efficient explicit reasoning models.
Critical weaknesses
W1) I think some of the gains over full-k in table 1 are due to the fact that GSM8k, SVAMP, and MultiArith generally need fewer reasoning steps than the number of latent steps. Most gold label solutions in GSM8k problems are 2-4 steps, SVAMP is 1-2 steps, and MultiArith is always 2 steps. The Coconut and CODI models are trained such that one latent step is roughly corresponds to 1 gold label step. So having 6 or 10 latent steps (depending on the model) is almost always more steps than needed, so it's not surprising to me that early termination in table 1 can beat full-k. What would the results in table 1 look like if you had a test set where the gold label number of solution steps was balanced, from 1-6 or 1-10 steps?
W2) Table 8 shows that two baselines (stability and confidence) perform about as well as your method. What is the main benefit of using your progress method over these baselines? Also, Table 8 should probably be in the main text.
Questions for authors
Q1) Frontier advancement makes sense for ProsQA since the model is searching through a DAG. But what does frontier advancement mean for math problems like those in GSM8k?
Q2) In Figure 3, inserting a discrete token should hurt the model's accuracy (and thus % gold candidates formed) since passing in a discrete token is OOD for Coconut and CODI during reasoning. Can you provide more detail on the motivation for this experiment? And how did you form the "matched control that perturbs the continuous state by a comparable magnitude" in panel (b)?
Q3) For table 8, can you add how stability, confidence, and convergence are calculated?