- Rating
- 3
- Confidence
- 4
- Informativeness
- 3
- Presentation
- 3
- Soundness
- 3
Read review
Paper summary
This paper proposes MicroMouse, a selective-SFT method for long reasoning traces. It splits each trace into segments and measures the relative support for the gold answer against a constructed incorrect answer, called a foil. Based on changes in this score, it masks losses for segments labeled as erroneous exploration or redundant wandering while keeping them in the context. Experiments report improved reasoning accuracy and shorter outputs compared with Full-CoT SFT and several selective-SFT baselines.
Critical strengths
- The paper addresses a useful problem: not every part of a successful reasoning trace provides equally valuable supervision. Keeping masked segments in the context also preserves the dependencies needed by later steps.
- The researchers categorize reasoning steps into Correct Progress, Necessary Exploration, Erroneous Exploration, and Redundant Wandering, which makes sense because sometimes in CoT, the model can spend some effort on exploring "what is wrong", and that is necessary.
- The scoring method is simple and relatively efficient. It uses answer likelihoods without requiring gradient-based attribution or a large model to annotate each step.
- The experiments cover three model sizes and seven reasoning benchmarks. Results under both greedy decoding and sampling provide useful evidence for the method’s effectiveness.
Critical weaknesses
- The maze analogy does not sufficiently justify the core classification rules. REP has not been shown to measure the actual distance to a complete solution, and useful exploration may involve several consecutive score decreases. Likewise, reaching the global REP peak does not establish that reasoning is complete or that all later steps are redundant.
- A single constructed foil may not reliably represent progress toward the correct answer. A numerically close answer is not necessarily a likely competing answer, and REP can increase even when the gold answer becomes less likely, provided the foil becomes less likely by a larger proportion. The paper does not establish whether its scores and masking decisions remain stable under different foil choices.
- The method’s sensitivity to the scoring model remains unclear. Replacing the default DeepSeek-R1-Distill-Qwen-1.5B scorer with Qwen3-4B-Instruct reduces greedy accuracy from 45.1 to 41.9 and Pass@1 from 50.9 to 49.0. Although some gains over Full-CoT SFT remain, testing one alternative scorer provides limited evidence of robustness across scoring models.
Questions for authors
- Can you provide independent step-level evidence that REP reflects actual reasoning progress? In particular, how reliably do consecutive decreases identify harmful exploration, and how often do steps after the REP peak still contain necessary reasoning?
- How sensitive are the masking decisions and downstream results to the choice of foil? Would the method remain effective with plausible model-generated wrong answers or multiple competing answers?
- What makes a model suitable for scoring, and why does the default 1.5B scorer perform better than the alternative 4B scorer? Can you evaluate more scorer families and sizes, and report how much their masking decisions agree?