- Rating
- 1
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
Authors propose SelVer-TTT that selectively performs test-time self-training. The method estimates uncertainty from multiple sampled solutions and synthesizes training examples + solutions of the same schema. If a problem is sufficiently uncertain and if synthetic supervision passes checks, then a LoRA adapter is trained with residuals discarded between tasks or decayed in persistent mode. Authors report 51.6% unconditional -> 52.4% selective mean accuracy across several reasoning benchmarks with Qwen2.5-7b-instruct. The conditional setup reduces queries adapted and overhead adaptation time by ~60%.
Critical strengths
The synthesized supervision is checked before used for weight updates-- even the rationale is checked apparently. This contributes to the selectivity that mitigates noise from self-supervised TTT. Overall seems somewhat effective in reducing adaptation time while preserving and slightly improving overall accuracy.
Critical weaknesses
The experimental details around verification are underspecified. Checkers and structural-validators are mentioned and support the selectivity of this approach but are not described or given concrete examples per benchmark. Moreover, the results do not sufficiently clarify each component's contribution or reliability. Tables lack uncertainty estimates and ablation experiments, limiting conclusions about specific interventions.
Questions for authors
- What structural validators and checkers produced tables 1-3?
- Can you share representative accepted/rejected synthetic task samples (especially for GPQA)
- How is retention/degradation computed for the multi-query/stream settings?