- Rating
- 1
- Confidence
- 3
- Informativeness
- 1
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
The paper shows that improving single-sample accuracy through RLVR does not necessarily improve majority-vote accuracy. It proposes adapter thickets, which split a fixed training budget across LoRA adapters trained on disjoint data shards and pool their responses under the same inference budget. Across four models, thickets consistently outperform fully trained single adapters in majority voting, with lower error correlation among voters. Their main benefit is preserving voting performance lost during concentrated training, while gains over the base model and early-stopped single adapters are smaller and less consistent.
Critical strengths
None
Critical weaknesses
- Limited conceptual novelty relative to prior work. The core recipe, partitioning training data, independently fine-tuning LoRA adapters, and ensembling their predictions, closely resembles existing adapter-ensemble approaches. Notably, ELREA [1] already includes a random, disjoint data-partitioning baseline, while Tang et al. [2] explicitly study the mismatch between single-sample training objectives and inference-time aggregation. The present paper’s distinct contribution is therefore primarily the controlled evaluation of this recipe under a fixed RLVR budget. It does not establish a substantially new mechanism or explain when and why this allocation should outperform existing alternatives, leaving the conceptual advance limited.
- The strongest improvements are measured against a fully trained single adapter whose voting performance has already deteriorated. At 160 samples, the average advantage falls to only 0.71 percentage points over the base model and 0.27 points over the early-stopped single-adapter control; thickets outperform the base model in only 9 of 16 settings. Moreover, the early-stopping control receives approximately 20% fewer updates than thicket members at (K=8) and (K=16), weakening even this small comparative advantage. These results primarily demonstrate recovery from training-induced degradation and do not yet establish a compelling practical benefit over properly selected single-policy baselines.
[1] Li, Y., Gao, V., Zhang, C., & Torkamani, M. (2025, May). Ensembles of low-rank expert adapters. In International Conference on Learning Representations (Vol. 2025, pp. 92940-92968).
[2] Tang, Y., Zheng, K., Synnaeve, G. & Munos, R.. (2025). Optimizing Language Models for Inference Time Objectives using Reinforcement Learning. <i>Proceedings of the 42nd International Conference on Machine Learning</i>, in <i>Proceedings of Machine Learning Research</i> 267:59066-59085 Available from https://proceedings.mlr.press/v267/tang25o.html.
Questions for authors
- What are the new insights and takeaways the authors bring? How do you think about the comparison between your work and previous works including two papers above?
- Beyond the empirical curves and correlations, can the authors provide a predictive explanation of when splitting the RLVR budget is beneficial, how the optimal number of adapters depends on the training and inference budgets, and why some models benefit while others do not?