- Rating
- 3(+1)
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
The paper revisits the Neural Thickets hypothesis, which proposes that pretrained LLMs are surrounded by dense neighborhoods of task-specific experts in weight space that can be discovered through random Gaussian weight perturbations. The authors challenge whether weight perturbations are actually necessary to obtain these performance improvements. They first demonstrate that perturbing only a single decoder block recovers much of the accuracy improvement achieved through full-weight perturbations. They then show that applying PSV directly to the output logits of a fully frozen model matches or even exceeds full-weight perturbation ensembles on most evaluated reasoning tasks. Finally, they extend their investigation to vision models, where output-logit perturbations similarly outperform weight perturbations under domain shift.
Critical strengths
-
The paper investigates an interesting and important question concerning the Neural Thickets Hypothesis, namely, whether the improvements actually require full weight space perturbations. For this, the authors propose a simple yet elegant alternative that achieves competitive performance without modifying the weights.
-
I appreciate that the authors extend their experiments beyond language models to vision classification under domain shift, broadening the scope of the analysis.
Critical weaknesses
-
The absence of uncertainty estimates for the main experimental results. Both full-weight and logit PSV are inherently stochastic. The central results in Figures 2-4 and Tables 1, 3, and 5 are reported without standard deviations or confidence intervals across independent runs. This is particularly concerning as several reported differences are relatively small.
-
The paper demonstrates strong ensemble performance, it remains unclear how well individual selected logit PSV members generalize to held-out examples. Although the ensemble-size sweeps include the single-member case, I would appreciate a more systematic analysis of the test accuracy distributions of individual randomly sampled and selected members.
-
The authors themselves acknowledge that larger models may exhibit weight-space effects that output-logit perturbations cannot capture. Evaluating at least one substantially larger model would provide valuable evidence about whether the proposed explanation continues to hold at larger scales.
Questions for authors
-
Can you provide standard deviations and/or confidence intervals across independent runs with different random seeds, for all the main experimental results?
-
Can you provide the systematic analysis of the test accuracy distributions of individual randomly sampled and selected members of the logit PSV methodology?
-
How large are the differences between the original full weight perturbation neural thicket methodology and your logit PSV methodology on a substantially larger model, please include standard deviations?
5 comments
Thank you for the detailed rebuttal and additional experiments. The new results address my main concerns, which inclines me to increase my score.
However, I cannot find the revised manuscript containing the referenced experiments and tables. Could the authors clarify where the updated PDF is?
Revised version now avilable
Dear Reviewer,
Thank you for your feedback. It appears that the revised manuscript was not uploaded correctly. This should now be fixed, and the updated PDF should be available.
Please do not hesitate to reach out if you have any further questions or need additional clarification.
Asnwer
We thank the reveiwer for the suggestion. Every table and figure is now repeated with three seeds of the perturbations (and of TT-MV's sampling) on the same selection and test items. The paper now reports mean ± s.d. and paired 95% CIs; Figures 2–4 show error bars (App. D.2). Spread across seeds:
Replicates that also redraw the selection set give the same picture (Table 10).
Yes, for both families: Tables 11–12 (App. E) cover every model and task. Logit PSV examples (N=5000, test accuracy):
Selected members are only slightly better than the base model on held-out data; across all cells, the median is +0.1 points. Random members are at or below the base. Most of the gain comes from the vote.
We also ran this analysis for vision: FMoW, DINOv3 S–7B, 10 pools of 3500 members each.
A random bias usually hurts, but selection reliably finds members that help. Their validation gain carries over almost unchanged to held-out test examples. Averaging the 10 selected biases then adds about one more point (+4.1 to +4.7).
Qwen2.5-32B-Instruct, three seeds, fixed selection and test sets (App. C.3; also our reply to E6hK):
Logit PSV matches or beats full-weight PSV on Countdown and GSM8K, and trails it by about 3 points on hard math. There, an LM-head-only perturbation recovers about 60% of the difference. We report this as a limitation.
In vision, DINOv3-7B (6.7B parameters) is the largest model of its family. Across the family, from 21M to 6.7B parameters, the logit-PSV gain stays flat (+4.1 to +4.7), and weight perturbations stay near zero even at the largest size (Table 3). We see no sign of a weight-space effect emerging with scale in vision.