- Rating
- 2
- Confidence
- 4
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper proposes architectural sampling for frozen VLMs, generating candidate responses by varying the location and repetition count of reused decoder blocks. Experiments show improved candidate coverage over standard sampling and benefits when using these candidates for test-time learning.
Critical strengths
S1. The method is simple, requires no additional training for candidate generation, and introduces no new parameters.
S2. The evaluation spans multiple models and benchmarks, with generally consistent improvements. The ablations help characterize effective configurations.
Critical weaknesses
W1. The main concern is missing comparisons with closely related methods. Although the paper discusses Inner Loop Inference, Training-Free Looped Transformers, and CoLa, it compares experimentally primarily against standard decoding strategies. This limits the assessment of its contribution beyond existing layer-reuse techniques.
W2. The authors should provide additional experimental evidence to distinguish the benefit of architectural diversity from that of layer looping itself. The current comparison changes both whether layers are looped and whether candidates use different looping configurations. It therefore remains unclear whether the gains require complementary computation paths or can be explained by a stronger individual looping configuration.
The authors should also update the anonymous GitHub repository with the code for any additional experiments, which I intend to reproduce. I would be willing to raise my score if the authors adequately address these concerns.
Questions for authors
please see weakness.
5 comments
Out of respect for the reviewer, we report them in the new Table 4. We do not, however, recommend adding them to the paper as baselines. Doing so would mislead readers into believing these sampling methods existed before this work. They did not.
| Method | CV-Bench | MMStar | AI2D | A-OKVQA | ScienceQA |
|---|---|---|---|---|---|
| Standard decoding | |||||
| Temperature | 88.48 | 58.93 | 84.89 | 88.65 | 88.75 |
| Beam Search | 86.09 | 52.60 | 80.78 | 87.16 | 86.22 |
| Top-k | 89.12 | 60.27 | 85.65 | 89.61 | 89.74 |
| Top-p | 88.91 | 59.84 | 85.37 | 89.28 | 89.41 |
| Architectural perturbations | |||||
| Layer skipping [19] | 93.78 | 62.24 | 92.20 | 93.97 | 95.93 |
| Layer dropout [10] | 88.86 | 60.12 | 87.69 | 90.13 | 90.48 |
| Hidden-state noise | 89.80 | 59.58 | 88.12 | 90.48 | 92.12 |
| Input-side test-time methods | |||||
| Plan-and-Solve [37] | 85.37 | 60.87 | 83.81 | 86.72 | 88.05 |
| Self-Refinement [23] | 76.38 | 52.80 | 77.45 | 77.47 | 83.19 |
| Describe-then-Answer [28] | 83.93 | 51.27 | 77.88 | 86.03 | 80.61 |
| Prompt Repetition [18] | 86.16 | 54.40 | 83.34 | 87.86 | 88.15 |
| Architectural sampling | 96.40 | 82.73 | 93.48 | 95.72 | 97.97 |
Table 4: Comparison of candidate diversity sources on Qwen3-VL-8B under an equal nine-candidate budget (). Best results are bolded.
W2: Separating architectural diversity from layer looping itself
We thank the reviewer for this constructive question. We have added a new analysis, “Do we need different architectural configurations?” (Section 5, Figure 5).
On MMStar, we drew all nine candidates from each single configuration and compared this with pooling one candidate per configuration, keeping the candidate budget and decoding temperature identical. Each configuration saturates quickly on its own, and the best one reaches only 67.53%. Pooling reaches 82.73%, which is 15.20 points higher, even though several of the pooled configurations are weak in isolation. The same holds on CV-Bench and MMMU (Supp. C). The gain therefore comes from diversity across computation paths, not from any individual looped path. This answers W2 directly and also settles W1.
We thank the reviewer for their time and energy in reviewing our paper.
W1: Missing comparisons with Inner Loop Inference, Training-Free Looped Transformers, and CoLa
We disagree with this concern. It rests on a mischaracterization of both our contribution and the cited works. Status of the cited works. Inner Loop Inference is a five-page, non-peer-reviewed arXiv preprint. Training-Free Looped Transformers is under review at ICLR 2027, the same venue as our submission. We cited both for transparency. Citing a paper does not make it an established baseline.
CoLa is not comparable. There are four independent reasons:
It studies text-only LLMs, not VLMs.
Its per-input MCTS is guided by answer correctness. It is therefore an oracle analysis that requires ground-truth labels at test time, which makes it unusable in our label-free setting.
It requires many forward passes per input. Our method needs a fixed set of nine configurations costing 10.5 decoder traversals.
It searches for one best path per input and produces no candidate pool.
Comparing a label-free sampling method against a label-guided oracle search is not a meaningful experiment.
And most importantly None of these works is a sampling method. All three aim to improve a single forward pass. Our contribution is a candidate-generation technique that uses computation paths as a sampling axis. To our knowledge, no prior work does this. Benchmarking against single-path inference methods would misrepresent our contribution, and we will not add comparisons that mislead readers about what the method is for even if it means acceptance.