- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper systematically studies test-time scaling for multi-view diffusion models, with a particular focus on the interplay among generation, verification, and search. The paper first shows that joint multi-view generation exhibits more favorable test-time scaling behavior than independent per-view generation, while also revealing a trade-off between set-level consistency and per-view improvement. Motivated by this observation, the authors propose DINO-centroid as a verifier, which leverages feature-level consensus across multiple generated candidates to perform both full multi-view set selection and uncertain-view identification. Building on this verifier, the paper further introduces E-R Search, which combines set-level exploration with view-level conditional refinement to improve generation quality. Experiments show that, under matched compute budgets, the proposed method outperforms baselines including Best-of-N, Beam Search, and IPR-style Refinement.
Critical strengths
-
The paper provides a systematic analysis of test-time scaling in multi-view diffusion models. By separately studying the generation paradigm, verification granularity, and search strategy, it identifies an important tension between preserving cross-view consistency and improving individual views. This analysis provides a clear motivation for the subsequent method design.
-
DINO-centroid is simple yet effective. By exploiting feature-space agreement across different generations, it supports both set-level selection and uncertain-view identification within a unified framework. The authors further observe that the similarity between the centroid and the ground truth increases as the number of sampled candidates grows, suggesting that the verifier itself also exhibits favorable scaling behavior with increased sampling.
-
The design of E-R Search is well aligned with the preceding empirical analysis: set-level exploration addresses global multi-view consistency, while view-level conditional refinement targets locally uncertain views. The experimental evaluation is relatively comprehensive, and the method is also shown to be effective in practical downstream settings.
Critical weaknesses
-
Although the paper compares against several standard baselines, a natural sequential search strategy is not considered. For example, one could generate multiple candidates for a single new view, use a DINO-based criterion to select a relatively reliable candidate, append this view to the current image set, and then continue generating subsequent views conditioned on the expanded set. Compared with the proposed strategy, which first explores multiple complete multi-view sets and then performs refinement, this would constitute a more incremental form of search. The current experiments therefore do not establish whether E-R Search exhibits better scaling behavior than such a sequential alternative. Including this baseline would help clarify the benefit of jointly exploring complete multi-view hypotheses before refinement.
-
DINO-centroid appears to rely on the assumption that consensus among generated samples in feature space is correlated with the quality or plausibility of the underlying multi-view hypothesis. This assumption may become problematic when the generative model itself has systematic biases or when the samples form multiple distinct modes in feature space. In such cases, the centroid may be dominated by a frequent but incorrect mode and potentially reject a less frequent yet more accurate hypothesis. The current evaluation mainly reports average performance and provides limited analysis of such failure cases. It would be useful to include challenging or out-of-distribution examples for which the underlying model has difficulty estimating the object's unseen views, and analyze how model bias or multi-modality affects the robustness of DINO-centroid and the overall search procedure.
-
There is some tension between the paper's motivation regarding the inherent ambiguity of single-view-conditioned multi-view generation and its evaluation protocol, which is largely based on a single ground-truth reference. A multi-view hypothesis that differs from the dataset reference but is nevertheless visually and geometrically plausible may still receive poor LPIPS, PSNR, or SSIM scores. Therefore, it is unclear whether these metrics fully capture the practical improvement provided by the proposed method. A human evaluation, or alternatively an evaluation protocol that better accounts for multiple plausible reconstructions, would help address this concern.
-
There remains a noticeable gap between the practical setting and the Oracle setting, but the source of this gap is not sufficiently analyzed. The reported results seem to suggest that DINO-centroid is already relatively close to the Oracle in identifying which views should be refined, while the final generation quality remains substantially below the Oracle result. If this is indeed the case, it would be valuable to further decompose the gap and determine whether the remaining bottleneck comes primarily from the refinement procedure itself, imperfect candidate selection, conditioning errors, or other factors. A more detailed analysis would help clarify where future improvements are most likely to be effective.
Questions for authors
- The paper shows that DINO-centroid becomes increasingly similar to the ground truth as the sampling budget increases, but the tested range of sample counts is still relatively limited. Does this scaling behavior eventually saturate? More specifically, as the number of sampled candidates continues to increase, does the gap between DINO-centroid and the Oracle continue to shrink, or does it plateau at a non-zero value?
10 comments
Author Response — Part 2/3
W3. Evaluation with a Single Ground-Truth Reference
We agree that single-image-conditioned multi-view generation is inherently ambiguous and that multiple geometrically plausible reconstructions may exist for the same input. Reference-based metrics cannot fully capture this ambiguity. Nevertheless, PSNR, SSIM, and LPIPS are widely adopted evaluation metrics in multi-view generation.
Importantly, our evaluation is not limited to reference-based image metrics. Appendix J (Table 5) additionally evaluates reference-free cross-view consistency using MEt3R, while Appendix K (Table 6) assesses downstream 3D reconstruction using Chamfer Distance and Volume IoU. At N=64 on the Normal subset, E-R Search reduces MEt3R from 0.441 to 0.434 compared with Best-of-N, while reducing Chamfer Distance from 0.0265 to 0.0252 and increasing Volume IoU from 0.551 to 0.568. Following your suggestion, we extended the downstream reconstruction evaluation in Appendix K (Table 6) to Easy and Hard at a sampling budget of . Across the three subsets, E-R Search reduces Chamfer Distance by 3.6–4.9% and increases Volume IoU by 1.5–1.7 percentage points relative to Best-of-N.
| Method | Easy Chamfer Dist. ↓ | Easy Volume IoU ↑ | Normal Chamfer Dist. ↓ | Normal Volume IoU ↑ | Hard Chamfer Dist. ↓ | Hard Volume IoU ↑ |
|---|---|---|---|---|---|---|
| SyncDreamer | 0.0255 | 0.580 | 0.0274 | 0.537 | 0.0305 | 0.508 |
| BoN | 0.0249 | 0.592 | 0.0265 | 0.551 | 0.0286 | 0.525 |
| E-R Search | 0.0240 | 0.607 | 0.0252 | 0.568 | 0.0275 | 0.540 |
Furthermore, we conducted a preliminary pairwise human evaluation involving 30 objects (10 per difficulty level) and three graduate students with computer science backgrounds, each independently evaluating all 30 objects. For each object, we generated four initial multi-view candidates using different random seeds. The baseline was randomly selected from these four candidates, while the E-R Search result was obtained by using the same candidate pool for exploration, followed by conditional refinement. E-R Search was preferred in 80.0%, 83.3%, and 90.0% of judgments on Easy, Normal, and Hard, respectively, achieving an overall preference rate of 84.4% (76/90 votes).
Together, these complementary evaluations support the effectiveness of E-R Search beyond reference-based reconstruction accuracy. Our preliminary human evaluation further shows a consistent preference for E-R Search over single-sample generation across all three difficulty levels. We will include these results and expand the human evaluation in the revised manuscript.
Author Response — Part 3/3
W4. Understanding the Gap Between Practical and Oracle Performance
We thank the reviewer for this insightful suggestion. To identify whether the practical–Oracle gap mainly arises from imperfect candidate selection, refinement decisions, or conditioning errors, we decompose E-R Search into three decision stages: (1) initial set selection (S), (2) uncertain-view identification (M), and (3) final candidate selection (F).
We conduct a controlled ablation by independently replacing DINO-centroid with Oracle verification at each stage under matched sampling budgets, while keeping the refinement procedure unchanged. The results at N=64 are summarized below.
| Initial Selection (S) | View Identification (M) | Final Selection (F) | Easy ↑ | Normal ↑ | Hard ↑ |
|---|---|---|---|---|---|
| DINO | DINO | DINO | 8.11 | 11.60 | 8.98 |
| Oracle | DINO | DINO | 15.72 | 18.09 | 15.63 |
| DINO | Oracle | DINO | 8.79 | 12.05 | 9.65 |
| DINO | DINO | Oracle | 9.27 | 13.06 | 10.24 |
| Oracle | Oracle | Oracle | 20.10 | 22.70 | 19.90 |
LPIPS improvement (%) relative to the four-sample average baseline; higher is better. Oracle verification uses ground-truth LPIPS for selection decisions without providing ground-truth observations during refinement.
The results indicate that imperfect initial candidate selection is the dominant source of the practical–Oracle gap. On the Normal subset, replacing DINO-centroid with Oracle at the initial selection stage improves LPIPS gain from 11.60% to 18.09%, an increase of 6.49 percentage points. In contrast, using Oracle only for uncertain-view identification or final candidate selection yields additional gains of just 0.45 and 1.46 percentage points, respectively. Similar trends hold across all difficulty levels.
Importantly, the refinement procedure and conditioning mechanism remain unchanged across these ablations, indicating that the largest observed gap arises from imperfect verification of the initial multi-view hypothesis rather than differences in the refinement algorithm itself. Nevertheless, conditioning quality also affects refinement performance, as demonstrated by the GT-observation experiments in Fig. 6. These experiments reveal substantial additional potential when more accurate observations become available, although their contribution cannot be quantitatively isolated by the present stage-wise ablation.
Overall, our analysis identifies initial candidate selection as the primary practical bottleneck, while highlighting observation quality as another important factor influencing refinement. These findings suggest that improving complete-set verification and reliable observation selection offers promising directions for further advances in E-R Search. We will include these additional ablation results and analyses in the revised manuscript.
Q1. Does the Scaling Behavior of DINO-centroid Eventually Saturate?
We thank the reviewer for this insightful question. We extended our experiments to N=256. While DINO-centroid continues to become more similar to GT, the incremental gains at larger sampling budgets are substantially smaller than those at smaller budgets, indicating diminishing returns and a tendency toward saturation rather than linear scaling with log(N).
We expect the similarity to eventually plateau below 1, since increasing the sampling budget can reduce estimation variance but cannot necessarily eliminate the underlying distributional bias. These observations suggest that DINO-centroid gradually approaches a saturation point, where further increasing the sampling budget yields diminishing improvements in similarity to GT.
Author Response — Part 1/3
We thank the reviewer for the constructive feedback and for recognizing the value of our systematic investigation of multi-view TTS, the effectiveness of DINO-centroid, and the motivation behind E-R Search. We address the concerns below.
W1. Comparison with Sequential Search
We appreciate the reviewer's suggestion and agree that sequential view-by-view search is a reasonable alternative.
We have explored this strategy in our preliminary experiments using SyncDreamer backbone and Posterior-DDIM sampler. At each of 16 sequential stages, previously selected views are retained as fixed observations, and a N=64 budget permits 4 candidate multi-view sets to be generated conditioned on these observations. We then select one candidate view at the current viewpoint using DINO-centroid and add it to the observation set for subsequent stage.
It is worth mentioning that SyncDreamer is a joint multi-view generator rather than a per-view generator. Even when regenerating only one target view, our conditional sampling procedure requires a full joint denoising trajectory over all 16 views, incurring approximately the same computational cost as generating a complete multi-view set. Consequently, under a budget of 64 trajectory-equivalent evaluations, a uniformly allocated sequential strategy can evaluate only four candidates per viewpoint.
We compare this strategy with E-R Search under the same computational budget on the Normal subset:
LPIPS improvement (%) relative to the four-sample average baseline; higher is better. Lower MEt3R indicates better cross-view consistency.
The results show that sequential conditioning does not sufficiently compensate for the limited candidate pool. Moreover, committing to individual views sequentially may propagate early errors into subsequent conditional generations and introduce cross-view inconsistencies. In contrast, E-R Search first explores complete multi-view hypotheses, allowing the verifier to assess globally coherent candidates before selectively refining uncertain views. With N=64, it allocates 48 trajectories to global exploration and 16 to conditional refinement, providing a more effective allocation of the sampling budget.
We will include this preliminary comparison and clarify the relationship between sequential search and our E-R strategy, together with their computational and consistency trade-offs, in the revised manuscript.
W2. Robustness of DINO-centroid to Systematic Biases and Multimodality
We thank the reviewer for raising this important concern and acknowledge that DINO-centroid relies on the assumption that cross-sample consensus provides a useful signal for generation quality, which is empirically supported by our results across different sampling budgets and difficulty levels.
One potential reason of DINO-centroid’s robustness to generator’s systematic biases and multimodality is that DINO-centroid aggregates view-level scores to select a complete multi-view set. Even if most candidates share a similar incorrect mode at an individual viewpoint, biasing that viewpoint’s consensus, their overall set-level scores depend on the remaining viewpoints. This can mitigate the influence of incorrect local consensus and highlights the advantage of set-level selection over independent view-level selection.
To directly address the reviewer's suggestion of evaluating challenging cases where the underlying model struggles to predict unseen views, our evaluation already includes a Hard subset, consisting of the 100 GSO objects with the highest single-sample LPIPS. These objects represent particularly challenging cases for the underlying generator to reconstruct unseen viewpoints. On these challenging cases, DINO-centroid achieves a 5.74% LPIPS improvement under Best-of-N at N=64, while E-R Search further improves this to 8.98%. These results demonstrate that our approach remains effective even when the underlying generator struggles to reconstruct unseen views, providing evidence of robustness under challenging generation conditions.