acceptodds
Under review as a conference paper at ICLR 2027

Test-Time Scaling for Multi-View Diffusion Models: Generation, Verification, and Search

Abstract

Test-time scaling (TTS) improves generative performance by allocating additional inference computation, yet its effectiveness for structured outputs such as multi-view diffusion models remains underexplored. In this work, we systematically investigate how generation, verification, and search interact when the generated views must remain consistent. We first find that joint multi-view generation exhibits stronger Oracle scaling than independent per-view generation across multiple evaluated models, while revealing a fundamental tension between preserving cross-view consistency through set-level selection and improving individual views through view-level selection. To address this challenge, we introduce DINO-centroid, a lightweight verifier that leverages cross-sample consensus for complete-set selection and uncertain-view identification. Our analysis shows that DINO-centroid becomes increasingly aligned with the ground-truth representation as the sampling budget grows, suggesting that the verifier itself exhibits a form of scaling behavior absent from conventional TTS verifiers. Building on these findings, we propose Exploration–Refinement Search (E-R Search), a strategy that combines set-level exploration with targeted view-level regeneration by formulating the latter as a diffusion inverse problem. Under matched inference budgets, our strategy achieves higher reconstruction quality than existing test-time search baselines. Our findings highlight the importance of accounting for component dependencies in generation, verification, and search for effective TTS of structured outputs.

Primary area:
generative models
Keywords:
test-time scaling, multi-view diffusion models, multi-view generation, verification, search, conditional refinement, inverse problems

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_6LJDOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper systematically studies test-time scaling for multi-view diffusion models, with a particular focus on the interplay among generation, verification, and search. The paper first shows that joint multi-view generation exhibits more favorable test-time scaling behavior than independent per-view generation, while also revealing a trade-off between set-level consistency and per-view improvement. Motivated by this observation, the authors propose DINO-centroid as a verifier, which leverages feature-level consensus across multiple generated candidates to perform both full multi-view set selection and uncertain-view identification. Building on this verifier, the paper further introduces E-R Search, which combines set-level exploration with view-level conditional refinement to improve generation quality. Experiments show that, under matched compute budgets, the proposed method outperforms baselines including Best-of-N, Beam Search, and IPR-style Refinement.

Critical strengths
  1. The paper provides a systematic analysis of test-time scaling in multi-view diffusion models. By separately studying the generation paradigm, verification granularity, and search strategy, it identifies an important tension between preserving cross-view consistency and improving individual views. This analysis provides a clear motivation for the subsequent method design.

  2. DINO-centroid is simple yet effective. By exploiting feature-space agreement across different generations, it supports both set-level selection and uncertain-view identification within a unified framework. The authors further observe that the similarity between the centroid and the ground truth increases as the number of sampled candidates grows, suggesting that the verifier itself also exhibits favorable scaling behavior with increased sampling.

  3. The design of E-R Search is well aligned with the preceding empirical analysis: set-level exploration addresses global multi-view consistency, while view-level conditional refinement targets locally uncertain views. The experimental evaluation is relatively comprehensive, and the method is also shown to be effective in practical downstream settings.

Critical weaknesses
  1. Although the paper compares against several standard baselines, a natural sequential search strategy is not considered. For example, one could generate multiple candidates for a single new view, use a DINO-based criterion to select a relatively reliable candidate, append this view to the current image set, and then continue generating subsequent views conditioned on the expanded set. Compared with the proposed strategy, which first explores multiple complete multi-view sets and then performs refinement, this would constitute a more incremental form of search. The current experiments therefore do not establish whether E-R Search exhibits better scaling behavior than such a sequential alternative. Including this baseline would help clarify the benefit of jointly exploring complete multi-view hypotheses before refinement.

  2. DINO-centroid appears to rely on the assumption that consensus among generated samples in feature space is correlated with the quality or plausibility of the underlying multi-view hypothesis. This assumption may become problematic when the generative model itself has systematic biases or when the samples form multiple distinct modes in feature space. In such cases, the centroid may be dominated by a frequent but incorrect mode and potentially reject a less frequent yet more accurate hypothesis. The current evaluation mainly reports average performance and provides limited analysis of such failure cases. It would be useful to include challenging or out-of-distribution examples for which the underlying model has difficulty estimating the object's unseen views, and analyze how model bias or multi-modality affects the robustness of DINO-centroid and the overall search procedure.

  3. There is some tension between the paper's motivation regarding the inherent ambiguity of single-view-conditioned multi-view generation and its evaluation protocol, which is largely based on a single ground-truth reference. A multi-view hypothesis that differs from the dataset reference but is nevertheless visually and geometrically plausible may still receive poor LPIPS, PSNR, or SSIM scores. Therefore, it is unclear whether these metrics fully capture the practical improvement provided by the proposed method. A human evaluation, or alternatively an evaluation protocol that better accounts for multiple plausible reconstructions, would help address this concern.

  4. There remains a noticeable gap between the practical setting and the Oracle setting, but the source of this gap is not sufficiently analyzed. The reported results seem to suggest that DINO-centroid is already relatively close to the Oracle in identifying which views should be refined, while the final generation quality remains substantially below the Oracle result. If this is indeed the case, it would be valuable to further decompose the gap and determine whether the remaining bottleneck comes primarily from the refinement procedure itself, imperfect candidate selection, conditioning errors, or other factors. A more detailed analysis would help clarify where future improvements are most likely to be effective.

Questions for authors
  1. The paper shows that DINO-centroid becomes increasingly similar to the ground truth as the sampling budget increases, but the tested range of sample counts is still relatively limited. Does this scaling behavior eventually saturate? More specifically, as the number of sampled candidates continues to increase, does the gap between DINO-centroid and the Oracle continue to shrink, or does it plateau at a non-zero value?
10 comments
AuthorsOpenReview ↗

Author Response — Part 1/3

We thank the reviewer for the constructive feedback and for recognizing the value of our systematic investigation of multi-view TTS, the effectiveness of DINO-centroid, and the motivation behind E-R Search. We address the concerns below.

W1. Comparison with Sequential Search

We appreciate the reviewer's suggestion and agree that sequential view-by-view search is a reasonable alternative.

We have explored this strategy in our preliminary experiments using SyncDreamer backbone and Posterior-DDIM sampler. At each of 16 sequential stages, previously selected views are retained as fixed observations, and a N=64N=64 budget permits 4 candidate multi-view sets to be generated conditioned on these observations. We then select one candidate view at the current viewpoint using DINO-centroid and add it to the observation set for subsequent stage.

It is worth mentioning that SyncDreamer is a joint multi-view generator rather than a per-view generator. Even when regenerating only one target view, our conditional sampling procedure requires a full joint denoising trajectory over all 16 views, incurring approximately the same computational cost as generating a complete multi-view set. Consequently, under a budget of 64 trajectory-equivalent evaluations, a uniformly allocated sequential strategy can evaluate only four candidates per viewpoint.

We compare this strategy with E-R Search under the same computational budget on the Normal subset:

MethodBackboneBudgetLPIPS Gain (%) ↑MEt3R ↓
Sequential searchSyncDreamer644.270.463
E-R SearchSyncDreamer6411.600.434

LPIPS improvement (%) relative to the four-sample average baseline; higher is better. Lower MEt3R indicates better cross-view consistency.

The results show that sequential conditioning does not sufficiently compensate for the limited candidate pool. Moreover, committing to individual views sequentially may propagate early errors into subsequent conditional generations and introduce cross-view inconsistencies. In contrast, E-R Search first explores complete multi-view hypotheses, allowing the verifier to assess globally coherent candidates before selectively refining uncertain views. With N=64, it allocates 48 trajectories to global exploration and 16 to conditional refinement, providing a more effective allocation of the sampling budget.

We will include this preliminary comparison and clarify the relationship between sequential search and our E-R strategy, together with their computational and consistency trade-offs, in the revised manuscript.

W2. Robustness of DINO-centroid to Systematic Biases and Multimodality

We thank the reviewer for raising this important concern and acknowledge that DINO-centroid relies on the assumption that cross-sample consensus provides a useful signal for generation quality, which is empirically supported by our results across different sampling budgets and difficulty levels.

One potential reason of DINO-centroid’s robustness to generator’s systematic biases and multimodality is that DINO-centroid aggregates view-level scores to select a complete multi-view set. Even if most candidates share a similar incorrect mode at an individual viewpoint, biasing that viewpoint’s consensus, their overall set-level scores depend on the remaining viewpoints. This can mitigate the influence of incorrect local consensus and highlights the advantage of set-level selection over independent view-level selection.

To directly address the reviewer's suggestion of evaluating challenging cases where the underlying model struggles to predict unseen views, our evaluation already includes a Hard subset, consisting of the 100 GSO objects with the highest single-sample LPIPS. These objects represent particularly challenging cases for the underlying generator to reconstruct unseen viewpoints. On these challenging cases, DINO-centroid achieves a 5.74% LPIPS improvement under Best-of-N at N=64, while E-R Search further improves this to 8.98%. These results demonstrate that our approach remains effective even when the underlying generator struggles to reconstruct unseen views, providing evidence of robustness under challenging generation conditions.

AuthorsOpenReview ↗

Author Response — Part 2/3

W3. Evaluation with a Single Ground-Truth Reference

We agree that single-image-conditioned multi-view generation is inherently ambiguous and that multiple geometrically plausible reconstructions may exist for the same input. Reference-based metrics cannot fully capture this ambiguity. Nevertheless, PSNR, SSIM, and LPIPS are widely adopted evaluation metrics in multi-view generation.

Importantly, our evaluation is not limited to reference-based image metrics. Appendix J (Table 5) additionally evaluates reference-free cross-view consistency using MEt3R, while Appendix K (Table 6) assesses downstream 3D reconstruction using Chamfer Distance and Volume IoU. At N=64 on the Normal subset, E-R Search reduces MEt3R from 0.441 to 0.434 compared with Best-of-N, while reducing Chamfer Distance from 0.0265 to 0.0252 and increasing Volume IoU from 0.551 to 0.568. Following your suggestion, we extended the downstream reconstruction evaluation in Appendix K (Table 6) to Easy and Hard at a sampling budget of N=64N = 64. Across the three subsets, E-R Search reduces Chamfer Distance by 3.6–4.9% and increases Volume IoU by 1.5–1.7 percentage points relative to Best-of-N.

MethodEasy Chamfer Dist. ↓Easy Volume IoU ↑Normal Chamfer Dist. ↓Normal Volume IoU ↑Hard Chamfer Dist. ↓Hard Volume IoU ↑
SyncDreamer0.02550.5800.02740.5370.03050.508
BoN0.02490.5920.02650.5510.02860.525
E-R Search0.02400.6070.02520.5680.02750.540

Furthermore, we conducted a preliminary pairwise human evaluation involving 30 objects (10 per difficulty level) and three graduate students with computer science backgrounds, each independently evaluating all 30 objects. For each object, we generated four initial multi-view candidates using different random seeds. The baseline was randomly selected from these four candidates, while the E-R Search result was obtained by using the same candidate pool for exploration, followed by conditional refinement. E-R Search was preferred in 80.0%, 83.3%, and 90.0% of judgments on Easy, Normal, and Hard, respectively, achieving an overall preference rate of 84.4% (76/90 votes).

Together, these complementary evaluations support the effectiveness of E-R Search beyond reference-based reconstruction accuracy. Our preliminary human evaluation further shows a consistent preference for E-R Search over single-sample generation across all three difficulty levels. We will include these results and expand the human evaluation in the revised manuscript.

AuthorsOpenReview ↗

Author Response — Part 3/3

W4. Understanding the Gap Between Practical and Oracle Performance

We thank the reviewer for this insightful suggestion. To identify whether the practical–Oracle gap mainly arises from imperfect candidate selection, refinement decisions, or conditioning errors, we decompose E-R Search into three decision stages: (1) initial set selection (S), (2) uncertain-view identification (M), and (3) final candidate selection (F).

We conduct a controlled ablation by independently replacing DINO-centroid with Oracle verification at each stage under matched sampling budgets, while keeping the refinement procedure unchanged. The results at N=64 are summarized below.

Initial Selection (S)View Identification (M)Final Selection (F)Easy ↑Normal ↑Hard ↑
DINODINODINO8.1111.608.98
OracleDINODINO15.7218.0915.63
DINOOracleDINO8.7912.059.65
DINODINOOracle9.2713.0610.24
OracleOracleOracle20.1022.7019.90

LPIPS improvement (%) relative to the four-sample average baseline; higher is better. Oracle verification uses ground-truth LPIPS for selection decisions without providing ground-truth observations during refinement.

The results indicate that imperfect initial candidate selection is the dominant source of the practical–Oracle gap. On the Normal subset, replacing DINO-centroid with Oracle at the initial selection stage improves LPIPS gain from 11.60% to 18.09%, an increase of 6.49 percentage points. In contrast, using Oracle only for uncertain-view identification or final candidate selection yields additional gains of just 0.45 and 1.46 percentage points, respectively. Similar trends hold across all difficulty levels.

Importantly, the refinement procedure and conditioning mechanism remain unchanged across these ablations, indicating that the largest observed gap arises from imperfect verification of the initial multi-view hypothesis rather than differences in the refinement algorithm itself. Nevertheless, conditioning quality also affects refinement performance, as demonstrated by the GT-observation experiments in Fig. 6. These experiments reveal substantial additional potential when more accurate observations become available, although their contribution cannot be quantitatively isolated by the present stage-wise ablation.

Overall, our analysis identifies initial candidate selection as the primary practical bottleneck, while highlighting observation quality as another important factor influencing refinement. These findings suggest that improving complete-set verification and reliable observation selection offers promising directions for further advances in E-R Search. We will include these additional ablation results and analyses in the revised manuscript.

Q1. Does the Scaling Behavior of DINO-centroid Eventually Saturate?

We thank the reviewer for this insightful question. We extended our experiments to N=256. While DINO-centroid continues to become more similar to GT, the incremental gains at larger sampling budgets are substantially smaller than those at smaller budgets, indicating diminishing returns and a tendency toward saturation rather than linear scaling with log(N).

We expect the similarity to eventually plateau below 1, since increasing the sampling budget can reduce estimation variance but cannot necessarily eliminate the underlying distributional bias. These observations suggest that DINO-centroid gradually approaches a saturation point, where further increasing the sampling budget yields diminishing improvements in similarity to GT.

Reviewer_KFPzOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

The paper studies how generation, verification, and search interact in multi-view diffusion. DINO-centroid compares candidates with their shared feature centroid to select a view set and identify views for refinement. Exploration-Refinement Search regenerates those views while retaining others, under a fixed number of diffusion-model evaluations. The paper also points out a contradiction: selecting the entire set preserves consistency across viewpoints, whereas selecting on a per-viewpoint basis—while superior when ground truth is available—compromises that consistency.

Critical strengths
  1. The study identifies a meaningful trade-off between selecting coherent view sets and independently choosing better-looking individual views.
  2. The problem framework is clear: "How should one design the generation, verification, and search processes when the output components are interdependent?" is a valuable question. The distinction that "the granularity used for scoring need not be the same as the granularity used for selection" is also insightful.
  3. The DINO-centroid method is simple and cost-effective; its scoring can be applied both to full-set screening and to targeted refinement. Experimental results confirm the effectiveness of the proposed search strategy.
Critical weaknesses
  1. The DINO-centroid approach essentially involves selecting the most "typical" sample within the feature space, which is a concept aligned with self-consistency, MBR decoding, and medoid selection. This relationship should be discussed and compared.

  2. Although the sample mean becomes more stable and closer to the true value as NN increases, this is a statistical property of averaging and does not necessarily imply that the accuracy of the selected sample improves. Selecting the sample closest to the mean inherently biases the result toward conservative outcomes near the conditional mean.

  3. The experimental section relies solely on the GSO dataset and is built primarily upon SyncDreamer (2023, at 256 resolution). Furthermore, 3D reconstruction is performed only on the "normal" subset, which fails to adequately demonstrate the general applicability of the proposed method.

Questions for authors
  1. How does consensus behave when candidates share an incorrect unseen geometry or contain several plausible modes?
  2. For the "Easy/Normal/Hard" categorization, is the single sample used for sorting the same as the N=1N=1 baseline employed for calculating gain? If so, would it be possible to re-partition the subsets using an independent sample—or, alternatively, to forgo partitioning altogether and report results on the full dataset?
4 comments
AuthorsOpenReview ↗

Author Response — Part 1/4

We thank the reviewer for the thoughtful and constructive feedback. We address the concerns and questions below.

W1. Relationship to Self-Consistency, MBR Decoding, and Medoid Selection

We thank the reviewer for pointing out these important connections. We agree that DINO-centroid is closely related to established consensus-based selection approaches, including self-consistency, minimum Bayes risk (MBR) decoding, and medoid selection. We will make these connections more explicit in the revised manuscript.

Relationship to MBR and medoid selection. In particular, for L2-normalized DINO features, ranking candidates by their cosine similarity to the centroid at a fixed viewpoint is mathematically equivalent to ranking them by their average pairwise cosine similarity to the other candidates. Specifically, let zi(v)\mathbf{z}_i^{(v)} denote the normalized feature of candidate ii at viewpoint vv, and define the empirical MBR similarity as

= \frac{1}{K-1}\sum_{j\neq i} \left(\mathbf{z}_i^{(v)}\right)^\top \mathbf{z}_j^{(v)}.$$ Since the features are unit-normalized, our centroid-based score can be written as $$S_{i,v}^{\mathrm{DINO}} = (\mathbf{z}_i^{(v)})^{\top} \frac{\sum_{j=1}^{K}\mathbf{z}_j^{(v)}}{\lVert\sum_{j=1}^{K}\mathbf{z}_j^{(v)}\rVert_2} = \frac{1+(K-1)S_{i,v}^{\mathrm{MBR}}}{\lVert\sum_{j=1}^{K}\mathbf{z}_j^{(v)}\rVert_2}.$$ The denominator is shared by all candidates at a fixed viewpoint. Therefore, the two criteria produce exactly the same candidate ranking. We thus acknowledge that the underlying single-view consensus-selection principle is not new and can be interpreted as empirical MBR or medoid selection in DINO feature space. **Multi-view aggregation and additional comparison.** For complete multi-view sets, however, our method independently normalizes the centroid at each viewpoint before averaging the viewpoint-level scores. This introduces viewpoint-dependent weights proportional to the inverse norm of the corresponding unnormalized centroid. In contrast, unweighted MBR simply averages pairwise cosine similarities across viewpoints. Consequently, the two methods are not generally equivalent for complete-set selection. To isolate the contribution of this aggregation strategy, we conduct an additional controlled comparison between DINO-centroid and unweighted average pairwise cosine similarity (MBR), using the same candidate pools, DINO features, and complete-set selection protocol. Table X reports LPIPS improvements (%) relative to single-sample generation, where higher values indicate better performance. | Difficulty | Method | N=4 | N=16 | N=64 | |:--|:--|--:|--:|--:| | Easy | Unweighted MBR | 2.12 | 2.61 | 3.15 | | | DINO-centroid | **3.24** | **3.78** | **4.86** | | Normal | Unweighted MBR | 2.80 | 3.37 | 5.29 | | | DINO-centroid | **3.71** | **5.83** | **8.21** | | Hard | Unweighted MBR | 2.04 | 4.25 | 4.70 | | | DINO-centroid | **2.69** | **4.64** | **5.74** | DINO-centroid consistently outperforms unweighted MBR across all three difficulty levels and sampling budgets, achieving an average improvement of **1.37%** in LPIPS gain across the nine settings. These results suggest that per-view centroid normalization provides beneficial adaptive weighting of viewpoint-level consensus, improving complete-set selection beyond simply averaging pairwise similarities. We will clarify the relationship to established consensus-selection methods, incorporate the additional comparison, and better distinguish the established selection principle from our contributions to multi-view verification and search.
AuthorsOpenReview ↗

Author Response — Part 2/4

W2. DINO-centroid, and Selection Bias

We agree that improved centroid-to-GT similarity alone does not guarantee better candidate selection. Fig. 3 shows that DINO-centroid-guided Best-of-N improves the LPIPS of selected generations, but increasing NN simultaneously enlarges the candidate pool and improves centroid estimation, making their individual contributions difficult to disentangle.

To isolate the effect of centroid estimation, we draw four fresh candidates for each object, independently of all three centroid-estimation pools. This selection pool is disjoint from every centroid-estimation pool. We construct centroids from 4, 16, or 64 original samples and use each centroid to rank the same four newly sampled candidates.

DifficultyCentroid N=4Centroid N=16Centroid N=64
Easy3.183.644.39
Normal3.735.476.90
Hard2.724.025.15
Average3.214.385.48

LPIPS improvement (%) relative to the four-sample average baseline; higher is better.

With the candidate pool fixed, increasing the centroid estimation budget consistently improves selection performance across all difficulty levels, increasing the average LPIPS gain from 3.21% to 5.48%. This directly demonstrates that improved centroid estimation independently contributes to better candidate selection, beyond the benefit of enlarging the candidate pool.

Regarding the tendency toward conservative outcomes, DINO-centroid selects an existing complete multi-view candidate rather than synthesizing an averaged output. Furthermore, our method aggregates scores across viewpoints for set-level selection. Even when the consensus at an individual viewpoint is biased toward an incorrect or dominant mode, the remaining viewpoints can provide complementary evidence for candidate selection, mitigating the influence of unreliable local consensus.

We will clarify these findings and limitations in the revised manuscript.

AuthorsOpenReview ↗

Author Response — Part 3/4

W3. Dataset Coverage, Backbones, and Downstream Reconstruction

We thank the reviewer for raising these concerns about the scope of our evaluation.

Dataset coverage. We acknowledge that our evaluation is limited to GSO. We chose GSO because its reference 3D geometry lets us evaluate multi-view generation and downstream reconstruction on the same objects under a consistent protocol. We agree that evaluation on additional datasets is needed to assess cross-dataset generalization.

Scaling across backbones. Appendix H (Table 2) extends the Oracle scaling analysis to Zero123-XL, Stable Zero123, Zero123++, and MV-Adapter. The results show that the scaling trends extend beyond the initial comparison between Zero-1-to-3 and SyncDreamer.

Verification and search across backbones. On MV-Adapter, DINO-centroid consistently outperforms the evaluated practical verifiers, and E-R Search achieves the strongest performance among the compared search strategies at matched budgets (Appendix I, Tables 3–4). At N=64N=64, E-R Search improves LPIPS by 11.41%, compared with 7.98% for DINO-centroid-guided Best-of-N. We further evaluate MV-Adapter on Easy and Hard. E-R Search improves over Best-of-N on both backbones across all three difficulty levels at N=4N=4 and N=16N=16.

BackboneMethodEasy N=4Easy N=16Normal N=4Normal N=16Hard N=4Hard N=16
SyncDreamerBoN (DINO-centroid)3.243.783.715.832.694.64
SyncDreamerE-R Search5.507.126.128.015.277.71
MV-AdapterBoN (DINO-centroid)3.513.834.065.722.424.60
MV-AdapterE-R Search5.617.105.878.384.957.89

LPIPS improvement (%) relative to the four-sample average baseline; higher is better.

Downstream reconstruction. E-R Search improves MEt3R over Best-of-N on Easy, Normal, and Hard (Appendix J, Table 5). Following your suggestion, we extended the downstream reconstruction evaluation in Appendix K (Table 6) to Easy and Hard at a sampling budget of N=64N = 64. Across the three subsets, E-R Search reduces Chamfer Distance by 3.6–4.9% and increases Volume IoU by 1.5–1.7 percentage points relative to Best-of-N.

MethodEasy Chamfer Dist. ↓Easy Volume IoU ↑Normal Chamfer Dist. ↓Normal Volume IoU ↑Hard Chamfer Dist. ↓Hard Volume IoU ↑
SyncDreamer0.02550.5800.02740.5370.03050.508
BoN0.02490.5920.02650.5510.02860.525
E-R Search0.02400.6070.02520.5680.02750.540

Human evaluation on GSO. Three graduate students independently evaluated all 30 objects (10 per difficulty level). For each object, the baseline was randomly drawn from four initial multi-view candidates, and E-R Search used the same pool for exploration before conditional refinement. E-R Search was preferred in 80.0%, 83.3%, and 90.0% of judgments on Easy, Normal, and Hard, respectively, with 84.4% overall preference (76/90 votes). We will report these preliminary results on perceived quality within GSO and expand the study in the revised manuscript.

AuthorsOpenReview ↗

Author Response — Part 4/4

Q1. How does consensus behave when candidates share an incorrect unseen geometry or contain several plausible modes?

We thank the reviewer for raising this important concern. We distinguish two scenarios:

(1) Shared incorrect unseen geometry. When multiple candidates share the same incorrect geometry at a particular viewpoint, their features may form a biased consensus, potentially assigning high scores to incorrect views. However, DINO-centroid does not independently select views based on local consensus. Instead, it aggregates scores across all viewpoints to select a complete multi-view set. Therefore, even if the consensus at one viewpoint is incorrect, information from the remaining viewpoints can help mitigate its influence on the final selection. Nevertheless, if the incorrect geometry is shared consistently across most candidates and viewpoints, consensus alone cannot reliably identify the error.

(2) Multiple plausible modes. When candidates represent different plausible geometries for unseen regions, their features may form multiple clusters. In this case, the centroid tends to reflect the more frequently sampled modes, potentially favoring dominant hypotheses over less frequent but equally plausible alternatives. However, DINO-centroid selects an existing complete multi-view sample rather than averaging generated images, thereby preserving the internal consistency of the selected hypothesis. This does not eliminate the preference for dominant modes, but avoids constructing an artificial mixture of incompatible geometries.

We will clarify these behaviors and the limitations of consensus-based verification in the revised manuscript.

Q2. Is the sample used for difficulty partitioning also used to calculate the single-sample baseline?

We thank the reviewer for raising this concern. We understand the potential bias: if the same sample used to identify Hard objects were also used as the evaluation baseline, its unusually poor reconstruction quality could artificially inflate the reported improvements.

However, the difficulty subsets are constructed using a separate generation draw. The evaluation baseline is estimated by averaging metrics over four additional independent draws per object, none of which is used for partitioning. Therefore, the reported improvements are evaluated against an independent baseline rather than the partition-defining sample, avoiding the potential bias described by the reviewer.

We will clarify this evaluation protocol in the revised manuscript.

We thank the reviewer again for the constructive suggestions. We will strengthen the discussion of consensus-based verification and its limitations, incorporate the additional verifier comparisons and human evaluation, and clarify the dataset construction and generalization scope.

Reviewer_VNswOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper studies test-time scaling for multi-view diffusion models conditioned on a single image, and splits the question into generation, verification, and search. For generation, it compares a per-view model, Zero-1-to-3, with a joint model, SyncDreamer, under Oracle Best-of-N selection that picks each view by its LPIPS to the ground truth. The joint model gains more as the number of samples grows. For verification, it proposes DINO-centroid. At each viewpoint, the DINOv2 features of all candidates are averaged into a centroid, each candidate view is scored by its cosine similarity to that centroid, and the view scores are averaged into a score for the whole set. Against an aesthetic score, RoMa matching and MEt3R, it gives the best Best-of-N selection at the lowest cost, and its centroid moves closer to the ground-truth feature as more samples are drawn. For search, E-R Search draws several complete sets, picks one with DINO-centroid, and marks a view as uncertain whenever a different sample scored higher at that viewpoint. The uncertain views are then regenerated with Posterior-DDIM. On three subsets of GSO objects and at matched sampling budgets, it beats Best-of-N, view composition, beam search and a random partial-refinement baseline adapted from IPR

Critical strengths
  1. A useful framing for structured outputs. The paper separates how candidates are scored from how they are selected, and asks whether parts of independent joint samples can be recombined. Its answer is no, and it supports this with evidence beyond LPIPS: composing views from different samples clearly worsens MEt3R consistency in Fig. 5 and Table 5, and it hurts downstream 3D reconstruction in Table 6.
  2. A simple method that consistently beats the listed baselines. DINO-centroid is cheap and needs no training. E-R Search uses the same number of denoising trajectories as Best-of-N and improves on every baseline on both SyncDreamer and MV-Adapter. The ablation in Table 7 shows that choosing which views to regenerate with the verifier helps compared with choosing them at random.
Critical weaknesses
  1. The gains are small, the backbone is dated, and the baselines are weak. The practical improvement is modest. With 64 samples, E-R Search beats Best-of-N by about three points of relative LPIPS gain on each subset, and its total improvement over a single sample stays around ten percent for sixty-four times the compute. The improvement in downstream 3D reconstruction over Best-of-N is also small. All main experiments use SyncDreamer, a 2023 model. MV-Adapter, the only other backbone, is tested on the Normal subset alone. No recent and stronger multi-view generator is evaluated, for example a model built on video diffusion such as SV3D, which the paper cites. Test-time gains usually shrink as the base model improves, so it is unclear whether the conclusions and margins hold for current models. None of the practical verifiers in the comparison looks at the input image, and two of them were never meant to measure correctness: an aesthetic predictor and a count of matched points. Stronger options are missing, such as DINO or CLIP similarity to the input image, a VLM judge, or a check that reconstructs the object and renders it back.

  2. The central tension and several Oracle results are guaranteed by how the evaluation is built. Quality is measured as PSNR, SSIM and LPIPS against a single ground truth for each view, averaged over views, and the Oracle verifier selects with the same LPIPS. When a metric is an average over views, picking the best candidate at each view can never score worse than picking the best complete set. The finding that the trend reverses under Oracle verification in Fig. 7, and the fundamental tension stated in the abstract, therefore hold by construction. Because the metric ignores consistency between views, the Oracle result also says nothing about real room for local improvement, yet much of the motivation for E-R Search rests on it. Fig. 6 has a similar problem. Its large potential comes from feeding ground-truth target views in as observations, which never happens at test time, and the realistic version with views chosen by DINO-centroid gains only about three percent. The paper argues that the task is ambiguous, with several plausible answers for the unseen views, as Fig. 11 illustrates. Yet every result is scored against one ground truth. Such a metric naturally rewards a verifier that seeks the consensus mode and penalizes plausible minority answers. The verifier scaling in Fig. 4, where the mean of many features lies closer to the ground-truth feature than any single sample, is the expected effect of averaging and adds little.

Questions for authors

Q1. Can you report absolute PSNR, SSIM and LPIPS for a single sample, Best-of-N and E-R Search? Q2. With a metric averaged over views, view-level Oracle selection is at least as good as set-level selection by construction. What evidence for the tension goes beyond this? Does view-level Oracle selection still win when judged by a consistency metric or by 3D reconstruction? Q3. Is the single-sample baseline the same sample that was used to sort objects into Easy, Normal and Hard? If so, please recompute the per-subset gains against a freshly drawn baseline. Q4. In the final selection of E-R Search, is the centroid computed from the explored sets only, or from all candidates including the refined ones? The refined candidates share their reliable views with the reference set, so including them would pull the centroid toward that set and make the choice self-reinforcing. Each candidate is also part of the centroid it is scored against. Does a leave-one-out centroid change the results when few samples are drawn?

5 comments
AuthorsOpenReview ↗

Author Response — Part 1/5

We thank the reviewer for the thoughtful and constructive feedback. We address the concerns and questions below.

W1. Performance Gains, Backbone Selection, and Stronger Verifiers

We thank the reviewer for these comments and would like to clarify three points.

(1) Performance gains. We agree that the additional gain of E-R Search over DINO-centroid-guided Best-of-N is relatively modest. However, the two comparisons reflect complementary contributions of our work: An effective verification mechanism DINO-centroid enables practical search strategies such as trivial Best-of-N competitive; (2) A brand-new search strategy E-R further improves the allocation of generation budget. Importantly, DINO-centroid-guided Best-of-N already achieves substantial improvements over single-sample generation (8.21% improvement, N=64N = 64), which E-R Search further builds upon.

The additional gain from E-R Search is smaller because it focuses on targeted local refinement (R=16R = 16trials) of an already selected multi-view hypothesis from K=48K = 48 trials, whereas Best-of-N benefits from exploring diverse complete hypotheses (N=64N = 64tirals). These two stages serve different purposes and naturally offer different improvement opportunities. Importantly, our key finding is that allocating part of the sampling budget to targeted refinement is more effective than spending the entire budget on independent global exploration. This suggests a promising direction for future research on adaptive allocation of test-time computation between exploration and refinement for structured output generation.

(2) Choice of backbone. We selected SyncDreamer not because it produces larger TTS gains, but to enable systematic TTS experiments within available computational budget. In our experiments, SV3D requires approximately 3–4 times the sampling time of SyncDreamer. Given the extensive comparisons across multiple search strategies and sampling budgets up to N=64, SyncDreamer allowed us to conduct a more comprehensive and controlled study under practical computational constraints. Furthermore, Appendices H and I provide additional experiments on MV-Adapter (ICCV 2025), a more recent joint multi-view generator. We further extend the evaluation to the Easy and Hard subsets and compare its TTS performance with SyncDreamer across different difficulty levels.

BackboneMethodEasy N=4Easy N=16Normal N=4Normal N=16Hard N=4Hard N=16
SyncDreamerBoN (DINO-centroid)3.243.783.715.832.694.64
SyncDreamerE-R Search5.507.126.128.015.277.71
MV-AdapterBoN (DINO-centroid)3.513.834.065.722.424.60
MV-AdapterE-R Search5.617.105.878.384.957.89

LPIPS improvement (%) relative to the four-sample average baseline; higher is better.

The results show that TTS gains remain comparable across both backbones, suggesting that these benefits do not necessarily diminish on newer generative models. This further supports the generalizability of our verification and search strategies beyond SyncDreamer. We will include the extended results in the revised manuscript.

AuthorsOpenReview ↗

Author Response — Part 2/5

(3) Stronger verifier comparisons. We appreciate the suggested alternatives. In fact, we have explored several additional verification strategies during the early stages of our study, including DINO similarity to the input image, Cosmos 3-based VLM verification, and reconstruction-and-rendering verification using InstantMesh. Their results were either inconsistent or showed limited improvements as the sampling budget increased. Therefore, we did not further extend these verifiers to N=64 and didn’t report them in our manuscript. We report their comparisons across all three difficulty levels at N=4 and N=16 below.

VerifierEasy N=4 ↑Easy N=16 ↑Normal N=4 ↑Normal N=16 ↑Hard N=4 ↑Hard N=16 ↑
DINO input similarity-0.571.161.581.902.033.27
Cosmos 33.411.253.590.742.880.70
InstantMesh-based verification0.891.471.631.912.723.04
DINO-centroid3.243.783.715.832.694.64

LPIPS improvement (%) relative to the four-sample average baseline; higher is better.

For reproducibility, we provide additional details of these verifiers:

  • DINO input similarity. We compute the DINO feature cosine similarity between each generated view and the input image, and average the similarities across all viewpoints to obtain the candidate score.

  • Cosmos 3. We provide the input image and a 4×4 grid containing the 16 generated views to Cosmos 3. The evaluation prompt assesses five aspects: object identity recognizability, color and brightness consistency, geometric and physical consistency, part-structure consistency, and semantic detail preservation. Each aspect is assigned a fine-grained score between 0 and 100, and the overall consistency score is used for candidate ranking. The full evaluation prompt and output format will be provided in the revised manuscript or supplementary material.

  • InstantMesh-based verification. We use the official InstantMesh implementation with its default reconstruction settings to obtain a 3D representation. We then render the reconstructed result from the corresponding viewpoints and compute the residuals between the rendered views and the generated views. These residuals are used to assess candidate consistency.

The results show that DINO input similarity provides limited improvements, as it relies only on information from the single input image. Cosmos 3 can be competitive with DINO-centroid at smaller sampling budgets, but its performance does not consistently improve as the budget increases. This may be because it mainly identifies samples with prominent errors and provides less fine-grained discrimination between views. InstantMesh-based verification also provides some improvements, but does not match DINO-centroid and incurs substantially higher computational costs. These limitations motivated our choice of DINO-centroid as a practical verifier for large-scale test-time sampling.

We also clarify the reviewer's concern that none of the practical verifiers considers the input image. Our Cross-view Matching verifier incorporates the input image through cyclic view matching, where the input image participates in the correspondence evaluation. We will clarify this implementation detail in the revised manuscript.

Overall, our contribution lies in jointly studying verification and search for multi-view diffusion, with DINO-centroid and E-R Search providing complementary improvements.

AuthorsOpenReview ↗

Author Response — Part 3/5

W2. Oracle Analysis, Evaluation Ambiguity, GT-Conditioned Refinement, and Verifier Scaling

We thank the reviewer for these insightful comments.

(1) Oracle selection and cross-view consistency. We agree that view-level Oracle selection is mathematically guaranteed to achieve no worse average LPIPS than set-level Oracle selection. However, our key finding is the reversal of this trend under practical verification: while view-level selection outperforms set-level selection with Oracle verification, view-level selection’s performance deteriorates substantially when using practical verifiers, making set-level selection more effective.

Moreover, independent view selection can compromise cross-view consistency. As shown in Appendix J (Table 5), on the Normal subset at N=64, View-composed Best-of-N achieves a worse MEt3R score of 0.508 compared with 0.441 for set-level Best-of-N. In contrast, E-R Search achieves 0.434 by preserving reliable views from a coherent reference set and conditionally refining uncertain views. These findings motivate our E-R search strategy, which seeks to improve individual views without sacrificing global consistency.

(2) Role of GT-conditioned refinement. We agree that the substantial gains in Fig. 6 rely on ground-truth observations unavailable at inference time. However, Fig. 6 serves as a controlled diagnostic experiment rather than a prediction of attainable test-time gains. By evaluating only the unobserved viewpoints, it demonstrates that the joint diffusion prior can exploit information from reliable observations to improve the remaining views, motivating conditional regeneration as an alternative to independent view composition.

Importantly, our exploration results show that additional sampling and verification improve the quality of selected multi-view hypotheses, providing a practical source of observations for refinement. To further investigate this connection, we conduct an Oracle ablation of the three verification stages in E-R Search:

Initial set selectionView identificationFinal selectionEasy ↑Normal ↑Hard ↑
DINODINODINO8.1111.608.98
OracleDINODINO15.7218.0915.63
DINOOracleDINO8.7912.059.65
DINODINOOracle9.2713.0610.24
OracleOracleOracle20.1022.7019.90

LPIPS improvement (%) relative to the four-sample average baseline; higher is better.

Notably, improving initial set selection alone substantially increases the performance of E-R Search across all three subsets, yielding additional gains of 6.49–7.61 percentage points without introducing GT views as observations. This provides more direct evidence that E-R Search can benefit from better candidate selection within the existing generator's distribution, rather than relying on hypothetical access to GT observations.

Finally, the practical effectiveness of E-R Search is established by the matched-budget comparisons in Fig. 8. We will clarify the diagnostic role of Fig. 6 and include this additional ablation in the revised manuscript.

(3) Single-reference evaluation. We acknowledge the inherent ambiguity of unseen views. However, LPIPS, PSNR, and SSIM are widely adopted evaluation metrics in multi-view generation, and our evaluation is not limited to these single-reference metrics. As reported in the manuscript, Appendix J (Table 5) additionally evaluates reference-free cross-view consistency using MEt3R, while Appendix K (Table 6) assesses downstream 3D reconstruction quality using Chamfer Distance and Volume IoU. On the Normal subset at N=64, E-R Search improves MEt3R from 0.441 to 0.434, reduces Chamfer Distance from 0.0265 to 0.0252, and increases Volume IoU from 0.551 to 0.568 compared with Best-of-N.

Furthermore, we conducted a preliminary pairwise human evaluation involving 30 objects (10 per difficulty level) and three graduate students, each independently evaluating all 30 objects. For each object, we generated four initial multi-view candidates using different random seeds. The baseline was randomly selected from these four candidates, while the E-R Search result was obtained by using the same candidate pool for exploration, followed by conditional refinement. E-R Search was preferred in 80.0%, 83.3%, and 90.0% of judgments on Easy, Normal, and Hard, respectively, achieving an overall preference rate of 84.4% (76/90 votes). These results provide additional evidence of the effectiveness of E-R Search beyond reference-based reconstruction accuracy. We will include these findings and expand the human evaluation in the revised manuscript.

Comments on the paper

1 comment
AuthorsOpenReview ↗

Summary of Manuscript Revisions and Reviewer Discussion

We thank all reviewers for their thoughtful feedback and the time they devoted to our work. We have responded to each point raised during the discussion and marked the revisions in blue.

P1. Difficulty splits and baseline. We clarified that we use LPIPS from one initial generation per object to identify the Easy, Normal and Hard subsets. We estimate single-sample baseline performance separately by averaging the scores of four independent samples per object.

P2. Backbone generalization. We added experiments with MV-Adapter (ICCV 2025), a more advanced joint multi-view backbone, on all three splits at three sampling budgets (N=4,16,64N=4, 16, 64). Across all nine settings, DINO-centroid improves candidate selection, and E-R Search further reduces LPIPS over DINO-centroid-guided Best-of-NN at the same budget.

P3. Absolute metrics. The main comparisons now report absolute PSNR, SSIM, and LPIPS alongside relative LPIPS gains.

P4. Magnitude of the gains. Our work contributes both the DINO-centroid verifier and the E-R Search strategy. On the Normal subset at N=64N=64, Best-of-NN with our verifier already reduces LPIPS by 8.21% relative to single-sample generation. E-R Search brings the total reduction to 11.60%, adding 3.39 percentage points at the same sampling budget.

P5. Additional analyses. We expanded the downstream 3D reconstruction evaluation, examined cases where DINO-centroid favors a frequent but incorrect hypothesis, and added a stage-wise Oracle ablation that identifies initial set selection as the critical stage at N=64N=64.

Following the discussion, Reviewer VNsw raised his/her rating to 3 (Weak Accept), and Reviewer KFPz maintained his/her rating of 3 (Weak Accept). We sincerely appreciate their positive recommendation. Reviewer 6LJD expressed some reservations about the magnitude of the gains (see P4), maintaining rating of 2 but indicating that he/she would be open to acceptance. We are grateful to all the reviewers for their efforts to improve the quality of the manuscript!

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.