acceptodds
Under review as a conference paper at ICLR 2027

WEAVE: Warp Every Angle via 3D Geometry for Reference-Guided Image Completion

10 views

Abstract

Reference-guided image completion uses auxiliary views of a scene to reconstruct missing image content faithfully. Existing approaches often require per-scene optimization, pairwise alignment, or a fixed number of reference images, limiting their flexibility and practical use. We introduce WEAVE, a feed-forward framework that supports variable numbers of references without per-scene fine-tuning. WEAVE estimates camera poses and depths in a single forward pass and uses them to reproject each reference into the target view. These projections form two complementary conditioning inputs: a merged query warp (Q-Warp) providing a target-aligned estimate, and separate reference warps (R-Warps) retaining view-specific details and disagreements that merging would discard. WEAVE combines these inputs with the original references to resolve cross-view discrepancies and synthesize the missing content. Experiments on MegaDepth and RealBench show consistent improvements with both MM-DiT and U-Net backbones. WEAVE achieves state-of-the-art reconstruction performance on both benchmarks, outperforming scene-finetuned methods on RealBench without test-time optimization.

Primary area:
generative models
Keywords:
multi-image, inpainting, outpainting, reference-guided

Reviews

Among:
Shape:
1234this paper, Top 15%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.00 over 3 reviews, above 85% of the others.
Reviewer_MUgpOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper presents WEAVE – a method for image completion with arbitrary number of reference images in a feed-forward manner. The method extracts scene geometry using VGGT-Omega then warps the query and reference images. Using the warped and original images it utilizes MM-DiT arch to predict the complete in painted image.

Critical strengths
  1. The novelty of the paper is in the later part of the method, using the warped images to create the inpainting. This part is justified well by the ablation study and the U-Net adaptation, that shows the improvement comes from the warping and not from the model.
  2. The method shows strong results compared to the baseline with a large gap in PSNR and LPIPS. The example in figure 3 demonstrates it well and supports the claim of improvement with number of reference images.
Critical weaknesses
  1. Over claiming – “WEAVE: Warp Every Angle via 3D Geometry for Reference-Guided Image Completion” , “WEAVE estimates the geometry relating these views”. The method is based on VGGT-Omega which is a strong and legitimate backbone, but that is not something the method contribute. The paper should not claim as its own contribution the improvements of a frozen backbone, I found it misleading.
Questions for authors

In figure 2 there is a “text tokens” input to the MM-DiT, but it is not clear what is this input and how it is used. I only found a study about it in the appendix, but it is still not clear how it is applied. Can you please elaborate?

6 comments
AuthorsOpenReview ↗

Response to Reviewer MUgp (1/2)

We thank the reviewer for recognizing the novelty of our conditioning design, the strong empirical gains, and the value of the ablations across both the MM-DiT and U-Net implementations.

W1. Clarifying role of geometry

We agree with the reviewer that the distinction between the pretrained geometry estimator and WEAVE's contributions should be explicit. Our core contribution lies in the multi-view conditioning framework for reference guided completion, rather than proposing a new geometry estimator. To prevent misunderstanding, we revised the wording throughout the manuscript to clarify that WEAVE relies on camera poses and depths predicted by a frozen VGGT-Ω model.

We updated this framing in the Abstract [Lines 32–33], Introduction [Lines 67–69], Related Work [Lines 111–113], and Method [Lines 126–128, 142]. For example, the revised text explicitly states that WEAVE uses geometry predicted by a frozen VGGT-Ω\Omega model. Our technical contribution focuses on how these geometric estimates are structured for the generative backbone. Specifically, we combine a merged, target-aligned Q-Warp with separate R-Warps and raw references, which allows the generator to leverage geometric alignment while retaining fine-grained information from individual views. The title of our paper, “Warp Every Angle via 3D Geometry,” thus refers directly to this multi-view conditioning strategy, particularly the use of R-Warps to preserve evidence across different viewpoints.

AuthorsOpenReview ↗

Response to Reviewer MUgp (2/2)

Q1. Clarifying text tokens

Thank you for pointing out the need for a clearer explanation of the text tokens. To clarify, the text tokens represent a static task instruction passed to the backbone's frozen text encoder, and this same prompt is shared across all evaluations without any per-scene tuning. We have added this clarification to the main manuscript [Lines 207–209], rather than leaving it only in Appendix B.

These tokens are embeddings of the prompt: “Fill in the blacked-out region of the first image, keeping it faithful and consistent with the content, lighting, and details of the reference images” (Appendix B [Lines 813–815]). The embeddings are generated once by the backbone's frozen text encoder and injected through standard text-conditioning layers during denoising. While this instruction remains constant for all test scenes, scene-specific appearance and geometry are supplied by the Q-Warp, R-Warps, and raw reference images.

Dear Reviewer MUgp,

Thank you again for your review. We’re following up on the two clarifications in our rebuttal.

  • Role of VGGT-Ω. We revised the abstract, introduction, related work, and method to explicitly credit frozen VGGT-Ω\Omega for predicting camera poses and depth. WEAVE’s contribution is how it uses those predictions to guide completion, combining a merged target-view Q-Warp with separate R-Warps and raw references to retain evidence from individual views.
  • Clarification on text tokens. Sec. 3.1 now explains that the backbone’s frozen text encoder embeds a single completion instruction for use during denoising. The text tokens tell the model what task to perform, while the warps and raw references provide the scene-specific evidence for the completion.

When you have a chance, we’d welcome your thoughts on whether these revisions clarify both points. Thank you for your time.

Reviewer_9SyNOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

WEAVE addresses reference-guided image completion using a variable number of reference images without per-scene optimization. A frozen geometry estimator predicts camera parameters and depths from the masked target and references. The references are reprojected into the target view to construct a merged Q-Warp and separate R-Warps, which condition the generator alongside the original references. The framework is instantiated with FLUX and Stable Diffusion 2 Inpainting backbones. Experiments on MegaDepth and RealBench demonstrate strong reconstruction performance, supported by component ablations and analyses of reference count, ordering, alternative conditioning mechanisms, and inference efficiency.

Critical strengths
  1. The method provides a clear and practical way to incorporate geometric evidence into reference-guided completion. Separately retaining reprojected references allows the generator to access observations that a merged image would discard, while avoiding costly per-scene optimization.

  2. The experimental validation is extensive and convincing overall. Improvements are demonstrated with both MM-DiT and U-Net backbones, including comparisons against methods using the same SD2 backbone and generalization from MegaDepth to RealBench. The component ablations support the effectiveness of geometry-aligned conditioning, particularly R-Warps.

Critical weaknesses
  1. The practical advantage of variable-reference conditioning over simple adaptations of fixed-reference models is not fully established. Table 6 provides useful comparisons of training reference counts, but an explicit repetition-padding baseline would clarify whether a fixed four-reference model can handle fewer references effectively by duplicating available images. This matters because variable-reference support is a central contribution, while the multi-reference comparisons with LeftRefill and CorrFill use only one reference for those baselines.

  2. Robustness to non-rigid changes between the target and references remains insufficiently characterized. RealBench includes subject-pose differences, whereas camera-and-depth reprojection alone cannot align independently moving or articulated subjects. In such cases, projected evidence may conflict with the visible target context. The current evaluation does not isolate these cases, leaving uncertainty about how reliably the proposed conditioning handles this important source of correspondence error.

Questions for authors
  1. Reference-count evaluation: Does Table 6 use repetition padding when evaluating fixed-count models with fewer references than their training count? If not, could you compare against this simple strategy, using a predefined duplication rule, to clarify the benefit of variable-count training?

  2. Non-rigid changes: How does WEAVE handle cases where the masked subject has a different pose in the target and references, particularly when visible target boundaries conflict with the warps? Representative examples or an analysis of existing test cases would help assess this limitation. Please also discuss the relationship to human-focused completion methods such as CompleteMe, which explicitly consider cross-pose appearance correspondence.

  3. Appendix B specifies the instruction and compositing procedure for WEAVE. Were the same instruction and outside-mask compositing used for GPT Image 2 and Nano Banana Pro, and how were masks provided to these models?

  4. Camera conditioning: Have you considered adding camera poses or Plücker embeddings alongside the warps? Could they provide complementary information, particularly for interpreting raw references?

10 comments
AuthorsOpenReview ↗

Response to Reviewer 9SyN (1/4)

We thank the reviewer for acknowledging the practicality of our geometry-guided approach, the benefits of retaining separately reprojected references, and our extensive experimental validation across backbones, baselines, and datasets.

W1, Q1. Fixed-reference model adaptations

Thank you for suggesting this comparison. In Table 7 (originally Table 6) of Appendix C.1 [Lines 889–903], we evaluate fixed-count models directly with the available references, without repetition padding.

To investigate whether repeating inputs helps a model trained exclusively with four references, we conducted an additional study evaluating this fixed-4 checkpoint with cyclic padding. We find that cyclic padding offers a minor improvement only at N=2N=2 while degrading performance at N=1N=1 and N=3N=3. Across all reference counts, WEAVE consistently outperforms the fixed-4 model under both padding protocols.

We have also clarified in Table 1 [Lines 216–217, 237–238], Table 2 [Lines 272–273, 279–282] and Sec. 4.1 [Lines 263–267] that LeftRefill and CorrFill are evaluated using their available single-reference implementations. CorrFill is designed for one reference, while only LeftRefill's single-reference checkpoint is publicly available. We detail our experiment below.

—

We perform cyclic padding as follows: when the number of available references N<4N<4, we cyclically repeat the ordered references r1,…,rNr_1,\ldots,r_N to construct a 4 reference input:

N=1:(r1,r1,r1,r1),N=1: (r_1,r_1,r_1,r_1), N=2:(r1,r2,r1,r2),N=2: (r_1,r_2,r_1,r_2), N=3:(r1,r2,r3,r1),N=3: (r_1,r_2,r_3,r_1), N=4:(r1,r2,r3,r4).N=4: (r_1,r_2,r_3,r_4).

For N≥4N\geq4, we use all available references without padding. All experiments use the same RealBench scenes, reference ordering, and inference configuration. We report RealBench PSNR$_{\mathrm{mask}}$ in the table below.

Max. eval. refs. NNFixed-4, no paddingFixed-4, cyclic paddingWEAVE
115.2115.1916.91
217.0517.3317.84
317.4817.2218.17
417.9617.9618.29
517.9117.9118.35

As shown, cyclic padding increases the fixed-4 score at N=2N=2 (17.33 vs. 17.05 dB), but lowers it at N=1N=1 (15.19 vs. 15.21 dB) and N=3N=3 (17.22 vs. 17.48 dB). WEAVE outperforms both protocols at every reference count. Furthermore, cyclic padding adds redundant conditioning tokens without introducing new visual evidence, while WEAVE uses each supplied reference only once. These results confirm the practical value of variable-reference training beyond the baseline architectural capacity of our backbones to accept varying input lengths.

AuthorsOpenReview ↗

Response to Reviewer 9SyN (2/4)

W2, Q2. Robustness to non-rigid changes

We agree that camera-and-depth reprojection cannot fully align articulated or independently moving subjects. To evaluate WEAVE under these conditions, we have added Appendix E.2 [Line 1333], which describes zero-shot results on CompleteMe's human-completion benchmark. We find that WEAVE outperforms or matches CompleteMe on five out of seven metrics, despite operating without human-specific training, pose representations, or non-rigid correspondence. We provide experimental details below.

—

Since CompleteMe's evaluation code is not publicly available, we reevaluate its released predictions and the other baselines using the same evaluation protocol as our paper. Similarly to others datasets evaluated in our paper, we only give WEAVE as input the masked query and raw reference images when evaluating human completion without any other human-specific information. We report performance in the table below, which is the same as Table 14 [Lines 1350–1358].

MethodPSNR$_{\mathrm{mask}}$ ↑SSIM$_{\mathrm{mask}}$ ↑LPIPS$_{\mathrm{mask}}$ ↓DINO ↑DreamSim ↓CLIP-I ↑CLIP-T ↑
AnyDoor9.720.1730.6290.9150.1010.9050.318
Paint-by-Example12.480.2860.4850.9540.0470.9590.332
MimicBrush10.960.2280.5570.9340.0660.9510.333
LeftRefill12.730.3370.4900.9480.0550.9530.335
CompleteMe12.570.3000.4700.9580.0400.9720.338
WEAVE13.210.3490.479‾\underline{0.479}0.9600.0400.970‾\underline{0.970}0.339

WEAVE achieves the top performance on four out of seven metrics, plus one tie for best. Our model attains higher PSNRmaskPSNR_\mathrm{mask}​ (13.21 vs. 12.57) and SSIMmaskSSIM_\mathrm{mask}​ (0.349 vs. 0.300) than CompleteMe. WEAVE also achieves the best DINO and CLIP-T scores, ties with CompleteMe for first on DreamSim, and ranks closely second on LPIPSmaskLPIPS_\mathrm{mask} and CLIP-I. Importantly, CompleteMe is trained for human completion and uses reference human masks, while WEAVE is evaluated zero-shot without human parsing, pose information, or human-specific training.

Our qualitative comparisons in Fig. 16 demonstrate that WEAVE can recover plausible human poses and appearances despite substantial pose differences. When the reference pose conflicts with the visible target, WEAVE can complete the masked region in a way that follows the target pose. In Fig. 16 (row 6), the reference subject is crouching, but WEAVE reconstructs the missing shoulder and arm in line with the visible standing pose. In row 7, WEAVE reconstructs the missing arm hanging at the subject’s side, consistent with the standing target even though the reference shows the subject seated with the arm angled toward his lap. These examples show that WEAVE can accommodate substantial pose changes without explicit non-rigid alignment or human-specific adaptation.

We have added a discussion of CompleteMe in Related Work [Lines 98–102] to contextualize prior research on reference-guided human image completion.

AuthorsOpenReview ↗

Response to Reviewer 9SyN (3/4)

Q3. Instruction and mask compositing for foundation models

We thank the reviewer for the insightful question, and agree that consistency is important during evaluation. We have added details on the foundation-model evaluation protocols to Appendix B [Lines 860–883], clarifying that API models are evaluated using their provided interfaces, and that the completion instruction (Appendix B [Lines 813–815]) is identical across both API models, WEAVE, and all other text-conditioned baselines. We perform outside-mask composition for all model and baseline outputs so that only changes in the mask region are evaluated.

As a general rule, we use the same masked query, reference images, and completion instruction across all models. We also apply the same outside-mask compositing rule, retaining only the generated content within the hole while preserving the known target pixels. Each model is evaluated with a fixed protocol across all examples, without per-scene prompt tuning or output selection.

For the proprietary API models, we follow their supported input interfaces rather than imposing a common mask representation. Specifically, GPT Image 2 accepts an explicit RGBA edit mask with a transparent hole and opaque known region, while Nano Banana Pro uses the blacked-out query image to identify the missing region. We do not impose additional input restrictions on either model. As noted above and in Appendix B, we then perform outside-mask composition to the outputs for fair comparison across all baselines.

Reviewer_wWiNOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

Reference-guided image completion is an existing and important problem that aims to recover missing image regions using other images/views of the same scene. Existing methods often require per-scene optimization or support only fixed numbers of reference images, limiting practical usage. The proposed method uses a pretrained feed-forward geometry network (such as VGGT-Ω), to first estimate depth and pose/viewpoint, which are then used to warp the reference images into the target viewpoint. At each target pixel location, the frontmost projected reference observation is selected to fill supported missing regions, forming a merged target-view estimate called the Q-Warp. The Q-Warp, together with warped references (R-Warps), and original raw reference images jointly condition the diffusion model alongside the noisy target latents. Keeping the R-Warps separate preserves view-specific details to further reduce any disagreements due to merging. The framework supports variable numbers of references without per-scene optimization and is demonstrated with both MM-DiT and U-Net diffusion backbones. Results on MegaDepth (in-domain) and RealBench (zero-shot) show improved performance. Ablation studies support the usefulness of the conditioning components.

Critical strengths

The results are good, with the proposed method outperforming the evaluated baselines even when the reference and target images have very similar viewpoints. The method is intuitively well motivated, and the experiments, including the ablation study, demonstrate the usefulness of its different components. It is also computationally efficient and addresses an important, practical problem of interest to the community.

Critical weaknesses
  1. Most of the examples are with substantial overlap and similar viewpoints between the target and reference images (or atleast one reference image). How does the method perform under extreme viewpoint changes and inaccurate depth estimates. Since the Q-Warp selects the frontmost projected observation at each target pixel location, depth errors could cause incorrect reference pixel to be selected. Explicit examples showing whether the model can correct these errors using the separate R-Warps and original references would help clarify its robustness.

  2. Since, depending on the mask’s size and location, important geometric cues may be missing, potentially degrading the estimated geometry. It is not properly discussed in the paper. An analysis of this sensitivity would help clarify the method’s robustness.

Questions for authors

Line 367: "We find that additional references can help, but only when models can reconcile their viewpoints." ==> Why does adding references worsen performance for FLUX and Qwen, and is this degradation caused by difficulties in reconciling viewpoints? Further analysis would strengthen this claim. If such analysis is not feasible, qualitative examples illustrating these failures would help readers better understand the issue.

10 comments
AuthorsOpenReview ↗

Response to Reviewer wWiN (1/3)

We thank the reviewer for recognizing the intuitive motivation behind our method, its efficiency and practical relevance, and the thorough experimental validation across different viewpoint configurations.

W1. Robustness to extreme viewpoint changes and geometry errors

Thank you for suggesting this experiment. To address this concern, we added a new evaluation titled “Robustness to viewpoint changes and geometry errors” in Section 4.4 [Line 403] and Appendix C.3 [Line 953]. In summary, our findings show that WEAVE remains robust under both substantial camera rotations and severe geometry perturbations. Even under our strongest noise setting, WEAVE outperforms the best baseline (Nano Banana Pro) by 2.77 dB.

—

To evaluate viewpoint changes, we group RealBench scenes by the rotation angle between the target and its nearest reference (ΔRmin\Delta R_\mathrm{min}). All entries indicate PSNR$_\mathrm{mask}$ performance in dB.

ΔRmin\Delta R_\mathrm{min}WEAVE$_\mathrm{[UN]}$WEAVE
All scenes15.5818.35
< 5$^\circ$14.8317.29
[5$^\circ$, 10$^\circ$)15.4818.38
[10$^\circ$, 15$^\circ$)16.3319.77
[15$^\circ$, 20$^\circ$)17.2719.91
≥ 20$^\circ$16.0318.55

We find no consistent performance degradation as the viewpoint separation increases. Across all RealBench scenes, WEAVE’s per-scene Pearson correlation between ΔRmin⁡\Delta R_{\min} and PSNR$_{\mathrm{mask}}$ is r=+0.23r=+0.23 (p=0.20p=0.20); thus, we find no statistically significant evidence that performance systematically declines as the nearest reference becomes more challenging.

Even when the nearest reference is at least 20∘20^\circ away, WEAVE achieves 18.55 masked-region PSNR, comparable to its overall average of 18.35. WEAVE${_\mathrm{[UN]}}$ shows a similar trend (16.03 vs. 15.58). We have also included qualitative examples with wide viewpoint changes in Fig. 10 [Lines 1026–1040].

To further test sensitivity to geometry errors, we introduce perturbations to the estimated camera poses and depth scales used to build the Q-Warp and R-Warps, keeping the input images unchanged. Each jitter column reports the result for a pair of perturbation scales σpose,σdepth{\sigma_\mathrm{pose},\sigma_\mathrm{depth}}. An analysis in Appendix C.3 [Lines 1048–1061] and Fig. 9 [Lines 1003–1014] show that these values are on a meaningful calibrated scale, and the jitter range covers the typical-to-high levels of perturbation inherent in VGGT-Ω\Omega outputs. We also simulate missing warped evidence by removing a subset of R-Warps, randomly masking cells of a 4×44\times4 grid, or removing all warp inputs. Nano Banana Pro receives the same images throughout and serves as a fixed comparison.

MethodCleanJitter {0.005, 0.040}Jitter {0.010, 0.080}Jitter {0.020, 0.160}R-Warp subset4 ×\times 4 patchesAll warps removed
Nano Banana Pro (all clean)13.5613.5613.5613.5613.5613.5613.56
WEAVE$_\mathrm{[UN]}$15.5815.2814.8314.4315.2714.0012.15
WEAVE18.3517.8217.1616.3318.0116.5714.31

Even under the strongest perturbation, WEAVE achieves 16.33 masked-region PSNR​. This still surpasses Nano Banana Pro (13.56 dB), the next best baseline without per-scene fine-tuning, by 2.77 dB. The additional ablation experiments indicate that WEAVE handles geometry errors gracefully, though removing all warp inputs leads to a noticeable drop in quality. These results show that WEAVE remains effective under the tested viewpoint changes and geometry errors and that our approach contributes significantly to our performance.

Our qualitative findings reinforce this. In Fig. 11 [Lines 1080–1089], controlled perturbations to Q-Warp and R-Warps produce only small numerical variations and minimal visible differences in the final completions. In Fig. 12 [Lines 1090–1106], we show how R-Warps help correct Q-Warp errors. Without R-Warps, the network relies on the faulty Q-Warp, which distorts a neon sign in a real misprojection case and stretches a painting under controlled jitter. With R-Warps included, WEAVE reconstructs the neon sign more closely to the ground truth, and preserves the geometric structure of the paintings.

AuthorsOpenReview ↗

Response to Reviewer wWiN (2/3)

W2. Sensitivity to mask size and location

We agree that masks can conceal visual cues needed for accurate geometry estimation. To investigate this effect, we evaluated WEAVE under systematically varied mask locations and sizes on MegaDepth. In summary, our findings show that WEAVE remains robust across different mask positions, with performance depending primarily on the total area of missing content rather than the mask placement location.

—

Our original evaluation on MegaDepth follows CorrFill's match-aware masking protocol, which explicitly masks regions containing feature correspondences between the target and references. This makes the missing content deliberately reference dependent, and the benchmark thus already poses a meaningful geometry estimation challenge.

To isolate the effects of mask location and size, we keep the target and reference images fixed across all MegaDepth test scenes and altered only the mask. We evaluated three equal area holes placed on the left, center, and right of the frame, alongside a larger centered hole. For each mask configuration, we reran VGGT-Ω\Omega on the masked target and unchanged references, and then recomputed the Q-Warp and R-Warps before inpainting. This setup captures the end-to-end impact of masking on both geometry estimation and final completion.

We run Kornia LoFTR-Outdoors once on the unmasked target--reference pairs and retain matches with confidence >0.8>0.8. Pooling counts across all examples, we report for each mask

= \frac{N_{\mathrm{hidden}}}{N_{\mathrm{matched}}}, \qquad \mathrm{Match\ density} = \frac{N_{\mathrm{hidden}}}{A_{\mathrm{hole}}}\times 10^4,$$ where $N_{\mathrm{hidden}}$ counts matched target-side keypoints inside the holes, $N_{\mathrm{matched}}$ counts all matched target-side keypoints, and $A_{\mathrm{hole}}$ counts hole pixels. Results are shown in the table below. | Mask | Hole area | LoFTR matches hidden | Matches per $10^4$ hole pixels | PSNR\$_{\mathrm{mask}}\$ ↑ | LPIPS ↓ | Warp coverage | |---|---:|---:|---:|---:|---:|---:| | Original CorrFill | 23.2% | 52.0% | 63.6 | — | — | — | | Left | 25.0% | 39.4% | 44.6 | 19.40 | 0.052 | 82.2% | | Center | 25.0% | 49.1% | 55.7 | 20.27 | 0.047 | 90.3% | | Right | 25.0% | 39.3% | 44.5 | 19.26 | 0.053 | 81.3% | | Center, larger | 56.25% | 82.7% | 41.6 | 17.63 | 0.135 | 82.9% | For equal-area masks (25.0% hole area), WEAVE maintains consistent reconstruction quality across all three positions. The center hole conceals more LoFTR matches (49.1%) than the left (39.4%) or right (39.3%) holes, yet it achieves the highest warp coverage (90.3%) and the highest masked PSNR (20.27 dB). Moving the hole to either side lowers masked PSNR by only 0.87 to 1.01 dB. Increasing the center hole size to 56.25% causes a larger drop of 2.64 dB (down to 17.63 dB), but estimated warp coverage remains high at 82.9%. Across all four rectangular masks, the warps cover 81.3% to 90.3% of the missing area. Therefore, under the tested conditions, enlarging the centered hole reduced WEAVE’s reconstruction quality more than shifting it to the sides, consistent with less visible target context and a larger area without projected reference evidence.
AuthorsOpenReview ↗

Response to Reviewer wWiN (3/3)

Q1. Adding references vs. foundation model performance

We agree with the reviewer that performance drops observed when adding more references do not definitively prove that baselines struggle specifically with viewpoint reconciliation. Several factors could contribute to this behavior, and isolating the exact cause is particularly challenging for proprietary API models. To avoid overstating the cause of this decline, we have revised the discussion in Sec. 4.4 [Lines 375–377].

We nevertheless investigate this behavior with an empirical study below. Our findings confirm that adding raw, unaligned references does not reliably improve reconstruction quality across the evaluated foundation models. This motivates WEAVE’s explicit alignment step: reference observations are reprojected into the target view before conditioning the generator, while separate R-Warps and raw references preserve complementary evidence and details that merging may discard.

—

To quantify how often and by how much performance drops, we compare the one- and four-reference predictions for each MegaDepth test example. We report the mean PSNR difference (four minus one reference), the percentage of examples where four references perform worse, and the percentage where the drop is at least 1 dB.

Zero-shot modelMean Δ\DeltaPSNR (4 − 1 refs)Worse with 4 refsDrop ≥ 1 dB
Qwen Image Edit−1.90 dB66.4%49.4%
FLUX.2 [Klein] 4B−1.41 dB65.8%47.4%
FLUX.2 [Klein] 9B−1.55 dB69.6%48.6%
FLUX.2 [dev] 32B−3.66 dB78.6%62.4%

As shown above, we find that four references reduce reconstruction quality on 65.8–78.6% of examples across the evaluated models, suggesting widespread degradation. Qualitative inspection reveals failures in maintaining consistency with the target: Qwen sometimes changes the target layout or generates content from a different viewpoint, while FLUX sometimes introduces foreground details or lighting inconsistent with the target. These observations show that supplying additional raw references does not reliably improve target-view reconstruction in zero-shot settings and motivate our proposed method.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.