- Rating
- 3(+1)
- Confidence
- 3
- Informativeness
- 1
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
The work targets the task of image analogy where a transition is specified through an input image pair (A, A'), and the same transition should be applied to an input image B in order to create a target B'. In this work, the authors highlight a gap in existing analogy work, which struggles when the pair A, A' specify both an attribute change as well as a change in camera pose. Ideally, a user should be able to decide if they want to transfer only the attribute change, only the camera pose change, or both.
To enable this control, the authors create a set of analogy pairs with known camera estimates using a pipeline based on qwen-based multi angle generation for a sparse initial camera grid, followed by a 3d-lifting approach to rotate the viewpoint to additional views. This 3d-lifting pipeline degrades photorealism, which is later restored via a render-to-real editing pipeline.
Finally, this set is used to tune an image editing model (FLUX.2 Klein) such that it can also be conditioned on the camera pose for the input analogy images and the target via additional tokens. Additionally, the authors train a camera posterior prediction model that attempts to predict the poses for the source images and is then used to estimate the right pose for the target.
On the experimental side, the approach is compared to a set of analogy baselines as well as to leading closed-source API models and it shows remarkable performance (even beating the closed-source models on some of the metrics like camera accuracy). There is a large range of metrics and a large set of qualitative examples and a user study to further back them up.
Critical strengths
The paper's core strength, as I see it, is its aim to help with a large issue that image analogy models have: its very hard to get an analogy pair that only differs in the one attribute you actually care about (unless you can easily describe this attribute in text and do image editing on A, but in this case you don't need an analogy pair since you can just edit image B directly). As such, there's a clear benefit to a method that can disentangle different attributes that may have changed in the A, A' pair, and allow the user to select which of them they want to keep. This work takes a step in disentangling the camera pose, which is an important part of this.
Two further strengths, albeit weaker, are the introduction of a new benchmark for analogy-based editing, and the obvious boost in performance that this approach gives over the baselines. Finally, the ablation into how to condition the resulting model on camera poses may be useful for future work on this topic, but please see weaknesses below.
Critical weaknesses
On the technical side, one glaring weakness here is that qualitative results seem to use the synthetic samples from the distribution on which the authors trained, and which often do not actually look like real images. I don't know how well this approach actually works on real images (at any part of the A, A', B triplet). Some parts of the pipeline, like the camera estimators, may be particularly sensitive to this.
A second weakness is that this work is practically a camera conditioning paper, but it is obviously not the first work to condition generation and editing models on camera poses, and I don't know how well this approach fares compared to other camera control alternatives (for example, image to video models with controlled camera trajectories?).
As a corollary to this, while I noted that I like the desire to disentangle different aspects of the analogy pair, this work is very much designed specifically for camera poses and generalizing it to other attributes is non trivial, so I'm not sure this is even a big step towards that goal.
Finally, this is mostly a systems paper, which essentially combines existing ideas in a pipeline. This seems like a much better fit for a conference like SIGGRAPH and less to ICLR where there is a bigger highlight on works that teach the reader something fundamentally new, provide some new theoretical or empirical understanding, or demonstrate new applications. I personally don't feel like I've gained any new understanding from this work, but this is subjective so I will not insist on a negative score only over this point.
Very minor and didn't affect my score, but didn't know where to list it: Some citations need to be changed from /citep to /citet, for example Reed et all in line 116.
Questions for authors
My main request for authors would be to provide us with results using real images (2+ examples, or better yet some numerical results on a small set) or specify which of the conditioning triplets in the existing paper already use real images if these already exist and I missed them.
Beyond this I'd like to see some comparison to alternative camera conditioning approaches from the existing litterature (perhaps MetaView?) or api-based camera conditioned models (even if they are video models and you take the end frame of the trajectory).
Real images You did not miss anything: all test images in the paper come from our generation pipeline. We evaluated camera-only analogies built from CO3D, which have real ground truth (App. A.2, Table 3). Our model estimates the cameras, while SEVA and MetaView are given the ground-truth move.
OA2 error is 66.4° for ours, 41.0° for SEVA and 58.3° for MetaView. OA2 is itself noisy on the real ground-truth images, so these numbers should be read with care. On the 168 analogies where our estimated move matched the label, the errors are 63.2°, 39.8° and 59.6°. That is a tie with MetaView and a clear loss to SEVA. Direction is correct 0.73 of the time (SEVA 0.74), but we under-rotate, with a rotation ratio of 0.61 vs 0.88. It gets worse when the estimate is wrong, so you are right that the estimator is a sensitive part. Image metrics are mixed: PSNR/SSIM slightly favour ours, while LPIPS and the VLM judge favour SEVA (judge, mean of Subject and Background: 6.95 vs 5.28).
How we read this: on real photos the model turns the right way but not far enough, and it is behind a dedicated novel-view model. A figure with selected CO3D cases, including one under-rotation failure, will be added to App. A.2 in the final version. App. A.2, Figure 8 shows analogies on our own phone photos, with A, A′ and B all real.
Other camera-control methods We compared with SEVA and MetaView on the CO3D set above (App. A.2, Table 3). We lose to SEVA, and we are behind MetaView overall; the two are only tied on the matched subset. These are camera-only analogies, so they measure only half of our task, since neither baseline transfers the edit. We did not run image-to-video models with camera trajectories.
Beyond the camera Yes, the method is built for the camera: the estimator, the composition rule and the tokens are all camera-specific. The general recipe might carry over: estimate an attribute for A, A′ and B, compose the target explicitly, then condition on it. But we have no evidence for any attribute other than the camera and will not claim it.
Systems paper Our main claim is that camera-aware image analogy needs one extra constraint: the target camera has to be composed explicitly from the estimated cameras of A, A′ and B and then given to the generator. Without it, analogy models fine-tuned on our data learn the edit but not the camera (judge Camera 3.50 and 3.94 against 7.39 for ours; to be added to the appendix in the final version). How the constraint is posed matters more than how it is encoded: as text, the composed absolute target beats the relative move by 11.35°, and text and camera tokens tie. The gap is widest when the query starts from a different view than the exemplar: commercial editors get 33 to 35° worse, ours 6.7°, and even with the oracle relative move in text the best editor stays about 30° behind. The fix is simple but very effective, and we will restructure the paper around it.
Citations Will fix, including Reed et al. at line 116.
MetaView and API models MetaView is in the CO3D comparison above (App. A.2, Table 3). It gets 58.3° and ours 66.4°, with MetaView given the ground-truth move. So we are behind it overall, and it is only a tie on the matched subset. We did not run API video models.