acceptodds
Withdrawn from ICLR 2027

LensMaster: All-in-one Photorealistic Depth-of-field Rendering

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Depth-of-field (DoF) editing remains challenging due to three limitations of existing methods: reliance on error-prone depth or segmentation priors, one-way rendering from all-in-focus inputs, and limited modeling of lens-dependent bokeh characteristics. We present LensMaster, an end-to-end framework that formulates DoF manipulation as a bidirectional optical-state transformation. Given a source image and its optical state, LensMaster continuously controls both focus distance and aperture to synthesize an arbitrary target state without explicit geometric priors. To support coherent editing across optical configurations, we introduce optical-state consistency with identity, cycle, and composition constraints. We further disentangle shared DoF transformation from lens-specific appearance through spatially varying lens-character modulation, enabling a single model to reproduce distinct real-lens Bokeh characteristics. To support cross-lens learning, we introduce FujiBokeh, a real-world multi-focus, multi-aperture dataset captured with a Fujifilm system. Experiments demonstrate that LensMaster achieves accurate continuous, bidirectional, and lens-aware DoF editing while preserving fine structures and realistic bokeh appearance.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
Bokeh Rendering, Depth-of-field Rendering

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_xi45OpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

LensMaster formulates depth-of-field editing as a transformation between source and target optical states. It unifies multiple editing tasks through focal and aperture conditions, identity, cycle, and composition consistency constraints, and spatially varying lens feature modulation. The authors also collect approximately 40,000 FujiBokeh images and train the model jointly with RealBokeh. The method is evaluated on bokeh synthesis, defocus removal, and refocusing.

Critical strengths
  1. Conditioning on both source and target states is a reasonable design. Inputs with different degrees of defocus may require different restoration processes even when the target aperture and focal setting are the same. Using the focal and aperture settings at both ends helps describe aperture opening and closing, refocusing, and joint editing within a unified framework. (Pages 3–4, Equations 1–6)

  2. The method does not require separately estimated depth, segmentation, or CoC maps. This reduces its dependence on intermediate estimates and provides an alternative approach to end-to-end depth-of-field editing. (Pages 3–4)

  3. The experiments cover a broad range of editing tasks. Tables 2–4 evaluate bokeh synthesis from all-in-focus images, defocus removal, and four types of arbitrary-state editing. Table 5 further examines source conditioning, consistency constraints, and lens modulation. The results show relatively consistent performance gains across these settings.

Critical weaknesses
  1. Missing implementation and training details are a major obstacle to reproducibility. The main text only broadly mentions a pretrained autoencoder, a Transformer, and conditional flow matching. It does not specify the backbone, pretrained weights, model size, number of training epochs, optimizer, sampling steps, or loss weights. Of the 12 pages provided, the last three are references, and a complete implementation configuration is still missing. These details should be included.

  2. The implementation of the consistency losses within the flow matching framework needs a more concrete explanation. Equations 14–16 treat the generator as a directly callable mapping, but generation typically involves multi-step integration, and the cycle and composition constraints require repeated generator calls. Do the authors backpropagate through the full sampling process, use a single-step endpoint approximation, stop gradients, or introduce a teacher branch? These choices substantially affect both training cost and the meaning of the constraints, and I cannot determine them from the current equations.

  3. It is unclear how random noise is handled across different generation paths. The model starts from Gaussian noise, but the identity, cycle, and composition consistency equations do not explicitly include random variables. If different paths use independent noise, pixelwise differences may reflect both editing errors and stochastic generation differences. If noise is shared, the sharing scheme should be specified. I suggest describing the noise settings and evaluating sensitivity to random seeds.

  4. The current experiments do not sufficiently establish the model's lens modeling capability. The quantitative results mainly report task performance separately on RealBokeh and FujiBokeh. There is no one-to-one comparison with real captures of the same scene under different target lenses, nor an evaluation on unseen lenses. The evidence therefore more clearly supports adaptation to two seen data domains, while cross-lens transfer still requires additional validation.

  5. The final configurations and evaluation tasks in the different ablation groups of Table 5 are unclear. The last rows of the three groups report 27.68/0.1548, 27.68/0.1648, and 28.19/0.1326, respectively, without explaining whether they correspond to different tasks, data subsets, or fixed components. If all three rows represent the full model, the differences should be explained. If the configurations differ, they should be listed directly in the table to make the comparisons easier to follow. (Page 9, Table 5)

  6. The individual contributions of the consistency constraints are not sufficiently separated. Table 5 adds the identity and cycle losses together, making it difficult to determine how much benefit comes from the cycle constraint itself. I suggest evaluating the identity, cycle, and composition losses separately and including a sensitivity analysis of their weights. Identity error, round-trip editing error, and drift after repeated edits would also provide more direct evidence that these constraints achieve their intended goals. (Pages 5 and 9)

  7. The claim of recovering extremely defocused regions needs more evidence of content fidelity. The examples show that the model can generate visually plausible details, but whether those details match the original structures remains unclear. I would like to see targeted evaluations on text and thin lines, along with failure cases involving transparent objects, reflections, occlusions, and inputs from unknown lenses. These experiments would help clarify the method's limitations.

  8. The reporting of computational cost and result stability is incomplete. The paper does not provide inference latency, peak memory usage, or training cost. It also lacks variation across random seeds or confidence intervals computed over test scenes. Given the multiple generation calls and regularization terms in this framework, these details are important for judging whether the performance gains justify the additional cost and whether small differences in the reported metrics are stable. (Pages 5–9)

Questions for authors

See weakness

Reviewer_dWPfOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

This paper presents the LensMaster system that enables the focus and aperture settings of a given image. In this work, optical-state oo of an image consists of the pair (focal length, aperture). Images captured using a source state oso_s can be converted to a target state oto_t using the LensMaster system. Additionally, the effects of the spatial characteristics of the lens is modelled indecent of the optics The system consists of a Variational Auto Encoder to transform the input image to an embedding, a transformer network that takes source and target optical states as well as the lens character conditioning as inputs and generates the refocussed image in the encoded space. A VAE decoder converts it into the output image.

The paper introduces a new dataset called FujiBokeh consisting of 4000040000 images of 3000×20003000\times 2000 resolution from 200 scenes. A focal stack of each scene is captured at 4 aperture settings, with upto 100 images in each stack. This and the previous dataset RealBokeh are used for quantitative and qualitative evaluations. The supplementary also contains several examples from real images including a couple of interactive focus adjustments.

The LensMaster model is trained using 4 loss terms: an identity LidL_{id} loss that prefers to create exactly the input image when the source and target optical states are the same, a cycle consistency loss LcycL_{cyc} that prefers the input image to be recreated when it transferred to a target state oto_t and then brought back to the source state oso_s, and a composition loss LcompL_{comp} to enforce compositonality of changes in optical states. These are in addition to the flow matching loss LFML_{FM}. A clear ablation study brings out the importances of each of the 3 additional losses used.

Quantitative results on RealBokeh and FujiBokeh datasets with focal length and aperture settings show the proposed method outperforming all prior works.

Critical strengths

The formulation of the problem is clear and straightforward. The identity, cycle, and compositional losses seem interesting, and are effective. The quantitative results appear to be very persuasive and strong. The dataset will be a very valuable contribution also.

Critical weaknesses

I am unconvinced and unclear of several aspects of the work after reading the paper (which is easy to read). Here are some of my points in that regard:

  • I couldn't find any description of how the LensMaster model was trained in the paper. The losses are mentioned, but it is not clear what the training dataset was. Given that training needs oso_s, oto_t, and lsl_s, ltl_t, I assume RealBokeh and FujiiBokeh were used for training. What are the other parameters or factors relevant to training? Was the VAE trained along with the main model, or was it trained separately? Paper mentions a "pretrained image autoencoder", but gives no details about it.
  • Apart from missing training information, there was no clear description of how a new oto_t image is generated. Pseudocode or steps to do this will help.
  • What is the training effort and refocussing effort in terms of compute needed and time taken? Those are important aspects of any practical algorithm and should be reported broadly in the paper.
  • It is not clear how arbitrary, "wild" images are processed by the system. Do we need to give the system oso_s and oto_t parameters as input? While oto_t can be a part of the specification, how do get oso_s from any image? The evaluations on the datasets are clear where all the information is available upfront. Or, can only all-in-focus or f/22 images be used as input? Isn't knowing oso_s a limitation on the applicability of the method?
  • The supplementary contains just 3 wild image. The middle one seems quite simple and perhaps such results can be obtained by other methods. The last room seems convincing though it appears like controlled blurring of the input; depth-of-field effects are not obvious. Given the title of the paper includes "DoF Rendering", I don't see sufficient demonstration of that on arbitrary images.
Questions for authors

Please see my observations under weaknesses and address them!

Reviewer_7WiJOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
1
Soundness
2
Read review
Paper summary

The paper suggests that existing depth-of-field rendering and editing methods are limited due to relying on depth or segmentations priors, only rendering bokeh from all-in-focus input images, or not modeling lens-specific bokeh appearance. It proposes an end-to-end flow-matching editing model that transforms the input image from a source focus-aperture state to a target state. Its contributions rely on two main components: (i) an editing model conditioned on both the source and target optical states, with a lens module to reproduce the bokeh of different lenses; and (ii) FujiBokeh, a real-world dataset with multiple focus distances and apertures per scene. The method is tested on RealBokeh and FujiBokeh.

Critical strengths
  • The formulation as an arbitrary-to-arbitrary optical-state transformation is reasonable and the proposed method shows consistent gains over baselines across all tasks as well as convincing qualitative results.
  • The FujiBokeh dataset is a useful contribution and could be a valuable resource for the community, with varying apertures and focus distances.
Critical weaknesses

The paper lacks enough details to reproduce the method and properly assess the reported results.

  • For instance, there are no implementation details in the paper or the appendix. The paper does not describe shared optical transformer architecture or its initialization (e.g., whether it is fine-tuned from a pretrained model). Similarly, other important details such as training data and schedule, loss weights (Eq. 17), and number of sampling steps and compute are also not reported. Additionally, the details of how the proposed consistency losses (Eq. 14-16) are trained, given that the losses are defined on the outputs images, are also not present.
  • The evaluation protocol is also not clear. The paper does not specify whether baselines are trained on RealBokeh and FujiBokeh or directly used from public checkpoints, which is highly relevant since LensMaster is trained in-domain for both benchmarks. The paper also does not mention which depth estimator is used for the depth-based methods. Moreover, in Tab. 5, the bottom row reports different scores across blocks for what seems to be the full model.

The method seems somewhat incremental and some of the claims are not well supported.

  • Several of the cited works already address the three main limitations in the field presented as the paper motivation, e.g., depth-free rendering (Bokehlicious, BokehFlow), editing defocused inputs (DiffCamera, GenRefocus), and lens appearance (LensStyle). The proposed individual components also build on established techniques, such as cycle consistency and feature modulation. Therefore, the main contribution seems to be the unification of aforementioned techniques and the new FujiBokeh dataset. The claims should more clearly reflect this, potentially with a small table that compares the capabilities of related works.
  • Given that the model is trained with only two lenses, each coming from a different dataset, it is not clear whether the lens module is actually learning the lens characteristics or the other differences between the two datasets (e.g., scenes, camera, processing). Furthermore, the paper only conditions the aperture on the nominal f-number (Eq. 4), but the amount of bokeh produced by a given f-number also depends on the sensor format and the focal length. Firstly, the two datasets are captured with different sensor formats (full-frame Canon for RealBokeh vs. APS-C Fujifilm for FujiBokeh), so the same f-number leads to different amounts of bokeh across datasets. Additionally, both datasets are captured with zoom lenses, so the focal length may also vary within each dataset, but it is not modeled. It is not clear how these factors are handled.
Questions for authors
  • Could the authors also evaluate on EBB! (Ignatov et al., 2020)? Since it is captured with a different camera than both training datasets, it would show how well the method generalizes
  • GenRefocus and DiffCamera are compared to in the editing tasks but they also support rendering from all-in-focus inputs. Could the authors also compare to them in Tab. 2?
  • For the defocus removal setting (Tab. 3), could the authors include a dedicated defocus deblurring model as a baseline?
  • The paper claims continuous control of focus distance and aperture. Could the authors evaluate on unseen apertures or focus positions and clarify how the values are parameterized?
  • Will FujiBokeh be publicly released?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.