- Rating
- 3
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This submission proposes an algorithm for depth-of-field synthesis in images using language guidance. The user provides an image alongside a text prompt specifying which object in the image should remain in focus, as well as a numerical defocus strength level. The authors propose to fine-tune a foundation model using a synthetic dataset of paired images, whose construction appears to be the main technical contribution of this work. Specifically, several foundation models are arranged into a pipeline to generate training data pairs for a large collection of all-in-focus images. Experiments show that the approach performs favorably compared to a reasonable selection of baselines.
Critical strengths
Being able to edit the depth of field in an image is a desirable capability in many image editing workflows, and numerous solutions have been proposed over the decades. Instead of relying on image-level manipulations, this work addresses the problem by allowing users to specify their desired results through text prompts. This is, of course, a very common paradigm these days. At least two other well-known approaches exist in this space, but they apparently do not provide source code, preventing direct comparisons. The results of this work are quite convincing. The evaluation suggests that the proposed approach advances the state of the art.
Critical weaknesses
I have two closely related concerns about this submission.
My first concern relates to the mathematical derivations in Sec. 3.1, which are listed among the main contributions in Sec. 1. The presented derivations are elementary analytical results that have long been known in the field. For example, the harmonic-mean relationship in Eq. 5 can be found in an optics-industry technical white paper (https://www.photigy.com/wp-content/uploads/2010/09/DOF-and-Bokeh-explained.pdf), where it is formulated in terms of metric depth rather than inverse depth. Similar arguments apply to most other parts of the corresponding subsection. I am concerned that the exposition in the paper presents these elementary analytical results as substantial algorithmic contributions. I would strongly suggest more clearly delineating the original contributions in the exposition.
My second, related concern is that the manuscript does not adequately position itself with respect to the extensive body of related work that has accumulated over decades in the field of depth-of-field synthesis. The paper refers to "early methods" (L102), but only cites works published since 2016, with most of the discussed works being less than three years old. For example, the thin-lens equations considered here are accompanied only by a reference from 2022. The original sources in this area should be cited to give proper credit. Starting points for a literature review could be the following two works:
- Potmesil & Chakravarty, A Lens and Aperture Camera Model for Synthetic Image Generation, SIGGRAPH 1981.
- Kraus & Strengert, Depth-of-Field Rendering by Pyramidal Image Processing, CGF 2007.
I strongly encourage the authors to more seriously consider and acknowledge the rich history of the field to avoid the perception of insufficient scholarly rigor and a lack of due credit to foundational contributions.
Questions for authors
I do not have any questions.