- Rating
- 2
- Confidence
- 3
- Informativeness
- 1
- Presentation
- 3
- Soundness
- 1
Read review
Paper summary
This paper proposes Dragometry, a point-based facial image editing framework that projects 2D user drags onto a 3D FLAME mesh to perform either local deformation or viewpoint rotation. The updated 3DMM parameters and camera poses are then fed as explicit conditions into a pretrained Morphable Multi-view Diffusion Model to synthesize the edited face image. The core technical focus of this explicit mesh manipulation lies in a Jacobian-based parameter weighting strategy designed to localize deformations and suppress unintended changes in distant facial regions. Additionally, the authors introduce DragFace, an FFHQ-based evaluation benchmark, alongside a Geometry Mean Distance metric. Compared to purely latent- or feature-based 2D drag editing baselines, the proposed framework demonstrates noticeably more robust and coherent results.
Methodologically, since image synthesis relies entirely on an off-the-shelf, frozen pretrained diffusion model, the core of this work is essentially solving for 3D facial mesh deformation or rigid rotation from 2D handle constraints (i.e., screen-space inverse kinematics). From this perspective, the paper requires much more thorough experimentation on the 3D deformation optimization itself—including comparisons against classic 3D/blendshape manipulation methods, as well as comprehensive ablation studies and hyperparameter sensitivity analyses. From the perspective of facial image editing, while comparing against generic 2D drag-based methods fits the paper's narrative, it is essential within the face editing domain to also benchmark against 3DMM- or ControlNet-based portrait editing approaches. Without such comparisons, the necessity and specific advantages of formulating facial manipulation as a point-based dragging problem over existing conditional face editing paradigms remain unsubstantiated.
Critical strengths
-
clearly illustrates the unreliability of implicit diffusion feature correspondence (e.g., DIFT) under large facial deformations or rotations (Fig. 2). Shifting the drag operation from feature space to an explicit 3D geometry condition space is intuitive and geometrically sound.
-
point out that standard DIFT-based Mean Distance (MD) can be misled by global face translation or feature mismatching (Fig. 4). The proposed GeoMD, which measures 3D surface correspondence while discounting global translation, is a more reliable metric for facial dragging.
-
The paper is well-written and easy to follow. The authors also provide an honest failure analysis in Appendix D (Figs. 11 & 12).
Critical weaknesses
Since both the front-end 3D reconstruction and the back-end image synthesis rely entirely on off-the-shelf pretrained models, the proposed method essentially reduces to optimizing 3D FLAME mesh deformation or camera pose from sparse 2D screen-space constraints.
In computer graphics, this is fundamentally classic screen-space inverse kinematics (IK) or direct manipulation of blendshapes (e.g., Lewis & Anjyo, 2010; Seo et al., 2011; Cetinaslan et al., 2015). Performing simple gradient descent over a low-dimensional linear parameter space offers relatively limited algorithmic novelty for ICLR.
Moreover, confining the deformation optimization strictly to a 66-dimensional FLAME subspace (65 expression PCA coefficients + 1 jaw parameter) inherently caps the method's expressive ceiling—it fundamentally cannot handle fine-grained, out-of-subspace local edits (e.g., asymmetric cheek puffing, tongue movement, or dragging non-FLAME regions like hair and glasses).
In addition, given that the core contribution lies in overcoming global basis entanglement for localized 3D facial mesh manipulation, the paper completely lacks discussion and experimental comparisons with established graphics methods on direct blendshape manipulation and localized facial rigs (e.g., Tena et al., 2011; Neumann et al., 2013), as well as any robustness evaluation against initial 3D reconstruction errors.
[Lewis & Anjyo, 2010] John P. Lewis and Ken-ichi Anjyo. "Direct Manipulation Blendshapes."
[Seo et al., 2011] Jaewoo Seo, Geoffrey Irving, John P. Lewis, and Junyong Noh. "Compression and Direct Manipulation of Complex Blendshape Models."
[Cetinaslan et al., 2015] Ozan Cetinaslan, Verónica Orvalho, and John P. Lewis. "Sketch-Based Controllers for Blendshape Facial Animation."
[Tena et al., 2011] J. Rafael Tena, Fernando De la Torre, and Iain Matthews. "Interactive Region-Based Linear 3D Face Models."
[Neumann et al., 2013] Thomas Neumann, Kiran Varanasi, Stephan Wenger, Markus Wacker, Marcus Magnor, and Christian Theobalt. "Sparse Localized Deformation Components."
More disscusions are in Questions part.
Questions for authors
-
As the primary algorithmic design of the paper, the Jacobian-based parameter weighting is under-evaluated (supported only by a two-row Table 3)
-
Given that Eq. (3) already includes a vertex preservation loss to penalize distant vertex displacements, why are its first-order gradients insufficient to prevent global deformation without introducing a heuristic Jacobian pre-filtering step with multiple hardcoded hyperparameters (e.g., )? Furthermore, why does the ablation study (Table 3) replace the Image Fidelity (IF) metric from Table 1 with Identity Similarity, rather than directly measuring the 3D Vertex-to-Vertex (V2V) error on the unedited mesh region to explicitly validate deformation locality?
-
Comparing a framework built on a domain-specific multi-view 3D portrait model (MMDM) against generic 2D drag baselines without 3D facial priors is inherently asymmetric (especially for viewpoint rotation); can the authors provide comparisons against recent 3D-aware drag methods (e.g., FlowDrag, DragIn3D) as well as running feature-based dragging directly on MMDM? More importantly, within the facial editing domain, what is the specific necessity and advantage of a point-based drag interface compared to existing ControlNet- or 3DMM-conditioned portrait editing methods?
-
Were the 410 handle-target pairs in DragFace manually annotated by human users, or synthetically generated by perturbing FLAME parameters and re-projecting (which would introduce severe subspace bias in favor of your method)?
13 comments
-
"Moreover, confining the deformation optimization strictly to a 66-dimensional FLAME subspace (65 expression PCA coefficients + 1 jaw parameter) inherently caps the method's expressive ceiling—it fundamentally cannot handle fine-grained, out-of-subspace local edits (e.g., asymmetric cheek puffing, tongue movement, or dragging non-FLAME regions like hair and glasses)."
We agree that the expressive capacity of FLAME imposes limitations on achievable deformations. However, manipulating arbitrary image regions, such as hair or glasses, is outside the scope of our claimed contributions.
As explicitly described in the Introduction and illustrated in Fig. 1, DRAGOMETRY focuses on two tasks: facial-expression deformation and viewpoint rotation. Consequently, editing hair, glasses, or other regions not represented by FLAME is not an intended capability of our current formulation.
We acknowledge that certain fine-grained facial deformations, such as tongue movement or highly localized asymmetric expressions, may also fall outside FLAME's representational capacity. This limitation arises from the chosen geometric representation and is already acknowledged in our Conclusion (Sec. 6).
-
"In addition, given that the core contribution lies in overcoming global basis entanglement for localized 3D facial mesh manipulation, the paper completely lacks discussion and experimental comparisons with established graphics methods on direct blendshape manipulation and localized facial rigs (e.g., Tena et al., 2011; Neumann et al., 2013), as well as any robustness evaluation against initial 3D reconstruction errors."
We agree that these classical graphics methods are relevant to the geometric optimization component of DRAGOMETRY and have incorporated them into the revised Related Work. However, fair direct experimental comparisons are nontrivial because these methods operate on different geometric representations and address different output objectives.
Tena et al. (2011) introduce region-based 3D facial models for interactive mesh manipulation and animation, while Neumann et al. (2013) extract localized deformation components from animated 3D mesh sequences. In contrast, DRAGOMETRY requires updated FLAME parameters as conditioning inputs to a pretrained diffusion model, with the final objective of generating an edited facial image rather than an edited 3D mesh. Neither method directly produces these FLAME conditioning parameters. Integrating them into our pipeline would require additional model adaptation or fitting their edited geometry back into the FLAME parameter space, making a direct comparison under identical conditions nontrivial.
Alternatively, these methods could be evaluated as independent baselines. However, they do not directly support the sparse 2D handle–target interface used in DragFace, instead operating on 3D facial geometry or deformation components. A fair comparison would require reconstructing 3D geometry from the source image, mapping 2D dragging constraints to corresponding 3D deformation targets, and synthesizing the edited geometry back into a 2D facial image while preserving the source appearance. These additional steps are nontrivial and would require substantial methodological adaptations beyond the original methods, rather than a direct application of existing baselines.
Regarding initial reconstruction errors, our work focuses on facial image editing rather than 3D reconstruction accuracy. We employ an existing pretrained FLAME fitting model to obtain the initial geometric conditions and evaluate the resulting edits using the same benchmark and evaluation protocol. DragFace contains real FFHQ images without ground-truth 3D meshes; therefore, absolute 3D reconstruction errors cannot be directly quantified. We acknowledge that inaccurate initial fitting can affect downstream editing, but improving or independently benchmarking the pretrained reconstruction model is outside the primary scope of this work.
We thank reviewer kC2q for their careful evaluation and constructive feedback, particularly for recognizing the advantages of explicit 3D geometric control and the effectiveness of our editing results. However, we respectfully note that several criticisms appear to be factual misunderstandings of our method, its technical contributions, and the scope of our evaluation. We clarify these distinctions and address the reviewer’s specific concerns below.
We respectfully disagree with the reviewer's characterization of DRAGOMETRY's contribution reducing to the classical 3D mesh manipulation method. This overlooks the central objective of our work: using explicit 3D geometry to control facial image generation by guiding the latent representations of a pretrained diffusion model.
First, performing drag-based editing at inference time using a pretrained generative model is an established paradigm. All four baseline models—DragDiffusion, DragNoise, GoodDrag, and DragLoRA—build upon pretrained Stable Diffusion backbones. Their contributions lie in how they manipulate the generative process at inference time to satisfy user-specified dragging constraints, whether through latent optimization, intermediate feature manipulation, or image-specific model adaptation. DRAGOMETRY follows the same general paradigm but introduces a different control mechanism: it optimizes explicit FLAME parameters to construct geometric conditions that guide MMDM toward the desired facial image. Therefore, using an existing generative backbone does not reduce our contribution to conventional mesh manipulation. Our contribution lies in how sparse 2D dragging constraints are translated into effective control of face image generation.
Second, using pretrained models to guide the generation of another pretrained diffusion model is an established research direction in computer vision. For example, Universal Guidance (Bansal et al., ICLR 2024) uses off-the-shelf pretrained networks, including face-recognition models, to guide pretrained diffusion generation. FreeDoM (Yu et al., ICCV 2023) similarly employs pretrained networks, such as face detectors, to construct guidance signals without additional training. Other representative works include classifier guidance (Dhariwal and Nichol, NeurIPS 2021), Diffusion Self-Guidance (Epstein et al., NeurIPS 2023), and Plug-and-Play Diffusion Features (Tumanyan et al., CVPR 2023). Although their specific mechanisms differ, these studies demonstrate that developing new control mechanisms for pretrained generative models is itself a meaningful research contribution.
Third, DRAGOMETRY differs fundamentally in its objective from the classical graphics methods cited by the reviewer. Lewis and Anjyo (2010), Seo et al. (2011), and Cetinaslan et al. (2015) investigate direct manipulation of 3D blendshape models or facial animation rigs, where the intended outputs are modified 3D geometry or animations. In contrast, DRAGOMETRY does not aim to output an edited 3D mesh. Its intermediate output consists of optimized 3DMM parameters, while its final output is an edited facial image generated by a pretrained diffusion model.
Moreover, in DRAGOMETRY, FLAME parameters provide explicit geometric conditions that guide MMDM's denoising process. Different geometric conditions influence the evolution of the diffusion latents and consequently the generated facial appearance. Our method therefore optimizes FLAME parameters not simply to produce a desired 3D deformation, but to guide a pretrained diffusion model toward the facial image specified by sparse 2D dragging constraints. This distinguishes our objective from classical blendshape manipulation, where the geometric deformation itself is the intended output.
We acknowledge that the cited graphics methods provide relevant foundations for our geometric optimization component. We revised the Related Work section to incorporate the suggested references and clarify how they differ from DRAGOMETRY. Please see the newly added text in blue (Lines 845-856) in the uploaded PDF. Nevertheless, reducing DRAGOMETRY to classical screen-space inverse kinematics does not accurately characterize the complete framework or its research objective. We hope these clarifications resolve the misunderstanding regarding our technical contribution.