acceptodds
Withdrawn from ICLR 2027

ForensicFace: Reliable 3D Face Texture Reconstruction via Confidence Guided Diffusion

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Reliable 3D face texture reconstruction from a single 2D facial image is of critical importance in criminal investigations. Existing methods usually use visible texture directly as a generative condition, but they overlook differences in observation confidence across facial regions. As a result, unreliable projections may be retained incorrectly, while genuine local details such as moles and skin tone variations may be smoothed or rewritten during generation. To address these problems, we present ForensicFace, a framework for reliable single image 3D face texture reconstruction using a canonical UV representation. First, we propose texel level observation confidence estimation to model the reliability of input texture continuously, addressing the inability of binary visibility masks to distinguish the reliability of different observed regions. We then design an observation confidence guided diffusion model that controls generative freedom according to observation confidence. Regions with high observation confidence preserve more of the input observation, whereas regions with low observation confidence or no direct observation rely more on the generative prior for completion. This design prevents reliable observations from being overwritten by the generative prior while reducing the risk of directly retaining unreliable projections. Finally, we estimate generation confidence from consistency across repeated stochastic samples to characterize the stability of generated regions. Experiments on FaceScape and Stirling ESRC show that ForensicFace outperforms existing methods in PSNR, SSIM, LPIPS, and CSIM while maintaining stable texture reconstruction quality under large pose variations.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
3D Face Reconstruction, Diffusion Models, Confidence Estimation, Forensic Face Analysis

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_MBvPOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

ForensicFace reconstructs photorealistic facial textures with high fidelity from a single facial image. Specifically, given an input facial image, ForensicFace first reconstructs the facial geometry and then obtains the visible facial textures through projection. It introduces a confidence scoring mechanism to assess the reliability of different facial texture regions. In general, regions with uniform illumination and clear visibility are more likely to receive higher confidence scores. ForensicFace then employs a diffusion model to refine the facial textures, reducing the extent of modification in high-confidence regions to preserve fidelity to the input image as much as possible. Experiments demonstrate the advantages of ForensicFace in high-fidelity facial texture reconstruction.

Critical strengths
  1. The paper is clearly presented, allowing readers to readily grasp its core ideas.
  2. The proposed confidence scoring mechanism is insightful, but I believe its potential warrants further exploration.
Critical weaknesses

Based on the current experimental results, the improvements attributable to the proposed innovations are rather limited. As shown in the ablation study in Table 2, removing the corresponding module leads to only marginal performance degradation, which somewhat weakens the paper's overall contribution. Nevertheless, I find the proposed confidence scoring mechanism interesting, and believe its value should extend beyond marginal numerical improvements. In particular, its significance would be substantially greater if it could effectively handle challenging scenarios involving facial occlusions, such as glasses and masks. This is especially relevant given that the authors position criminal investigations as a key application scenario in the introduction. However, the current visualizations and quantitative evaluations do not sufficiently investigate this aspect.

Minor Weaknesses: The citation format does not appear to follow the official ICLR template. In addition, the layout of some figures could be improved, as Figures 5 and 6 occupy a considerable amount of space.

At this stage, I lean toward rejection. Nevertheless, I believe the proposed confidence scoring mechanism represents an interesting research direction, and I encourage the authors to explore its potential further.

Questions for authors

As discussed in the Critical Weaknesses section, I would particularly like to understand whether the confidence scoring mechanism can improve reconstruction under facial occlusions and similar challenging conditions.

Reviewer_7WaeOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper proposes ForensicFace, a diffusion-based framework for single-image 3D face texture reconstruction in canonical UV space that aims to preserve trustworthy input evidence while plausibly completing unobserved regions. The core ideas are: (1) a texel-level observation confidence map that continuously models the reliability of input projections, used to adaptively inject observed UVs during every reverse diffusion step, and (2) a generation confidence map estimated from the dispersion across multiple diffusion seeds to quantify stability in prior completed regions. Experiments on FaceScape and Stirling ESRC show improvements over recent baselines on PSNR, SSIM, LPIPS and identity similarity (CSIM), with ablations indicating complementary benefits from the confidence modeling and the UV diffusion prior.

Critical strengths

Bridges the Gap Between Realism and Forensic Fidelity The paper addresses a genuine gap in UV completion by introducing a continuous observation confidence framework. By adaptively controlling generative freedom during diffusion, it successfully preserves genuine identity cues while completing missing textures, solving the critical trade-off required for forensic applications. Enhances Reliability via Explicit Uncertainty Quantification It provides a practical mechanism for interpretability by estimating generation confidence through multi-seed consistency. This explicitly distinguishes observed evidence from synthesized content, offering a necessary safeguard against hallucination.

Critical weaknesses

Lack of a dedicated metric for evidence preservation Although the paper emphasizes preserving genuine observed details, it does not provide a fidelity metric specifically for visible/observed texels. Identity similarity is a global measure and may fail to reflect whether local evidential details have been altered. Hand-crafted observation confidence The observation confidence is composed of multiple factors, such as geometry, correspondence, boundary, and illumination. However, their precise definitions, calibration procedures, weight settings, and sensitivity are not clearly specified. More importantly, these weights are not learned or data-driven, raising my concerns about the stability and reliability of the confidence estimation.

Questions for authors

1.Can you quantify “observation preservation” by reporting metrics restricted to observed texels (PSNR/LPIPS on observed regions only) and show that your method rewrites them less than baselines? 2.How calibrated is generation confidence? Can you report correlation between Cgen and per-texel reconstruction error (reliability diagrams)? I would consider increasing my rating if the authors satisfactorily address my questions and justify the identified weaknesses.

Reviewer_RaUsOpenReview ↗
Rating
1
Confidence
4
Informativeness
1
Presentation
2
Soundness
2
Read review
Paper summary

The paper presents ForensicFace for single-image facial texture reconstruction and completion. Existing geometry and correspondence estimators project the input image into UV space. The method assigns observation confidence to each texel and uses it to control observation injection during diffusion, starting from a StyleGAN2 texture initialization. It also measures variation across five generated samples to describe completion stability. Experiments on FaceScape and Stirling ESRC include comparisons with existing methods, component ablations, and pose analysis.

Critical strengths

Preserving observed facial details for face reconstruction is a meaningful objective. The paper clearly motivates why texture completion should retain genuine input details and distinguish them from generated content.

The reported aggregate results are positive. Table 1 shows improvements over the compared methods on both datasets across reconstruction and identity metrics. Evaluating textures with shared geometry and rendering settings helps make these comparisons meaningful.

Critical weaknesses

The added value over existing texture completion and refinement methods is not well established. Relightify already preserves observed UV texture during diffusion-based completion, while FaceRefiner explicitly refines generated textures to preserve input details through differentiable rendering. Neither is discussed. AvatarTex is cited and supplies initialization weights, but its complete reconstruction pipeline is not compared. The main distinction therefore needs stronger evidence. In Table 2, continuous confidence improves PSNR by only 0.091 dB and LPIPS by 0.001 over the binary variant, without measuring identity or local-detail preservation. A comparison with simple boundary smoothing would help determine whether the proposed confidence calculation adds more than smoother transitions. Several factors in Equation 1 also lack concrete computational definitions.

The evaluation does not convincingly establish the claimed reliability of the reconstructed faces. Although Table 1 evaluates rendered images, Figure 3 presents only unfolded UV textures. These are insufficient to judge final facial appearance, identity, and texture transitions; matched frontal and side-view renderings are needed. Figure 4 provides one useful mole example, but there is no systematic evaluation of retained or incorrectly generated local features. Generation confidence is also shown without testing its relationship to reconstruction errors. The authors correctly define it as sampling stability, but consistently generating the same texture does not establish that the completion is accurate. This leaves a substantial gap between the reported results and the paper’s emphasis on reliable reconstruction for forensic use.

Questions for authors

What does the proposed confidence calculation improve beyond binary visibility or simple boundary smoothing? With the same initial texture, diffusion prior, and sampling budget, please compare these alternatives on identity and local-detail preservation. Please also provide the definitions and parameters needed to compute the factors in Equation 1.

Can you provide matched rendered comparisons that demonstrate the practical benefit? Please show frontal and side views of the predictions and ground truth under shared rendering settings, with close-ups of observed details and completed regions. A comparison with an available observation-preserving refinement method, such as FaceRefiner or the relevant AvatarTex pipeline, would help establish the contribution.

Do the confidence maps identify reconstruction errors? Using the available ground truth, please evaluate observation confidence against errors in the projected texture, and generation confidence against errors in unobserved regions. Showing whether higher confidence corresponds to lower error, including confidently incorrect examples, would directly address the reliability claim.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.