acceptodds
Under review as a conference paper at ICLR 2027

GlassHead: Learning 3D Eyeglass-Wearing Heads from a Single Portrait via Pseudo-Multiview Supervision

Abstract

Single-image 3D reconstruction of eyeglass-wearing heads is challenging because important structures, particularly the temples and their attachments around the ears, are poorly observed from a frontal portrait. Learning to recover these regions requires diverse viewpoint supervision, yet large-scale multiview captures with varied eyeglasses are expensive and difficult to obtain. We address this data bottleneck by using generative video as a scalable source of pseudo-multiview supervision. We introduce a scalable pipeline that leverages a video generation model to transform frontal portraits into view-diverse pseudo-multiview observations, recover their camera parameters, and construct the Eyeglass Portrait Pseudo-Multiview Dataset (EPPM-20K). Building on this supervision, we propose GlassHead, a feed-forward reconstructor that predicts a complete 3D Gaussian head from a single portrait through hierarchical coarse-to-fine decoding. To accommodate inconsistencies in generated views, we combine reliable frontal reconstruction with perceptual and camera-conditioned adversarial supervision on pseudo-multiview observations. Experiments demonstrate consistent improvements over recent single-image head reconstruction methods, with particularly clear gains in lateral eyeglass structures under novel viewpoints. At inference time, GlassHead requires only a single frontal portrait and reconstructs the 3D head in one forward pass within one second.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
3D Gaussian Splatting, 3D Head Reconstruction, Generative Data

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_fQR7OpenReview ↗
Rating
1
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper studies single-image 3D reconstruction of eyeglass-wearing heads, with particular emphasis on recovering lateral eyeglass structures such as temples and ear-side attachments. The core idea is to use a video generation model to synthesize view-diverse observations from frontal portraits, recover approximate camera parameters for these generated frames, and use them as pseudo-multiview supervision.

Critical strengths
  1. The strongest contribution is the data-supervision paradigm. Instead of requiring calibrated real multiview capture, the paper converts frontal portraits into approximately multiview observations using video generation and then estimates corresponding camera trajectories.
  2. On the held-out EPPM evaluation, GlassHead improves substantially over the reported baselines. For example, Table 1 reports PSNR 16.00 versus 12.90 for the strongest baseline on that metric, LPIPS 0.230 versus 0.347, and DreamSim 0.061 versus 0.120, while also improving both eyeglass-specific metrics. The method reconstructs a Gaussian head in a single forward pass in under one second and uses 4.75 GB GPU memory at inference.
Critical weaknesses
  1. The most important concern is that the main quantitative evaluation is performed on EPPM-20K itself. The training and evaluation identities are disjoint, but the target views are nevertheless produced using essentially the same portrait/video-generation pipeline that defines the training distribution. Therefore, the results convincingly demonstrate that GlassHead models the EPPM pseudo-multiview distribution better than the evaluated baselines, but they do not yet establish equally strongly that the recovered lateral structures are geometrically correct for real photographs.
  2. The paper states that GlassHead is trained on the newly constructed EPPM dataset, whereas the competing methods are evaluated using their released implementations. Based on the current manuscript, I do not find evidence that the strongest baselines were retrained. This is a substantial confound because the paper's central premise is precisely that existing methods suffer from insufficient multiview eyeglass supervision. If GlassHead is given a large targeted training set while competing models retain their original training distributions, the very large improvements in Table 1 cannot be attributed cleanly to the proposed reconstruction architecture or supervision strategy.
  3. The most novel conceptual claim is that imperfect video-generated views provide effective supervision for learning complete 3D eyeglass structure. However, the ablations currently focus on the GAN loss, hierarchical decoder, hierarchy depth, image encoder, and initialization strategy. I do not find a controlled experiment that holds the GlassHead architecture fixed while removing the generated non-frontal supervision.
Questions for authors
  1. Can the authors provide quantitative evaluation on genuinely captured multiview data, for example using calibrated NeRSemble views, rather than only qualitative real-world examples?
  2. Can the authors report a controlled ablation in which the same GlassHead architecture is trained without pseudo-multiview supervision, e.g. using only the frontal reconstruction objective?
1 comment

Thank you for the thoughtful review. We appreciate your constructive feedback and have substantially revised the manuscript with additional experiments addressing your concerns point by point.

Critical Weaknesses

  1. Evaluation limited to generated data and insufficient real-world validation.

We agree that evaluation on EPPM-Test alone cannot establish generalization to real-world data. We therefore introduce quantitative comparisons on two real captured datasets (Table 1 and Figure 4):

  • NeRSemble: 40 eyeglass-wearing identities with 600 synchronized real target views.
  • FaceScan: Our independently collected dataset containing 38 subjects wearing five different eyeglass styles, yielding 190 cases and 1,520 selected views.

On FaceScan, which was not used for training or fine-tuning, GlassHead achieves the best results on five of seven metrics. Although UIKA and FlexAvatar outperform GlassHead on several NeRSemble metrics, both methods have used NeRSemble data during training, whereas GlassHead is trained only on EPPM-20K.

We also expand our in-the-wild evaluations (Figure 5 and Appendix F, Figure 14) with portraits captured under diverse lighting, backgrounds, and head poses. These results provide further evidence of generalization beyond our synthetic training distribution.

  1. Unfair baseline comparisons due to training-domain mismatch.

We agree that evaluating only released pretrained baselines on EPPM-Test may introduce a training-domain mismatch. To address this concern, we additionally fine-tune UIKA and GAGAvatar on EPPM-20K, retaining their original architectures and training objectives. Both methods are fine-tuned for 100,000 iterations on two NVIDIA A800 80GB GPUs. The updated results are reported in Table 1, with implementation details in Appendix E. Both baselines improve substantially on EPPM-Test after fine-tuning. We do not fine-tune FlexAvatar because its training code is not publicly available.

  1. Missing controlled ablation of pseudo-multiview supervision.

We have added the requested controlled ablation in Table 2 and Figure 6. Specifically, we train a variant using only frontal reconstruction losses, removing the multiview perceptual and adversarial losses while keeping the GlassHead architecture and training settings unchanged. Without pseudo-multiview supervision, the model produces flattened, billboard-like reconstructions with severe distortions under novel viewpoints. Quantitatively, PSNR decreases from 16.00 to 9.70, while LPIPS increases from 0.230 to 0.498. These results demonstrate that pseudo-multiview supervision is essential for recovering plausible 3D head geometry and lateral eyeglass structures, rather than relying solely on the hierarchical reconstruction architecture.

Questions for Authors

Q1. Quantitative evaluation on captured multiview data.

We now provide quantitative comparisons on NeRSemble and FaceScan (Table 1), using synchronized real camera views rather than generated pseudo-ground-truth images. For NeRSemble, we select 40 eyeglass-wearing identities and evaluate against 600 real target views. We also report additional results on eyeglass-free subjects in Appendix G (Table 4). The evaluation protocol, subject selection, and camera alignment details are provided in Appendix C. For FaceScan, we collect 190 eyeglass-wearing cases and 1,520 multiview images using a synchronized 30-camera capture system.

We further include magnified eyeglass comparisons (Figure 4 and Appendix F, Figures 13–14) to highlight the preservation of temples, hinges, and ear-side attachments.

Q2. Controlled ablation without pseudo-multiview supervision.

We have added precisely this experiment in Table 2 and Figure 6. We retain the same GlassHead architecture but train it using only the frontal reconstruction objective. The frontal-only model produces flattened, billboard-like reconstructions and suffers substantial degradation in novel-view quality. In contrast, the full model recovers more complete and visually coherent head and eyeglass structures. This experiment directly supports our central claim that imperfect H3-generated pseudo-multiview observations provide valuable supervision for learning 3D structures that cannot be reliably inferred from frontal reconstruction losses alone.

Reviewer_GaPrOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper studies single-image 3D reconstruction of eyeglass-wearing heads. The authors construct EPPM-20K from FFHQ portraits by using FLUX.2 Klein to standardize portraits and add eyeglasses, then MiniMax H3 to generate pseudo-multiview videos. They train a feed-forward, hierarchical Gaussian reconstructor using frontal reconstruction and perceptual and adversarial supervision from the generated views. The method is evaluated on held-out EPPM identities with and without eyeglasses, and qualitative results are shown on NeRSemble.

Critical strengths
  1. The paper addresses a concrete difficulty in single-image head reconstruction: the eyeglass temples and their attachment near the ears are poorly observed in a frontal image. EPPM-20K is a potentially useful way to obtain supervision for this setting.

  2. The reported EPPM results and lateral-view examples suggest improved reconstruction of eyeglass-wearing heads.

Critical weaknesses
  1. The comparison does not adequately separate the contributions of data and method. Section 4.1 evaluates released implementations and checkpoints of prior methods on EPPM, but does not report training or adapting them on EPPM. Their poorer performance could therefore partly reflect a training-domain mismatch. Table 2 provides a useful matched-size comparison with a flat Gaussian decoder, but there is no control that isolates the effect of H3-derived pseudo-multiview supervision. This matters for assessing the incremental contribution: hierarchical Gaussian generation has prior art in GSGAN, while Portrait4D-v2 and FaceLift have explored related forms of pseudo-multiview or synthetic supervision for heads, albeit in different settings.

  2. Real-world and eyeglass-specific validation is insufficient for the central claim. Section 4.3 and Appendix E show qualitative NeRSemble results, but report neither quantitative comparisons against its real target camera views nor the number of tested subjects. The paper also states that FLUX.2 preprocessing improves results while potentially changing identity, without quantifying either effect. Glass-DINO and Glass-CLIP measure feature similarity over crops containing both eyes and eyeglasses; they do not directly establish whether the temples, hinges, and ear-side attachments are geometrically correct across views.

Questions for authors
  1. Can you evaluate single-frontal-image reconstruction against synchronized, held-out NeRSemble camera views? Please report the number of subjects, results for eyeglass-wearing and non-eyeglass subjects separately, and results from both raw and FLUX.2-processed inputs. How much does preprocessing change input identity, and were the test subjects checked for overlap with EPPM training identities?

  2. Can at least representative released baselines be trained or adapted on EPPM under a stated data and compute budget? Please also specify the exact checkpoints used in Tables 1 and 4.

  3. With the architecture and training budget held fixed, what is the effect of adding H3-derived pseudo-multiview supervision? This would help distinguish the value of the proposed data pipeline from that of the reconstruction architecture.

  4. Can you provide more direct evidence for eyeglass structure in novel views, particularly the temples and attachments near the ears? For example, region-separated results or an assessment of the side-view structures would clarify what the current cropped feature similarities capture.

1 comment

Thank you for the thoughtful review. We have substantially revised the manuscript with additional experiments and clarifications to address your concerns point by point.

Critical Weaknesses

  1. Difficulty separating the contributions of data and architecture.

We agree that evaluating only pretrained baselines on EPPM-Test may introduce a training-domain mismatch. We therefore additionally fine-tune UIKA and GAGAvatar on EPPM-20K retaining their original architectures and training objectives. The updated results are reported in Table 1, with training details in Appendix E.

We acknowledge that hierarchical Gaussian generation has been explored in prior work. Our main contribution lies in constructing scalable pseudo-multiview supervision from generated videos and effectively learning complete 3D head structures from these imperfect observations.

  1. Insufficient real-world quantitative evaluation.

We agree that evaluation on EPPM-Test alone cannot establish real-world generalization. We therefore introduce two additional quantitative evaluations (Table 1 and Figure 4):

  • NeRSemble: 40 eyeglass-wearing identities with 600 synchronized real target views.
  • FaceScan: Our independently collected dataset containing 38 subjects, five eyeglass styles per subject, and 1,520 selected multiview images.

On FaceScan, which was not used for training or fine-tuning, GlassHead achieves the best results on five of seven metrics. Although UIKA and FlexAvatar outperform GlassHead on several NeRSemble metrics, both methods have used NeRSemble data during training, whereas GlassHead is trained only on EPPM-20K. These additional evaluations provide stronger evidence of generalization beyond our synthetic training distribution.

  1. Insufficient evidence for eyeglass-specific reconstruction.

We strengthen eyeglass-specific evaluation in two ways. First, we use Glass-DINO and Glass-CLIP to quantitatively measure eyeglass-region appearance consistency. Second, we provide magnified eyeglass comparisons in Figure 4 and Appendix F (Figures 13–14), demonstrating better preservation of frame contours, temples, hinges, and ear-side attachments.

  1. Unquantified effects of FLUX.2 Klein preprocessing.

We introduce a controlled comparison between raw and FLUX.2 Klein-processed inputs (Appendix I, Table 7, Figure 20). Although preprocessing can produce visually cleaner reconstructions, it reduces identity preservation. Specifically, CSIM decreases from 0.588 to 0.468 on NeRSemble and from 0.564 to 0.277 on FaceScan.

Questions for Authors

Q1. NeRSemble evaluation, eyeglass-free subjects, and identity overlap.

We now evaluate GlassHead against synchronized real NeRSemble target views rather than relying only on qualitative examples. The eyeglass-wearing subset contains 40 identities and 600 target views, with the identity list, camera alignment, and evaluation protocol provided in Appendix C.

We additionally report quantitative and qualitative comparisons on eyeglass-free subjects (Appendix G, Table 4; Figures 15–17), demonstrating that GlassHead is applicable to both wearing conditions.

We evaluate raw versus FLUX.2 Klein-processed inputs (Appendix I, Table 7, Figure 20). Preprocessing can improve visual cleanliness but reduces CSIM from 0.588 to 0.468 (NeRSemble) and 0.564 to 0.277 (FaceScan).

Regarding identity overlap, EPPM-20K uses disjoint training and test identities. NeRSemble and FaceScan are independently captured real-world datasets, with no identity overlap with EPPM-20K.

Q2. Baseline fine-tuning and training budget.

We additionally fine-tune UIKA and GAGAvatar on EPPM-20K, retaining their original architectures and training objectives (We do not fine-tune FlexAvatar because its training code is not publicly available). Both are fine-tuned for 100,000 iterations using two NVIDIA A800 80GB GPUs. The updated Table 1 reports both pretrained and fine-tuned variants, denoted UIKA (FT) and GAGAvatar (FT). The fine-tuning configurations and frozen modules are described in Appendix E.

Q3. Controlled ablation of H3-derived pseudo-multiview supervision.

We have added the requested ablation in Table 2 and Figure 6. Specifically, we train a variant with the same architecture using only frontal reconstruction losses, removing both the multiview perceptual and adversarial losses. Without pseudo-multiview supervision, GlassHead produces flattened, billboard-like reconstructions. This directly demonstrates the importance of H3-derived pseudo-multiview supervision for recovering plausible 3D head and eyeglass structures.

Q4. Direct evidence for temples, hinges, and ear-side attachments.

We expand our qualitative comparisons with magnified eyeglass regions in Figure 4 and Appendix F (Figures 13–14). These results highlight more complete and visually consistent temples, hinges, and ear-side attachments under novel viewpoints.

Reviewer_c31NOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper proposes a feed-forward generator which maps a single frontal portrait image of a person to a 3DGS scene of the complete head. The paper consists of two major parts:

  1. half-synthetic dataset generation
  2. feed-forward architecture and training

Dataset generation: The paper uses FFHQ images, then uses FLUX-2 to canonicalize the images to a frontal view and studio lighting. Then the video diffusion model MiniMax-H3 is used to generate phone-scan-style videos, generating novel views of the face. The cameras are then estimated post-hoc, which completes the training data.

Architecture: A hierarchical-cascaded MM-DiT architecture generates the Gaussians from coarse to fine. The depth of the hierarchy is determined experimentally. The architecture has similarities, e.g. to the GSGAN architecture, where the point features replace the role of the noise vector.

Training: To accomodate for view-inconistencies in the generated pseudo-g.t. images, the authors mainly uspervise using perceptual and adversarial losses. on the frontal FLUX-2 image is used for per-pixel metrics. The adversarial losses increases sharpness at a small PSNR cost.

The paper uses a held-out subset of their generated dataset for quantative evaluation. Different data is only used for a qualitative example, without comparisons, and without metrics. For this the authors choose NeRSemble, which is a studio dataset, and as such would allow for metrics computation. Also, as a studio dataset, those examples are demonstrating real-world generalization.

Critical strengths
  • The paper shows great feed-forward 3DGS prediction for in-distribution examples. There it significantly ourperforms baselines. Furthermore, stable training using an adversarial loss is not trivial.
  • The generated dataset would be a valuable extension of the FFHQ dataset for the community. Roughly 40k executions of the MiniMax-H3 model have an extremely high computational budget. However, I believe that the generated data looks quite synthetic, likely stemming from the "frontalization" stage, which greately reduces the dataset diversity.
Critical weaknesses
  • The evaluation of the method is lacking. The authors exclusively report metrics computed on a subset of their generated dataset, putting other methods at a significant disadvantage. Even more importantly, the experimental section does not show that the model generalizes to in-the-wild or real world images. The images from the NeRSemble dataset are recorded in a studo dataset with very uniform lighting. Furthermore, computing metrics on that dataset would be possible, so why not compute them and compare against baselines? Additionally, demonstrating performance on some real-world images with non-unifrom lighting would be very important, especially since most of the method is a relatively straight forward extension of existing work, in combination with the proposed (new and powerful) dataset. I think that it is especially important to show out-of-distribution results for papers that work with generated data as training data.
  • Initially, the paper puts a lot of focus on glasses, stating that they are strongly under-represented in current datasets, and that they are interesting due to occlusion, however, they do not play any sepcial role in the method (only a small role in dataset generation). Furthermore, I am wondering why the dataset is paired, i.e. each image has an multi-view image set with and without glasses. This pairing is not really used, the training is only 20k steps, and generating a MiniMax-H3 sequence must be quite expensive, computationally speaking.
  • Finally, the dataset and method does not suppurt animation of the face, which is one of the most important capabilities of 3D avatars. E.g. the strongest baseline, FlexAvatar, also supports animation of the Gaussians.
Questions for authors
  • I am wondering about the alignment to FlexAvatar in the main result figure. It seems that FlexAvatar is not really well aligned. I am not sure whether this is just a visualization issue, or a flaw in the experimental execution, but I am sure that it could be aligned much better.
  • How does the method perform on non-frontal input images? and on images with harsh lighting, and or shadows? Can this be absorbed with Flux? But then the method could still not reproduce the appearance of the input image.
  • This brings me to the next question: Why not keep the photometric variation from FFHQ?
1 comment

Thank you for the thoughtful review and for recognizing the value of our pseudo-multiview dataset, feed-forward reconstruction, and stable adversarial training. We have revised the manuscript and address your concerns point by point.

Critical Weaknesses

  1. Evaluation only on generated data; missing real-world comparisons.

We agree that EPPM-Test alone does not establish real-world generalization. We now compare against the baselines on NeRSemble (40 eyeglass-wearing identities; 600 real synchronized target views) and our independently captured FaceScan (38 subjects, five eyewear styles each; 190 cases and 1,520 selected views). Results appear in Table 1 and Figure 4. On our independently collected FaceScan dataset, which was not used for training or fine-tuning, GlassHead achieves the best results on five of seven metrics. Although UIKA and FlexAvatar outperform GlassHead on several NeRSemble metrics, both methods have used NeRSemble data during training, whereas GlassHead is trained on EPPM-20K. These results provide further evidence of GlassHead's generalization to unseen real-world data.

  1. Lack of truly in-the-wild evaluation.

We have expanded Figure 5 and Appendix F (Figure 14) with uncontrolled portraits exhibiting varied lighting, backgrounds, poses, and eyewear styles. These results qualitatively demonstrate plausible novel views and lateral eyewear structures beyond the synthetic training distribution.

  1. Eyeglass-specific evaluation.

Eyeglasses are a central focus of our training and evaluation. First, we introduce Glass-DINO and Glass-CLIP to quantitatively evaluate eyeglass-region consistency. Second, we provide magnified eyeglass comparisons in Figure 4 and Appendix F (Figures 13–14), demonstrating better preservation of fine-grained structures, including frame contours, temples, hinges, and ear-side attachments. These results highlight the effectiveness of our eyeglass-focused pseudo-multiview supervision.

  1. Purpose of paired glasses/no-glasses data.

Although our work focuses on eyeglass-wearing heads, we do not intend to restrict GlassHead to this setting. Since eyeglass-free portraits are naturally available as an intermediate step in our data generation pipeline, we also generate pseudo-multiview videos from them for training. This allows the model to handle both conditions. Results in Appendix G (Table 4; Figures 15–17) further demonstrate its capability to reconstruct heads without eyeglasses.

  1. Lack of facial animation.

We agree that facial animation is an important capability for 3D avatars. We explored generating expression-varying pseudo-multiview data using video generation models. However, we observed that introducing expression changes often led to identity drift and unstable camera motion, making it difficult to disentangle facial deformation from viewpoint changes and obtain reliable supervision. Therefore, we constrain the generated sequences to static expressions. GlassHead does not currently support expression-driven animation, which we leave for future work.

Questions for Authors

Q1. FlexAvatar alignment.

We confirmed that the original FlexAvatar results were incorrectly aligned. We have corrected the alignment, rerun the evaluation, and recomputed all quantitative metrics accordingly. The updated results and qualitative comparisons are presented in Table 1, Figure 4, and Appendix F.

Q2. Non-frontal portraits, harsh lighting, and FLUX.2 Klein preprocessing.

We add oblique-input results (Figure 4, rows 4 and 6) and unconstrained real-world examples (Figure 5; Appendix F). We also evaluate raw versus FLUX.2 Klein-processed inputs (Appendix I, Table 7, Figure 20). Preprocessing can improve visual cleanliness but reduces CSIM from 0.588 to 0.468 (NeRSemble) and 0.564 to 0.277 (FaceScan). Consequently, all reported evaluations use raw portraits, without inference-time generative preprocessing.

Q3. Why remove FFHQ photometric variation?

The original FFHQ images exhibit substantial variations in head pose, making it difficult to generate consistent camera trajectories using a unified video-generation prompt. We therefore standardize the portraits to frontal views to improve the consistency of video gernation. We agree that preserving the original photometric variation during standardization could improve training diversity and generalization, and consider this a valuable direction for future improvement.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.