acceptodds
Under review as a conference paper at ICLR 2027

FLARE: Fast Lightweight Avatars with Real-time Expression

Abstract

The deployment of interactive digital avatars, particularly for low-latency applications like LLM-driven interaction, is often hindered by a trade-off between rendering speed, visual quality, and generalization. While many methods achieve real-time frame rates, they can fall short of the high-throughput demands required for truly scalable and responsive systems. To address this, we present Fast Lightweight Avatars with Real-time Expression (FLARE), a one-shot 3D head avatar framework designed for high-speed synthesis. Our approach uses a decoupled Core-Shell Gaussian representation, with a dynamic FLAME-based Core and a static Shell for finer details. Crucially, we replace heavy per-frame neural deformation with a Hybrid Kinematic-Neural Deformation architecture. This relies on a one-time geometric binding based on barycentric coordinates as a gross kinematic base, paired with an lightweight Multi-Layer Perceptron (MLP) for high-frequency residual deformation and opacity control. This decoupled design effectively eliminates key computational bottlenecks in prior work. Consequently, our method achieves rendering speeds of over 600 FPS for motion reenactment and over 350 FPS for end-to-end speech-driven generation, measured at resolution on a single NVIDIA A100 GPU. Extensive experiments show FLARE achieves state-of-the-art inference speed while maintaining competitive reconstruction quality, offering a practical and scalable solution for deploying high-fidelity avatars in latency-sensitive applications. Code will be available upon publication.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
Talking head

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_GUUUOpenReview ↗
Rating
2
Confidence
4
Informativeness
1
Presentation
2
Soundness
2
Read review
Paper summary

FLARE proposes a one-shot gaussian head avatar framekwork, consisting of Core gaussians on FLAME vertices and Shell gaussians on a plane lifted with 3D offsets. Each shell point is bound once to its KNN(K = 3) core vertices and follows the core gaussian’s deformation. FLARE adds a MLP for per-frame offset, color and opacity residuals. FLARE achieves GAGAvatar-level reenactment at 630 FPS and can be extended to audio-driven synthesis modality at 353 FPS on A100.

Critical strengths
  1. Practical design enables competitive visual quality and inference speed. FLARE matches GAGAvatar’s self-reenactment quality in terms of PSNR and LPIPS without per-frame neural renderer at about 9 times of the speed.
  2. FLARE enables seamless extension to audio-driven talking head generation with the same logic. FLARE remains competitive in terms of quality while it accelerates FPS about 20 times from the second-fastest model.
Critical weaknesses
  1. Novelty is incremental over GAGAvatar (line 820). The Core and L_attach come from GAGAvatar and the Shell is its lifting with one plane instead of two without proving its sanity for even semi-side views. Some ablations related to Shell guassians(w/o SG, w/o GB in Table 3) are expected given that Core tightly bounded to the mesh, while LAM already animates without a per-frame network and achieves 547.5 fps. The gain from KNN shell binding and residual MLP is modest, so I take these as the strong weaknesses for my rating.

  2. Latency is not measured and the quality improvement is marginal. Throughput is only reported while latency is the stated motivation(line 038). FLARE loses to best metric on 3/7 and 1/3 evaluation metrics for each self/cross reenactment and nuanced expressions to GAGAvatar(3, 6 and 8th rows in Figure 2.), which is not “almost imperceptible” (line 440). Additionally, evaluation is in-distribution for training data although held-out without HDTF evlaution. Audio-driven modality also loses in two metrics and such sync and naturalness inferiority look explicit compared to KDTalker(supplementary demo, especially in ad_driven_with_text_1.mp4). These weaknesses are supportive to rating.

Questions for authors

Q1. In Table 1, metrics on LAM reported from own LAM-20K running while others are all copied from GAGAvatar publication without explanation. Can the protocol be unified to authors’ own run for each baseline at least for some appeared in qualitative results, with additional evaluation on HDTF? Q2. Could concerns in W1 be resolved? Q3. No AI use statement while it is a mandatory section for ICLR 2027 submission?

Reviewer_hk5tOpenReview ↗
Rating
2
Confidence
4
Informativeness
1
Presentation
2
Soundness
2
Read review
Paper summary

The paper proposes FLARE, a generalized avatar generation method. The input is a source image from which the model predicts an animatable 3D gaussian representation. Once the representation is generated it can be animated with a 3DMM base (LBS rigged) and then images can be rendered.

Technically, the method is relatively clear and follows existing works: first predict a base FLAME representation, then add gaussians on top of it that can then be animated with a skinning / MLP combination. The source animation can either be from Audio or visual inputs.

Note that I reviewed the paper before at Neurips. The paper contains no changes aside from a newly submitted video. Hence, my review will be similar as well. To my understanding, no issues in the main paper were addressed by the authors despite being promised in the rebuttal.

Critical strengths

Strength: the paper addresses an important problem (fast and visually appealing avatar generation from an image that can then be rendered in real time). The main focus of this work is to make existing generalized avatar generation methods faster.

Critical weaknesses
  • the overall pipeline is not that novel and similar variations have been used in the literature. It's essentially a generalized 3dgs avatar pipeline with known components.
  • the paper should discuss the difference to other 3dgs avatar generators; the most similar one would be GGHead: Fast and Generalizable 3D Gaussian Heads - Kirschstein et al. 2024. To my understanding, that's the same setting and a very similar method idea. How does it compare?

In my previous review I also asked for a video to properly evaluate the method better. While the authors addressed none of the reviewers comments in the main paper, there is now a submission of a supplemental video; however, the quality is not overwhelming.

Another open question whether the method is run on a managed cluster which takes the real-time / fast storyline somewhat away. To my understanding it cannot be run in an interactive setting / demo.

Questions for authors

No open questions.

Reviewer_vPKuOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper proposes FLARE, a one-shot (single source image input) 3D Gaussian head avatar pipeline that supports both video-driven reenactment and audio-driven animation. FLARE uses a core-shell representation, the FLAME-based core provides facial geometry and the shell generates separate appearance details. For animation, FLARE combines geometries binding with a lightweight neural residual model. The core component is driven by FLAME motion, and the shell component follows geometric anchors with expression-conditioned learnt adjustments of positions, color and opacity of the Gaussian splats. A Wav2Vec-TCN is used to predicted motion parameters from audio in case of audio input.

The experiments include qualitative and quantitative comparisons, covering self-, cross-identity reenactment, and audio-driven generation. The proposed method presents competitive results to state-of-the-art method in case of selected reconstruction, identity and motion metrics, with a significantly improved generation speed (630 fps for reenactment and 353 fps for audio-driven).

Critical strengths
  1. This practical design and engineering provides fast inference, which is quite useful for real-time applications. The reported inference speed is significantly higher than most representative baselines, achieving a similar and higher speed level compared to LAM-20K.

  2. On the selected qualitative metrics, the method achieve competitive performance. Compared to representative baseline of human head avatars, FLARE achieve SOTA performance on PSNR, SSIM, LPIPS and identity similarity (self-reenactment), and best CSIM and AED scores on the audio-drive task. These results suggest a useful speed–quality trade-off under the reported evaluation protocol.

Critical weaknesses
  1. Concerns about technical novelty and inaccurate discussion of the related work. The proposed method largely combines existing ideas, including a similar representation closely compared to GAGAvatar, also the geometric binding and lightweight residual deformation. Although the authors have discussed this similarity, what new technical insights is provided in the design of FLARE is not fully discussed. Specifically, Appendix A incorrectly describes GaussianAvatars as binding each Gaussian to a parent vertex, whereas it uses the local coordinate frame of a mesh triangle. This makes the claimed distinction misleading. Similar geometric parameterizations with barycentric coordinates and normalized displacement are also used in SplattingAvatar [1]. Overall, the work appears to be a useful engineering optimization of existing approaches. But the paper provides limited new insights into the representation or animation challenges of this task.

  2. Concerns about dynamic detail quality and the speed-quality trade-off. In the supplementary demos, FLARE’s mouth movements appear less expressive than KDTalker, with less appearance detail change inside the mouth. Besides, GAGAvatar produces more expression dynamic details, such as forehead wrinkles, while these details appear relatively static in FLARE, while the architecture designs of FLARE and GAGAvatar are actually very close.

These differences may partially reflect the trade-off from simplifying neural refinement. However, the paper describes the loss of micro-expressions as “almost imperceptible,” while the demos suggest a more noticeable difference. The paper should discuss this trade-off more clearly and evaluate dynamic detail quality more directly (e.g., with more fine-grained expression scores or user study), rather than relying mainly on aggregate metrics.

[1] SplattingAvatar: Realistic Real-Time Human Avatars with Mesh-Embedded Gaussian Splatting, Shao et al., CVPR 2024.

Questions for authors

Q1. Could the authors clarify the technical novelty and contribution beyond combining existing representation, binding and refinement techniques? In particular, how does the proposed design advance in Gaussian representations, after correcting the description of GaussianAvatars?

Q2. LAM shows noticeable head jitter in the sup.mat. videos. Could the authors explain its source? Are the driving motion estimates, camera alignment and temporal processing consistent between LAM and FLARE? Differences in these settings could also affect reconstruction metrics.

Q3. LAM also provides a public audio-driven pipeline through its Audio2Expression module. Could the authors include this baseline in the audio-driven evaluation?

Q4. Could the authors report the first-frame delay of the framework? This would help assess the proposed interactive use case beyond steady-state FPS.

Q5. Appendix E.2.3 averages expression parameters over the entire video before computing audio-driven AED. A video often contains multiple expression changes, and this averaging can smooth them out, overlook the temporal alignment, and hide effects of temporally strong expressions. Could the authors additionally report a frame-wise or a more temporally sensitive expression metric?

Q6. Appendix B.3 describes a 69-dimensional motion output, while Appendix B.2 specifies 100 expression dimensions, 6 jaw/head pose dimensions and 6 eye pose dimensions as the dynamic input to the residual network. How are these parameter spaces mapped to each other?

Q7. Could the authors visualize separate Core-only, Shell-only and combined renderings across different expressions? Understanding which components actually model facial appearance, peripheral details and expression-dependent changes could provide extra technical insights and help us better understand the method.

Q8. Could the authors provide more complete descriptions and labels for the supplementary files? In particular, what does sync_improv.mp4 demonstrate? And what do its left and right panels represent? Which setting corresponds to the main results?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.