acceptodds
Withdrawn from ICLR 2027

Fre-Flow: Offline Multiple Appropriate Facial Reaction Generation with Frequency-Aware Flow Matching

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Facial Reaction Generation (FRG) aims to synthesize contextually appropriate listener facial reactions conditioned on speaker behaviors in dyadic interactions. Since the same speaker behavior may elicit multiple appropriate reactions, FRG is inherently a one-to-many generation problem, motivating the Multiple Appropriate Facial Reaction Generation (MAFRG) task. However, existing MAFRG approaches typically employ unified spatio-temporal modeling despite the inherently multi-scale temporal structure of facial reactions. Such frequency-agnostic learning can cause cross-scale entanglement between global reaction trajectories and local variations, leading to unstable global dynamics and over-smoothed local details, while diffusion-based MAFRG approaches additionally incur costly iterative denoising for long-sequence generation. To address these challenges, we propose Fre-Flow, a frequency-aware Conditional Flow Matching (CFM) training strategy for offline MAFRG. Fre-Flow introduces two key components: (i) Frequency Flow Matching (FFM), which constructs frequency-aware spectral states and target velocities with adaptive noise scales, and (ii) Frequency Loss Optimization (FLO), which supervises velocity learning through frequency-weighted spectral matching and high-frequency energy regularization. During inference, Fre-Flow generates diverse listener facial reactions through efficient few-step ODE integration. Experiments on the MARS dataset show strong generation quality and inference efficiency, with a favorable balance among appropriateness, diversity, and structural fidelity.

Primary area:
generative models
Keywords:
Multiple Appropriate Facial Reaction Generation

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_YhUMOpenReview ↗
Rating
2
Confidence
2
Informativeness
1
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes a frequency-aware conditional flow matching method for offline multiple appropriate facial reaction generation. It introduces two components: frequency flow matching, which treats different frequency bands separately, and frequency loss optimization, which adds frequency-based supervision. The experiments compare the method with existing approaches in terms of reaction quality, diversity, synchrony, and inference speed.

Critical strengths
  1. The paper is well organized. The introduction gives a clear explanation of the task and its challenges, and describing FFM and FLO separately makes the method easier to follow.
  2. The inference speed is a practical advantage: Table 2 reports a reduction from 17.46 s to 3.44 s compared with PerFRDiff. Table 1 also shows the best FRDist among the compared methods and competitive FRCorr. These results make conditional flow matching worth considering for this task, although the benefit of the frequency-aware components is less clear.
Critical weaknesses
  1. I am not fully convinced by the novelty of FFM and FLO. FFM divides the spectrum into predefined frequency bands with manually chosen coefficients, while FLO adds frequency-weighted velocity supervision and high-frequency energy regularization. Given the existing work on frequency-aware modeling, I would like a clearer explanation of what is new here and why these particular choices are needed for facial reaction generation. The paper also does not test how sensitive the results are to the band boundaries and noise coefficients, so it is hard to judge how well these choices generalize.
  2. The ablation results leave me unsure whether FFM and FLO offer a clear advantage over plain CFM. In Table 3, plain CFM actually has a better FRDist than the full model, at 136.69 versus 137.78. Adding FFM worsens it to 141.94, and FLO recovers only part of that gap while reducing FRDiv from 0.1404 to 0.1324. The FRCorr improvement from 0.60 to 0.70 is encouraging, but the results show a trade-off rather than consistent gains. Separate ablations of FLO’s two loss terms and results across multiple runs would help establish whether the added complexity is worthwhile.
Questions for authors
  1. What is the main technical difference between FFM/FLO and existing frequency-aware methods, beyond applying them to MAFRG?
  2. How did you choose the frequency-band boundaries and noise coefficients? How much do the results change with different choices?
  3. Since plain CFM achieves a better FRDist in Table 3, where do you see the clearest benefit of adding FFM and FLO?
  4. Could you separate the effects of FLO’s two loss terms and report results across multiple random seeds?
  5. Could you show more comparisons with strong baselines, including failure cases, to help readers see when the frequency-aware components help and when they do not?
  6. Please also check the citation formatting for consistency with the conference style.
Reviewer_Q4suOpenReview ↗
Rating
1
Confidence
4
Informativeness
1
Presentation
1
Soundness
1
Read review
Paper summary

The paper studies offline multiple appropriate facial reaction generation, i.e., generating several plausible listener reactions from the speaker's audio, 3DMM and emotion features. It proposes Fre-Flow, a conditional flow matching training strategy with two parts: (i) Frequency Flow Matching (FFM), which builds the interpolation state and target velocity in the rFFT domain with a per-band coefficient, and (ii) Frequency Loss Optimization (FLO), a frequency-weighted velocity loss plus a high-frequency energy regularization. Experiments are on the MARS dataset, with comparisons to previous MAFRG methods.

Critical strengths
  • The idea of treating low- and high-frequency facial dynamics differently is intuitive for facial reaction sequences.
  • The method is simple and does not change the network architecture, so it is easy to plug into existing CFM models.
Critical weaknesses
  • The ablation does not support the main claims. The plain CFM baseline already has the best FRDist (136.69 vs 137.78), so the headline FRDist improvement over prior methods comes from CFM itself, not from the frequency components.
  • The "efficiency gain" problem is a generic property of flow matching. In Appendix Table 4, adding FLO makes the high-band error worse (1.2359 -> 1.4206), which contradicts the claim that FLO reduces high-frequency over-smoothing. Likewise, the "cross-scale entanglement" problem is only stated. There is no analysis showing that standard CFM actually suffers from it.
  • Fre-Flow is not best on FRCorr, FRDiv/FRVar, or FRSyn. Favorable balance is also not quantified.
  • There is only one dataset (MARS), no qualitative comparison with baselines, and no user study. Some recent methods cited in the paper (e.g., the two ReactDiff papers) are missing from Table 1.
  • The writing is repetitive (Sec. 3 overview repeats 3.1/3.2; Eq. 3 = Eq. 8-9, Eq. 4 = Eq. 17). There are also some inconsistencies, e.g., Appendix C.3 lists an L_CFM term that is not in Eq. 17, and image preprocessing is described although the model does not use images.
Questions for authors

Please refer to weaknesses.

Reviewer_ZZyhOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

Paper proposes Fre-Flow, conditional flow matching method for offline multiple appropriate facial reaction generation from multimodal speaker behavior with few-step inference. Method consists of 2 main components: FFM: spectral-domain training states, target velocities, and per-band coefficients over 4 bands (DC, Low, Mid, High); FLO, a loss combining frequency-weighted spectral velocity matching and high-frequency energy regularization. Main contributions are frequency-aware flow matching training per frequency band, and few-step ODE sampling instead of iterative diffusion denoising. Evaluation covers Trans-VAE, PerFRDiff, REGENN, GAFRG, SC-Diff methods, and CFM ablations

Critical strengths
  1. Fre-Flow method is well-motivated: facial reactions contain slow global trajectories and fast local variations, few-step flow matching gives lower inference cost compared to iterative diffusion denoising.
  2. Paper is easy to follow, implementation is detailed, authors report sigma_k values, band boundaries, architecture, number of steps and latency. Extensive ablations: isolates FFM and FLO over CFM baseline, includes spectral and temporal analysis (PSD-JSD, High-band error, Quarter drift)
  3. Comparison with 5 offline MAFRG baselines on MARS: lower FRDist than prior baselines and ~5x lower latency than PerFRDiff.
Critical weaknesses
  1. Ablation results are not fully clear. CFM baseline achieves the best FRDist (136.69, Table 3), better than the full model (137.78), while contributions attribute lower FRDist than prior baselines to Fre-Flow
  2. Significance of FFM is not clear. In Eq. 8 sigma_k seems to only slightly scale the data term; not clear how it affects the noise scale, and how it gives more flexibility to high frequencies? (Sec. 3.1)
  3. Over-smoothing claim is not supported. High-band error worsens with FLO from 1.2359 to 1.4206 (Table 4 supp.), worse than plain CFM (1.3432)
  4. Latency of the CFM baseline is not included in the ablations, the contribution of each component and the efficiency compared to the CFM baseline are not clear. Main improvement may come from CFM, not from the proposed components
Questions for authors
  1. Additional ablations. Could latency of the CFM baseline be added to the ablations to quantify the contribution of each component? This would clarify how the efficiency compares to the CFM baseline
  2. Clarification. In Eq. 8 sigma_k seems to only slightly scale the data term; how does it affect the noise scale, and how does it give more flexibility to high frequencies?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.