- Rating
- 2
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper proposes MindAU for editing facial action units (AUs) from an EEG segment and a reference face. An AU-aware encoder learns signal representations, and dual-stream alignment connects them to AU text semantics and identity-reduced visual displacements. The resulting tokens condition a diffusion editor with reference masking and regional supervision. The authors curate E-CAFE from BU-EEG, containing 5,000 pairs across ten individually performed AUs from 28 healthy participants. Evaluation uses two held-out participants and also transfers their signals to synthetic reference identities. The method improves AU accuracy and identity similarity over the reported pipelines, with additional ablations and perceptual evaluation.
Critical strengths
-
A structured formulation with useful empirical support. AU-level supervision provides an interpretable link between biosignals and localized facial edits. The dual-stream design combines semantic alignment with visual displacement supervision, and the ablations support the value of AU supervision and supervised contrastive alignment. The curated benchmark and separate reference-identity protocol provide a useful starting point for further work.
-
Attention to shortcuts and limitations. Reference masking directly addresses the risk of copying the input expression: removing it raises identity similarity but reduces AU accuracy from 27.71% to 5.02%. Fixed-reference and EEG-free controls further examine signal dependence. The frequency/channel analysis and explicit limitations acknowledge unresolved EEG/EMG separation, which is valuable for interpreting the results.
Critical weaknesses
- The neural interpretation and application motivation exceed the demonstrated setting. The experiments use recordings acquired while healthy participants actually perform facial actions. The original BU-EEG paper describes sensors covering the forehead and cheeks and explicitly retains EMG-like and EOG-like activity as useful information for facial analysis [1]. Thus, successful conditioning does not establish decoding of brain-origin facial intent. Appendix I.1 is helpful: after 0.1-50 Hz filtering, removing face-adjacent channels reduces Stage-1 classification accuracy from 32.93% to 28.95%. However, channel removal can also remove neural information, and residual muscle activity may remain. These classifier results do not establish the physiological source driving the final editor. Section 4.5 appropriately acknowledges this uncertainty and the absence of evidence for imagined or attempted movements. Nevertheless, the introduction still positions the contribution as control through brain activity and a step toward assistance for facial neuromuscular disorders. The demonstrated contribution should be framed consistently as editing from recordings during performed actions, with a clearer rationale for this acquisition setting. The title's use of EEG is defensible as a recording modality; the substantive concern is the gap between the neural-control motivation and the available evidence.
[1] X. Li et al. An EEG-Based Multi-Modal Emotion Database with Both Posed and Authentic Facial Actions for Emotion Analysis. arXiv:2203.15829, Sections II.B and III. https://arxiv.org/abs/2203.15829
- The comparisons do not isolate the benefit of the proposed conditioning representation. Appendix B states that the LongCat-Image and FireRed editors remain frozen, whereas MindAU receives task-specific training with reference masking and regional losses. Using the same Stage-1 AU classifier controls the recognition component, but does not isolate the benefit of continuous EEG alignment from adaptation of the editor itself. The results establish an advantage over these off-the-shelf pipelines, while the stronger interpretation that continuous conditioning yields more faithful or intensity-aware control remains insufficiently tested. A baseline using the same editor, training pairs, and optimization with predicted AU labels or embeddings would materially strengthen the comparison. Calibrated intensity evaluation would also help substantiate the claimed advantage beyond AU-category recognition.
Questions for authors
-
Signal interpretation and scope. Which claims are intended to depend on brain-origin information? Please clarify the intended scope throughout the front matter. Is there any end-to-end generation analysis using the face-adjacent-channel exclusion or other artifact-reduction settings, beyond the Stage-1 classifier results?
-
Matched comparison. Can the authors provide a comparison using the same task-adapted editor and training losses, conditioned on predicted AU labels or AU embeddings? What evidence separates the benefit of continuous EEG conditioning, including expression intensity, from editor adaptation?
-
Signal specificity and robustness. Can the authors report paired versus within-participant, cross-AU shuffled EEG results while holding the reference and generation settings fixed? The zero-token control changes the input distribution substantially. Per-participant results for the two held-out subjects would also help assess whether gains are consistent across people.
-
Future validation. A future study of facial motor imagery or attempted movements, with facial EMG and video monitoring to check for overt or covert contraction, would directly test the assistive motivation. A new clinical study is beyond a reasonable rebuttal request; the immediate need is to align the claims with the demonstrated setting.