- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
MAM-Reactor generates ten facial reactions using a frozen deterministic anchor and nine query-conditioned residual candidates. It uses a bidirectional Mamba backbone, expression-specific queries, diversity constraints, balanced Sinkhorn alignment to a set of same-session reactions, and a final restricted valence–arousal refinement. The method targets the quality–diversity trade-off in multiple appropriate facial reaction generation.
Critical strengths
Preserving an anchor while learning alternatives is a reasonable approach to this trade-off. The staged experiments show how diversity and reaction distance change as components are added. The paper reports both target-to-prediction and Hungarian coverage diagnostics, rather than relying only on prediction-to-target distances. Producing the set in one forward pass is practically attractive. The supplement includes model code and a checkpoint.
Critical weaknesses
1. The experiments do not establish why balanced transport is the right set-alignment mechanism. The central idea uses established ingredients: parallel learned queries, entropy-regularized transport, and a sequence-alignment loss. These need to be positioned against work on set prediction, Sinkhorn transport and Soft-DTW [1–3]; the Mamba backbone also needs its original citation [4]. These are methodological references, not additional facial-reaction baselines.
More substantially, the current ablation compares the set-alignment stage with removing the alignment package. It does not compare balanced transport against nearest-target matching, a bidirectional/Chamfer objective, or hard Hungarian matching using the same costs, regularizers and training budget. P2 to P3 also adds training. Without these controls, the gains could come from the added style/FRD supervision rather than balancing the correspondence. Balanced marginals constrain transport mass, but do not guarantee distinct outputs: identical candidates can split their mass over several targets.
There is also a modeling assumption to test. Uniform weighting treats the available same-session reactions as equally representative, and eight fixed expression queries encourage particular expression types irrespective of their appropriateness in a given context. The compatibility head and mixture query help, but do not by themselves establish that all eight explicit candidates are valid alternatives. Please measure per-query appropriateness and compare uniform with context-dependent target weighting.
2. The evidence remains too dependent on one benchmark's scoring geometry. Several objectives directly reproduce aspects of FRD and FRDiv, including AU rounding, channel allocation and a diversity budget. Optimizing these quantities is legitimate, but an independent evaluation is needed to show that the resulting diversity corresponds to plausible reactions. The reported coverage gains are useful, yet remain distances to the same finite session-level target set. There is no human evaluation or second dataset establishing that the additional candidates are natural and contextually appropriate, and no variability across training seeds or sampled target sets.
The synchrony evaluation is especially limited. Appendix A.1 computes FRSyn using only the first candidate, which is the frozen anchor; it therefore cannot assess the nine learned alternatives. The reported 49-frame value is also the boundary of the lag search, while ground truth is already close to that boundary. This raises a question about the diagnostic, rather than proving that the generated motion is delayed by exactly 49 frames. Please validate it and report synchrony for the complete set. The supplementary README further identifies the checkpoint as evaluated against session-all-GT targets, while training samples ten targets. This can be a valid protocol, but the exact target pool and evaluator version must be specified for every comparison, including imported baseline numbers. The archive deliberately omits the evaluator and training launchers, so the provided checkpoint alone does not make the reported table reproducible.
I lean toward rejection because the central alignment choice and the validity of the generated alternatives need stronger evidence.
Questions for authors
- How does balanced OT compare with nearest-target, bidirectional/Chamfer and Hungarian matching under identical costs and training budgets?
- Are all explicit expression queries appropriate for each input? Please report per-query failures and test target weighting rather than assuming uniform mass.
- Can independent raters judge the complete reaction set, and do the conclusions survive different random seeds and target-set samples?
- Can you clarify FRSyn's use of the anchor only, its frequent boundary optimum, and the exact all-GT evaluation protocol? Please provide the evaluator/configuration needed to reproduce the checkpoint results.
Presentation
The main text repeats the anchor/diversity/alignment motivation, while many decisions that actually determine the objective appear only in the appendix. A compact loss/stage table would be more useful. Please fix references such as “Eq. equation 31” and add the missing methodological citations. These writing issues are secondary to the experimental concerns above.
References
[1] Cuturi. Sinkhorn Distances: Lightspeed Computation of Optimal Transportation Distances. https://arxiv.org/html/1306.0895
[2] Carion et al. End-to-End Object Detection with Transformers. Sec. 3.1, parallel query-based set prediction and bipartite matching. https://arxiv.org/html/2005.12872
[3] Cuturi and Blondel. Soft-DTW: a Differentiable Loss Function for Time-Series. https://arxiv.org/html/1703.01541
[4] Gu and Dao. Mamba: Linear-Time Sequence Modeling with Selective State Spaces. https://arxiv.org/html/2312.00752