acceptodds
Withdrawn from ICLR 2027

WorldMER: Recognizing Micro-Expressions through Temporal Closure

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Existing micro-expression recognition (MER) methods typically aggregate facial motion cues across the onset-to-apex and apex-to-offset phases, overlooking the predictive biomechanical relationship between muscle activation and relaxation. To explicitly model this underlying physical coherence, we draw inspiration from latent world models, which excel at learning transition dynamics in state spaces. We introduce temporal closure as a bidirectional predictive constraint between onset-to-apex activation and apex-to-offset relaxation. Given either observed phase, the model predicts the latent representation of its counterpart. Based on this formulation, WorldMER learns two direction-specific transitions between onset-to-apex and apex-to-offset latent states. Their predictions encode phase dynamics shared across expressions, while the residuals between predicted and observed states expose sample-specific departures from those dynamics. A reliability-aware gate then regulates predicted and residual cues at the token-channel level while retaining observed motion as the evidence anchor. Under subject-independent evaluation, WorldMER achieves 90.33% UF1 and 90.30% UAR on the 3DB Composite benchmark, and obtains the best UF1 across the three-, four-, and seven-class CAS(ME) protocols. Ablations show that transition-derived predictions outperform observed motion alone and that bidirectional prediction, residual reasoning, and adaptive fusion provide complementary gains. Our code is available in an anonymous repository at https://anonymous.4open.science/r/WorldMER/.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
Micro-expression recognition, Temporal modeling, Bidirectional Latent World Model, Affective computing

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_sJAaOpenReview ↗
Rating
1
Confidence
3
Informativeness
2
Presentation
3
Soundness
1
Read review
Paper summary

This paper proposes WorldMER, a micro-expression recognition framework that models the relationship between onset-to-apex activation and apex-to-offset relaxation through bidirectional latent prediction. Two direction-specific predictors estimate the latent representation of each phase from the other. The resulting predictions and transition residuals are integrated with observed motion features through adaptive gating. The method is evaluated on CASME II, SMIC, SAMM, their 3DB Composite benchmark, and CAS(ME)3 under multiple class protocols. The paper reports competitive recognition performance and includes ablations examining bidirectional prediction, transition residuals, and adaptive fusion.

Critical strengths

1.The paper addresses a meaningful aspect of micro-expression recognition by explicitly modeling the relationship between activation and relaxation phases. The bidirectional latent-prediction formulation is motivated by the temporal structure of facial motion.

2.The proposed framework integrates predicted phase representations, transition residuals, and observed motion through adaptive fusion. The experimental design includes several component and prediction-direction ablations, which are useful for examining the proposed architecture.

Critical weaknesses

1.A major concern is the LOSO evaluation protocol. According to Appendix F.2, the model is evaluated on the held-out subject after every epoch, and predictions from the epoch with the highest held-out accuracy are retained for final metric computation. If this procedure was used in the reported experiments, it amounts to test-label-based checkpoint selection. Although the held-out subject is excluded from gradient-based training, its labels would influence model selection, introducing potential optimistic bias into the reported results. Consequently, the main performance claims and affected ablation results cannot be considered reliable estimates of subject-independent generalization without evaluation under a genuinely independent checkpoint-selection protocol. This evaluation concern is the primary reason for my rejection recommendation. I would reconsider my assessment if the authors can establish that held-out labels were not used for model selection, or provide results under an independent checkpoint-selection protocol.

2.The contribution of the cross-phase prediction objective is not fully isolated. Although the ablations examine individual components and Table 5 varies the prediction-loss weight from 0.01 to 1.00, they do not clearly separate the benefit of the auxiliary prediction objective from that of the additional predictor branches. A capacity-matched control that retains both predictors and their inference-time features while setting the prediction-loss weight to zero would help establish whether the explicit prediction objective provides benefits beyond the additional model capacity and recognition supervision. In addition, Equations (6) and (9) appear to use different scaling factors for the bidirectional prediction loss, which should be clarified for reproducibility.

Questions for authors

1.Appendix F.2 states that the epoch with the highest held-out subject ACC is selected within each LOSO fold. Were the ground-truth labels of the held-out subject actually used for checkpoint selection in the reported experiments?

2.Can the authors provide results using a checkpoint-selection procedure based exclusively on training subjects, such as an inner subject-disjoint validation split? Please clarify whether the same selection procedure was used for the CAS(ME)3 results and the ablation studies.

3.Table 5 evaluates prediction-loss weights from 0.01 to 1.00 but does not include a zero-weight control. Could the authors provide a capacity-matched ablation that retains both predictor branches and their inference-time features while removing the explicit prediction loss?

4.Could the authors clarify which loss formulation is implemented in practice, given the apparent scaling difference between Equations (6) and (9)?

Reviewer_4R1rOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

WorldMER studies micro-expression recognition using onset-to-apex(OA) and apex-to-offset(AO) motion representations. It learns two separate latent predictors between these phases, trained jointly with classification using stop-gradient MSE targets. Predicted states and absolute prediction residuals are incorporated into observed states through token-channel gates. The method is evaluated using leave-one-subject-out cross-validation on the three-dataset Composite benchmark and on CAS(ME)³ under three-class, four-class, and seven-class classification settings. The supplementary material specifies preprocessing, architecture, optimization, metric aggregation, and additional fixed-fusion direction comparisons.

Critical strengths

(1) A coherent mechanism for using cross-phase prediction at inference. Predictions and absolute discrepancies from observations are used directly for recognition, while an additive observed-state path is retained (Sections 3.3–3.5). This makes the approach more concrete than simply adding an auxiliary predictive loss and provides a testable design for exploiting activation–relaxation relationships. (2) Useful component analysis and implementation transparency. Main-paper Tables 3–5 examine evidence sources, directionality, gates, and loss weights. Supplementary Table 2 further compares fixed observed-plus-predicted fusion while retaining both observed phases in every setting. Bidirectional fusion achieves 89.47% UF1, compared with 88.73% and 88.87% for the two one-way variants. This comparison provides clearer evidence about directionality than prediction-only variants that use different phase inputs. Supplementary Sections D–F also provide useful details on input construction, network architecture, and metric aggregation.

Critical weaknesses

(1)The contribution of the predictive constraint could be better isolated. The component and direction ablations are useful, but introducing the transition predictors also increases model capacity. A comparison using the full architecture with the prediction loss disabled would help establish how much of the improvement specifically comes from cross-phase predictive supervision. (2)Robustness to key-frame localization errors remains unclear. Both phase representations depend on the selected apex frame, so localization errors may affect the observed motion as well as the resulting prediction residuals. The current evaluation does not establish how sensitive the proposed mechanism is to such errors, which limits the conclusions about its applicability when phase boundaries are uncertain.

Questions for authors

(1)Could you report results for the full architecture with the prediction loss disabled, while retaining the transition predictors and fusion mechanism? This comparison would help distinguish the contribution of cross-phase predictive supervision from that of additional model capacity. (2)How sensitive is WorldMER to errors in apex-frame localization? Could you evaluate controlled shifts of the selected apex frame and compare the resulting performance degradation with the observed-only baseline? This would help determine whether cross-phase prediction and residual reasoning improve robustness or introduce additional sensitivity to localization errors.

Reviewer_T3ysOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The authors proposed WorldMER, which learns two direction-specific transitions between onset-to-apex and apex-to-offset in latent space. A temporal closure is proposed as a bidirectional predictive constraint. Two networks predict each phase’s features from the other. The observed features, predictions, and prediction errors are then combined through learned gates for classification. Experiments were conducted on four MER datasets. While the proposed method achieved the best performance on three datasets, its inferior performance on the SMIC dataset, which does not provide groundtruth apex labeling, clear demonstrate its major weakness, i.e., over-reliance on the accuracy of onset/apex/offset detection.

Critical strengths
  1. The method models the relationship between expression formation and relaxation through cross-phase feature prediction. Prediction residuals are then used as classification features, allowing the model to use discrepancies between the two phases as well as their observed motion.
  2. The model uses both predicted features and absolute prediction residuals at inference, with separate gates controlling their contributions. A direct path preserves the observed features while allowing predictions and residuals to contribute through learned gates.
  3. The reported experiments cover Composite and three-, four-, and seven-class CAS(ME)3 recognition. Tables 3–5 examine the main components, while Figure 3 exposes the remaining class-specific errors. The reported UF1 gains on CAS(ME)3 are promising, although they need confirmation after correcting the checkpoint-selection procedure.
Critical weaknesses
  1. The proposed WorldMER heavily rely on the detection accuracy of onset, apex, and offset frames. However, detecting the onset, apex, and offset remains challenging for MER/FER. This has been demonstrated from the experimental results on the SMIC, which is the only dataset without groundtruth apex labeling. The performance of the propose method is inferior to D-FACE, UGPT, and CausalNet. Furthermore, it is unclear how the onset, apex, and offset frames are identified for the experimental datasets. The authors should make it clear for which datasets the ground truth annotations were used.
  2. The world-model claim is overstated. Sections 3.1–3.3 mainly add cross-phase feature reconstruction to a bidirectional network. The model maps between two observed phase representations, without showing that it learns temporal evolution beyond these feature mappings. The current method and experiments do not justify presenting this as a world model.
  3. Appendix F.2 states that each fold retains the epoch with the highest test-subject accuracy. This uses test labels to choose the model. The UF1 gain over UGPT in Table 1 is only 0.08 points, so the results need to be rerun with a separate validation set or a fixed training schedule.
  4. In Table 4, prediction-only AO→OA drops to 78.95% UF1, compared with 87.71% for OA→AO, while using both reaches 89.10%. The reason for this gap is unclear. A weaker branch may still help, but the bidirectional setting also receives both phases. Direct OA-only and AO-only baselines are needed to explain the gain.
  5. The gains over Ra-ICL differ between Tables 1 and 2: 8.17 UF1 points on three-class CAS(ME)3 versus 2.28 on Composite. This is not necessarily an error, but the difference deserves analysis. Please clarify whether each comparison uses matched inputs and evaluation rules, and which classes account for the larger gain on CAS(ME)3.
  6. Equation (6) uses λ/2 for the summed prediction losses, while Appendix F.1 uses λ. Both set λ to 0.50, so the loss weight differs by a factor of two. Please state which version was used, including in Table 5.
Questions for authors
  1. Does the checkpoint rule in Appendix F.2 apply to all reported experiments? Please provide the main results and ablations without using test labels to select the epoch.
  2. What distinguishes this method from a bidirectional network with feature reconstruction, and what justifies calling it a world model? Please include the same network without the prediction loss (λ = 0).
  3. Why does AO→OA perform much worse in Table 4? What are the OA-only and AO-only classification results?
  4. Were the Ra-ICL comparisons made under the same data and evaluation settings? Why is the gain larger on CAS(ME)3?
  5. Which loss formula was used in the experiments?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.