acceptodds
Withdrawn from ICLR 2027

Cross-Domain Micro-Action Learning via Inter-Space Alignment and Prototype Calibration

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Prototype distances learned independently on heterogeneous datasets are defined relative to dataset-specific geometries, so the same class-relative score need not carry the same meaning across datasets. We study cross-dataset anchor generality: whether one fixed class-reference system can provide a common class-reference coordinate for SMG and iMiGUE under a prespecified binary polarity ontology while retaining recognition utility. This is a reference-comparability problem rather than a prototype-storage or unseen-domain generalization claim. We realize it through three coupled requirements: a domain-shared class geometry is constructed, frozen source features are aligned to that reference geometry, and residual sample-level deviations are calibrated with the same fixed anchors. Cross-domain Class-conditional Shared Geometry (CCSG), a soft-gated source-to-reference mapper, and Prototype-Guided Anchor-Pair Calibration (PG-APC) implement these roles. On SMG, dataset-specific PG-APC obtains Accuracy/Macro-F1, while CCSG-PG-APC obtains . On iMiGUE, the same five seeds fixed a priori give for CCSG-PG-APC and for PG-APC, a descriptive -point Accuracy gap. Naive pooling also degrades substantially, consistent with reference-geometry mismatch.

Primary area:
unsupervised, self-supervised, semi-supervised, and supervised representation learning
Keywords:
Cross-Domain Learning, Micro-Action Recognition, Class-Conditional Alignment, Inter-Space Alignment, Shared Representation Learning, Prototype-Guided Calibration

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_kk8MOpenReview ↗
Rating
1
Confidence
3
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

This paper asks whether a single class-reference system can be reused across two heterogeneous micro-action datasets, SMG and iMiGUE, once their labels are mapped onto a prespecified binary polarity ontology. The setting is deliberately not unseen-domain generalization: both datasets are observed during training, one anchor pair is shared, and the downstream predictors remain dataset-specific. The pipeline has three stages. Cross-domain Class-conditional Shared Geometry (CCSG) builds a common class-conditional reference from Wasserstein barycenters, bidirectional bridge matching, and alignment, compactness, and separation terms. A supervised soft-gated whitening–coloring mapper then aligns frozen TrackMAE features to this reference geometry, and Prototype-Guided Anchor-Pair Calibration (PG-APC) applies sample-dependent attraction and opposite-anchor separation around the same fixed anchor pair. Over five prespecified seeds, CCSG-PG-APC reaches 68.00% accuracy on SMG and 70.38% on iMiGUE, that is, 1.00 point below dataset-specific PG-APC on SMG and 1.73 points above it on iMiGUE. Directly swapping dataset-specific anchors, or using naively pooled anchors, degrades performance substantially. A four-class HEROES experiment serves as a proof of concept for the multiclass extension. My overall concern is that the evidence does not establish that the learned anchor-distance coordinate is genuinely comparable across the two datasets, nor that the bridge-based CCSG construction is necessary.

Critical strengths
  1. The paper draws a useful distinction between sharing prototype vectors and sharing a compatible reference geometry. The anchor-swap and naive-pooling controls show that independently estimated prototypes in equal-dimensional spaces are not automatically interchangeable, and the paper is careful not to oversell the setting as unseen-domain generalization.

  2. The main train/test protocol is described clearly. Validation and test samples are excluded from reference construction, and the central comparison between dataset-specific PG-APC and CCSG-PG-APC uses the same prespecified seeds with validation-selected checkpoints. The appendix contributes component diagnostics, sensitivity sweeps, and post-feature inference costs, which makes the empirical section easy to audit.

Critical weaknesses
  1. The empirical scope is too narrow for the claimed reference portability. The primary evaluation covers a single dataset pair, one frozen TrackMAE representation, and small test sets. HEROES uses a different feature configuration and is explicitly not a cross-dataset evaluation, so the paper offers little evidence that the proposed common coordinate extends beyond SMG and iMiGUE.

  2. The recognition gains are small and inconsistent, and the statistical support is thin. CCSG-PG-APC trails dataset-specific PG-APC by one accuracy point on SMG and leads by only 1.73 points on iMiGUE; both gaps are modest relative to run-to-run variation, yet no paired confidence intervals or significance tests are reported, and most component ablations are single-run diagnostics. Table 3 further mixes a prespecified diagnostic seed with post hoc maxima from the five-seed pool for two iMiGUE rows, so it cannot be read as a controlled comparison.

  3. The method reads as an elaborate integration of established components rather than an isolated new mechanism. Wasserstein barycenters, bridge/flow matching, kernel distribution alignment, moment matching, supervised soft routing, prototype attraction and separation, ranking losses, and dataset-specific prediction heads are all combined. The added complexity is substantial relative to the small, inconsistent empirical gains, and the dense terminology makes it hard to tell which component actually matters.

  4. No experiment directly verifies that a common reference coordinate has been learned. Both datasets are seen during training, the routing gate is class-supervised, and the final classifiers are dataset-specific; taken together, these can absorb domain mismatch even if signed anchor-distance scores carry different meanings across the two datasets. The swap experiment only shows that independently estimated prototypes are not portable — it does not show that CCSG produces a calibrated common coordinate. Nearest-anchor prediction, a single shared classifier, cross-dataset classifier transfer, or calibration of the signed distance difference would all probe the central claim more directly, and none of them are reported.

  5. The CCSG construction itself is under-specified. The empirical Wasserstein barycenter comes without details on its support, solver, transport regularization, initialization, convergence behavior, or update schedule. The coupling between child and root endpoints in the bridge objective is also unclear: if ξ₀ and ξ₁ are independently sampled, the objective resembles stochastic interpolation or conditional flow matching more than a Schrödinger bridge in any precise sense.

  6. The necessity of CCSG is never isolated against simpler alignment methods. There is no control that replaces CCSG with a simpler reference while keeping the mapper, PG-APC, losses, and prediction heads fixed — for example, class-wise pooled means, Gaussian or empirical Wasserstein barycenters without learned bridge fields, or class-conditional CORAL. This omission matters especially because the selected mapper uses α=1, which discards off-diagonal covariance information and reduces the moment transform to diagonal rescaling and shifting.

Questions for authors
  1. How exactly are the Wasserstein barycenter and the bridge endpoint pairs constructed numerically? It would help to know the root support size, domain weights, ground cost, regularization, solver, stopping criterion, initialization, update schedule, and whether ξ₀ and ξ₁ are independently sampled or coupled through a transport plan. If the endpoints are independent samples, some clarification of the term "Schrödinger bridge" is in order.

  2. Could you report the five paired per-seed differences between CCSG-PG-APC and dataset-specific PG-APC, along with a paired confidence interval or a permutation test? The mean differences are small compared with the marginal standard deviations, so the current reporting makes the comparison difficult to judge.

  3. Is there a direct way to test that the signed anchor-distance coordinate carries the same predictive meaning on both datasets? Nearest-anchor prediction, one shared linear classifier, cross-dataset classifier transfer after mapping, or calibration curves for the signed distance difference on each dataset would all address the central claim more directly than separate dataset-specific classifiers.

  4. How does CCSG fare against simpler shared-reference constructions under a matched downstream pipeline? Replacing CCSG with class-wise pooled means, a Gaussian or empirical Wasserstein barycenter without learned bridge fields, class-conditional CORAL, or diagonal whitening–coloring — while holding the mapper, PG-APC, losses, and predictors fixed — would isolate its contribution.

Reviewer_1JgBOpenReview ↗
Rating
1
Confidence
1
Informativeness
1
Presentation
1
Soundness
1
Read review
Paper summary

The paper combines class-conditional shared geometry, source-to-reference alignment, and prototype-guided calibration to reuse class anchors across SMG and iMiGUE.

Critical strengths
  • The shared-reference question is potentially interesting.
Critical weaknesses
  • The presentation needs substantial revision. And I found the paper difficult to follow. The three “consistency” requirements are explained repeatedly, while the actual training procedure remains unclear. Figure 1 also leaves ambiguity about which components are shared across datasets.
  • Using the same anchors does not necessarily make distance scores comparable across datasets. Each dataset still has its own predictor, which could compensate for differences in these scores. Classification accuracy alone therefore does not establish the claimed comparability.
Questions for authors
  • Can the authors directly evaluate whether anchor-relative scores have a consistent meaning across datasets, for example by transferring a decision threshold without fitting a separate classifier?
  • How is the Wasserstein barycenter in Eq. (3) computed, and how are source and reference samples paired for bridge training? These details are difficult to recover from the current description.
Reviewer_gAp3OpenReview ↗
Rating
3
Confidence
1
Informativeness
2
Presentation
1
Soundness
3
Read review
Paper summary

The paper investigates cross-dataset anchor generality, examining whether heterogeneous datasets can share a fixed class-reference system. It addresses the problem that prototype distances learned independently on different datasets. The paper proposes a framework combining Cross-domain Class-conditional Shared Geometry (CCSG), a soft-gated source-to-reference mapper. CCSG builds the shared reference, the source-to-reference mapper aligns frozen features, and PG-APC calibrates them with the same fixed anchors. Experiments on SMG and iMiGUE demonstrate that the shared-reference framework can retain competitive recognition performance while enabling anchor reuse across datasets.

Critical strengths

The paper carried systematic study of the problem related to "prototype distances learned independently on heterogeneous datasets". It provided good insight on the related issues. The paper seems mathematically strong and it has demonstrated good understanding of the current subject area.

Critical weaknesses

The overall presentation of the paper could be improved. Mathematic equations will need clear notation and explanation. Some figures appear to be placed inappropriately and are not well aligned with the surrounding text or relevant discussions, which affects the clarity and readability of the paper.

Questions for authors

None.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.