- Rating
- 3
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 3
Read review
Paper summary
The authors propose to use meta-learning with a domain discriminator to alleviate the issues with JMMD and contrastive learning in having multi-model features for multiple domains. They provide a theoretical framework to show these issues as well in theory and how their method does not suffer from the same issues. On multiple datasets they outperform the state-of-the-art
Critical strengths
The paper is well motivated, solves an important problem and is thoroughly evaluated on multiple datasets. The authors also include a theoretical analysis and ablations.
Critical weaknesses
Measurement of Sub-Modal dispersion: They measure this parameter using a GMM distribution assumption. Now their model assumption is also that they following some kind of anisotropic gaussians with sigma_ij as learnt bandwiths. This naturally makes their method perform better than competing methods without this assumption. There is no other metric be this quantitative like OT-based or qualitative like t-sne to gather some more insight.
The inner loop updates are computed using a finite difference style formulation, however it is unclear how close this is approximation is to the real gradient even in praxis. There is no theory behind it and no error estimate. There is not even a numerical experiment showing that during trainig the error is in the ballpark of x% of the norm of the gradient.
Questions for authors
Can you also visualize confirm that the modes are actually separated and do you have some insight into which features they found as different modes?
Can you elaborate on the finite style derived gradient computation and how close they are to the real gradient?
Minor: Check the citations, the core competitor JMMD has the wrong year.
5 comments
Response to Reviewer gYt5 W1
3. Additional metrics distinguish compactness from separation.
We appreciate the suggestion of OT-based evaluation. We prioritized the added metrics because they directly test the failure mode at issue: whether distinct same-class groups remain separated. So we introduced additional metrics in Appendix E.4: , the minimum distance between two same-class sub-mode centers; , that distance relative to within-sub-mode spread; Silhouette, within-group cohesion versus separation; Submode Match, agreement with the nearest same-class source sub-mode center; and Merge Rate, the fraction of same-class reference pairs assigned to the same dominant non-noise cluster.
Fig. 4 reports averages over ten transfer scenarios per dataset. Each entry below is / .
| Dataset | Source | JAN | w/o MetaMMD | META |
|---|---|---|---|---|
| HHAR_SA | 0.0257 / 0.3104 | 0.0563 / 0.2782 | 0.0296 / 0.3222 | 0.0261 / 0.3439 |
| WISDM | 0.0331 / 0.5320 | 0.0506 / 0.2631 | 0.0314 / 0.3062 | 0.0184 / 0.3527 |
| HAR | 0.0398 / 0.1668 | 0.0568 / 0.2609 | 0.0288 / 0.3784 | 0.0239 / 0.3604 |
| Sleep-EDF | 0.0602 / 0.4825 | 0.0265 / 0.4376 | 0.0210 / 0.5643 | 0.0049 / 0.5886 |
Table 4 provides a task where compactness, absolute separation, and correspondence improve together:
| EMG | Silhouette ↑ | Submode Match ↑ | Merge Rate ↓ | ||||
|---|---|---|---|---|---|---|---|
| w/o MetaMMD | 0.1289 | 0.0840 | 0.0083 | 0.0256 | −0.0289 | 43.43% | 56.60% |
| META | 0.0907 | 0.0727 | 0.0478 | 0.1543 | 0.0386 | 58.49% | 18.87% |
Here, the groups become tighter while even their closest centers move farther apart. Improved Silhouette, Submode Match, and Merge Rate provide additional evidence that the lower dispersion is accompanied by retained distinction between groups.
4. Training dynamics and loss-balance analysis provide complementary evidence (Fig. 5b–c).
For WISDM , Fig. 5b tracks , the minimum same-class sub-mode center distance. Selected values are shown below, averaged over three seeds; discriminator pretraining ends at epoch 20.
| Epoch 0 | Epoch 20 | Epoch 24 | Epoch 50 | Epoch 70 | |
|---|---|---|---|---|---|
| w/o MetaMMD | 0.0290 | 0.0155 | 0.0034 | 0.0105 | 0.0174 |
| META | 0.0290 | 0.0155 | 0.0040 | 0.0148 | 0.0276 |
Both methods initially contract, but META subsequently recovers greater separation, reaching 0.0276 versus 0.0174 at epoch 70. This supports improved separation during later joint optimization, rather than an absence of transient contraction.
On WISDM , Fig. 5c varies while keeping the sum of these weights fixed. The table reports all five configurations for the plotted seed.
| w/o MetaMMD: | w/o MetaMMD: | META: | META: | |
|---|---|---|---|---|
| 0 | 0.0313 | 0.0082 | 0.0407 | 0.0092 |
| 0.25 | 0.0444 | 0.0078 | 0.0345 | 0.0221 |
| 0.5 | 0.0386 | 0.0152 | 0.0376 | 0.0258 |
| 1 | 0.0439 | 0.0093 | 0.0406 | 0.0105 |
| (w/o TCon) | 0.0384 | 0.0070 | 0.0348 | 0.0103 |
META achieves both lower dispersion and greater separation at four of the five matched settings. At , separation improves but dispersion increases. These results show why compactness and separation should be evaluated jointly.
5. Assignment-based analyses assess merging across transfer scenarios.
Appendix G.3, Fig. 9 reports pooled, all-class Merge Rate (%):
| WISDM | 26→2 | 21→31 | 7→30 | 20→30 | 2→32 | 12→7 | 25→29 | 12→19 | 18→20 | 4→15 | Mean |
|---|---|---|---|---|---|---|---|---|---|---|---|
| w/o MetaMMD | 18.18 | 15.00 | 13.89 | 12.50 | 10.26 | 10.00 | 7.50 | 4.76 | 4.76 | 0.00 | 9.69 |
| META | 6.82 | 15.00 | 2.78 | 4.17 | 10.26 | 0.00 | 0.00 | 4.76 | 14.29 | 0.00 | 5.81 |
| EMG | 1→3 | 1→2 | 0→3 | 0→2 | 0→1 | 3→1 | 2→3 | 2→1 | 2→0 | Mean |
|---|---|---|---|---|---|---|---|---|---|---|
| w/o MetaMMD | 56.60 | 52.94 | 42.11 | 25.00 | 21.62 | 16.22 | 8.16 | 8.11 | 7.32 | 26.45 |
| META | 18.87 | 23.53 | 2.63 | 19.64 | 10.81 | 16.22 | 8.16 | 16.22 | 21.95 | 15.34 |
Mean Merge Rate decreases from 9.69% to 5.81% on WISDM and from 26.45% to 15.34% on EMG. On WISDM, average noise changes only from 33.65% to 33.82%. Improvements are not universal, as the individual scenarios show.
The Sankey diagrams (Fig. 6) and heatmaps (Appendix G.2, Fig. 8) trace individual reference groups: merged pairs decrease from 3/10 to 0/10 on HAR , from 1/3 to 0/3 on WISDM (both class 5), and from 10/10 to 3/10 on HHAR_SA (cycling), with noise reported separately. The class-specific improvement on WISDM coexists with an increased all-class merge rate, reflecting different outcomes across classes.
Response to Reviewer gYt5 W2
W2. The inner loop updates are computed using a finite difference style formulation, however it is unclear how close this is approximation is to the real gradient even in praxis. There is no theory behind it and no error estimate. There is not even a numerical experiment showing that during trainig the error is in the ballpark of x% of the norm of the gradient.
Response. We thank the reviewer for requesting a quantitative assessment of the approximation. We have added gradient diagnostics in Appendix G.5, Eqs. (57)–(59), and Fig. 11. We clarify that finite differences approximate the mixed derivative used in the outer bandwidth update and the inner model updates use backpropagation and Adam.
1. Why an approximate update can be useful, and what finite differences approximate.
Optimization does not require every update to reproduce the exact gradient in both direction and magnitude. For example, for a nonzero gradient , the estimate , with , has the same direction and changes the effective step size, although its relative error is . Nevertheless, this observation does not establish the quality of our estimator: its direction and magnitude must be measured against a specified reference.
Our approximation uses a centered finite difference to estimate a local mixed-derivative–vector product, following the computational principle used in DARTS (Liu et al., ICLR 2019). To state its error precisely, fix the snapshot, the support/query batches, and the outer-gradient direction . Let denote the predicted support bandwidths, and hold their input embeddings fixed during the two perturbations, as in the implementation. With , define
The local automatic-differentiation reference and its finite-difference approximation are
\widetilde g_\epsilon=-\eta\frac{F(\epsilon)-F(-\epsilon)}{2\epsilon}.$$ If $F$ is three times continuously differentiable on the perturbation interval and $\sup_{|t|\leq\epsilon}\|F'''(t)\|\leq M$, Taylor's theorem gives $$\|\widetilde g_\epsilon-g_{\rm local}\| \leq\frac{\eta M}{6}\epsilon^2.$$ This is a conditional local truncation-error bound, rather than a global smoothness guarantee for a network with piecewise-smooth operations. Finite precision also matters: if each evaluation of $F$ has error at most $u$, the additional error is bounded by $\eta u/\epsilon$. Thus, making $\epsilon$ arbitrarily small need not improve numerical accuracy. **2. Experimental verification of the finite difference supergradient** In **Fig. 11**, on WISDM $2\to32$, gradients are measured every five epochs on fixed probe batches with $\epsilon=0.001$. We compare $g_{\rm local}$ (local mixed derivative) with $\widehat{g}_K$ (differentiating through $K$ inner Adam updates from the snapshot's optimizer state), a reference for the finite-$K$ computation. We distinguish: - $g_{\rm local}$: the local mixed derivative computed by automatic differentiation. - $\widehat g_K$: the reference hypergradient computed by differentiating through $K$ inner Adam updates, initialized with the snapshot’s optimizer state. For an estimate $\widetilde g$ and a specified reference $g_{\rm ref}$, **Appendix G.5, Eqs. (58)–(59)** defines \$ \\text{Cosine}(\\tilde{g}, g_{\\text{ref}}) = \\frac{\\tilde{g}^\\top g_{\\text{ref}}}{\\Vert \\tilde{g} \\Vert_2 \\Vert g_{\\text{ref}} \\Vert_2} \$ Values closer to **1** indicate stronger directional agreement between the finite-difference estimate and the local reference gradient. Local directional agreement: **Fig. 11(a)**. The reference in this panel is $g_{\rm local}$. | $\epsilon$ | Epoch 0 | Epoch 20 | Epoch 40 | Epoch 45 | Epoch 50 | |---|---:|---:|---:|---:|---:| | **0.001 (training)** | 0.9615 | 0.9811 | 0.9136 | 0.9999 | 0.9975 | | 0.01 | 0.9769 | 1.0000 | 0.9977 | 1.0000 | 1.0000 | | 0.1 | 0.9998 | 1.0000 | 0.9988 | 1.0000 | 0.9998 | These measurements support that **the centered finite-difference estimator shows strong directional agreement with its local mixed-derivative reference**
Response to Reviewer gYt5 W1
We sincerely thank the reviewer for the positive assessment of our motivation, theoretical analysis, and experimental evaluation, and for the insightful suggestions on assessing sub-modal structure. Your comments helped us strengthen the evaluation beyond within-sub-mode compactness. We have uploaded the revised manuscript, with all changes highlighted in blue. Below we provide our detailed response.
Response. Thank you for raising this concern. We clarify the distinction between the Gaussian kernel used for training and the GMM used for evaluation, and provide additional density-based clustering, geometric measurements, and visualizations.
1. A Gaussian kernel is not a Gaussian-mixture density assumption.
Since σij is an MMD interaction scale rather than a fitted GMM covariance, and MetaMMD never optimizes a GMM likelihood, the Gaussian kernel confers no built-in advantage under GMM evaluation, though we agree an independent evaluation is indeed valuable. Our revised evaluation separates fixed-reference geometry from independent cluster discovery:
Fixed-reference geometry. We construct fixed reference groups across all methods by applying PCA to standardized time–frequency descriptors and fitting a classwise GMM (BIC-selected K∈{1,2,3,4,5}). We then compute empirical covariance on ℓ2-normalized embeddings within these frozen groups.
Independent cluster discovery. We evaluate classwise GMM/BIC clustering and HDBSCAN on the same normalized embeddings, using identical settings across methods. HDBSCAN uses minsamples=5 and minclustersize=max(10,⌈0.02nc⌉).
Without GMM, HDBSCAN independently detects multiple populated clusters. The following examples show that the observed multiplicity of clusters is not solely an artifact of fitting Gaussian components.
HHAR_SA 1→6, cycling/class 0: 483 samples; Appendix G.2, Fig. 8.
Without fitting a Gaussian mixture to discover these clusters, HDBSCAN identifies four clusters containing 346/483 samples under META.
WISDM 4→15, class 5: 107 samples; additional classwise heatmap analysis.
HDBSCAN identifies two populated clusters containing 98/107 samples under META. The two reference groups have distinct dominant destinations, with 46/53 and 52/54 samples entering their respective clusters.
HAR 12→16, class 5: 91 samples; Fig. 6.
HDBSCAN independently detects four clusters containing 80/91 samples under META. In particular, all 18 R1 samples and 16 R4 samples enter one cluster without MetaMMD but occupy distinct clusters with META.