- Rating
- 3
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 3(+1)
Read review
Paper summary
The submission proposes SGS for DD across domains. It computes DM gradients for each source, then uses Fourier phase agreement to form a consensus update, and adds a domain-specific residual for each synthetic image. Experiments on Digits-DG, PACS, and CORe50-Hard show improved OOD accuracy with DM and HDD-DM. The paper also studies pseudo-domains, component ablations, gradient alignment, and distillation cost.
Critical strengths
SGS consistently improves DM and HDD-DM across the evaluated benchmarks and budgets without changing downstream classifier training. The pseudo-domain results further support its applicability when domain labels are unavailable. Controls with matched gradient norms, individual components, and pixel-space agreement suggest that the gains are not simply due to increasing the update size.. The cost analysis also helps assess the additional work required during distillation.
Critical weaknesses
I find the results promising, but the explanation of why SGS works needs to be more precise:
- Phase agreement is a useful heuristic, but it does not necessarily identify transferable class information. Shared nuisance signals can pass the filter, while spatial shifts can reduce agreement for the same pattern. A large gradient from one domain can also dominate the score. The random-phase analysis does not establish when these effects matter, so the claimed class/domain separation needs stronger support.
- The domain residual in eq9 is . Its reported zero similarity with target gradients needs clarification, since averaging the residuals first would cancel them.
- The efficiency evidence is also limited: the main setting has higher distillation cost, while the comparison at approximately matched compute covers only a single-source setting. I encourage the authors to clarify the diagnostic and separate downstream efficiency from total distillation cost.
Questions for authors
- What allows SGS to distinguish transferable class information from a class-correlated nuisance shared by all source domains?
- Fourier phase changes under translation. How does this affect the consensus update when the same useful pattern appears at different positions across domains?
- Could you compare against an equal-weighted phase score to separate cross-domain agreement from dominance by one large gradient?
- Is the residual–target similarity in tab7 computed per domain or after averaging residuals? The residuals sum to zero by construction, so the aggregation order is essential to interpreting this diagnostic.
5 comments
Response to Reviewer 2GCo (2)
W1. Scope of the random-phase analysis
We agree that Appendix C establishes attenuation only under the random-phase condition and does not establish when the three effects raised in W1 matter. A nuisance shared by all sources is preserved in the same way as class information, which we now state as a limitation (Q1). For the other two cases, the diagnostics on real DM gradients in Q2 and Q3 show that translations barely change except for extreme shifts and that one domain rarely dominates.
We discussed this in Section 4.4 and Appendix C.3.
W2 / Q4. Aggregation in Table 7
We acknowledge the reviewer's concern. The reported value was the signed cosine averaged over the three sources and all synthetic images of all classes. Since the residuals sum to zero, this average largely cancels. Therefore, the near-zero entries did not indicate orthogonality. As reported below, when evaluated per source, the residuals are not orthogonal to the held-out target gradient.
| Target (sources 1/2/3) | Source 1 | Source 2 | Source 3 |
|---|---|---|---|
| Digits M (N/V/Y) | +0.64 | −0.62 | −0.25 |
| Digits N (M/V/Y) | +0.18 | −0.20 | −0.11 |
| Digits V (M/N/Y) | −0.61 | +0.03 | +0.67 |
| Digits Y (M/N/V) | −0.49 | −0.04 | +0.51 |
| PACS P (A/C/S) | +0.57 | +0.08 | −0.47 |
| PACS A (P/C/S) | +0.54 | +0.12 | −0.51 |
| PACS C (P/A/S) | −0.02 | +0.12 | −0.04 |
| PACS S (P/A/C) | −0.27 | −0.12 | +0.27 |
| CORe50-H s5 (s7/s9/s10) | −0.02 | −0.03 | +0.03 |
| CORe50-H s7 (s5/s9/s10) | −0.20 | +0.15 | +0.05 |
| CORe50-H s9 (s5/s7/s10) | −0.10 | +0.28 | −0.16 |
| CORe50-H s10 (s5/s7/s9) | −0.02 | +0.18 | −0.14 |
Each entry denotes the cosine similarity between the residual of one source and the gradient of the held-out target. For example, with M as the target, the residual of N has a similarity of +0.64 and that of V has −0.62. We observe that the signs differ across sources on every target. The residual of a source close to the target points toward it, while that of a distant source points away. This aligns with the visual similarity between domains: M and N in Digits-DG, and P and A in PACS, are highly similar. This also suggests the role of : assigning the source domains uniformly across the synthetic samples diversifies the synthetic set, which may help cover unseen domains.
We have revised the discussion and Table 7 (now Table 9) in Section 4.4 ("Alignment with Unseen Domains"), and added the full table above as Table 17 (Appendix D.1).
Response to Reviewer 2GCo (3)
W3. Downstream efficiency and distillation cost
We consistently distinguish between the two throughout the manuscript:
- Downstream training cost: It is the same as DM, because IPC, resolution, and the training procedure do not change.
- Offline distillation cost: It scales roughly with the number of source domains, about 3.25× DM with three sources (Table 6).
We also extended the compute-matched comparison to the main multi-source setting by running DM for 66k iterations, which matches the wall-clock cost of SGS.
| IPC=10 | Cost | PACS | CORe50-Hard |
|---|---|---|---|
| DM (20k iter.) | 1.0× | 35.6 | 27.4 |
| DM (66k iter.) | 3.3× | 35.0 | 27.5 |
| SGS (20k iter.) | 3.25× | 38.6 | 28.9 |
Running DM 3.3× longer barely changes accuracy on either benchmark, so the gain of SGS is not explained by additional distillation compute. This comparison is added to Appendix D (Table 16).
We are glad to provide further detail on any of these points.
Response to Reviewer 2GCo (1)
We thank the reviewer for the meaningful and valuable comments.
W1 / Q1. Shared class-correlated nuisance
We acknowledge that SGS cannot isolate semantic class information but instead identifies information shared across the source domains. Thus, we have replaced "class-relevant" with "source-consistent" throughout the manuscript. However, note that this limitation is common to other invariance-based methods, such as IRM, Fish, and Fishr. Nevertheless, the source-consistent component is empirically better aligned with unseen domains: gclass is closer to the held-out target gradient than g on every PACS target (cosine 0.27 vs. 0.21 on average; Table 9).
We have discussed this limitation in Appendix A.
W1 / Q2. Translation and the consensus update
We agree that translating an image changes its Fourier phases. However, SGS applies the Fourier transform not to real images but to domain-wise gradients, all computed with respect to the same synthetic image and network. Thus, image-level translation has a limited effect on phase agreement because real images influence these gradients only through the class-mean embedding of each domain, which aggregates them into a coarse feature map. We verified this through controlled shifts, independently translating the real images of each source domain in a random direction at the beginning of distillation and reporting the ranges across target domains.
The mean r remains stable in most settings; the largest change occurs for the 16 px shift on Digits-DG, which corresponds to half of the image width. Furthermore, although Digits-DG does not allow a controlled pose study, CORe50 objects are dynamically moved during capture, so their position and pose vary across images and sessions. SGS still improves DM on CORe50-Hard from 27.4 to 28.9 (Table 1).
We added these results in Appendix C.3 (Table 14).
W1 / Q3. Dominating domain, Comparison with equal-weighted scoring
We first measured how often one domain dominates the score using real DM gradients at IPC=10 at the beginning of distillation, reporting the ranges across the four target domains of each dataset.
Domain grad norm ratio denotes the median over classes of the max/min source-domain gradient norm, and Actual consensus denotes, among the frequency components with r>0.8, the percentage in which every domain is within 90° of the mean phase. These results suggest that:
We added these results in Section 4.4 (Table 8).
We replaced r with an equal-vote score req, computed in the same way as r but from unit-length phase vectors so that every domain receives the same weight, and kept everything else unchanged (Digits-DG, IPC=10).
These results show that the gain of SGS does not depend on magnitude dominance: SGS with req still improves DM on all four targets (65.1 to 66.9 on average). Nevertheless, the magnitude can be helpful for weighting the update: with unit-length phase vectors, a low-magnitude coefficient that contributes little to the update receives the same vote as a dominant one, so components that should be suppressed can be emphasized, which may explain the remaining gap to r.
We discussed these results in Section 4.3 ("Equal-vote Agreement Score") and Table 5.