acceptodds
Under review as a conference paper at ICLR 2027

Meta-Diffusion: Transferable Reverse-Dynamics Coordinates for Data-Asymmetric Generation

Abstract

Many deployments require a model to move rapidly from a data-rich source to a related target observed only sparsely. This is especially consequential where target data are difficult to collect, as in healthcare settings with persistent population or acquisition imbalances. Retraining or fine-tuning a high-capacity generator for every target is costly and can be statistically unreliable when target data are scarce. We introduce Meta-Diff, which instead learns a recurring source–target relation once in a compact task space. A shared encoder maps each distribution to a low-dimensional coordinate; a learned relation transports a new source coordinate to its predicted target, and a shared diffusion backbone generates from that prediction without target-specific retraining. Controlled Gaussian-mixture experiments identify when this works: task identity must survive the representation, the relation must generalise across tasks, and the source–target shift must leave useful headroom for transport. On held-out CIFAR-100 validation tasks, correct source + transport achieves 13.9% lower mean SW than transport from the wrong source. In the final-test one-target comparison, its mean SW is 14.7% lower than target-only modelling and 32.7% lower than full fine-tuning; the main route itself uses no target image. Office-Home exposes a complementary boundary: transport can align target-domain statistics without reliable object identity. Together, these results recast extreme-few-shot adaptation as reusable transport in task space, while making the conditions for its success and failure directly measurable.

Primary area:
transfer learning, meta learning, and lifelong learning
Keywords:
meta-learning, diffusion models, transfer learning

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_mja4OpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

Meta-Diff learns a source-to-target relation from several training tasks. A set encoder turns source images into a short vector called a task coordinate. A learned map predicts the target coordinate. A shared diffusion model then generates target images. For a new task, the main method uses source images and the known relation. It uses no target images and does not update model weights. The paper studies this method with Gaussian mixtures, CIFAR-100 image changes, and Office-Home domain transfer. It reports useful distribution changes and studies failures to preserve object identity.

Critical strengths
  1. The problem is clearly set and the approach has practical motivations. The paper uses rich source data and cross-task shared source-target relationships to generate target domain samples without target image or backbone updates during deployment, providing a reasonable idea for scenarios where target data is extremely scarce.

  2. Controlled experimental system with adequate failure analysis. The GMM experiment distinguishes task identifiability, relationship predictability and transmission benefits through multi-seed and full factorial designs; the authors also truthfully report semantic generation failures in the natural domain, which helps to understand the applicable boundaries of the method.

Critical weaknesses
  1. The number-taking question is inconsistent with the final submitted question. The title when picking up the number was "Meta-Diffusion: Transferable Reverse-Dynamics Coordinates for Data-Asymmetric Generation", but the final PDF title was "Bridging the Data Gap: Source-Informed Task-Space Transport for Few-Shot Diffusion". The author needs to explain the change and check whether the title information of the submission system is consistent with that of the paper document; whether it involves procedural violations should be determined according to the specific rules of the conference.

  2. There is insufficient evidence for method-independent contributions and semantic generation effects. The lack of a conditional diffusion baseline that also uses source support sets and relation identification, but does not explicitly transfer task coordinates, makes it difficult to determine the independent value of the transfer mechanism. In addition, the distance metric improvement has not been fully translated into semantic generation capabilities, for example, the Office-Home object recognition rate is only 1.67%, limiting the practical significance of the method.

Questions for authors

See weakness.

3 comments
AuthorsOpenReview ↗

Response to Reviewer mja4

W1: Title discrepancy

We thank the reviewer for raising this, and we acknowledge the change of title. The title registered with the abstract was “Meta-Diffusion: Transferable Reverse-Dynamics Coordinates for Data-Asymmetric Generation”; the title of the submitted PDF is “Bridging the Data Gap: Source-Informed Task-Space Transport for Few-Shot Diffusion”. We changed the title between abstract registration and the full-paper submission and did not update the registered title, so the OpenReview metadata and the PDF currently differ; we apologise for this inconsistency. The change is editorial: the new title is a more precise description of the same source-informed task-space transport problem, and the method and research question were not changed by it. We will align the metadata with the anonymised PDF through the process the conference permits, and we defer to the Area Chair and Programme Chairs on whether the change is procedurally acceptable. The abstract of the revised PDF has also gained one sentence, which states the result of the transport-free control (E25X W1); we will update the registered abstract to match.

W2: Independent method contribution and semantic usefulness

The concern about a transport-free comparator is valid, and the control we have now run bears it out. The new objective ablation shows that adding a source denoising branch improves CIFAR SW over transport-only training (+0.0224,[0.0183,0.0265]+0.0224\\,[0.0183,0.0265]), with the same direction across three seeds, FiLM and DomainNet. But both arms still use AzS+bcAz_S+b_c, so that experiment does not isolate the effect of explicit affine transport from equally informed, equally supervised direct conditioning. We therefore describe this arm as transport-only training, and we have run the transport-free control that isolates the effect (E25X W1). Conditioned directly on (zS,c)(z_S,c) and trained with the supervision of Meta-Diff, it is at least as good as Meta-Diff in energy MMD and identity and is better in SW (control minus Meta-Diff −0.0088,[−0.0122,−0.0056]-0.0088\\,[-0.0122,-0.0056]; −0.0140-0.0140 and −0.0057-0.0057 at two further training seeds, each interval excluding zero), and it shows the same dependence on the correct source. We therefore do not claim an independent contribution of explicit affine transport on CIFAR-100. What the evidence supports is source-informed joint training in a shared task space, of which the affine map is one parameterisation. Paper: Existing mechanism and controls: Section 3, Eqs. (2)–(4), pp. 3–4; Section 5.2, Tables 1–2, p. 8. New ablations and the transport-free control: Appendix I.1, Tables 51–52, p. 46. New rebuttal experiment; now reported in Appendix I of the revised manuscript.

The semantic limitation is equally explicit. On Office-Home, generated object identity is 1.671.67\\% versus 52.0052.00\\% on matched real images; bypassing transport with abundant-target coordinates does not repair the failure (1.561.56\\% on held-out Clipart categories). Transport error is therefore not a sufficient explanation, but capacity, conditioning, optimisation and data have not been disentangled. We consequently claim source-specific domain/distribution alignment, not object-level semantic transfer. Paper: Section 5.3, pp. 9–10, and Figure 6, p. 10; Appendix D.4, Tables 34–39, pp. 36–38; Appendix F.4, Table 49, p. 43.

This response didn't solve my problem, so I'll keep the score as it is.

We appreciate the reviewer's assessment. We would like to clarify the contribution established by the revised experiments. Our central question is whether source–target relationships learned across training tasks can support generation for unseen semantic tasks without target observations or adaptation. The results demonstrate source-specific distributional effects on held-out tasks and characterise when those effects depend on source identifiability, relation predictability and source–target discrepancy (Section 5.1, Figures 2–3; Sections 5.2–5.3).

The requested transport-free control provides an important additional finding: the benefits of source-informed joint training do not require the explicit affine map. We have incorporated this result into the paper and present affine transport as a compact target-coordinate parameterisation rather than an independently superior generator (Appendix I.1). The control retains the shared task-space framework, with its set encoder and source-informed joint training, and replaces only the affine parameterisation. The semantic diagnostics likewise distinguish information retained in the task representation from its expression in generated images. A new class-centroid analysis on DomainNet (Appendix I.7, Table 61) shows that class-specific source information is expressed in the generated distribution: in two frozen feature spaces, samples generated from the correct source align with the requested class more than samples from a wrong source or from the relation alone, on each of the three relations and with intervals that exclude zero. The effect is small (3.6% and 4.6% of the real-image alignment in the two feature spaces). These results clarify the scope and contribution of the framework while retaining the zero-target deployment setting and the experimentally demonstrated source-dependent transfer.

Regarding the title, the change was editorial, and the registered title and abstract have been updated to match the PDF.

Reviewer_CFKSOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper studies how to generate data for unseen semantic categories when source-domain data are abundant but target-domain data are limited. The authors propose Meta-Diff, which learns a shared source-to-target relationship across categories. Specifically, the method represents each category using a low-dimensional coordinate derived from its support set, learns an affine mapping between source and target coordinates, and uses the predicted target embedding to guide a shared diffusion model. Once trained, Meta-Diff can generate target-domain samples for unseen categories without requiring target examples or further fine-tuning. The authors evaluate the method on Gaussian mixtures, CIFAR-100 under blur, noise, and contrast shifts, and Office-Home Product-to-Clipart transfer. The results show that Meta-Diff performs well on CIFAR-100 when target-domain data are scarce. However, on Office-Home, although the generated images better match the target domain, they often fail to preserve the identity of the original objects.

However, several key methodological and experimental details remain unclear (see weaknesses and questions), limiting reproducibility and the assessment of the claimed contributions. Therefore, I recommend rejection (2/4).

Critical strengths

The method reuses a learned source–target relation through compact task coordinates and a small affine mapping, enabling a shared diffusion model to generate target-domain samples for unseen categories without target examples or test-time optimization.

Critical weaknesses
  1. Insufficient implementation details. The paper does not adequately specify the encoder architecture, model initialization, loss weights, or complete training configuration. The adaptation objectives, conditioning settings, and parameter update scope of the baselines are also unclear, limiting reproducibility and assessment of the comparisons.

  2. Limited comparison with existing methods. The experiments primarily compare against target-only adaptation, full fine-tuning, and internal controls, without directly evaluating relevant few-shot diffusion or adaptation methods. This leaves the method’s competitiveness unclear.

  3. Weak semantic generation under natural domain shifts. On Office-Home, improved domain alignment does not translate into reliable object generation: requested-object accuracy is only 1.67%, and the displayed samples are difficult to recognize as the intended categories. This limits the evidence for practical effectiveness beyond controlled shifts.

  4. Limited dataset coverage. Image experiments are limited to CIFAR-100 with synthetic corruptions and a single Product-to-Clipart relation on Office-Home. Additional datasets and domain-transfer directions are needed to establish generalization beyond these settings, particularly given the weak semantic results on Office-Home.

  5. Unclear backbone initialization and generalization to unseen categories. The paper does not clearly explain how the diffusion backbone is initialized and trained, particularly whether it is trained from scratch or pretrained on additional data. This distinction is important for understanding how Meta-Diff generalizes to unseen categories. If the backbone is trained only on the 60 training classes of CIFAR-100, it is unclear how the model can generate images that preserve the semantic identity of the 20 unseen test classes, given that it has never observed examples from these categories. If the backbone is pretrained on a broader dataset that includes these categories, the authors should clarify this setup, as it would affect the interpretation of the claimed generalization to unseen categories. A clearer description of the training data and backbone initialization, along with further evidence of semantic preservation on unseen categories, would help address this concern.

Questions for authors
  1. What are the exact architectures of hψh_\psi and ρψ\rho_\psi? Are these models pretrained or trained from scratch?

  2. Is the diffusion backbone trained from scratch or initialized from pretrained weights? If pretrained, which datasets and categories are used, and do they include any held-out evaluation categories?

  3. What are the values of λT\lambda_T, λtr\lambda_{\mathrm{tr}}, and λz\lambda_z in Eq.(4), and how are they selected?

  4. Could the authors provide more details of the meta-training configuration, including the optimizer, learning rate, batch construction, support/query sizes, and whether the encoder, transport, and diffusion network are trained jointly?

  5. Could the authors provide more details of the implementation of other baselines, such as target-only adaptation, full fine-tuning, relation-only and etc.?

  6. Could the authors compare against stronger few-shot generation methods, such as FSDM [1] or CDM [2] or explain why these comparisons are unsuitable?

  7. Could the authors evaluate additional datasets and source–target relations to establish generalization beyond CIFAR corruptions and Product-to-Clipart transfer?

[1] Giannone, G., Nielsen, D., & Winther, O. (2022). Few-shot diffusion models. arXiv preprint arXiv:2205.15463. [2] Gupta, P., Hayat, M., Dhall, A., & Do, T. T. (2024, December). Conditional distribution modelling for few-shot image synthesis with diffusion models. In Asian Conference on Computer Vision (pp. 3-20). Singapore: Springer Nature Singapore.

Suggestions. The abbreviation SW is used before it is defined on Page 5, Line 226. It would be helpful to define it at its first occurrence.

7 comments
AuthorsOpenReview ↗

Response to Reviewer CFKS (part 1 of 4)

W1 and Q1–Q5: Reproducibility, architectures, training and baseline scopes

We agree that the submitted description was insufficient. The shared Deep Sets encoder is rψ(D)=ρψ(∣D∣−1∑x∈Dhψ(x))r_\psi(D)=\rho_\psi(|D|^{-1}\sum_{x\in D}h_\psi(x)). The documented implementation of hψh_\psi has three 3×33\times3 stride-2 convolutions (64/128/128 channels), GroupNorm (8 groups), SiLU, global average pooling and a linear projection to 256 dimensions; ρψ\rho_\psi is Linear(256,256)–SiLU–Linear(256,16). The encoder has approximately 0.327M parameters and is trained from random initialisation. CIFAR uses a 128-base-channel U-Net (approximately 27.8M parameters with basis conditioning); Office-Home uses a 64-base-channel, FiLM-conditioned U-Net (approximately 7.0M parameters). No diffusion component is pretrained on external image data. Paper: Eqs. (1)–(4), pp. 3–4; Appendix G, pp. 43–45. The layer-by-layer encoder configuration, the training settings and the baseline update scopes listed here are now given in Appendix G.2–G.4 (pp. 44–45).

The same joint meta-training objective uses (λT,λtr,λz)=(1,1,10−4)(\lambda_T,\lambda_{\mathrm{tr}},\lambda_z)=(1,1,10^{-4}), AdamW at 2×10−42\times10^{-4}, no weight decay, a 500-update linear warm-up then constant learning rate, gradient clipping at 1.0, EMA 0.9990.999, bf16 autocast and 50,000 updates. Each CIFAR training episode uses MS=64M_{S}=64 source-support images, a target support set of KTK_{T} images with KTK_{T} drawn uniformly from 1,2,5,10,20\\{1,2,5,10,20\\} (the first KTK_{T} of that class's 20 reserved target-support images), and 32 source plus 32 target queries; Office-Home uses MS=16M_{S}=16, KT=10K_{T}=10, and 16/20 source/target queries with two-noise-draw accumulation. These values are taken from the training configuration files and logs, and will be released with the anonymised code and configurations.

The comparison protocols differ as follows: target-only adaptation updates a 16-dimensional coordinate for 25 steps while freezing the U-Net; full fine-tuning updates the U-Net with its coordinate fixed; wrong-source transport substitutes another semantic class under the correct relation; relation-only decodes bcb_c; strict source reuse decodes zSz_S without transport. The validation-selected CIFAR full-fine-tuning schedules use 25/25/400 updates for KT=1/5/20K_{T}=1/5/20, with a separate fixed-100-update sensitivity panel. These are not identical information or compute budgets, and the revised manuscript states them explicitly. Paper: Section 5.2, Table 1, p. 8; Appendix C.3, Table 19, p. 29; Appendix F.2, Tables 44–46, pp. 40–41; Appendix G.4, p. 45, and Table 50, p. 44.

AuthorsOpenReview ↗

Continued Response to Reviewer CFKS (part 2 of 4)

W2 and Q6: External few-shot and adaptation baselines

We have extended the low-target-data comparison, while distinguishing matched-information mechanism controls from target-supported practical competitors. We have run the official FSDM on CIFAR-100. Architecture: the model, diffusion and sampling modules of the authors' repository, imported unmodified, in its CIFAR-100 configuration: a ViT set encoder (patch 8, width 256, depth 6, mean pooling) conditioning an improved-DDPM U-Net (64 base channels, attention at 16×1616\times16 and 8×88\times8, learned variance) through FiLM, 29.65M parameters in total. Training: from scratch with our own training loop on our 60 meta-training classes (clean images and the three corruptions) for the 100,000 updates of that configuration (AdamW, learning rate 10−410^{-4}, batches of 32 sets of six images, one seed); the published weights were not used. The checkpoint at 80,000 updates was selected on validation classes only, by a rule fixed before any test result. Resolution and budgets: 32×3232\times32, sampled with the official ancestral sampler (250 steps) on the same 180 final-test episodes, with KT=1,5,20K_{T}=1,5,20 target images (60 episodes each). FSDM conditions on the KTK_{T} target images and uses neither source images nor the relation label, whereas Meta-Diff uses no target image. Because FSDM is sampled with 250 steps, we compare the two models at 250 sampler steps for both.

Both models were evaluated using 250-step ancestral-type sampling with their respective sampling implementations. In this setting, zero-target Meta-Diff is significantly better than the official FSDM given one target image: FSDM minus Meta-Diff SW is +0.0211,[0.0040,0.0383]+0.0211\\,[0.0040,0.0383] and energy MMD is +0.640,[0.075,1.201]+0.640\\,[0.075,1.201] at KT=1K_{T}=1, and both intervals exclude zero. At KT=5K_{T}=5 the two are not distinguishable (+0.0065,[−0.0037,0.0179]+0.0065\\,[-0.0037,0.0179]), and at KT=20K_{T}=20 FSDM is better (−0.0076,[−0.0134,−0.0017]-0.0076\\,[-0.0134,-0.0017]). FSDM requires at least one target image, whereas Meta-Diff's transport route uses none (it uses a source set and the label of a relation seen in training). Meta-Diff therefore has a significant advantage when target-domain data are scarce. The same holds on CIFAR-100 against the routes that adapt to a single target image: at KT=1K_{T}=1 their SW is higher than that of zero-target Meta-Diff, and each paired 95% interval excludes zero, for target-only adaptation (+0.0306,[0.0152,0.0467]+0.0306\\,[0.0152,0.0467]), full fine-tuning (+0.0867,[0.0623,0.1108]+0.0867\\,[0.0623,0.1108]), LoRA (+0.0401,[0.0281,0.0525]+0.0401\\,[0.0281,0.0525]), the CDM-inspired transplant (+0.0277,[0.0134,0.0430]+0.0277\\,[0.0134,0.0430]) and our internal target-set-only arm (+0.0348,[0.0243,0.0452]+0.0348\\,[0.0243,0.0452]). DINO-100 identity is low for both models at KT=1K_{T}=1 (FSDM 2.532.53\\%; Meta-Diff 2.622.62\\% at 250 steps). Every target budget and sampler setting is in Appendix I.5 (Table 59).

Our internal target-set-only arm, which uses the Meta-Diff encoder and backbone with only a target-support coordinate, is not the published FSDM architecture and should not be read as an estimate of FSDM; its SW is higher than Meta-Diff's by +0.0348+0.0348, +0.0325+0.0325 and +0.0281+0.0281 at KT=1,5,20K_{T}=1,5,20. The official model was trained with one seed. Paper: Section 5.2, p. 7; Appendix I.3, Tables 55–56, p. 49; Appendix I.5, Table 59, pp. 50–51. New rebuttal experiment; now reported in Appendix I of the revised manuscript.

Additional LoRA experiments show that plain LoRA has worse SW at KT=1,5K_{T}=1,5 and an unresolved difference at KT=20K_{T}=20. We could not evaluate the published CDM model itself: to our knowledge no official implementation has been released, and a faithful re-implementation (a VQ-VAE, a classifier and a latent diffusion model trained from scratch, followed by per-class inversion) did not fit the rebuttal period. We therefore report a CDM-inspired test-time transplant, which applies CDM's test-time steps to our own frozen generator and coordinates; it is not a faithful evaluation of the published CDM model. It loses at KT=1K_{T}=1. Each method is identified by its actual architecture, support inputs and update budget in Appendix I.3, which reports every method at every target budget (Tables 55–56) and says which are external implementations and which are internal or adapted baselines. New rebuttal experiment; now reported in Appendix I of the revised manuscript. Paper: Existing target-only/full-fine-tuning reference: Section 5.2, Table 1B, p. 8. New baselines: Appendix I.3, Tables 55–56, p. 49.

AuthorsOpenReview ↗

Continued Response to Reviewer CFKS (part 3 of 4)

W3: Semantic generation under natural shifts

Our results demonstrate that Meta-Diff learns reusable source–target relationships and applies them to unseen semantic tasks without target observations or target-time optimisation. This extends beyond controlled CIFAR transformations to natural domain shifts. On Office-Home Product$\to$Clipart, correct-source transport achieves feature-space SW of 0.038969\mathbf{0.038969}, compared with 0.039394\mathbf{0.039394} for wrong-source transport, 0.039251\mathbf{0.039251} for relation-only prediction, and 0.041641\mathbf{0.041641} for strict source reuse, with all paired differences favouring Meta-Diff. Transport also increases predicted Clipart share from \mathbf{63.8\\%} to \mathbf{86.5\\%}. These results demonstrate that the learned relation transfers target-domain characteristics while retaining useful source-specific distributional information. Paper: Section 5.3, pp. 9–10, and Figure 6, p. 10; Appendix D.3, Table 30, p. 34, for the route means and Clipart shares; Tables 32–33, pp. 35–36, for the paired differences.

We distinguish this distributional transfer from faithful semantic generation. Requested-object identity on Office-Home remains \mathbf{1.67\\%}, compared with \mathbf{52.00\\%} on real target queries. To investigate this, we bypassed transport and generated directly from coordinates estimated using abundant target observations. We also evaluated separately supervised class embeddings. Neither restored reliable semantic fidelity, and the same abundant-target decoding test on the new relations and datasets produced similarly low identity rates (\mathbf{1.5\\%}–\mathbf{6.5\\%}). Thus, transport error alone does not explain the semantic limitation. Paper: Section 5.3, pp. 9–10; Appendix D.4, Tables 34–39, pp. 36–38; Appendix F.4, Table 49, p. 43; Appendix I.4, p. 50, for the new benchmarks.

These findings support our central claim: Meta-Diff enables reusable, source-specific distributional transfer across unseen tasks and natural domain shifts without target-specific adaptation. Reliable object-level generation remains distinct from this established capability, and the additional diagnostics help identify where the current generative pipeline requires improvement. New rebuttal experiment; now reported in Appendix I of the revised manuscript.

W4 and Q7: Additional benchmarks and relation diversity

We have tested three additional Office-Home relations (Clipart$\to$Product, Product$\to$Real-World and Product$\to$Art) and three DomainNet relations. The values below are paired differences on held-out classes with 95% class-bootstrap intervals. SW is DINOv2 feature-space SW, and a positive SW difference means that correct-source transport has the lower (better) SW.

  • Office-Home Clipart$\to$Product (10 held-out classes, eight repeats). Wrong-source minus correct-source SW is +0.0006,[0.0002,0.0011]+0.0006\\,[0.0002,0.0011]. Predicted Product share rises from 21.221.2\\% under source reuse to 43.843.8\\% (+22.7+22.7 percentage points [19.8,25.6][19.8,25.6]). The SW differences from relation-only prediction and from source reuse are unresolved.

  • Office-Home Product$\to$Real-World (same protocol). Wrong-source minus correct-source SW is +0.0007,[0.0002,0.0010]+0.0007\\,[0.0002,0.0010], and relation-only minus correct-source SW is +0.0007,[0.0004,0.0010]+0.0007\\,[0.0004,0.0010]. Identity is higher than under the wrong source by 0.510.51 percentage points [0.04,1.25][0.04,1.25]. Transport gains nothing over source reuse in SW (−0.0000,[−0.0002,0.0002]-0.0000\\,[-0.0002,0.0002]).

Reviewer_E25XOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper proposes Meta-Diff, which encodes source distributions into compact task coordinates, applies a learned affine transport, and conditions a shared diffusion model on the predicted target coordinate. The main deployment route requires neither target examples nor additional optimization. Controlled Gaussian-mixture experiments investigate conditions for transfer, while CIFAR experiments report an advantage at small target budgets. Office-Home demonstrates improved domain alignment but weak preservation of the requested object identity.

Critical strengths

The formulation explicitly separates learning a recurring source–target relation from adapting to each new target. The deployment procedure is simple, and the CIFAR split holds out entire superclasses rather than allowing closely related sibling classes across training and evaluation.

Critical weaknesses
  1. The specific contribution of explicit task-space transport is not isolated from ordinary source-conditioned generation. In the additive model, substituting AzS+bcAz_S+b_c into Equation (2) can be absorbed into a transformed residual basis and relation-dependent offset. Wrong-source and relation-only controls establish source dependence, but not an advantage over an equally informed direct-conditioning baseline.

  2. CIFAR’s headline metrics operate in pixel space, while its target shifts are fixed blur, noise, and contrast transformations. Improvements may therefore reflect matching corruption statistics more than preserving category content.

  3. Propositions 1–2 provide concentration and continuity guarantees, but neither bounds transport prediction error on unseen tasks nor explains the observed advantage over target adaptation. The second proposition also assumes uniform regularity constants that are not quantified for the trained networks.

Questions for authors
  1. What evidence distinguishes the benefit of explicit coordinate transport from directly conditioning the same generator on the source representation and relation? Were any equally informed direct-conditioning controls evaluated?

  2. Given the weak direct-target generation and the low accuracy of the explicit class-embedding control, what distinguishes a limitation of the coordinate interface from insufficient generator capacity or optimization?

  3. Can the authors quantify the assumptions in Proposition 2 for the trained models, or clarify whether the bound is intended only as a qualitative stability statement?

  4. Please provide the meta-training loss weights, optimizer and schedule, SW projection and normalization settings, and the complete definition and calibration of headline semantic consistency.

5 comments
AuthorsOpenReview ↗

Reponse to Reviewer E25X (part 1 of 3)

W1 and Q1: Does explicit transport improve on equally informed source conditioning?

We agree with the algebraic observation. In the additive model, substituting z~T=AzS+bc\widetilde z_T=Az_S+b_c into ϵ^z=ϵ^0+Rz\widehat\epsilon_z=\widehat\epsilon_0+Rz can be absorbed into a transformed residual basis and relation-dependent term. Explicit affine transport alone therefore does not establish a distinct function class. The empirical question is whether the structured representation and training protocol provide a useful inductive bias.

We have performed a matched training-objective ablation, keeping the encoder, U-Net, additive basis, affine map, initialisation, episode stream, optimiser and 50,000 updates fixed. Crucially, the ablated arm still receives AzS+bcAz_S+b_c: it removes the source and target-set denoising branches, not the affine map. We therefore call it transport-only training, and do not present it as the requested transport-free baseline. The arms, together with the transport-free control described below, are:

ArmTraining branchesCIFAR SW at KT=1/5/20K_{T}=1/5/20
Meta-DiffLS+LT+Ltr\mathcal L_S+\mathcal L_T+\mathcal L_{\mathrm{tr}}0.1781/0.1834/0.18440.1781/0.1834/0.1844
Transport-only trainingLtr\mathcal L_{\mathrm{tr}}0.1996/0.2074/0.20600.1996/0.2074/0.2060
No target-set branchLS+Ltr\mathcal L_S+\mathcal L_{\mathrm{tr}}0.1678/0.1720/0.17240.1678/0.1720/0.1724
Transport-free controlLS+LT+Ltr\mathcal L_S+\mathcal L_T+\mathcal L_{\mathrm{tr}} on (zS,c)(z_S,c)0.1691/0.1745/0.17570.1691/0.1745/0.1757
Transport-free control, no target-set branchLS+Ltr\mathcal L_S+\mathcal L_{\mathrm{tr}} on (zS,c)(z_S,c)0.1673/0.1729/0.17230.1673/0.1729/0.1723

Regularisation is suppressed in this shorthand. The pooled paired SW differences (comparator minus Meta-Diff) are +0.0224,[0.0183,0.0265]+0.0224\\,[0.0183,0.0265] for transport-only training and −0.0112,[−0.0166,−0.0057]-0.0112\\,[-0.0166,-0.0057] without the target-set branch (95% class-cluster intervals). The no-target-set arm is therefore better than the headline route in SW, and also in pooled DINO-100 identity (3.153.15\\% against 2.582.58\\%; +0.58+0.58 percentage points [0.06,1.27][0.06,1.27]); the identity difference of the transport-only arm (2.932.93\\%) is unresolved. These results support a benefit of source-branch supervision relative to a transported-target-only objective, not the necessity of the target-set branch or of explicit affine transport.

Role of the target-set branch. The term LT(zT)\mathcal L_T(z_T), with zT=rψ(DT)z_T=r_\psi(D_T), trains the encoder and the generator on directly encoded target sets. Its intended role is to make target sets inputs of the same task space: it defines the target coordinate that the affine map is meant to predict, and the target-supported routes of the same model (target-only adaptation starts from the target-support encoding) and the coordinate-level diagnostics (target-coordinate retrieval, abundant-target decoding, and the coordinate errors in W3) depend on it. In the no-target-set arm the encoder is never applied to a target set during training, so its target-set encodings are not trained inputs of its generator. We retain the branch for these uses. We do not claim that it is necessary for, or beneficial to, zero-target generation (Appendix I.1, Table 51).

The transport-only–minus–Meta-Diff SW differences over three CIFAR training seeds are +0.0224+0.0224, +0.0135+0.0135 and +0.0172+0.0172. A separate FiLM comparison reports +0.035,[0.028,0.043]+0.035\\,[0.028,0.043], and the same direction is observed in feature-SW on DomainNet Real$\to$Clipart/Painting/Sketch (+0.0003+0.0003 to +0.0006+0.0006). Class-identity differences are unresolved. Nearest-neighbour, ridge and kernel-ridge coordinate predictions, decoded with the frozen Meta-Diff generator, also perform worse in generated distributional distance despite all three achieving smaller target-coordinate prediction error (Appendix I.1, Table 53). This is consistent with predictor–generator co-adaptation, but the post-hoc predictors are not jointly trained with that generator.

Thus the objective ablation strengthens the case for joint source-informed training, but it does not answer the reviewer's question. We have therefore run the precise counterfactual requested: a transport-free control trained with the supervision of Meta-Diff but conditioned directly on (zS,c)(z_S,c), with no predicted target coordinate. zSz_S enters the additive basis unchanged, the relation enters as a learned token on the time embedding, and no AzS+bcAz_S+b_c is constructed. The encoder, U-Net, initial weights, training episodes, update budget and source/target denoising supervision are those of Meta-Diff, and the relation tokens add 2,048 parameters against 304 for the affine map.

AuthorsOpenReview ↗

Continued Reponse to Reviewer E25X (part 2 of 3)

The control matches Meta-Diff in energy MMD and identity and is better in SW. Its pooled SW is 0.17310.1731 against 0.18190.1819: control minus Meta-Diff is −0.0088,[−0.0122,−0.0056]-0.0088\\,[-0.0122,-0.0056], with the same sign at each target budget (table above). The differences in energy MMD (+0.028,[−0.052,0.111]+0.028\\,[-0.052,0.111]) and DINO-100 identity (2.572.57\\% against 2.582.58\\%; Meta-Diff minus control +0.01,[−0.21,0.28]+0.01\\,[-0.21,0.28] percentage points) are unresolved. Without the target-set branch the two conditionings have the same SW (+0.0001,[−0.0032,0.0029]+0.0001\\,[-0.0032,0.0029]), and energy MMD is marginally lower with the affine map (+0.085,[0.001,0.162]+0.085\\,[0.001,0.162] pooled; unresolved at KT=1K_{T}=1 and 55). The control is also better than transport-only training (−0.0312,[−0.0380,−0.0247]-0.0312\\,[-0.0380,-0.0247]), and it depends on the source as Meta-Diff does: on these 180 episodes, wrong-source minus correct-source SW is +0.0162,[0.0051,0.0276]+0.0162\\,[0.0051,0.0276] for the control and +0.0166,[0.0074,0.0263]+0.0166\\,[0.0074,0.0263] for Meta-Diff.

Our answer to Q1 is therefore that, on CIFAR-100, explicit affine transport does not improve on equally informed and equally supervised source conditioning. The improvements reported above do not depend on that choice: both models condition on the same source coordinate, both are better than transport-only training, and both lose a similar amount of SW when the source is wrong. They come from source-informed joint training. The abstract, the introduction and Sections 3, 5.2 and 6 of the revised manuscript now state this, and Appendix I.1 presents the affine map as one parameterisation of source conditioning. It is the more compact of the two (304 parameters against 2,048) and yields an explicit predicted target coordinate, on which the coordinate-level diagnostics of W3 rely. The result holds over three training seeds, each control being paired with the Meta-Diff model of the same seed: control minus Meta-Diff SW is −0.0088,[−0.0122,−0.0056]-0.0088\\,[-0.0122,-0.0056], −0.0140,[−0.0182,−0.0099]-0.0140\\,[-0.0182,-0.0099] and −0.0057,[−0.0082,−0.0030]-0.0057\\,[-0.0082,-0.0030] for seeds 12345, 23456 and 34567, and each interval excludes zero. Energy MMD favours the control at one seed (−0.129,[−0.198,−0.061]-0.129\\,[-0.198,-0.061]) and is unresolved at the other two, and identity is unresolved at every seed. Paper: Section 3, Eqs. (2)–(4), pp. 3–4; Section 5.2, Tables 1–2, p. 8, for the existing controls; Appendix I.1, Tables 51–53, pp. 46–47, for the new ablations and the transport-free control. New rebuttal experiment; now reported in Appendix I of the revised manuscript.

W2: Pixel-space corruption statistics versus category content

The correct-source route improves CIFAR pixel SW over wrong-source and relation-only controls by 0.03330.0333 and 0.03460.0346, respectively (paired 95% intervals: [0.0163,0.0525][0.0163,0.0525] and [0.0224,0.0477][0.0224,0.0477]). With the relation fixed, 20-way requested-class consistency rises from 0.04700.0470 (wrong source; chance 0.050.05) to 0.10750.1075 (correct source), though its difference from strict source reuse is unresolved. This is evidence for limited source-specific content, not faithful category generation. Paper: Section 5.2, Table 1A, p. 8, for all four values; Appendix C.2, Table 18, p. 28, for the separate 180-episode validation sweep.

The matched DINO-100 audit gives Meta-Diff requested-class identity of 2.53/2.85/2.342.53/2.85/2.34\\% at KT=1/5/20K_{T}=1/5/20 versus approximately 5050\\% on real target queries. We now state the CIFAR result as source-specific distributional transfer under controlled shifts with weak semantic identity, and present class identity alongside SW/MMD. Paper: Section 5.2, p. 9; Appendix F.3, Table 47, p. 42; Section 6, p. 10; Appendix H, Table 3, p. 45.

W3 and Q3: Theoretical scope and numerical stability

We agree that Propositions 1–2 neither guarantee unseen-task transport prediction nor explain which generative method is more accurate. Proposition 1 bounds source-coordinate sampling error; Proposition 2 gives stability of an idealised continuous-time additive-score process assuming global regularity constants. These assumptions are not numerically certified for the trained U-Nets or the DDIM sampler, and the theorem does not automatically apply to arbitrary FiLM conditioning. Proposition 2 is therefore intended as a qualitative stability statement: we do not quantify LxL_x, LzL_z or CτC_\tau for the trained models. Paper: Section 3.1, p. 4; Appendix A, pp. 14–15; Section 6, p. 10.

AuthorsOpenReview ↗

Continued Reponse to Reviewer E25X (part 3 of 3)

For clarity, let zS∗z_S^* and zT∗z_T^* be population coordinates and let ξ=zT∗−(AzS∗+bc)\xi=z_T^*-(Az_S^*+b_c) with εrel=(E∣ξ∣22)1/2\varepsilon_{\rm rel}=(\mathbb E\\|\xi\\|_2^2)^{1/2}. Under the propositions' assumptions, the conditional implications are

EW2(Pz~T,PzT∗)≤Cτ(∣A∣opLρtr⁡(ΣS)/MS+εrel),EW2(Pz^T,PzT∗)≤CτLρtr⁡(ΣT)/KT.\begin{aligned}\mathbb E W_2(P_{\widetilde z_T},P_{z_T^*})&\leq \sqrt{C_\tau}\Bigl(\\|A\\|_{\rm op}L_\rho\sqrt{\operatorname{tr}(\Sigma_S)/M_{S}}+\varepsilon_{\rm rel}\Bigr),\\\\ \mathbb E W_2(P_{\widehat z_T},P_{z_T^*})&\leq \sqrt{C_\tau}L_\rho\sqrt{\operatorname{tr}(\Sigma_T)/K_{T}}.\end{aligned}

These are upper bounds on distances to the distribution generated using a population target coordinate, not to the true target-data distribution. Comparing two upper bounds cannot establish the ordering of the actual generative errors. Nor is εrel\varepsilon_{\rm rel} itself bounded for unseen tasks without additional assumptions; we consequently make no theorem-level crossover claim.

We have separately measured empirical normalised squared coordinate errors on held-out CIFAR tasks. Direct target-set encoding yields 2.90/1.49/0.59/0.30/0.162.90/1.49/0.59/0.30/0.16 at KT=1/2/5/10/20K_{T}=1/2/5/10/20, while transported source encoding at MS=64M_{S}=64 gives 0.440.44. These estimates cross between five and ten target observations, at the coordinate-error level. This crossover is empirical; it is not a guarantee that one generative method outperforms another. The 0.390.39 estimate at MS=300M_{S}=300 is not identified as an irreducible relation floor. Same-noise DDIM perturbation measurements are reported only as local empirical sensitivity diagnostics, not as estimates of the continuous-time stability constant. New rebuttal experiment; the coordinate errors are now reported in Appendix I.2 (Table 54, p. 48) of the revised manuscript.

Q2: Coordinate interface versus generator capacity or optimisation

On Office-Home, Product-coordinate same-class retrieval is 91.091.0\\% among ten held-out classes, whereas decoding a coordinate estimated from abundant actual Clipart images produces only 1.561.56\\% held-out requested-class identity (matched real target queries: 52.0052.00\\%). A separately trained class-embedding control reaches only 6.906.90\\% on seen categories. Thus transport error alone cannot explain the poor semantic fidelity; bypassing it does not restore object identity. The failure could reflect the learned coordinate, the denoiser's use of it, model capacity, data or optimisation, and these causes have not been uniquely separated. Paper: Section 5.3, pp. 9–10 (the 91.091.0\\% retrieval figure is on p. 10 and in Appendix H, Table 3, p. 45); Appendix D.4, Tables 34–39, pp. 36–38; Appendix F.4, Table 49, p. 43.

Q4: Training, SW and semantic calibration

For the CIFAR model the loss weights are (λT,λtr,λz)=(1,1,10−4)(\lambda_T,\lambda_{\mathrm{tr}},\lambda_z)=(1,1,10^{-4}). The joint encoder/transport/U-Net optimisation uses AdamW, learning rate 2×10−42\times10^{-4}, zero weight decay, EMA 0.9990.999, a cosine 1,000-step diffusion schedule, 50,000 updates, and deterministic DDIM-50 at evaluation. Pixel SW uses 256 shared unit-norm Gaussian projection directions on normalised [−1,1][-1,1] pixels; energy MMD uses the stated matched sample protocol. Headline 20-way consistency is the fraction of generated samples that a 20-way classifier assigns to the episode's own fine class. The classifier is a small CNN: three blocks of two 3×33\times3 convolutions (the second with stride 2), each followed by GroupNorm and SiLU, then global average pooling and a linear head. It is trained from scratch on real images of the 20 classes of the evaluated split, which the generator never sees during meta-training. Training uses 550 images per class, AdamW (learning rate 2×10−32\times10^{-3}, weight decay 10−410^{-4}), a cosine schedule, 8 epochs at batch size 256 and random horizontal flips. Each batch is shown clean or under one of the three corruptions, chosen at random, so that the classifier reads the class rather than the corruption. Calibration is measured on 50 withheld real images per class. For the Table 1A evaluation, fine-class accuracy is 39.339.3\\% on average (38.538.5–40.040.0\\% across the clean and three corrupted presentations; chance 55\\%), and superclass accuracy is 67.067.0\\% (chance 2525\\%). This instrument is distinct from the independent DINO-100 audit. Paper: Section 4, p. 5; Section 5.2, pp. 7–9; Appendix C.1, p. 27, for the SW, MMD and class-consistency definitions; Appendix F.3, p. 42; Appendix G, pp. 43–45, and Table 50, p. 44. The optimiser warm-up, gradient clipping, encoder-layer definitions and classifier training details are now given in Appendix G.2–G.3 (pp. 44–45).

Comments on the paper

2 comments
AuthorsOpenReview ↗

Overview and collective response

We thank the reviewers for their constructive feedback. The revised manuscript now provides substantially stronger evidence for our central contribution: learning recurring source–target relationships across semantic tasks to generate for unseen tasks without target observations or target-time optimisation. The additional experiments also establish important boundaries on this contribution.

1. Distinction from existing methods (CFKS W2/Q6; mja4 W2).

Methods such as FSDM, target-only diffusion adaptation, LoRA and full fine-tuning use observations from the new target task to condition or adapt generation. Meta-Diff instead exploits a complementary information source: abundant examples of the new task in a source domain, together with a source–target relation learned across other tasks. At deployment, it requires only source observations and the identity of a previously learned relation—no target images, coordinate refinement or backbone updates.

This is a different information setting, not a claim of universal superiority over target-supported methods. Nevertheless, on CIFAR-100 at (K_T=1), zero-target Meta-Diff achieves 0.1781 SW versus 0.2087 for target-only adaptation and 0.2648 for full fine-tuning. It also outperforms the official FSDM under matched 250-step ancestral-type sampling (+0.0211 [0.0040, 0.0383]); that advantage is not resolved under other tested sampler configurations, and FSDM performs better with twenty target images (Section 5.2; Appendix I.5).

2. Is explicit transport responsible for the improvement? (E25X W1/Q1; mja4 W2).

We implemented the reviewers’ requested equally supervised transport-free control, which conditions directly on the source coordinate and relation identity. The result clarifies the mechanism: direct conditioning achieves lower CIFAR SW across three training seeds, while pooled DomainNet feature-SW differences are unresolved and pixel SW and identity favour direct conditioning (Appendix I.1, Table 51; Appendix I.6, Table 60).

Thus, the evidence supports source-informed joint training, not an independent generation-quality advantage from the affine map. The affine map provides a compact, explicit target-coordinate representation (304 versus 2,048 conditioning parameters in the evaluated implementations). We have revised the abstract, contributions and discussion accordingly.

3. Does transfer preserve meaningful semantic information? (E25X W2; CFKS W3/W5; mja4 W2).

On CIFAR-100, correct-source transport improves SW over wrong-source transport by 0.0333, while requested-class consistency increases from 4.70% to 10.75%. We extended evaluation to six additional Office-Home and DomainNet relations, distinguishing source-specific effects from improvements over direct source reuse.

A new DomainNet analysis further finds source-specific class alignment in generated distributions using CLIP and DINOv2 features (Appendix I.7, Table 61). However, the effects are small (3.6–4.6% of real-image alignment) and reliable object generation remains unachieved (1.67% Office-Home identity versus 52.0% on real images). We therefore claim measurable source-specific distributional transfer, not faithful semantic synthesis.

4. Reproducibility, generalisation and theoretical scope (CFKS W1/W4 and Q1–Q7; E25X W3/Q3).

The revision now provides detailed architectures, training configurations, baseline update scopes, calibrated semantic evaluations and additional natural-domain tests (Appendices G and I). The GMM experiments systematically distinguish source identifiability, relation predictability and source–target headroom, identifying when transfer is useful (Section 5.1, Figures 2–3). Propositions 1–2 establish conditional coordinate-estimation and stability results, not guarantees of semantic generation or practical superiority.

In summary, the rebuttal strengthens the paper in three ways: it validates source-dependent transfer on held-out tasks, identifies source-informed joint training as the source of the observed improvements, and establishes the limits of explicit transport and semantic fidelity. The contribution is a zero-target transfer formulation and shared task-space framework, supported by a controlled characterisation of when cross-task information is useful, rather than another method for adapting a generator to a handful of target images.

The anonymised implementation and reproduction materials are available at https://anonymous.4open.science/r/The-Repo-8169/. The registered title and abstract have also been aligned with the revised manuscript.

AuthorsOpenReview ↗

The Anonymous Repo is Available Now

The code to reproduce the results of the paper can be got through this anonymous repo: https://anonymous.4open.science/r/The-Repo-8169/

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.