acceptodds
Withdrawn from ICLR 2027

MOSAIC: Multi-Scale Object-Centric Slot Adaptation with Image-Guided Prototype Conditioning for Remote Sensing Image Super-Resolution

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Pretrained diffusion Transformers provide powerful generative priors for image restoration, yet adapting them to structure-sensitive tasks remains challenging. Existing methods mainly rely on dense low-resolution features as conditions, which often entangle salient objects with dominant backgrounds and provide limited abstraction for recurring structures across scales and samples. We propose MOSAIC, an object-centric representation learning framework that learns compact structural conditions for generative model adaptation. MOSAIC organizes degraded observations into structural slots that capture complementary object-level and layout-level evidence, integrates them across resolutions, and retrieves recurring priors from a shared prototype memory to complement incomplete image-specific information. The resulting representations are injected into selected layers of a frozen diffusion Transformer through lightweight residual modulation, preserving its pretrained generative prior. We instantiate MOSAIC for remote sensing image super-resolution, where small objects, repetitive structures, and large-scale variation provide a challenging testbed. Experiments on UCMerced, AID, and RSSCN7 show consistent PSNR improvements across ×2 and ×4 settings and favorable LPIPS results at ×4. Qualitative results further demonstrate improved preservation of small objects, structural boundaries, and repetitive spatial patterns.

Primary area:
unsupervised, self-supervised, semi-supervised, and supervised representation learning
Keywords:
Representation Learning, Object-Centric Learning, Diffusion Transformers, Image Super-Resolution, Remote Sensing

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_yUoDOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
1
Soundness
3
Read review
Paper summary

This paper addresses the problems of small objects being overwhelmed by background information, significant scale variations, and insufficient structural information under severe degradation in existing RSISR methods. It proposes the MOSAIC framework integrating multi-scale OCL extraction, MSF, and PMC, and evaluates it on the UCMerced, AID, and RSSCN7 datasets.

Critical strengths
  1. The motivation targets several critical challenges specific to RSISR.
  2. The proposed components are well motivated and correspond to the identified problems.
  3. The method is extensively evaluated on three remote sensing datasets: UCMerced, AID, and RSSCN7.
Critical weaknesses
  1. The terminology and abbreviations are not clearly organized, which hurts readability.
  2. Some key components, particularly PMC, provide only marginal performance gains.
  3. The current experiments do not sufficiently validate some of the key motivations behind the proposed method.
Questions for authors
  1. The terminology is chaotic, which affects the readability of the methods. The article introduces a large number of custom names such as OCL, MSF, PMC, SlotFiLM, etc., but lacks unified and clear definitions and corresponding relationships (such as Multi-Scale Slot Fusion (MSF)). Although the full name of PMC (Prototype Memory Conditioning) is provided, it only appears on Page 8, which is too late.The content of Figure 2 only uses the full names, while the caption only uses the abbreviations. Here, SlotFilM appears directly in the caption, but this module is only formally introduced and appears in the content of Figure 3. Overall, this makes readers need to frequently refer back to the main text to understand the relationships between the various modules.
  2. PMC is a title-level core contribution, but the actual performance improvement it brings is very limited. In Table 3, after adding PMC to the basis of OCL + MSF, the PSNR only increased by +0.0079 dB; SSIM only increased by +0.0002; LPIPS improved by -0.0033. Compared to the improvements brought by OCL and MSF, this gain is very limited. However, Prototype Conditioning not only appears in the title of the paper, but also serves as one of the main components of MOSAIC. Therefore, the current results are not sufficient to prove that PMC is an indispensable core module in the entire framework, rather than just an auxiliary design that brings marginal benefits.
  3. The main motivation for proposing PMC is as follows: When the LR input is severely degraded, the structural information of the current image may have been lost (Section 3.4), and the shared structures repeatedly appearing in the training data can serve as additional priors for supplementation. However, the current experiments have not directly verified this assumption. For instance, further research could be conducted on the following aspects: Whether the benefits of PMC become more significant as the degree of degradation increases; Whether PMC brings more significant improvements to small target areas. Currently, relying solely on overall image metrics is not sufficient to prove that PMC truly functions through the mechanism claimed in the paper.
  4. The paper repeatedly emphasizes that small objects, structural boundaries, repetitive spatial patterns and object geometry are the key issues in remote sensing super-resolution. However, quantitative experiments mainly still use conventional global image metrics such as PSNR, SSIM, LPIPS, MANIQA and CLIP-IQA, and evaluate them on standard validation/test sets. These metrics cannot directly measure the structural properties emphasized by the paper. If reconstruction metrics based on target size, boundary-related metrics, or downstream performance of detection/segmentation on SR images could be added, it would more effectively support the core claim of the paper.
  5. MOSAIC introduced additional components such as multi-scale OCL extraction, MSF, prototype cross-attention, and SlotFiLM beyond the DiT backbone. However, the paper did not report inference latency, training/inference memory usage, FLOPs, nor did it provide a comparison of computational costs with baselines like DiT4SR. Therefore, it is currently difficult to determine the additional computational cost corresponding to these performance improvements.
  6. The paper description states that the OCL extractor and MSF are pre-trained first, then the slots are used to construct and freeze the prototype memory, and subsequently the prototype cross-attention and SlotFiLM are optimized. However, in Figure 2, the OCL extractor is frozen while the other modules such as MSF are trainable. Is this corresponding to another fine-tuning stage? Which parameters are optimized in each stage; what objective is adopted; and does there exist joint fine-tuning in the future? It is still difficult to fully reproduce the specific training process based solely on the text.
Reviewer_4pvbOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper proposes MOSAIC, a framework for adapting a frozen pretrained diffusion Transformer (DiT4SR) to remote sensing image super-resolution. The authors argue that conditioning on dense low-resolution features entangles small foreground objects with dominant backgrounds. Instead, they condition the backbone through a set of lightweight adapter modules:

  • An object-centric slot extractor (OCL) that compresses the LR control latent into a small set of slots.
  • A multi-scale slot fusion module (MSF) that combines slots extracted at two scales.
  • A prototype memory (PMC), built offline by KMeans clustering of training slots, that adds retrieved context to the slots via cross-attention.

The resulting slots are mean-pooled into a global vector, which a FiLM-style adapter (SlotFiLM) uses to modulate selected DiT blocks through gated residual scale-and-shift terms.

The paper follows a standard structure. It introduces the method, evaluates it at ×2 and ×4 on three established remote sensing datasets (UCMerced, AID, RSSCN7), and compares against existing real-world SR methods including DiT4SR. It then analyzes the contribution of each component and its hyperparameters through ablation studies. The authors report the best PSNR across all datasets and scales, with gains over DiT4SR of 1.5–5.4 dB, and the lowest LPIPS on the two datasets evaluated for perceptual quality. The main novelty lies in the design of the adapter modules and how they condition the frozen backbone, rather than in the backbone or training paradigm.

Critical strengths

I see two critical strengths in this paper:

  • The paper uses object-centric representations (slot attention), multi-scale fusion and prototype memories to condition a pretrained generative model. The motivation is domain-specific and convincing: small objects in large backgrounds, strong scale variation and recurring structures make dense conditioning a plausible bottleneck. If the reported gains hold up, this points to object-centric conditioning as a promising direction for structure-sensitive restoration, which is the main source of the paper's potential value.

  • The method is clearly described and conceptually straightforward. The backbone stays frozen, and only lightweight adapters are trained. Their gated, zero-initialized residual modulation starts as an identity mapping. This makes the approach cheap to train and, in principle, easy to transfer to other DiT backbones, which lowers the barrier to adoption.

Critical weaknesses

I see several weaknesses in the submission, where the following have a critical level:

  • No result reports variance, seeds or confidence intervals, yet all values are given to four decimals. The paper’s own numbers suggest substantial run-to-run variation. The same default configuration scores between 24.07 and 24.44 dB PSNR across Table 3 and Figs. 6–8, on the same validation set. Most ablation differences are far smaller: PMC adds 0.008 dB in Table 3, M = 16 beats M = 32 by 0.005 dB, and full MSF beats inter-scale-only fusion by 0.007 dB. Fig. 8 is also non-monotone, with M = 8 scoring below no memory. As a result, the ablations cannot attribute gains to individual components, and the PMC contribution in particular is unsupported.

  • The gains over DiT4SR (up to 5.4 dB) are very large for a lightweight adapter on a frozen backbone. The main text does not state whether the baselines were retrained or fine-tuned on these datasets and degradations, or run with matched sampling steps. The paper also lacks a same-budget control without slots, such as a trained adapter on pooled dense features or a fine-tuned DiT4SR. Without it, the gains cannot be attributed to object-centric conditioning rather than in-domain adaptation. Finally, the dedicated PSNR-oriented RSISR methods the paper cites (e.g., TTST, HAT, SwinIR) are not compared. Since the central claim is that object-centric conditioning drives the improvement, this gap is decisive for the assessment.

Questions for authors

I have the following key questions for the authors that could change my view:

  • Were the main results and ablations obtained from a single training run and a single sampling seed? Please report mean and std over multiple training seeds for the ablations (Table 3, Figs. 6–8), and per-image confidence intervals or a paired significance test against DiT4SR for Tables 1–2. If the differences in Table 3 and Figs. 6–8 remain distinguishable from seed variation, particularly for PMC and the memory-size and MSF-variant comparisons, the concern about the ablations would be resolved.

  • The same default configuration (K = 8, blocks [2, 8, 12, 16], M = 16, full MSF + PMC) yields 24.44, 24.24, 24.12 and 24.07 dB across Table 3 and Figs. 6–8 on the same validation set. What differs between these runs (seed, schedule, checkpoint rule)? Relatedly, Fig. 7’s [4, 8, 12, 16] result is identical to Fig. 6’s K = 8 result (24.0658 dB). Is this a shared run?

  • Were the baselines, in particular DiT4SR, retrained or fine-tuned on the same training splits and degradation pipeline as MOSAIC, or evaluated with publicly released weights? Were they run at the same resolution and with the same number of sampling steps?

  • Does the “no OCL” row in Table 3 correspond to unmodified DiT4SR? If so, can you provide a same-budget control that replaces the slots with a non-object-centric condition, such as SlotFiLM driven by pooled dense LR features, or DiT4SR fine-tuned (e.g., with LoRA) on the same data? This is the comparison most likely to change the assessment, since it directly tests the paper’s central claim.

Reviewer_viquOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper presents MOSAIC, an object-centric conditioning framework for adapting a frozen Diffusion Transformer (DiT4SR) to remote sensing image super-resolution. The method extracts compact structural slots at two resolutions, combines them through multi-scale slot fusion (MSF), and augments the fused representations with a shared prototype memory constructed from training data. The resulting slots are pooled and injected into selected DiT blocks through a residual FiLM adapter.

Experiments on UCMerced, AID, and RSSCN7 show that MOSAIC achieves the highest PSNR among the compared methods at both ×2 and ×4 scales. It also obtains favorable LPIPS results on the two datasets where this metric is reported. Component ablations indicate that object-centric learning (OCL) accounts for most of the improvement, with additional contributions from MSF and prototype memory conditioning (PMC).

Critical strengths

The paper addresses a relevant problem: how to organize low-resolution observations into compact structural conditions for super-resolution. Combining object-centric representations, multi-scale fusion, and residual prototype retrieval provides a coherent alternative to conventional dense conditioning. The use of a frozen DiT backbone also makes the framework potentially useful for parameter-efficient adaptation.

MOSAIC achieves the best PSNR on all six dataset-scale combinations in Table 1. The improvements over DiT4SR are particularly notable on AID (+5.40 dB at ×2 and +4.54 dB at ×4). Table 2 also reports the lowest LPIPS on RSSCN7 and AID. The ablation results in Tables 3 and 4 support the effectiveness of the OCL module and show that learned inter-scale fusion performs better than simple averaging. These findings provide meaningful empirical support for the overall framework.

Critical weaknesses

The main contribution is framed around object-centric structural representations, but the current comparisons do not clearly establish whether the improvements arise specifically from object-centric organization rather than additional trainable conditioning capacity. In Table 3, OCL contributes the largest PSNR gain (+1.8820 dB), yet comparisons with parameter-matched non-object-centric feature aggregation or alternative lightweight adapters would be important for attributing this improvement.

Two design choices further merit clarification. First, MSF combines slots with the same index across resolutions (Eq. 10), although Slot Attention does not inherently guarantee consistent slot ordering. Shared extractor parameters may encourage consistency but do not by themselves establish cross-scale correspondence. Second, SlotFiLM mean-pools all slots into a single global condition (Eq. 2), followed by channel-wise modulation shared across spatial tokens. This may limit the extent to which object-specific spatial information is directly preserved during conditioning.

These concerns do not invalidate the method, but they weaken the evidence for its central object-centric mechanism.

Moreover, prototype memory is presented as a major component, but its incremental contribution in Table 3 is small: +0.0079 dB PSNR, +0.0002 SSIM, and −0.0033 LPIPS. While Figure 8 reports favorable results for 16 prototypes under a separate ablation protocol, it remains unclear how consistently prototype retrieval contributes beyond the image-specific slots.

Furthermore, although the paper emphasizes the preservation of small objects, structural boundaries, and repetitive patterns, the evaluation relies primarily on general image-quality metrics. The results are also mixed across metrics: at ×4, MOSAIC has lower SSIM than the best competing method on UCMerced and RSSCN7, and lower CLIP-IQA than DiT4SR on both datasets reported in Table 2. These results do not negate the PSNR gains, but they warrant a more qualified interpretation of structural fidelity and perceptual quality.

More direct evidence of object-level and geometric preservation would strengthen the connection between the proposed method and its stated motivation.

Questions for authors
  1. Cross-scale slot correspondence. How is slot correspondence maintained between the two resolutions in MSF? Since Eq. 10 fuses slots using matching indices, could the authors clarify whether slot initialization or training provides any explicit alignment? Evidence of correspondence, or a comparison with permutation-aware matching, would help assess the validity of this design.

  2. Attribution of OCL improvements. Could the authors provide comparisons with parameter-matched alternatives, such as pooled dense features, learned latent tokens, or a non-object-centric conditioning adapter? Such comparisons would help determine whether the substantial OCL improvement in Table 3 is attributable to object-centric representation learning.

  3. Contribution of prototype memory. Given the small incremental gains in Table 3, how robust are the PMC improvements across random seeds and datasets? Could the authors clarify how the prototype controls referenced in the supplementary material distinguish meaningful structural retrieval from generic feature augmentation?

  4. Evaluation of structural preservation. Could the authors provide more direct evidence that MOSAIC improves the fidelity of small objects and spatial structures, for example through boundary consistency or downstream object recognition metrics? It would also be helpful to discuss why the strong PSNR gains do not consistently translate into higher SSIM or no-reference perceptual quality.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.