acceptodds
Under review as a conference paper at ICLR 2027

UniFlow-RS: Unified Generative Prediction for Heterogeneous Remote-Sensing Tasks

Abstract

Remote-sensing foundation models increasingly unify representations across sensing modalities and downstream tasks, yet their prediction mechanisms remain largely task-specific. Separate decoders, prediction heads, and objectives leave prediction fragmented across tasks. This naturally raises a question: can heterogeneous remote-sensing tasks share the prediction process? UniFlow-RS answers this question with a single latent flow-matching predictor shared across tasks. This requires addressing two key challenges: unifying heterogeneous prediction spaces and harmonizing joint optimization across tasks. First, we cast heterogeneous task outputs into structured visual targets in a common latent space, allowing them to be modeled by the same pretrained flow predictor. Task-dependent conditioning further accommodates differences in input configurations. Second, we optimize all tasks with a common Flow Matching objective, eliminating task-specific prediction losses. To mitigate optimization mismatch across tasks and modalities, we introduce Task–Modality Adaptive Timestep Sampling (TMAS), which dynamically allocates training timesteps according to the online learning difficulties of each task–modality group. Using a single shared model, UniFlow-RS handles six tasks and thirteen datasets, surpassing recent task-specific state-of-the-art methods on several benchmarks while remaining competitive on the others. The same checkpoint also transfers strongly to two unseen datasets, demonstrating generalization ability of the unified model.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
Generative Models, Flow Matching, Multi-Task Learning, Remote Sensing, Foundation Models, Unified Visual Prediction

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_obhJOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes UniFlow-RS, which converts heterogeneous remote-sensing task outputs into visual targets, performs prediction using a shared latent Flow Matching model, and coordinates joint training through Task–Modality Adaptive Timestep Sampling (TMAS). Experiments cover six tasks across thirteen datasets.

Critical strengths
  1. The framework accommodates different modalities, input configurations, and output structures within a single predictor, demonstrating the feasibility of unified modeling for heterogeneous remote-sensing tasks.
  2. The experiments are extensive, and the method is relatively simple, using a shared LoRA adapter and training objective, with ablations examining target encoding, source-state initialization, and timestep-sampling strategies.
Critical weaknesses
  1. The benefits of task unification are insufficiently justified. The paper motivates its approach by the fragmentation of task-specific prediction mechanisms, but does not sufficiently explain what learning problems this fragmentation causes or how unified prediction addresses them. Performance gaps remain relative to specialist methods on some tasks, and the model does not achieve SOTA performance across all tasks; the benefits and trade-offs of unification therefore require clearer analysis.
  2. The methodological distinction from prior work is insufficiently established. Encoding heterogeneous task outputs as visual targets is closely related to generative generalist approaches such as DICEPTION and InstructCV, while adaptive timestep sampling also has precedents. Although the source-state design, task–modality grouping, and dual-EMA signal constitute specific extensions, the current analysis does not sufficiently establish their independent methodological value beyond existing approaches.
  3. The generalization evidence is limited in task coverage. Both OOD datasets concern referring segmentation, so the current results are insufficient to support broad generalization claims for the full six-task model.
Questions for authors
  1. What specific learning problems across tasks does unified prediction alleviate? Could you clarify why unified prediction is needed?
  2. What are the method’s core technical advances? Can experiments demonstrate which capabilities these advances enable that existing approaches cannot achieve?
  3. The generalization experiments cover only referring segmentation; is there transfer evidence for other tasks? If not, could the generalization claims be narrowed accordingly?
Reviewer_1NVKOpenReview ↗
Rating
2
Confidence
4
Informativeness
1
Presentation
1
Soundness
2(−1)
Read review
Paper summary

This paper proposes UniFlow-RS, a unified generative framework for multiple vision tasks using a pretrained generative model. The main idea is to formulate different tasks within a common generation framework through structured visual target representations. The authors also introduce Task–Modality Adaptive Timestep Sampling (TMAS) to improve the training process across different tasks and modalities. Experiments on several vision benchmarks show promising results, including comparisons with existing methods and zero-shot transfer evaluations. However, the extent to which the improvements come from the proposed framework rather than the pretrained generative backbone remains unclear.

Critical strengths

The paper explores an interesting research direction by unifying heterogeneous remote sensing tasks within a shared generative prediction framework, rather than relying on conventional multi-task architectures with separate prediction heads. This is a relevant direction for developing generalist remote sensing models.

The experiments cover six tasks and thirteen datasets involving different sensing modalities and prediction targets. The reported results are generally competitive, suggesting that a shared generative framework can handle diverse remote sensing tasks with reasonable effectiveness.

Critical weaknesses

Limited methodological novelty. The proposed framework mainly integrates existing techniques, including pretrained Flow Matching models, structured visual representations, task-dependent conditioning, and LoRA adaptation. Similar ideas of representing heterogeneous vision tasks through generative prediction have been explored in Painter, InstructDiffusion, and InstructCV. The paper does not sufficiently establish a fundamentally new learning mechanism beyond adapting these existing concepts to remote sensing. Furthermore, TMAS appears to be an incremental extension of existing adaptive timestep sampling strategies.

Insufficient experimental justification of the proposed unified generative framework. Although the paper reports competitive performance, the experiments do not convincingly demonstrate that unified generative prediction is preferable to conventional discriminative multi-task architectures. Table 1 mainly compares single-task and joint adaptation within the same generative framework. Moreover, Table 2 shows that TMAS improves the unified task score from 69.03 to 69.35 over group-adaptive sampling, suggesting a relatively modest independent contribution. The reliance on a large pretrained generative backbone also raises concerns about computational efficiency and practical applicability, which are not sufficiently addressed by the current comparisons.

Questions for authors

What is the fundamental methodological difference between UniFlow-RS and existing generative generalist vision frameworks, such as Painter, InstructDiffusion, and InstructCV? What new learning principles are introduced?

The proposed TMAS mainly extends existing adaptive timestep sampling by incorporating task–modality-specific statistics and dual EMA updates. Why is this particular formulation necessary, and what advantages does it provide over simpler group-adaptive sampling?

Table 2 shows that TMAS improves UTS from 69.03 to 69.35 over group-adaptive sampling. Are these improvements statistically significant across multiple training runs? How sensitive are the results to the timestep grouping and sampling hyperparameters?

How does UniFlow-RS compare with strong discriminative multi-task architectures under matched training data, pretrained model capacity, and computational budgets? Can the authors demonstrate that the benefits originate from the unified generative prediction mechanism rather than the pretrained backbone?

What are the end-to-end inference latency, peak GPU memory consumption, and throughput of UniFlow-RS compared with conventional task-specific predictors? How does the large pretrained backbone affect deployment efficiency?

The zero-shot experiments on EarthReason and RISBench both evaluate referring segmentation. What evidence supports generalization to genuinely unseen task types or modality combinations?

How do VAE reconstruction errors and the deterministic recovery process affect the accuracy of small-object detection, rotated bounding boxes, and fine-grained segmentation boundaries?

Reviewer_2ySfOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes UniFlow-RS, a unified generative framework for heterogeneous remote-sensing perception. It represents diverse task outputs as structured visual targets and uses a shared pretrained latent flow predictor with task-dependent conditioning. The proposed Task–Modality Adaptive Timestep Sampling further adjusts timestep sampling according to online learning dynamics. Experiments across six tasks and thirteen datasets demonstrate competitive performance and promising transfer to unseen referring-segmentation datasets.

Critical strengths
  1. The paper presents a well-motivated unified prediction framework that handles six heterogeneous remote-sensing tasks using a shared latent flow predictor and training objective, without learned task-specific output heads.
  2. The experiments cover thirteen datasets with diverse modalities and output structures. The controlled joint-training and TMAS ablations, together with zero-shot referring-segmentation evaluations, provide meaningful empirical support for the framework.
Critical weaknesses
  1. Representing heterogeneous task outputs as structured visual targets and recovering predictions through deterministic post-processing may introduce information loss, particularly for instance-level tasks requiring precise geometric localization.
  2. TMAS is designed to improve joint optimization, its benefits may vary across tasks. The paper does not sufficiently explain the task-specific trade-offs introduced by adaptive timestep allocation.
  3. The zero-shot evaluation focuses on referring segmentation. It remains unclear whether the proposed framework generalizes effectively to unseen datasets from other task families or to different sensing modalities.
Questions for authors
  1. Could the authors further analyze how the structured target representation and deterministic recovery process affect geometric precision in instance-level prediction?
  2. Could the authors clarify why the benefits of TMAS vary across tasks and explain how adaptive timestep allocation affects the optimization of individual tasks?
  3. Could the authors provide further evidence of zero-shot generalization to unseen datasets from other task families or sensing modalities?

Comments on the paper

1 comment
Area_Chair_ukeVOpenReview ↗

Author Reviewer Discussion

Dear reviewers,

Thanks again for agreeing to review for ICLR.

We are approaching the end of the discussion period. Therefore, I would appreciate your acknowledgment of the authors' rebuttal and any feedback you would like to share with the authors.

This will help to improve the submissions, streamline decision making, and keep ICLR's original idea of having such a discussion period valuable to all participants in the review process.

Best, Your AC

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.