acceptodds
Under review as a conference paper at ICLR 2027

The Change You Want To Talk About: Enabling spatial and narrative Change Detection in Earth Observation with a single VLM architecture

Abstract

Bi-temporal change detection (CD) of Earth Observation images is fragmented into disconnected spatial and textual prediction tasks. While recent Vision Language Models (VLM) explore open-vocabulary and query-driven challenges, their semantic flexibility is often constrained by decoupled, post-hoc matching pipelines. Besides, they often struggle to capture fine-grained temporal dynamics and remain restricted to isolated tasks, such as dense prediction, captioning, or grounding. We propose LOSCAR, a multi-task VLM capable of addressing diverse CD tasks under a unique paradigm, through specific prompting. By integrating a spatio-temporal architecture with a spatial decoder that translates multimodal features into dense image masks, our model jointly generates natural language descriptions and explicit spatial outputs under a continuous open-vocabulary regime. After multi-dataset training, our experiments show that LOSCAR achieves state-of-the-art results in change captioning while delivering accurate spatial localization and robust reasoning on complex semantic change trajectories.

Primary area:
unsupervised, self-supervised, semi-supervised, and supervised representation learning
Keywords:
remote sensing, change detection, visual question answering, large language models, open vocabulary

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_xJSzOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper introduces LOSCAR, which fuses visual features with LLM hidden states. Three spatial decoding branches produce BCD masks, semantic masks, and relevance masks alongside textual answers. The authors construct multi-task training data through automated question-answer generation and Gemini-based vocabulary expansion, and evaluate the model on multiple tasks, such as spatial and localization tasks and narrative tasks.

Critical strengths
  1. The architecture covers a broad range of tasks. Separating the Change Decoder from the Relevance Decoder is a sensible design: the model can retain global change information while localizing the changes requested by the user.

  2. Some results are competitive, such as on QAG-360K, LOSCAR achieves a CVG mIoU of 54.9, compared with 44.9 for VisTA. The ablations provide some evidence that multi-task supervision helps.

Critical weaknesses
  1. The proposed design needs to be evaluated more clearly. The current ablations remove entire task branches, changing the architecture. They do not show whether the gains come from the proposed fusion mechanism. I would like to see comparisons that remove the LLM hidden states or replace cross-attention with a simpler fusion operation, while keeping the visual backbone, decoders, training data, and training budget the same. Open-vocabulary change detection is a central contribution, but it lacks quantitative evaluation. A vocabulary of ~ 4000 concepts that accepts arbitrary text queries does not demonstrate that the model can localize categories or change trajectories absent from its training supervision. The gains from vocabulary expansion are small. Also, removing the vocabulary expansion data (in the ablation study for now) and replacing the Embedding Decoder with a fixed-category classification head tests different things. The current experiments do not isolate the contribution of the open-vocabulary representation itself.

  2. Some baselines are missing. The comparisons within each task are limited. There are no results from strong general-purpose VLMs, nor comparisons with modular systems that combine a VLM with grounding or segmentation tools. Closed-source VLMs that accept multiple images are also reasonable baselines for change captioning and VQA. Some baseline results differ from those reported in the original papers, such as MCI. If the authors reran these experiments, they should provide enough detail about the training and evaluation. The reported results need to be consistent across tables. For example, the CVG mIoU in Table 4 matches the result w/ pretraining in Table 6, while the textual answer OA matches the result w/o pretraining.

Questions for authors

See the questions in the weakness part.

Reviewer_jYAgOpenReview ↗
Rating
2
Confidence
3
Informativeness
1
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes LOSCAR, a multi-task vision-language framework for change understanding of bi-temporal remote sensing images. Built upon VideoLLaMA3, the method leverages cross-attention between visual tokens and the hidden states of the language model, along with multiple spatial decoding branches, to produce natural language responses, global change masks, bi-temporal semantic embeddings, and query-relevant regions. The authors also construct training data by combining existing remote sensing datasets, automated question-answer templates, and fine-grained concepts generated by Gemini. Open-vocabulary semantic prediction is enabled via alignment between pixel embeddings and CLIP text embeddings. Experiments encompass change captioning, visual question answering, change localization, as well as binary and semantic change detection. The paper reports performance gains on several caption generation and localization tasks. However, its core open-vocabulary capability is demonstrated mainly through qualitative examples, without clear partitioning of unseen concepts and quantitative generalization validation.

Critical strengths
  1. The application requirements addressed by this paper are of practical value. Integrating change descriptions, global change maps, semantic information, and query-relevant regions into a unified architecture facilitates interactive remote sensing change analysis. Automatically constructing question-answering and relevant-region supervision from existing semantic annotations also offers a scalable data construction paradigm.
  2. The paper reports several meaningful experimental gains. For instance, the localization mIoU on QAG-360K reaches 54.9, outperforming VisTA’s 44.9 listed in the table; the CIDEr score for change description on LEVIR-MCI is 136.0, exceeding MCI’s 118.0. After removing the correlation branch in Table 7, the SCD mIoU drops from 78.5 to 72.7, which provides preliminary evidence for the synergy among different supervised tasks. Nevertheless, the architectural origins of these gains still need to be verified through more adequate controlled experiments.
Critical weaknesses

The main newly added structures of LOSCAR consist of linear projection, vision-language cross-attention, 2D restoration of tokens, U-Net-style upsampling, and multiple task decoding branches (Section 3.2; Appendix A.1). Mapping the latent representations of language models to pixel masks has precedents such as LISA. In remote sensing change analysis, DVLChat has combined natural language question answering with pixel-referenced change detection; UniChange has unified binary and semantic change detection via language-conditioned spatial decoding; Delta-LLaVA has also jointly performed change understanding and mask generation. This paper needs to more clearly illustrate what new change modeling mechanisms or verifiable new capabilities its latent state fusion and multi-branch organization introduce compared with these methods.

Currently, Table 7 mainly conducts ablation by removing entire output branches. Such experiments alter both task outputs and supervision signals simultaneously. They can show that multi-task supervision may be beneficial, but are insufficient to prove that the proposed fusion method or decoding structure outperforms conventional alternative designs. The paper lacks comparisons with standard spatial decoding heads or other language conditioning approaches under the same VideoLLaMA3 backbone, identical training data, consistent supervision, and a similar training budget. Therefore, existing results cannot fully disentangle the respective contributions of the newly added architecture, pre-trained backbone, additional generated data, and auxiliary supervision.

Expanding task coverage carries certain systematic value, yet the paper has not provided sufficient evidence to establish this expansion as a notable methodological advance. For a work claiming a "novel spatial decoding mechanism" and a "unified multi-task architecture" as its core contributions, this gap directly affects the judgment of its novelty.

The paper repeatedly emphasizes open-vocabulary semantic understanding and generalization to unseen change types during training, yet Section 4.2 clearly states that OVCD only undergoes qualitative evaluation. Replacing discrete classification outputs with CLIP-aligned continuous embeddings and expanding existing categories to approximately 4,000 descriptive concepts alone are insufficient to prove that the model can accurately localize unseen semantics or identify unseen change relationships. The manuscript does not present quantitative results on a clear split of seen/unseen concepts, unseen attributes or change combinations, nor does it include corresponding open-vocabulary comparisons. Figure 8 mainly demonstrates text-guided localization of several concepts across two time phases and cannot independently verify open-vocabulary change generalization. Issues such as a large vocabulary size, imbalanced concept frequencies and synonymous terms can serve as reasons to refine the evaluation protocol, but they do not justify completely omitting quantitative testing of this core capability.

In addition, in Appendix B.1–B.3, Gemini generates fine-grained descriptions conditioned on existing coarse categories, and these descriptions are then used to replace the original pixel labels. However, when the building class in the same image includes both residential buildings and industrial warehouses, the manuscript does not specify how different fine-grained concepts are reliably assigned to their respective pixels or instances, nor does it provide corresponding label quality verification. Therefore, it remains unclear whether the current data construction pipeline truly adds reliable fine-grained spatial semantic knowledge.

In Table 7, after removing vocabulary expansion, the BCD IoU, SCD mIoU and CIDEr change only from 64.6, 78.5 and 83.5 to 64.4, 77.6 and 83.1 respectively, while the RCD IoU changes from 72.3 to 72.4. These results on existing tasks do not compensate for the lack of evaluation on unseen concepts. Since open-vocabulary capability serves as a key basis that distinguishes this work from existing multi-task change understanding frameworks, the absence of its verification constitutes a deficiency at the level of core contributions.

Questions for authors
  1. What are the key novel mechanisms proposed in this paper compared with UniChange, DVLChat and Delta-LLaVA? Under the same VideoLLaMA3 backbone, training data and supervision conditions, can you provide controlled comparisons between conventional spatial decoding heads or other language conditioning approaches and the design in this paper, so as to disentangle the gains brought by architecture, data and multi-task supervision?
  2. Which concepts, attribute combinations or change relations in open-vocabulary evaluation are absent from the task training data and the extended vocabulary of Gemini? Can you provide a clear seen/unseen split together with quantitative results? For the issues of vocabulary sparsity and synonyms, can subset manual verification, frequency grouping or semantic-tolerant evaluation be adopted?
  3. When Gemini returns multiple fine-grained concepts for one coarse category, how is the correspondence established between concepts and pixels/instances? For example, how are residential houses and industrial warehouses within the same image distinguished from the original building mask? Is there manual sampling inspection or other methods to verify the accuracy of these fine-grained labels?
  4. Are the results of each task obtained from a single checkpoint, or from models fine-tuned separately for different datasets or tasks? Please explain why the localization mIoU of 54.9 and the answer OA of 74.4 for QAG-360K in Table 4 correspond respectively to the pre-trained and non-pre-trained settings in Table 6; meanwhile, clarify the relationship between the IoU of 80.8 for LEVIR-MCI in Table 2 and the value of 81.9 under the full pre-training setting in Table 6.
  5. Since the pre-training data and downstream benchmarks such as QAG-360K share the same raw data source, please specify the rules for isolating images and geographic regions across datasets, and report the overlap checking results between the pre-training set and each test set.
  6. For which images and quantitative results are SLIC, majority voting and frequency-based weighting mentioned in Appendix C applied? Please report the raw decoder outputs and post-processed outputs separately to clarify the contribution of the proposed model itself to spatial prediction quality.
Reviewer_RJJ9OpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes a method for generating textual descriptions alongside detected change regions in remote sensing images of the same location captured at different times. The proposed approach successfully generates descriptions of change regions directly from an LLM backbone while incorporating an additional image encoder to extract the corresponding change regions. The method is evaluated using datasets and metrics that have been extended beyond those used in previous studies, and the experimental results demonstrate improvements over baseline methods.

Critical strengths
  • As shown in Table 1, the proposed framework supports a relatively wide range of tasks compared with previous approaches. The reviewer believes this is an important perspective in remote sensing and represents a meaningful step toward developing more general-purpose applications in this field.

  • The reviewer also finds the overall system implementation to be well balanced. The proposed method provides a complete framework that integrates an LLM with multiple components, while achieving improved experimental performance over the baseline methods.

Critical weaknesses
  • While the reviewer appreciates the proposed approach as an application that combines an LLM with paired images and an image encoder for change localization, the reviewer, at the same time, would like to better understand what the authors consider to be the essential methodological contribution of this work. The framework integrates well-established foundation models and achieves superior performance compared with previous methods. However, the reviewer is interested in understanding where the main academic novelty and contributions lie, or whether the primary contribution is the development of an effective overall framework. The reviewer believes clarifying this distinction is important for further improving both the methodological contribution and the overall quality of the paper.

  • The reviewer has some concerns regarding the presentation. For example, Figures 1 and 2 appear to illustrate essentially the same framework at different levels of detail. Is it more effective to integrate these figures to communicate the main contributions more clearly? Furthermore, Figure 1 indicates that VideoLLaMA3 is used as the VLM, whereas Figure 2 specifies Qwen 2.5 as the LLM. This appears to introduce an inconsistency between the two figures. Since both figures describe the framework or overview of LOSCAR, the reviewer believes the paper should clarify which description is correct, or whether the difference simply reflects different experimental configurations.

Questions for authors
  • The paper places considerable emphasis on the use of a single VLM. Is it common in this research field to perform change detection by integrating multiple models? The reviewer would like to understand why the authors particularly emphasize the ability of a single VLM to perform both change detection and textual description generation.

  • Regarding the overview in Figure 2, the reviewer understands that the image encoder connected to the LLM performs change detection and outputs the corresponding change regions. However, does the LLM also generate textual descriptions? Given that the experimental evaluation appears to include VQA-related tasks, the reviewer assumes that the framework produces both visual outputs and textual descriptions. The reviewer believes Figures 2 and 3 could be improved by illustrating the text generation outputs more explicitly, making the overall functionality of the proposed framework easier to understand.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.