acceptodds
Under review as a conference paper at ICLR 2027

Chain-of-Change Detection for Multimodal Large Language Models

Abstract

Conventional remote-sensing change detection (CD) models typically follow a common design: Siamese encoders extract features from each date, an interaction module compares them, and a task-specific decoder produces a single output. We refer to this observe-then-compare structure as the CD-specific inductive bias. In practice, these models remain task-specific: binary change detection, semantic change detection, and change captioning are typically addressed by separate models. Multimodal large language models (MLLMs) unify these tasks in a single model, yet retain this bias only through encoder-side feature interaction. Within the LLM, supervision defines the target outputs but leaves observation and comparison implicit, allowing shortcut learning and disagreement between a model's masks and captions about the same change. To address this, we introduce Chain-of-Change Detection (CoCD), which formulates each step of the CD process as an explicitly supervised task and passes the decoded prediction of each stage to the next as evidence. CoCD first observes each date by predicting semantic masks for the queried classes, then compares the image pair to localize change guided by these masks, and finally describes the change from the predicted change mask and its semantic transitions. CoCD connects these stages with two proposed modules, the Semantic State Painter routing the observed semantic states to comparison and the Change Evidence Injector routing the localized change and its transitions to description, extending the CD-specific inductive bias beyond encoder-side feature interaction. Since no existing benchmark provides all three outputs for the same queried classes, we construct CoCD-Dataset from HRSCD and WUSU with aligned per-date semantic masks, change masks, and grounded captions. Experiments show improved change localization, caption quality, and cross-output consistency over unified baselines and a sequencing-only control. Inference-time interventions further demonstrate that downstream predictions use the routed evidence. Code is available at https://anonymous.4open.science/r/cocd_release-3CE3/.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
remote sensing, change detection, multimodal large language models

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_gGpKOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

To address the inconsistency between a model's masks and captions about the same change, the authors propose Chain-of-Change Detection (CoCD), which formulates each step of the CD process as an explicitly supervised task and passes the decoded prediction of each stage to the next as evidence. Specifically, the Semantic State Painter routes the observed semantic states to the comparison stage, and the Change Evidence Injector routes the localized change and its transitions to the description stage, thereby extending the CD-specific inductive bias beyond encoder-side feature interaction. Since no existing benchmark provides all three outputs for the same queried classes, the authors construct CoCD-Dataset from HRSCD and WUSU, containing aligned per-date semantic masks, change masks, and grounded captions. However, CoCD's core contribution is closer to an engineering combination, as components such as task chaining, evidence injection, and mask decoding have precedents in MLLM and remote-sensing change detection. Second, the experimental scope is narrow, with evaluation only on CoCD-Dataset constructed from HRSCD and WUSU, insufficient baseline adaptation, and captions synthesized by Qwen3-VL-32B, raising concerns about evaluation reliability. In addition, the method details and training protocol are incomplete.

Critical strengths
  1. An anonymous code link is provided, documenting data construction, training objectives, evaluation protocols, and the caption generation and verification pipeline.
  2. Based on HRSCD and WUSU, it provides aligned per-date semantic masks, change masks, and grounded captions, totaling 14,014 queries (11,898 / 510 / 1,606). Since no existing benchmark provides all three outputs simultaneously, CoCD-Dataset fills this gap and serves as a reusable resource.
  3. CoCD decomposes change detection into three stages—observe, compare, and describe—which aligns with the human cognitive process of analyzing changes and assigns clear responsibilities to each stage.
  4. The ablation experiments are relatively comprehensive, essentially covering the core design choices of the method.
Critical weaknesses
  1. Task chaining, visual CoT, evidence injection, and mask decoding all have precedents in MLLM and remote-sensing change detection.
  2. Training and evaluation are conducted only on CoCD-Dataset, which is constructed from HRSCD and WUSU, leaving it unclear whether the conclusions generalize to other remote-sensing datasets. A full comparison on standard benchmarks such as the original HRSCD, WUSU, LEVIR-CD+, and SECOND is recommended.
  3. The paper explicitly states that synthetic change captions are generated by Qwen3-VL-32B and then verified by rules. While this is reasonable, it introduces bias from the generative model.
  4. The operations in Equations 4 and 5 are written in a highly non-standard way, and the weight coefficients preceding the losses in Equation 9 appear without justification. A complete definition, along with convergence and sensitivity analyses, should be provided.
  5. Comparison with conventional CD methods is missing.
  6. The method mainly adopts common operations such as convolution, and its novelty is insufficient to support an ICLR main-conference paper.
Questions for authors
  1. Equation (3) for SSP is not fully given in the main text. Could the authors provide the complete formulation?
  2. The evidence construction in Equation (7) for CEI is only partially specified. Could the authors provide a complete definition?
  3. MMC and CMC only measure internal consistency, not correctness. A model can produce a fully consistent but incorrect mask-caption pair. Have the authors considered reporting correctness metrics alongside consistency metrics?
  4. The caption verification rules only check consistency with the fact tuples. Do they also check language naturalness, diversity, or semantic nuance?
  5. Training and evaluation are conducted only on CoCD-Dataset, which is constructed from HRSCD and WUSU. Can the conclusions generalize to other remote-sensing datasets?
  6. Could the authors provide a full comparison on standard benchmarks such as the original HRSCD, WUSU, LEVIR-CD+, and SECOND?
  7. Could the authors add comparisons with conventional methods in terms of mIoU, inference FLOPs, and parameter count?
  8. Provide a complete definition of Equation (9), along with corresponding convergence and sensitivity analyses.
  9. Standardize the notation in Equations (3) and (4).
9 comments
AuthorsOpenReview ↗

Response to Reviewer gGpK (1/6)

We thank Reviewer gGpK. We provide the complete equations, the standard-benchmark comparison, model size and inference cost, and clarify the contribution.

Q1. "Equation (3) for SSP is not fully given in the main text" — Sec. 3.3.1, Appendix A.2

The SSP formulation is spread over Eq. 2–5, so Eq. 3 alone does not show the full path from class probabilities to the decoder prompt. The revision presents it as one connected derivation, which we summarize here.

Each class name is embedded as u_k = W_e e_k, with e_k the mean LLM input embedding of its tokens (Eq. 2). The state of date t is F_t(x) = Σ_k P_{t,k}(x) u_k + τ_t, a 64-dimensional vector per pixel at 128×128 (Eq. 3). The two states are concatenated, resized to 64×64, and passed through the state adapter (3×3 conv 128→256, LayerNorm, GELU, 1×1 conv 256→256) to give Z_S (Eq. 4), which is added to the visual prompt of the change decoder (Eq. 5). W_e, τ_t and the adapter are the only trained parameters (0.52 M).

Q2. "The evidence construction in Equation (7) for CEI is only partially specified" — Sec. 3.3.2, Appendix A.2

We agree that the selection rules and the text format were missing. They are now in Appendix A.2.

  • R renders the predicted change mask as a 256×256 white-on-black image.
  • f_cell lists the cells of a 3×3 grid that hold at least 2% of the changed pixels.
  • f_tr labels each changed pixel by its class pair, keeps pairs whose two labels are different and not background, drops pairs covering less than 0.4% of the image, and lists the three largest pairs with the cells holding at least 2% of their pixels.

The evidence reads "Changed (top-right, right): <IMG> Transitions: excavation -> low building (top-right); grassland -> road (right)."

Q3. "MMC and CMC only measure internal consistency, not correctness"

We agree, which is why every table pairs them with correctness metrics (oIoU, sem-mIoU, BLEU/CIDEr) and Appendix B.1 reports caption accuracy against ground truth (Cap-Acc). The two do come apart in our results: in Table 2, Naive-Chain raises MMC from 37.98 to 48.74 and CMC from 35.18 to 45.11 over Parallel, while oIoU (25.00 → 25.59) and BLEU-4 (21.98 → 21.96) stay similar. Consistency alone is therefore not taken as evidence of correctness in the paper.

AuthorsOpenReview ↗

Response to Reviewer gGpK (2/6)

Q4 (and W3). "The caption verification rules only check consistency with the fact tuples" and "it introduces bias from the generative model" — Appendix B.14 [also: CJgm W1, Q4]

The rules verify the facts of each caption. For this response we also measured the quality of the wording, in naturalness and diversity.

Naturalness. An independent judge from a different family (OpenAI GPT-5.5) compared each of the 1,002 LLM-written test references with a template sentence built from the same facts, in random order. It preferred the LLM reference in 75.7% of pairs (naturalness 4.18 vs. 3.17 on a 1–5 scale).

Diversity. The references are more varied than a template and less varied than free-form human captions, by design: class names and location words are fixed to the dataset vocabulary.

MetricCoCD referencesTemplate (same facts)LEVIR-CC (human)
Vocabulary size39941592
MATTR (window 50).541.448.643
Bigram entropy (bits)7.135.698.89
Unique sentences56.3%35.4%87.9%

Each column uses 4,766 captions: one per scene and fact set for ours, one per changed image for LEVIR-CC [1]. MATTR is the share of distinct words within a window of 50 consecutive words, averaged over all windows, and higher values mean less repetition. Bigram entropy measures how evenly the pairs of consecutive words are distributed, and higher values mean more varied phrasing. LEVIR-CC captions are free-form human descriptions of a different domain and are shown only as a reference scale.

Bias from the generator. The facts come from the masks and only the wording comes from the 32B model, so as a scoring reference the wording can affect only the n-gram metrics. When the test references are rewritten by a non-Qwen generator (OpenAI GPT-5.4), the n-gram scores of every model drop, but CoCD stays above Parallel on all of them. The training captions remain Qwen-written, and for that side the evidence is that the gains hold with non-Qwen backbones (LLaVA-1.5-7B [2], InternVL3-2B [3], Appendix B.4). The tables are in our reply to Reviewer CJgm (Q4).

References

[1] Liu, C., Zhao, R., Chen, H., Zou, Z., and Shi, Z. Remote Sensing Image Change Captioning With Dual-Branch Transformers: A New Method and a Large-Scale Dataset. IEEE TGRS, 60:1–20, 2022. [2] Liu, H., Li, C., Li, Y., and Lee, Y. J. Improved Baselines with Visual Instruction Tuning. CVPR, 2024. [3] Zhu, J., Wang, W., Chen, Z., Liu, Z., et al. InternVL3: Exploring Advanced Training and Test-Time Recipes for Open-Source Multimodal Models. arXiv:2504.10479, 2025.

AuthorsOpenReview ↗

Response to Reviewer gGpK (3/6)

Q5, Q6 (and W2). "Can the conclusions generalize to other remote-sensing datasets?" and "a full comparison on standard benchmarks such as the original HRSCD, WUSU, LEVIR-CD+, and SECOND" — Appendix B.11 [also: YJYF Q2]

We trained CoCD and Parallel on each benchmark alone (SECOND [4], HRSCD [5], WUSU [6]), with the same backbone, decoder and schedule, and scored them with the standard SCD protocol. Each model is tested on the benchmark it was trained on, so these results show whether the gain reproduces on other benchmarks, not zero-shot transfer to an unseen domain.

Splits. These runs use the splits released with recent SCD work, not our CoCD-Dataset splits. For HRSCD this is the split of TaCo [7] and SemDINO [8] (256×256 patches, patches with less than 5% changed pixels removed, 9,463 training and 2,366 test patches). For WUSU it is the split of GSTM-SCD [9] (512×512 tiles of the 2015→2018 pair, 977 training and 137 test tiles), which, unlike Eq. 1 of the paper, also treats a pixel as changed when it is labelled at one date and unlabelled at the other. For SECOND we train on 2,671 of the 2,968 official training pairs, keep the remaining 297 out of training, and test on the official test set (1,694 pairs). Both models are trained for 10 epochs on SECOND and HRSCD, as in the paper, and for 30 epochs on WUSU, whose training set is about a third the size of SECOND's, and we report the final checkpoint of each run.

Labels. HRSCD and WUSU label every pixel at both dates, so no pseudo labels are needed. SECOND labels classes only inside the changed regions. CoCD observes the full scene at each date, and SSP passes that full-scene state to the change decoder, so training needs a semantic label for the unchanged area as well. We therefore trained a segmenter on the changed-region labels of the SECOND training set and used its prediction as a pseudo label outside the change mask, keeping the ground truth inside it. Both models are trained on the same labels, and the test set is not used. The standard protocol scores semantics only inside the changed regions, so the pseudo labels do not enter the reported metrics.

BenchmarkModelOAmIoUF_scdSeK
SECONDParallel87.3072.8960.5520.98
CoCD87.7073.3161.2521.82
HRSCDParallel82.3067.6764.0316.94
CoCD82.3166.5766.7618.58
WUSUParallel93.9075.7154.0117.23
CoCD95.1478.6563.7626.07

CoCD is above Parallel on F_scd and SeK, the two metrics that score the semantic change, on all three benchmarks. The gain is largest on WUSU (SeK 17.23 → 26.07) and smallest on SECOND. On HRSCD, mIoU is lower for CoCD (66.57 vs. 67.67) and OA is equal.

These splits have much larger changed areas than CoCD-Dataset, which keeps scenes with little or no change and pairs adjacent years for WUSU: the median changed area per test scene is 20.0% (HRSCD) and 7.2% (WUSU) here, against 3.7% and 1.8% in CoCD-Dataset. The absolute values here are therefore not comparable with those in our reply to Reviewer YJYF (Q2), where the same protocol is applied to CoCD-Dataset.

References

[4] Yang, K., Xia, G.-S., Liu, Z., Du, B., Yang, W., Pelillo, M., and Zhang, L. Asymmetric Siamese Networks for Semantic Change Detection in Aerial Images. IEEE TGRS, 60:1–18, 2022. [5] Daudt, R. C., Le Saux, B., Boulch, A., and Gousseau, Y. Multitask Learning for Large-Scale Semantic Change Detection. Computer Vision and Image Understanding, 187:102783, 2019. [6] Shi, S., Zhong, Y., Liu, Y., Wang, J., Wan, Y., Zhao, J., Lv, P., Zhang, L., and Li, D. Multi-Temporal Urban Semantic Understanding Based on GF-2 Remote Sensing Imagery: From Tri-Temporal Datasets to Multi-Task Mapping. International Journal of Digital Earth, 16(1):3321–3347, 2023. [7] Guo, H., Liu, C., Zhang, H., Chen, B., Zou, Z., and Shi, Z. TaCo: Capturing Spatio-Temporal Semantic Consistency in Remote Sensing Change Detection. arXiv:2511.20306, 2025. [8] Tong, X., Zhou, M., Sun, J., Tang, Y., and Wang, L. SemDINO: A DINOv3-Driven Network for Cross-Temporal Semantic Alignment in Change Detection. arXiv:2606.09772, 2026. [9] Liu, X., Dai, C., Ding, L., Zhang, Z., Li, Y., Zuo, X., Li, M., Wang, H., and Miao, Y. GSTM-SCD: Graph-Enhanced Spatio-Temporal State Space Model for Semantic Change Detection in Multi-Temporal Remote Sensing Images. ISPRS Journal of Photogrammetry and Remote Sensing, 230:73–91, 2025.

Reviewer_YJYFOpenReview ↗
Rating
2
Confidence
3
Informativeness
1
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes CoCD, a change detection chain framework for remote sensing change understanding, which organizes bi-temporal semantic segmentation, binary change localization and change description into a supervised execution sequence of "Observation-Comparison-Description". Its core idea is to explicitly pass the decoded prediction results of each stage to subsequent stages, establishing direct information links between different outputs. Specifically, the Semantic State Painting (SSP) module combines predicted semantic class probabilities with class name embeddings to construct dense spatial prompts for change decoding; the Change Evidence Injection (CEI) module feeds the predicted change masks, along with class transition and location information extracted from predicted semantic maps and change maps, into the language model to generate the final descriptions. During training, the model uses prediction caches from the previous training round to build input evidence for the description stage. In addition, the authors construct CoCD-Dataset based on HRSCD and WUSU, ensuring that bi-temporal semantic annotations, change masks and automatically generated descriptions are aligned within the same query class scope. Experiments include comparisons with existing models, matching controls of parallel and sequential execution, component ablation studies, and evidence intervention during inference. The results demonstrate that the proposed method improves change localization, factual accuracy of descriptions and consistency among outputs, and intervention experiments verify that downstream predictions are indeed affected by the transmitted intermediate evidence.

Critical strengths
  1. A unified change understanding model can simultaneously output semantic maps, change maps and textual descriptions, yet these outputs may not refer to one another or represent the same set of changes. To address this issue, this paper implements information dependencies among different tasks into the actual execution flow of the model by explicitly passing supervised intermediate predictions.
  2. The controlled design of Table 2 and Table 4 is an important strength of this paper, as it helps distinguish the respective effects of task order, encoder-side difference modeling, and decoding state feedback.
  3. The semantic XOR ablation study, experiments with different multimodal large language model backbones, and comparisons of different training evidence sources in the appendix further validate the effectiveness of the proposed method.
Critical weaknesses
  1. In Appendix A.4, the reference descriptions are generated from category transition and location facts extracted by masks and are subject to strict lexical constraints; CEI, by contrast, feeds the same type of structured facts extracted from predicted masks into the language model. Table 7 presents evaluations against ground-truth facts, which is valuable and goes beyond the verification of internal consistency. Nevertheless, the existing experiments have not fully clarified how much of the factual accuracy of the final descriptions stems from improved dense prediction and explicit fact extraction, and how much is further contributed by the subsequent language generation stage.
  2. An important and low-cost missing control is as follows: fix the exact same predicted semantic maps, change maps and fact extraction processes, generate descriptions directly via deterministic templates, and compare their factual accuracy. Table 4 also lacks a CEI setting that only provides category shift and location text, so it remains unclear how much additional benefit can be brought by adding masked images after textual facts have already been provided.
  3. The intervention results in Table 3 support that "downstream predictions utilize intermediate evidence", but are insufficient to prove that the model has "eliminated shortcuts". The model still retains a direct visual input path, and replacements with ground-truth annotations and blank states alone cannot establish the stronger conclusion of shortcut elimination. It is recommended that the authors qualify the relevant statements accordingly.
  4. Section 4.1 states that semantic change detection results can be obtained by combining two semantic prediction maps and a predicted change mask, so SCD metrics are not reported separately. However, the ability to construct such an output does not mean its joint correctness has been verified. Semantic change detection requires that the change location, the pre-change category, and the post-change category are all correct simultaneously, while the separate binary change IoU and bi-temporal semantic mIoU cannot directly measure this requirement.
Questions for authors
  1. To clearly characterize the source of the gains in caption generation, could we fix the predicted semantic maps, change maps, category transition and location extraction procedures to be exactly identical to those of CoCD, and add a baseline that generates captions directly via deterministic templates? Among these, Cap-Acc and the accuracy of no-change captions are particularly informative. If the mask images themselves are also regarded as a key contribution of CEI, a baseline retaining only category transitions and location text would also help illustrate its additional benefits. 2. Could we use the predictions already generated on the CoCD-Dataset to report a joint SCD metric that requires both the change location and the before-and-after categories to be correct? Separate results for HRSCD and WUSU would further help assess whether such gains hold across both data sources. 3. How is the prediction cache from the previous training round kept spatially consistent with the random geometric augmentations described in Appendix A.1? Is the cache stored under the original image coordinate system and transformed to the current augmented view before mask images and location text are generated? Please elaborate on this coordinate handling pipeline, because merely masking the supervision loss for orientation-related words in captions cannot automatically align the previously augmented masks with the current input image. 4. The structured output example in Figure 2 generates mask tokens only for categories that actually appear in each time phase, whereas Section 3.3 and Equation 13 define queries for all K query categories. For categories absent in the current time phase, how does the model construct the corresponding queries and probability channels? During inference, how are cases handled where tokens for a certain category are missing or duplicated?
5 comments
AuthorsOpenReview ↗

Response to Reviewer YJYF (1/3)

We thank Reviewer YJYF for a review that identified exactly the controls we needed.

Q1 (and W1, W2). "Add a baseline that generates captions directly via deterministic templates" — Appendix B.7

We fixed the predicted semantic and change maps of the CoCD checkpoint and the fact extraction, and produced the final sentence either by a deterministic template or by the model's caption stage. The template renders exactly the transition text that CEI gives the model. Both rows use the masks of the same inference run, which reproduces the paper's numbers.

Caption (same masks, same extracted facts)Cap-Accno-change acc.mentions a changehallucinated transition
Template37.9295.3461.94%8.98%
Caption stage (CoCD)49.9284.1591.93%17.07%

"Mentions a change" is measured on the 1,177 queries with a ground-truth change. The hallucination rate is the share of stated transitions that are not in the ground truth, averaged over the queries that state at least one transition.

Replacing the template with the caption stage raises Cap-Acc by 12.0 points. The template can only verbalize the extracted transition text, which is empty for 38.1% of the queries with a ground-truth change: in 18.9% every nameable transition covers less than 0.4% of the image, in 12.2% the predicted change contains only same-class or background pairs, and in 7.0% no change is predicted. The caption stage also sees the rendered change mask, its own observation spans and the images, and it states a transition in most of these cases (mention rate 61.94% → 91.93%), recovering more ground-truth transitions than the template. The cost is more hallucination (8.98% → 17.07%) and lower accuracy on no-change scenes (95.34 → 84.15).

Q1, second part (and W2). "A baseline retaining only category transitions and location text" — Appendix B.7

We trained this setting with the same recipe as Table 4 and report it in Appendix B.7. Only the last row uses SSP.

CEI evidence to the caption stagelocation texttransition textrendered maskoIoUBLEU-4CIDErMMCCMC
None (Naive-Chain)25.5921.96125.148.7445.11
Text only (new)✓✓26.6326.73145.248.3261.68
Full✓✓✓27.3727.79148.548.4667.27
CoCD (Full + SSP)✓✓✓31.3330.00170.355.7766.71

The textual facts alone raise BLEU-4 from 21.96 to 26.73 and caption–mask agreement (CMC) from 45.11 to 61.68. Adding the rendered mask on top of the text, which is the reviewer's question, gives a small further gain in caption quality (BLEU-4 26.73 → 27.79) and a larger one in agreement (CMC 61.68 → 67.27). Change localization changes little across these CEI settings (oIoU 25.6–27.4), and the larger gain appears when SSP is added (31.33).

AuthorsOpenReview ↗

Response to Reviewer YJYF (2/3)

Q2 (and W4). "A joint SCD metric that requires both the change location and the before-and-after categories to be correct" — Appendix B.8

The SCD output is formed from the model's outputs by reading the per-date semantics inside the predicted change mask (Sec. 4.1). We agree that forming it does not establish its correctness, and that the binary change IoU and the per-date semantic mIoU, reported separately, do not measure it. We therefore now report the standard SCD metrics of Bi-SRNet [1] and SCanNet [2]. The per-date semantic prediction is kept only where the model predicts change and set to no-change elsewhere, the ground truth is treated in the same way with the ground-truth change, and the two maps are compared over all pixels. We re-ran the same checkpoints on the full-vocabulary query of each test scene.

ModelHRSCD mIoUF_scdSeKIoU_cchange recallWUSU mIoUF_scdSeKIoU_cchange recall
Parallel57.4237.291.8224.4028.259.3426.021.2522.0624.6
CoCD59.9741.332.7329.2635.061.4831.731.7326.4031.5

CoCD is above Parallel on every metric on both sources.

The absolute F_scd and SeK are lower than the values usually reported on HRSCD [3] and WUSU [4] because CoCD-Dataset is a different benchmark built from the same sources: it keeps scenes with little or no change and pairs adjacent years for WUSU, whereas the splits released with recent SCD work keep only HRSCD patches with at least 5% change and pair 2015 with 2018 for WUSU. The median changed area per test scene is 3.7% (HRSCD part) and 1.8% (WUSU part), against 20.0% and 7.2% on the released splits, so the two sets of scores are not comparable. On the released splits the same architecture reaches SeK 18.58 and 26.07 (reply to Reviewer gGpK, Q5–Q6).

References

[1] Ding, L., Guo, H., Liu, S., Mou, L., Zhang, J., and Bruzzone, L. Bi-Temporal Semantic Reasoning for the Semantic Change Detection in HR Remote Sensing Images. IEEE TGRS, 60:1–14, 2022. [2] Ding, L., Zhang, J., Guo, H., Zhang, K., Liu, B., and Bruzzone, L. Joint Spatio-Temporal Modeling for Semantic Change Detection in Remote Sensing Images. IEEE TGRS, 62:1–14, 2024. [3] Daudt, R. C., Le Saux, B., Boulch, A., and Gousseau, Y. Multitask Learning for Large-Scale Semantic Change Detection. Computer Vision and Image Understanding, 187:102783, 2019. [4] Shi, S., Zhong, Y., Liu, Y., Wang, J., Wan, Y., Zhao, J., Lv, P., Zhang, L., and Li, D. Multi-Temporal Urban Semantic Understanding Based on GF-2 Remote Sensing Imagery: From Tri-Temporal Datasets to Multi-Task Mapping. International Journal of Digital Earth, 16(1):3321–3347, 2023.

AuthorsOpenReview ↗

Response to Reviewer YJYF (3/3)

Q3. "How is the prediction cache kept spatially consistent with the random geometric augmentations?" — Appendix A.1

Yes, exactly as the reviewer describes. Before a prediction is saved to the cache, we undo the augmentation of that step, so the cache always holds masks in the original image frame. When the sample comes back with a different augmentation, we apply that augmentation to the cached masks first, and only then build the mask image and the location words. The evidence therefore always matches the current input. The loss masking of orientation words is unrelated to the cache: it applies to the ground-truth caption, which is written in the original frame.

Q4. "For categories absent in the current time phase … tokens missing or duplicated?" — Sec. 3.3.1 [also: gGpK Q9]

The training trace lists only the queried classes present at each date (Figure 2), so omitting a class is the model's "absent" prediction, and Eq. 3 now states that such a class has P_{t,k} ≡ 0. Each reported class token yields one query, a background query is appended, and pixel labels are the argmax over these channels. A missing token means the class is not predicted and its pixels are scored as misses. If a class token is duplicated, each copy yields its own query, the copies compete in the same softmax like any other queries, and their segments are merged under that class. No special rule is applied. Such cases are rare: 1 of the 1,606 test queries (0.06%) has a malformed trace. Omitting an absent class is the intended behaviour and is not counted here.

W3. "Insufficient to prove that the model has 'eliminated shortcuts'"

We agree that "closing the shortcuts" was too strong. The direct visual path remains: with the state route switched off, the model still reaches oIoU 27.58, above Parallel. What the interventions in Table 3 support is that the downstream stages depend on the routed evidence: replacing the semantic state with an all-background state lowers oIoU from 31.33 to 2.47, and blanking the change evidence lowers BLEU-4 from 30.00 to 16.11. The revision replaces “closing the shortcuts” in Sec. 1, 3.3 and 5 with the weaker claim that each downstream stage depends on the prediction routed from the stage before it.

What changed for this review. On the same masks and extracted facts, the caption stage raises Cap-Acc by 12.0 points over a template, at the cost of more hallucination. Appendix B.7 adds the text-only CEI setting. The SCD output is now scored with the standard SCD metrics, where CoCD is above Parallel on both sources. The shortcut claim is replaced by the statement that the downstream stages depend on the routed evidence, and the cache handling and unreported classes are documented.

Reviewer_CJgmOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes Chain-of-Change Detection (CoCD), which organizes remote-sensing change understanding into an explicit chain of intermediate predictions. The model first predicts semantic masks for the two time points, then feeds the semantic states back into change detection, and finally routes the predicted change mask and semantic transitions to caption generation. The authors further construct CoCD-Dataset based on HRSCD and WUSU. Experimental results show that explicit state feedback improves change localization, caption quality, and consistency across different outputs.

Critical strengths

1 The explicit intermediate-state feedback is well motivated. CoCD provides a unified sequential framework for multiple change-understanding tasks. 2 The experimental analysis is relatively comprehensive. The ablations help distinguish the effect of simple task chaining from that of decoded-state feedback.

Critical weaknesses
  1. The method has a strong dependence on dense supervision, and the caption construction may introduce bias. CoCD requires dense semantic labels for both time points, change masks, and captions, which substantially increases annotation cost and limits its applicability to weakly supervised or open-vocabulary settings. In addition, the captions are generated by Qwen3-VL-32B, while the main model is based on Qwen3-VL-4B. The paper lacks sufficient human evaluation of caption quality, and using models from the same family may introduce systematic linguistic or model-family bias.
  2. The necessity and optimality of the observe–compare–describe chain are not sufficiently justified. Predicting a change mask or caption does not necessarily require first obtaining complete semantic masks for both time points. The current design explicitly couples downstream tasks to upstream semantic predictions, so semantic errors may propagate and be amplified. The paper also lacks comparisons with other reasonable dependency structures, such as direct image-to-change prediction, change-to-caption generation, or introducing semantic states only within predicted change regions.
Questions for authors

1 Please clarify the task formulation. Does each query necessarily require the model to simultaneously output two semantic masks, one change mask, and one caption? How is this setting conceptually different from using a single model to jointly support BCD, SCD, and RSICC? The current formulation places all outputs under the same queried-class scope, but the distinction from conventional multi-task change detection is not sufficiently clear. 2 Why is the full observe–compare–describe ordering necessary? Could the authors compare against alternative dependency structures, such as directly predicting the change mask from the bi-temporal images and then generating the caption from the change mask, or using semantic states only within predicted change regions? It would also be helpful to compare performance with representative conventional change detection and change captioning methods. 3 Could the authors report the sensitivity of downstream change localization and caption generation to noise or errors in the semantic predictions? 4 Were the captions generated by Qwen3-VL-32B manually inspected or validated by an independent model? In addition, do the conclusions remain consistent if captions are generated using a model from a different family? 5 The mathematical notation and terminology should be further standardized and made more consistent throughout the paper.

5 comments
AuthorsOpenReview ↗

Response to Reviewer CJgm (1/4)

We thank Reviewer CJgm for a precise review. The two main concerns were whether the observe–compare–describe ordering is necessary and whether captions generated by a model of the same family bias the results. We address both with new experiments.

Q1. "Please clarify the task formulation" and the "distinction from conventional multi-task change detection" — Sec. 3.1

Yes, every query produces all four outputs in one autoregressive pass: the two semantic maps for the queried classes, the change mask defined on them (Eq. 1), and the caption describing the same transitions. A multi-task model reads these outputs out in parallel, so they can disagree: for example, the masks can show an excavation-to-structure change while the caption describes a low-building-to-excavation transition (Figure 5, left). In CoCD each later stage receives the decoded predictions of the earlier stages as additional evidence. The parallel read-out with the same backbone, decoders and data is the Parallel row of Table 2, so the comparison asked for is the first versus the last row of that table (oIoU 25.00 → 31.33, CMC 35.18 → 66.71).

Q2 (and W2). "Why is the full observe–compare–describe ordering necessary?" and "other reasonable dependency structures" — Appendix B.10

Following the reviewer's suggestion, we trained a change-first model, one of the alternatives listed, with the same backbone, decoder, data and budget as Table 2. It decodes the change mask before the semantics, so the semantic states are used only inside the predicted change, as caption evidence. We compare it with CoCD and with the CEI-only model of Table 4, which observes the semantics first but does not pass them to the change decoder (SSP removed), so that its change mask is predicted from the images alone. In neither model does the change decoder receive the semantic states.

StructureoIoUsem-mIoUCap-AccMMCCMC
CEI-only (Table 4)27.3750.5048.1248.4667.27
Change-first: change → semantics → caption (new)27.4949.9945.7445.8165.11
CoCD31.3352.6149.9255.7766.71

The two alternatives give similar change IoU (27.37 and 27.49), and CoCD is about 4 oIoU above both. Changing the order had little effect on mask quality, and the gain came with routing the decoded semantic states into the change decoder through SSP, which requires that they are decoded before the change step. This supports the observe-first order over the alternatives we tested.

AuthorsOpenReview ↗

Response to Reviewer CJgm (2/4)

Q2, second part. "Compare performance with representative conventional change detection and change captioning methods" — Appendix B.12 [also: gGpK W5, Q7]

We trained representative conventional models on CoCD-Dataset: ChangeMamba [1] and Change3D [2] for dense prediction, and RSCaMa [3] and the captioning variant of Change3D [2] for captioning. They cannot take a class query, so the comparison uses the 357 full-vocabulary test queries, which is why the CoCD and Parallel numbers differ from Table 1.

Dense predictionchange IoUchange F1sem-mIoUκ
ChangeMamba27.7343.4247.7862.59
Change3D20.8934.5545.0259.64
Parallel24.6739.5844.9058.11
CoCD30.5846.8450.5063.11
CaptioningBLEU-4CIDErMETEORROUGE-LCap-Acc
RSCaMa23.3595.0525.5738.0331.67
Change3D18.3074.2822.5535.5615.67
Parallel16.2087.7620.4131.1824.93
CoCD25.70123.4425.7637.8044.14

CoCD is above the conventional change detectors on all four dense metrics, and above the conventional captioners on every caption metric except ROUGE-L, where RSCaMa is slightly higher (38.03 vs. 37.80). Parallel, which uses the same MLLM without the dependency structure, is below ChangeMamba on all four dense metrics and below both captioners on BLEU-4. Using an MLLM alone therefore does not give the advantage. The matched controls of Table 2, where only the structure differs, attribute it to the dependency structure.

ChangeMamba (VMamba-Small backbone) and Change3D (X3D-L) use their official SCD architectures, which have a semantic head per date and a change head, and were supervised with the same full-scene semantic labels as our models. All conventional models were trained with their official schedules, with checkpoints selected on the validation set. CoCD and Parallel use the final checkpoint.

Q3 (and W2). "Sensitivity of downstream change localization and caption generation to noise or errors in the semantic predictions" — Appendix B.9

At inference we overwrote a fraction p of 32×32 blocks of the decoded semantic states with a wrong class, and let SSP, CEI and the metrics consume the corrupted state (same checkpoint, no retraining). The sem-mIoU column shows the size of the corruption.

psem-mIoU after corruptionoIoUBLEU-4CIDErCap-AccMMCCMC
052.6131.3330.00170.349.9255.7766.71
0.144.0530.0429.60168.747.5650.9363.12
0.333.4627.4429.28162.046.2247.2560.67
0.522.4324.1329.26163.245.9444.0056.82
Parallel, no corruption45.7825.0021.98131.630.5737.9835.18

Downstream performance falls gradually as the corruption grows, with no abrupt collapse in the tested range. At the strongest corruption, where sem-mIoU has fallen by 30 points, the caption and consistency metrics are still above the uncorrupted Parallel model (Cap-Acc 45.94 vs. 30.57, CMC 56.82 vs. 35.18), while change IoU falls slightly below it (24.13 vs. 25.00). This is consistent with the downstream stages combining the routed state with the direct visual path.

References

[1] Chen, H., Song, J., Han, C., Xia, J., and Yokoya, N. ChangeMamba: Remote Sensing Change Detection With Spatiotemporal State Space Model. IEEE TGRS, 62:1–20, 2024. [2] Zhu, D., Huang, X., Huang, H., Zhou, H., and Shao, Z. Change3D: Revisiting Change Detection and Captioning From a Video Modeling Perspective. CVPR, 2025. [3] Liu, C., Chen, K., Chen, B., Zhang, H., Zou, Z., and Shi, Z. RSCaMa: Remote Sensing Image Change Captioning With State Space Model. IEEE GRSL, 21, 2024.

AuthorsOpenReview ↗

Response to Reviewer CJgm (3/4)

Q4 (and W1, second part). "Were the captions manually inspected or validated by an independent model? Do the conclusions remain consistent if captions are generated using a model from a different family?" — Appendix B.14 [also: gGpK W3, Q4]

Were the captions inspected or validated? When the dataset was built, each caption was checked automatically against the ground-truth fact tuples by the parser used for CMC, and regenerated or replaced by a template if a fact was missing or contradicted. We inspected samples by hand during development, but there was no systematic inspection and no validation by another model. For this response, a judge from a different family (OpenAI GPT-5.5) checked the LLM-written references against the fact tuples.

LLM-written referencesnmajor facts statedcontradictedhallucinated transition
Test, all1,002100.0%32
Train, random sample1,000100.0%00

The flags fall on three test captions in total. We checked them by hand and all three are errors of the judge, not of the captions. In two, the caption writes "low buildings" where the class name is "low building", and the judge treated them as different classes. In the third, the caption states two transitions in one clause and the judge read them as one.

What the 32B model contributes. The facts come from the masks, and the 32B model only turns them into more natural sentences: against a template built from the same facts, the judge preferred the LLM reference as more natural in 75.7% of the 1,002 test pairs. The full naturalness and diversity measurements are in our reply to Reviewer gGpK (Q4).

Do the conclusions hold with captions from a different family? We tested this in two ways. For the test references, we rewrote them with a non-Qwen generator (OpenAI GPT-5.4) from the same tuples and prompt, and re-scored the fixed checkpoints.

ReferencesModelBLEU-4CIDErMETEORROUGE-LCap-AccCMC
Qwen3-VL-32B (paper)Parallel21.98131.625.038.330.5735.18
Qwen3-VL-32B (paper)CoCD30.00170.328.843.049.9266.71
GPT-5.4Parallel13.8969.623.133.230.5735.18
GPT-5.4CoCD18.7178.825.936.849.9266.71

The n-gram scores drop for every model when the reference style changes, but CoCD stays above Parallel on all of them, so its advantage does not depend on the wording of the Qwen-written references. Cap-Acc and CMC are unchanged, as expected, because they are computed from parsed tuples and do not use the reference sentences.

This test changes the test references only. The training captions are still written by the 32B model. For that side, the evidence is that the gains persist with non-Qwen backbones trained on the same captions (LLaVA-1.5-7B [4]: oIoU 22.66 → 27.12, CMC 35.11 → 65.55. InternVL3-2B [5]: 26.87 → 29.72, 39.50 → 66.60, Appendix B.4).

References

[4] Liu, H., Li, C., Li, Y., and Lee, Y. J. Improved Baselines with Visual Instruction Tuning. CVPR, 2024. [5] Zhu, J., Wang, W., Chen, Z., Liu, Z., et al. InternVL3: Exploring Advanced Training and Test-Time Recipes for Open-Source Multimodal Models. arXiv:2504.10479, 2025.

Comments on the paper

2 comments
AuthorsOpenReview ↗

General response

We thank the reviewers for careful and constructive reviews. The reviews converge on two requests: (i) stronger evidence that the observe–compare–describe dependency structure, rather than multi-task training or the data, is responsible for the gains, and (ii) evaluation beyond our own benchmark and metrics. We ran new experiments and uploaded a revised PDF, with all new text in blue. No training recipe or main-table number changed. The new results are additional controls and evaluations of the reported checkpoints, together with new training runs for the comparisons the reviewers asked for (alternative structures and CEI settings, standard benchmarks, and conventional models).

New experiments

  1. Template-captioning control, and a text-only CEI setting (YJYF): Appendix B.7.
  2. Standard SCD metrics per source (YJYF): Appendix B.8.
  3. Sensitivity of downstream outputs to corrupted semantic states (CJgm): Appendix B.9.
  4. Alternative dependency structures, including a newly trained change-first chain (CJgm): Appendix B.10.
  5. Validation of the reference captions: facts, naturalness, diversity, and re-scoring with references from a non-Qwen generator (CJgm, gGpK): Appendix B.14.
  6. Standard benchmarks (SECOND, HRSCD, WUSU), and conventional change detection and captioning models trained on CoCD-Dataset (gGpK, CJgm): Appendix B.11, B.12.
  7. Model size, FLOPs and measured inference latency (gGpK): Appendix B.12, B.13.

Changes to the paper

  • Sec. 1, 3.3 and 5: "closing the shortcuts" replaced by the supported statement that the downstream stages depend on the routed evidence.
  • Sec. 3.3.1: SSP presented as one derivation (Eq. 2–5), a queried class the LLM does not report at a date has zero probability in Eq. 3, and Eq. 4–5 are written as functions with kernel sizes and channel dimensions.
  • Appendix:
    • New sections for the seven experiments above (B.7–B.14).
    • Added details: cache coordinate handling (A.1), unreported classes and CEI construction rules (A.2), full loss definition (A.7) and training curves (B.15).

We reply to each reviewer in the order of their questions. Where two reviewers asked about the same result, the tables are in the reply to the reviewer who asked most directly.

Dear Reviewers,

As the discussion period is coming to an end, I kindly ask all reviewers who have not yet responded to the authors' rebuttal to carefully review their responses and provide feedback or update their reviews accordingly.

Thank you for your time and efforts!

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.