acceptodds
Under review as a conference paper at ICLR 2027

BiGenCap: Joint Bitemporal Generative Modeling via Reciprocal Prediction for Remote Sensing Image Change Captioning

Abstract

Remote sensing image change captioning (RSICC) requires interpreting semantic changes between temporally separated images while preserving scene context and cross-temporal relationships. Existing methods mainly construct change representations through direct comparison and interaction of bitemporal visual features. Beyond such pairwise comparison, large-scale collections of bitemporal images contain diverse visual semantics and cross-temporal dependencies. Modeling these distributions can provide a generative prior for interpreting specific image pairs. We propose BiGenCap, which learns a joint generative prior over bitemporal images at scale and instantiates it for each image pair through reciprocal generative prediction. Specifically, we pretrain a Diffusion Transformer (DiT) with flow matching on approximately 230K image pairs from 21 public data sources, assigning independent flow times to the two temporal streams to jointly model within-stream semantics and cross-temporal dependencies. During captioning, reciprocal generative prediction alternately perturbs one temporal feature stream while keeping its counterpart clean. We exploit the resulting predictions in two complementary ways: generative discrepancy fusion (GDF) derives prediction-based discrepancy cues from the final DiT responses, while reciprocal relation modeling (RRM) uses intermediate representations to model local cross-temporal relations and aggregate visual content from the temporal counterpart. BiGenCap achieves aggregate captioning scores of 82.55 and 94.17 on LEVIR-CC and WHU-CDC, respectively. Further analyses highlight the value of genuine bitemporal pairing and the complementarity of discrepancy and relational cues.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
remote sensing, change captioning, generative modeling

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_rs8GOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

This author proposed BiGenCap, which learns a joint generative prior (pretrained on 230K image pairs) over bitemporal visual features and designs reciprocal prediction residuals and intermediate representations for RSICC.

Critical strengths

The idea of exploiting rich representations of generative modeling is compelling and well motivated. The proposed method (e.g., GDF and RRM) achieves consistent improvements over the compared methods on two benchmarks (with caveats discussed in the Weaknesses section), with ablation studies supporting the contributions of both modules.

Critical weaknesses

I outline the weaknesses that limit the paper’s completeness, listed in descending order of importance.

[Limited Evaluation of Caption Quality] Although conventional metrics (e.g., BLEU and METEOR) have long been a standard in the change captioning community, their limited alignment with human judgments raises concerns about whether improvements in these scores reflect genuine gains in output quality (as a result, recent works tend to employ LLM-as-Judge metrics). Complementary LLM-as-judge protocols---such as in [1,2]---would strengthen the result that the reported gains translate into more accurate descriptions. Given the marginal improvements (below 1.0 point in BLEU-4) in Table 3, I am not convinced that the current results demonstrate a meaningful improvement in caption quality.

[Insufficient Insights on GDF] GDF uses the difference between an actual feature and its reconstructed estimate as an additional cue. Yet, it remains unclear "what useful information" is encoded in this residual. The residual can encode reconstruction error, which is not necessarily specific to semantic changes; it may also reflect texture complexity, appearance variations, or denoising uncertainty. Conversely, semantic changes that are accurately reconstructed may yield small residuals. Although Table 2 demonstrates the empirical benefit of incorporating GDF, it offers limited insight into the source of this improvement. Beyond numerical gains, the authors should provide additional insights into what the residual captures and strengthen their claim.

[Missing Relevant Baselines] Several methods discussed in Related Work---including Diffusion-RSCC---are omitted from the main table. I believe Diffusion-RSCC is particularly relevant because it also employs generative modeling for RSICC; The authors should clarify why they do not compare with it and, if possible, provide a controlled comparison with a representative strong baseline under matched data access and conditions.

[1] The STVchrono Dataset: Towards Continuous Change Recognition in Time (CVPR 2024)

[2] C3-Bench: A Context-Aware Change Captioning Benchmark (ECCV 2026)

Questions for authors

Please refer to the Weaknesses section.

I currently lean toward a Rejection in its current form. Yet, if the authors can satisfactorily clarify and address the weaknesses outlined above, I would be happy to reconsider the assessment during the discussion period.

1 comment

I greatly thank the authors for their thoughtful responses and additional experiments.

Although the semantic interpretability of GDF remains somewhat unclear to me (beyond empirical gain), I will carefully consider the insights thus far during the AC---Reviewer discussion period.

Thank you again for your time and effort.

Reviewer_a4m1OpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes BiGenCap for remote sensing image change captioning, addressing the limited use of priors from large collections of temporal pairs in methods centered on comparing features within an image pair. It pretrains a joint generative model of bitemporal features through flow matching on approximately 230K image pairs. During captioning, it alternately perturbs one temporal stream while retaining the other as clean context, then incorporates prediction residuals and relations derived from intermediate representations into the change features. The reported aggregate scores (S_avg) are 82.55 on LEVIR-CC and 94.17 on WHU-CDC, exceeding FST-Net, the prior method with the highest aggregate scores in Table 1, by 1.96 and 2.05 points. However, the residuals lack direct validation as semantic change cues, the contribution of the generative objective is not separated from that of genuine temporal pairing, and the necessity of reciprocal prediction remains untested. The ablations also leave uncertainty about how these contributions carry through the full training procedure.

Critical strengths

(1) BiGenCap uses independent flow times and boundary samples with one clean stream to train the Diffusion Transformer (DiT) for the asymmetric conditional prediction used in captioning. Only the perturbed stream is supervised at these boundaries, matching the downstream prediction setting.

(2) BiGenCap exceeds the listed methods on all seven metrics on both datasets, with S_avg gains over FST-Net of 1.96 points on LEVIR-CC and 2.05 on WHU-CDC (Table 1).

(3) Ablations support the contributions of generative discrepancy fusion (GDF) and reciprocal relation modeling (RRM) in BiGenCap, with their combination performing best (Table 2). The comparison of genuine and shuffled pairs controls the architecture, images, and data scale to assess temporal correspondence (Table 3).

Critical weaknesses

(1) The semantic interpretation of GDF residuals lacks direct evidence. Eqs. (1) and (5) express them as scaled velocity prediction errors, which could also respond to illumination differences. Figure 3 measures whole-feature reconstruction and does not establish which semantic change cues the residuals contain.

(2) Table 3 supports the value of genuine temporal pairing, but does not establish an advantage specific to flow matching. A non-generative pretraining control using the same genuine pairs at comparable capacity and budget is missing, leaving the contribution of the generative objective unclear.

(3) BiGenCap requires two DiT passes, yet its accuracy and inference cost are not compared with a single-direction variant. Eq. (7) compares clean and perturbed roles of the same stream, mixing corruption and cross-temporal effects. Removing RRM as a whole cannot isolate the benefits of role reversal or the second pass.

(4) Component and initialization ablations stop after cross-entropy (CE) training, whereas the main results include self-critical sequence training (SCST). Freezing the visual modules does not guarantee unchanged relative gains after decoder optimization; the best flow regression weight on WHU-CDC changes after SCST (Table 5). This leaves the final gains incompletely attributed.

Questions for authors

Please refer to the Critical Weaknesses section above.

1 comment

Thank you for adding the GDF residual analysis and the post-SCST ablations, which resolve my earlier questions on those components. However, two core attribution gaps remain. First, Table 3 only verifies the benefit of genuine bitemporal pairing under a fixed flow-matching objective; without a non-generative baseline trained on the same paired data (even with necessary head adaptations), the actual contribution of flow matching itself is still unverified. Second, the single-pass baseline in Table 10 perturbs both temporal streams simultaneously, confounding the number of DiT passes with the availability of an unperturbed temporal context. We still cannot tell whether the gain comes from reciprocal two-pass generation or simply from conditioning on a clean counterpart. Since the core design choices and the extra compute of the two-pass setup are not sufficiently justified, I will maintain my score of 2.

Reviewer_bXRmOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes BiGenCap, a joint bitemporal generative modeling framework for remote sensing image change captioning (RSICC). The central idea is to learn a generative prior over paired temporal visual features through large-scale flow-matching pretraining and subsequently exploit this prior for instance-specific change representation learning.

Specifically, the authors pretrain a Diffusion Transformer (DiT) on 229,642 image pairs collected from 21 public remote sensing datasets, using independently sampled corruption times for the two temporal streams. During downstream captioning, reciprocal generative prediction is performed by alternately perturbing one temporal stream while keeping its counterpart clean. The resulting predictions are exploited through two complementary modules: Generative Discrepancy Fusion (GDF), which utilizes prediction-derived residuals from the final DiT responses, and Reciprocal Relation Modeling (RRM), which extracts local cross-temporal relations from intermediate DiT representations and aggregates visual information from the temporal counterpart.

Experiments on LEVIR-CC and WHU-CDC demonstrate competitive performance, with aggregate scores (Savg) of 82.55 and 94.17, respectively, outperforming previous methods. The authors further conduct ablation studies and feature-prediction analyses to investigate the contributions of GDF, RRM, and genuine bitemporal pairing during generative pretraining.

Overall, the paper presents an interesting attempt to introduce joint generative modeling into remote sensing change captioning.

Critical strengths
  1. A new joint generative modeling framework. The paper introduces a joint bitemporal generative prior and reciprocal prediction mechanism, providing complementary discrepancy and relational information for change representation learning.

  2. BiGenCap achieves great results on both LEVIR-CC and WHU-CDC, with ablation studies supporting the effectiveness of its major components.

Critical weaknesses
  1. Insufficient evidence for the claimed generative pretraining contribution. Although genuine-pair pretraining consistently outperforms shuffled-pair pretraining, the improvements in Savg are relatively modest (0.46 on LEVIR-CC and 0.37 on WHU-CDC). Moreover, the final comparisons incorporate SCST optimization, whereas the component ablations are primarily conducted under CE training. Without more controlled comparisons, it remains unclear how much of the overall performance gain can be attributed specifically to the proposed joint generative modeling rather than the architectural design and additional caption optimization.

  2. Insufficient experimental validation and data transparency. The pretraining corpus includes datasets related to the downstream benchmarks, but the exact procedure for identifying and excluding overlapping imagery is not sufficiently detailed. Furthermore, SCST directly optimizes the same four metrics used to compute Savg. Therefore, improvements in automatic evaluation scores alone do not sufficiently establish better semantic correctness. These limitations weaken the reliability of the experimental claims.

Questions for authors
  1. Could the authors provide additional controlled comparisons between random initialization, shuffled-pair pretraining, and genuine-pair pretraining under the same complete caption training protocol, including SCST? In particular, do the observed improvements from genuine-pair pretraining remain consistent after SCST optimization?

  2. Since SCST directly optimizes the same four metrics used to calculate Savg, could the authors provide additional independent evaluations, such as human assessment or semantic correctness analysis, to verify whether the improvements in automatic metrics correspond to more accurate descriptions of changed objects, temporal direction, and spatial relationships?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.