acceptodds
Under review as a conference paper at ICLR 2027

PairTrans: Real-Pair-Grounded Semantic-Visual Transformation for Semantic Change Detection

Abstract

Semantic change detection (SCD) learns land-cover semantics and temporal transitions from densely annotated bi-temporal image-mask pairs. Such paired annotations are expensive to acquire. Existing generative augmentation methods construct additional change data from prompts, simulation rules, noise manipulation, or single-temporal states. Although effective, they do not explicitly construct new supervision by conditioning each synthesized temporal side on the opposite observation of the same annotated pair. We formulate SCD augmentation as bidirectional source-conditioned target-side generation from real annotated pairs and propose PairTrans. For target semantics, PairTrans derives a class-to-class transition matrix from the paired masks and perturbs it to propose new states with structured class transitions. For target appearance, it realizes each generated state from both its mask and the opposite-time real image. Applying the two directed generators symmetrically produces a fully synthetic pair whose semantic transitions are pair-guided and whose visual realizations are conditioned on observed scene context. Across three datasets and five SCD architectures, PairTrans achieves strong gains on SECOND and Landsat in low-data regimes, improves change-region metrics on HiUCD, and remains competitive as real supervision increases.

Primary area:
generative models
Keywords:
Dataset Generation, Semantic Change Detection, Diffusion Models

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_EjnhOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
3
Soundness
1
Read review
Paper summary

PairTrans addresses a SCD problem and shows improvements in the low-data settings on three datasets among five architectures. A SCD augmentation pipeline is clearly formulated as bidirectional source-conditioned target-side generation from real annotated pairs that enhances the method understanding.

Critical strengths
  1. PairTrans show practical significance for SCD problem. Since obtaining paired annotations for semantic change detection is costly, it is worth investigating how to generate training data from a limited amount of real-world paired data.

  2. The experiments encompass multiple downstream architectures, suggesting that the utility of the proposed method.

Critical weaknesses
  1. The novelty should be clearly discussed. The distinctive contribution appears to be the conditioning arrangement built around a real annotated pair. The two-stage semantic-to-visual pipeline and the underlying diffusion-conditioning mechanisms largely build on existing approaches (https://arxiv.org/abs/2503.15683).

  2. The formulation does not adequately explain how it handles the datasets’ annotation conventions. Sec. 3 assumes complete land-cover maps and defines changed area through semantic-label inequality in Eq. (6) and (16). However, SECOND explicitly distinguishes unchanged pixels from changes within the same land-cover category, such as building demolition and reconstruction. Its annotation convention therefore cannot be fully represented by label inequality alone.

  3. The connection between transition-matrix perturbation and the final synthetic supervision is insufficiently validated.

  4. The comparison with real-only training confounds augmentation with additional optimization. All downstream models are trained for 50 epochs, while augmentation increases the dataset size by a factor of six under the 1:5 real-to-synthetic ratio.

  5. PairTrans improves binary F1 and F_{scd} but reduces Score and SeK relative to real-only training across all five architectures. The authors appropriately acknowledge this trade-off, but do not analyze its cause.

Questions for authors

Since PairTrans only employs remote sensing datasets in the main experiment, how well does it generalize to street-view or SAR data?

2 comments
AuthorsOpenReview ↗

Response to Reviewer Ejnh

We thank the reviewer for the thorough review. In short:

  1. With the same number of updates, real-only training does not catch up with PairTrans.
  2. PairTrans exceeds HySCDG on every backbone under the same data and budget.
  3. We traced the Score/SeK decrease on HiUCD to the chance term of SeK; Kappa, which keeps true negatives, is highest for PairTrans on all five backbones.

All changes are marked in red in the revised PDF.

W4. Training-budget confound.

We agree that the fixed-epoch protocol gives augmented runs more updates. We therefore trained real-only for the same 5,550 updates that PairTrans receives in 50 epochs on the 1:5 set, about 300 epochs over the real pairs (Appendix C.1, Table 15):

ScoreSSCDLBiSRNetTEDA2NetSCanNetMean
Real only, 50 epochs28.6828.7128.7230.0929.5729.15
Real only, 5,550 updates28.3828.2728.4330.2329.4028.94
Ours30.7831.2830.9331.3232.0231.27

The additional updates do not improve real-only training on average, and PairTrans stays higher on every backbone, by 1.09–3.01 Score. HySCDG, with the same synthetic volume and budget, falls below real-only training on all five backbones (Table 16), so more samples or steps alone do not explain the gain. This control covers SECOND; the Evaluation Scope paragraph now notes that Landsat-SCD and HiUCD still use the fixed-epoch protocol.

W3. Transition-matrix validation.

We added this validation (Appendix C.4; see also our reply to HVXA).

  • After removing scene-level means, realized transitions move in the requested direction for 78.38% / 76.44% of 2,368 candidates, with 95% intervals above 73%.
  • The final pairs have the transition distribution closest to real data: TV 0.196, against 0.348 for ChangeDiff and 0.689 for Noise2Change.

As one seed sets both the request and the noise, this shows association, not a controlled intervention.

W1. Novelty and relation to HySCDG.

We agree that two-stage semantic-to-visual generation and diffusion conditioning are not our contribution. The contribution is reciprocal source conditioning: each endpoint of a real pair conditions the opposite side, with target semantics proposed from the pair's directed transition matrix. The two directions form a pair with two generated endpoints.

HySCDG instead edits only man-made object instances of a densely labeled single-temporal map, following its released code (Appendix A.3.3), and forms one-real-one-generated pairs. In our experiments:

  • PairTrans exceeds HySCDG by 2.04–4.49 Score on all five backbones (Table 16).
  • Pairs that keep one real endpoint, as in HySCDG, stay below real-only training (26.67–28.19 vs. 29.57), whereas PairTrans pairs reach 32.02 (Table 17).

HySCDG is added to Table 1 and the related work.

W2. Annotation conventions.

We agree that label inequality cannot represent SECOND's same-class changes, such as building reconstruction. Our setting follows the baseline ChangeDiff, whose released code defines change as pre_target != post_target in training and evaluation; we use this code unchanged for all methods and datasets, consistent with Eqs. (6) and (16). Same-class changes therefore count as unchanged for every method, so all comparisons share one convention. Sec. 4.1 and Appendix A.5 now state this, and Appendix A.1.1 describes the SECOND annotations accordingly.

W5. Score and SeK on HiUCD.

We agree that this trade-off needed an analysis; Appendix C.7 (Table 23) now provides it. Only 1.56% of the HiUCD test pixels change. SeK removes true negatives but keeps the chance term FP·FN/(N−q₀₀)² of the unchanged class. With fewer false alarms, this term rises (A2Net: 0.056 → 0.113), and SeK decreases although the predictions improve. A model that predicts (almost) no change gets SeK ≈ 0, as real-only SSCDL and BiSRNet do. Kappa, which keeps true negatives, is highest for PairTrans on all five backbones:

KappaSSCDLBiSRNetTEDA2NetSCanNet
Best other (incl. real only)16.0815.0023.8322.2823.35
Ours21.5118.7325.2428.9629.59

Among all compared methods, PairTrans also has the highest mIoU, FscdF_{scd}, IoU, and F1 on all five backbones (Table 14). Sec. 4.2 now states this.

Q1. Street-view or SAR data.

The transition and mask stages operate in label space and do not depend on the sensor; the pipeline already spans 0.1 m to 30 m optical imagery. Street-view and SAR also require adapting the image generator and class space and training on new data, which was not feasible within the rebuttal period. We therefore focused the new experiments on our existing benchmarks and will complete the street-view and SAR training and evaluation after the rebuttal.

We hope the added analyses address the reviewer's concerns.

Thank you again for your careful review. As the discussion period ends soon, we wanted to check whether our response and the revised manuscript address your concerns. If so, we would be grateful if you would consider updating your score. We are happy to clarify anything else before the deadline.

Reviewer_UNNXOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

The paper proposes a new dataset augmentation strategy to increase bitemporal remote sensing change detection dataset with new additional synthetic pairs. Existing methods either generate both images from masks only, or generate the post-change image from masks and pre-change image. Instead, PairTrans consists in a reciprocal pair generation, where the pre-change image and an alterated post-change mask is used to condition the generation of a post-change image, and conversely. The method is compared to competing baselines on 3 datasets and 5 architectures, achieving better performance in most cases. Important design choices and core technical contributions are correctly ablated, showing their actual contribution to overall performance. Qualitative visualisations of generated pairs are provided as well.

Critical strengths
  • The paper is sound, clear and easy to follow. Careful ablations confirm the positive impact of technical contributions on the performance. Extensive appendices report scores of all methods under different settings, and accross multiple metrics. Conclusion honestly distinguish settings where PairTrans outperforms competing baselines with those where its benefits are less clear, highlighting a possible sensor dependency.
  • The idea behind PairTrans is simple. Yet, its implementation and evaluation require a careful analysis which is performed in the paper. Explaining how to augment a CD dataset with reciprocal conditionned generation of both image, and showing that it actually gives satisfying performance is an valuable contribution to the field.
Critical weaknesses
  • Missing quantitative comparison of generated images: Synthetic data generation pipelines usually report FID or IS image quality metrics. The paper visually compares the images of several methods on the same datasource, but it is important to also compare them quantitatively.
  • Missing baselines and related work: Benidir et al. [1] also recently proposed a synthetic data generation pipeline (HySCDG) to augment change detection datasets. Their method should be added in the comparison. Also, few words could be added in the related works on methods beyond those based on generative diffusion, like Self-Pair (GAN) [2], STAR (Pair shuffling) [3]. These methods also seek to augment change detection datasets, and it is not obvious how STAR for example compare to PairTrans, while having a zero generation cost.
  • Missing limitation: While PairTrans starts from an existing annotated change detection dataset, Changen2, STAR or HySCDG only require single-temporal annotated datasets. And there are substantially more single-temporal semantic segmentation annotated datasets than change detection datasets, simply because they are less than half as costly to produce. This limitation of PairTrans should be acknowledged. Also, an equal-budget evaluation could compare all methods under the same data budget (acquisition, annotation and generation).

[1] Benidir et al., The Change You Want To Detect: Semantic Change Detection In Earth Observation With Hybrid Data Generation, CVPR 2025
[2] Seo et al., Self-Pair: Synthesizing Changes from Single Source for Object Change Detection in Remote Sensing Imagery, WACV 2023
[3] Zheng et al., Change is Everywhere: Single-Temporal Supervised Object Change Detection in Remote Sensing Imagery, ICCV 2021

Questions for authors

Clarification requests:

  • Add a colored legend for Figures 2, 3, and 4. It is difficult to visually assess the quality of generated images without knowing their expected semantic classes.
  • Acknowledge the dependency of PairTrans on bitemporal annotated data, which is not the case of HySCDG and Changen2

Suggestions for future work:

  • Add a quantitative comparison of generated images (FID, IS, ...)
  • Add a comparison to HySCDG
2 comments
AuthorsOpenReview ↗

Response to Reviewer UNNX

We thank the reviewer for the positive assessment and the concrete suggestions. In short:

  • We reproduced HySCDG and STAR-style re-pairing under the same data and budget; both stay below PairTrans on every backbone.
  • We agree that PairTrans depends on annotated bi-temporal pairs, and Appendix D now states this limitation.
  • FID/KID are now reported for all three datasets.

All changes are marked in red in the revised PDF.

W2. HySCDG, STAR, and Self-Pair.

We reproduced both methods on SECOND with the same 148 real pairs, 740 synthetic pairs, and 5,550 updates (Appendix A.3.3, C.2, Table 16):

  • HySCDG. Generation runs the authors' released code. The generator is trained on the subset, with 45k ControlNet iterations as in the paper.
  • STAR-style. Each scene is re-paired with five others, at zero generation cost.
Training dataMean Score (5 backbones)
Real only29.15
+ HySCDG27.69
+ HySCDG, labeled-only repaint†28.81
+ STAR-style18.42
+ STAR-style, masked change loss†28.54
+ Ours31.27

PairTrans is higher on every backbone, by at least 1.78 Score (Table 16). We see two likely reasons:

  • Sparse semantic labels. Both methods were designed for a dense semantic map per single-temporal image, whereas SECOND labels land cover only within changed regions (Appendix A.1.1), about 19% of the pixels (Table 19).
    • HySCDG repaints buffer rings and random rectangles while keeping their labels (Appendix A.3.3). In unlabeled regions the ControlNet has no class condition, so new objects can appear where the label states no change.
    • STAR-style re-pairing labels a pixel as unchanged whenever both sides are unlabeled, which covers most pixels of a cross-scene pair even though the two scenes differ.
  • Fixed generation rules. Restricting each method to labeled pixels (†) removes this noise but does not exceed real-only training on average (28.81 and 28.54 vs. 29.15, Table 16). Following the released code, HySCDG edits only man-made object instances (Appendix A.3.3), so the synthesized changes are few and narrow. We kept these rules so that the comparison stays with the original methods.

PairTrans instead learns change and appearance variety from generators trained on the real pairs, without hand-designed change rules. The related work now covers HySCDG, STAR, and Self-Pair, and HySCDG is added to Table 1. Changen2 lacks official training code (Sec. 4.1).

W3 and clarification. Dependency on bi-temporal annotation.

We agree, and we added an "Annotation Requirement" paragraph to Appendix D. PairTrans starts from an existing set of annotated bi-temporal pairs, whereas Changen2, STAR, and HySCDG can be trained on single-temporal segmentation data, which is more abundant and cheaper to produce. When a dataset is collected from scratch, these methods can draw on larger annotation pools.

PairTrans targets the setting where a small annotated SCD set already exists and more paired supervision is needed. In that setting, with the same 148 pairs and update budget, it improves over real-only training on every backbone (mean Score 31.27 vs. 28.94, Table 15).

A comparison under a common acquisition, annotation, and generation budget would require collecting new data, and we list it as future work.

Clarification. Color legends. Figures 2–4 now include a HiUCD class legend.

W1. FID / IS.

We added FID and KID for all datasets (Appendix C.6, Table 22). On SECOND, PairTrans has the highest FID and KID of the three methods; on Landsat-SCD and HiUCD, it has lower FID and KID than ChangeDiff.

We read FID as a measure of single-image realism rather than of value as paired supervision (Sec. 1), and in our results the two do not coincide:

  • Noise2Change has the lowest FID on SECOND and Landsat-SCD, yet trails PairTrans on every backbone.
  • On Landsat-SCD, Noise2Change also falls below real-only training on four backbones.

We do not report IS, which relies on ImageNet classes and is less suited to aerial images.

We hope these additions address the reviewer's suggestions.

Thank you again for your careful review. As the discussion period ends soon, we wanted to check whether our response and the revised manuscript address your concerns. If so, we would be grateful if you would consider updating your score. We are happy to clarify anything else before the deadline.

Reviewer_HVXAOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes PairTrans, a generative data augmentation framework for semantic change detection under limited labeled supervision. The framework constructs synthetic bi-temporal image-mask pairs using real annotated image pairs as conditioning references.PairTrans consists of two main components: (1) a semantic generation module that perturbs class-transition matrices and generates target masks through category-constrained diffusion, and (2) a visual generation module that synthesizes target images conditioned on the generated masks and opposite-time real observations. The framework applies these components in both temporal directions to produce complete synthetic training pairs.Experiments on SECOND, Landsat-SCD, and HiUCD evaluate the proposed augmentation strategy across five downstream SCD architectures.

Critical strengths
  1. The paper addresses the practical challenge of learning semantic change detection from limited densely annotated bi-temporal data. The integration of transition-matrix perturbation, category-constrained semantic generation, and source-conditioned image synthesis provides a well-motivated framework for synthetic data augmentation.
  2. The experiments cover three datasets and five downstream architectures, showing consistent improvements on SECOND and Landsat-SCD under limited supervision. The ablation studies and supplementary implementation details further support the empirical evaluation.
Critical weaknesses
  1. The framework uses perturbed semantic transition matrices to guide mask generation, but does not explicitly enforce the specified classwise transitions or target-class proportions. Without a quantitative evaluation of the discrepancy between intended and generated transitions, the effectiveness of this mechanism remains insufficiently validated.
  2. The candidate-pair selection criterion primarily relies on the global changed-area ratio, which does not explicitly ensure local spatial correspondence or semantic consistency between the generated temporal observations. The lack of a quantitative consistency evaluation raises concerns about the reliability of the synthetic supervision.
  3. The unidirectional and bidirectional ablations differ not only in generation direction but also in pair composition. Consequently, the reported improvements cannot be attributed solely to bidirectional generation, despite the additional reference-matching ablation.
Questions for authors
  1. Could the authors quantify the discrepancy between the perturbed transition matrices and the empirical classwise transitions realized by the generated masks?
  2. Beyond matching the global changed-area ratio, could the authors provide quantitative evidence of local spatial consistency between the two generated temporal observations and semantic alignment between the generated images and their corresponding masks?
  3. Could the authors provide a more controlled ablation that disentangles the contribution of bidirectional generation from the effect of using fully synthetic pairs, while keeping the training-pair composition and synthetic data volume comparable?
2 comments
AuthorsOpenReview ↗

Response to Reviewer HVXA

We thank the reviewer for the precise questions, which led to three new analyses. In short:

  1. Generated masks move in the requested direction in about three quarters of the cases, and the final pairs have the transition distribution closest to real data.
  2. The two generated endpoints show a changed/unchanged contrast close to that of real pairs.
  3. We agree that the original direction ablation also changed pair composition. With composition and volume fixed, the gain is associated with pairs whose two endpoints are both generated.

All changes are marked in red in the revised PDF.

W1/Q1. Requested vs. realized transitions.

We agree that this needed to be quantified. Since the transition matrix acts as a soft condition (Sec. 3.1; Appendix D), we measure how closely each output follows its own request rather than entry-wise equality (Appendix C.4, Table 18).

We use all 148 SECOND scenes, both directions, and eight seeds per scene. Requests and outputs are de-meaned within each scene to remove scene-level bias.

DirectionSame direction as request95% intervalCloser to own request
1→278.38%[75.08, 81.59]66.22%
2→176.44%[73.56, 79.56]64.86%

The response is already present before candidate selection. At the dataset level, the final PairTrans pairs have a foreground transition distribution with a TV distance of 0.196 to real data, against 0.348 for ChangeDiff and 0.689 for Noise2Change (Table 19); the 148-pair real subset itself is at 0.150. Since one seed sets both the request and the noise, we describe this as association rather than a controlled intervention.

W2/Q2. Local consistency and image–mask alignment.

We added two measurements (Appendix C.5):

  • Temporal consistency (Table 21). Within a pair, we divide the appearance difference inside changed regions by that inside unchanged regions. PairTrans gives 1.29, against 1.30 for the SECOND test split and 1.32 for real training pairs, so the two endpoints differ where their masks change, with a contrast similar to real data.
  • Image–mask alignment (Table 20). A segmenter trained only on real SECOND images reaches 40.66 mIoU and 25.28 boundary F1 against the generated masks on PairTrans images, against 34.85 and 14.48 on Noise2Change images. ChangeDiff scores higher on this proxy (44.34 / 30.29).
  • Downstream. At a matched update budget, PairTrans improves Score on every backbone (Table 15), so the remaining local deviations do not appear to prevent the gain.

We agree that matching the global change ratio does not by itself ensure local correspondence; explicit local consistency objectives remain a future direction stated in our conclusion.

W3/Q3. Bidirectional generation vs. fully synthetic pairs.

We agree that the original unidirectional and bidirectional variants also differ in pair composition. Appendix C.3 (Table 17) therefore fixes composition and volume: every augmented setting adds 740 pairs to the same 148 real pairs (SCanNet). The two single-direction rows are the Uni. rows of Table 5, and the mixed-direction row is a new run.

Added pairsScoremIoUSeK
None (real only)29.5767.0413.51
Real I1I_1 + generated I2I_226.6764.1410.61
Generated I1I_1 + real I2I_227.8765.4911.74
Both directions, 370 each28.1965.8612.04
Both endpoints generated (Ours)32.0268.8216.25

Covering both directions while keeping a real endpoint does not recover the gain and stays below real-only training, whereas pairs with two generated endpoints improve every metric. We therefore attribute the gain to the fully synthetic pair rather than to direction coverage alone; in PairTrans, such pairs are produced by the two directional generators. Table 10 shows the same ordering on all backbones, and Sec. 4.3 now points to this control.

We hope these results address the reviewer's concerns.

Thank you again for your careful review. As the discussion period ends soon, we wanted to check whether our response and the revised manuscript address your concerns. If so, we would be grateful if you would consider updating your score. We are happy to clarify anything else before the deadline.

Comments on the paper

1 comment
AuthorsOpenReview ↗

General Response to All Reviewers We thank all reviewers for their careful reading and constructive comments.

Summary. With the same real data and training budget, PairTrans stays ahead of:

  • real-only training;
  • two additional single-temporal baselines, HySCDG and STAR-style re-pairing;
  • pairs that keep one real endpoint.

The added analyses also show that the transition request is reflected in the generated masks. We uploaded a revised PDF with all changes in red; the new results are in Appendix C. Real-only and PairTrans rows there are the results of our original experiments, and each new run uses SECOND with 5% real data and 5,550 updates, the budget of the 1:5 runs in the main table.

Main new results

QuestionResultWhere
Is the gain only longer training?Real-only training with the same 5,550 updates reaches a mean Score of 28.94; PairTrans reaches 31.27 and is higher on every backbone.C.1, Table 15
HySCDG and STAR-stylePairTrans is higher on every backbone, by at least 1.78 Score (mean 31.27 vs. 27.69 HySCDG, 18.42 STAR-style).C.2, Table 16
Bidirectional generation vs. fully synthetic pairsWith composition and volume fixed, pairs that keep one real endpoint stay below real-only training (26.67–28.19 vs. 29.57 on SCanNet), also when both directions are mixed; PairTrans pairs reach 32.02.C.3, Table 17
Does the transition request reach the masks?Realized masks move in the requested direction for 78.38% / 76.44% of candidates, and the final pairs have the transition distribution closest to real data (TV 0.196 vs. 0.348 and 0.689).C.4, Tables 18–19
Pair consistencyThe changed/unchanged appearance contrast is close to real pairs (1.29 vs. 1.30), and image–mask alignment lies between Noise2Change and ChangeDiff (40.66 vs. 34.85 and 44.34 mIoU).C.5, Tables 20–21
Score/SeK on HiUCDScore and SeK are lower on HiUCD. We trace this to the chance term of SeK when only 1.56% of pixels change; Kappa, mIoU, FscdF_{scd}, IoU, and F1 are highest for PairTrans on all five backbones.Table 14; C.7, Table 23

We also report FID/KID (C.6, Table 22). We read FID as a measure of single-image realism; in our results it does not track downstream value: on SECOND and Landsat-SCD, Noise2Change has the lowest FID yet trails PairTrans on every backbone.

Text changes

  • HySCDG is added to Table 1, and HySCDG, STAR, and Self-Pair to the related work.
  • Figures 2–4 now include a HiUCD color legend.
  • The change definition, which follows ChangeDiff, is stated in Sec. 4.1 and Appendix A.5.
  • The description of the SECOND annotations is corrected in Appendix A.1.1.
  • Appendix D now states that PairTrans requires annotated bi-temporal pairs ("Annotation Requirement") and updates the Evaluation Scope.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.