acceptodds
Under review as a conference paper at ICLR 2027

PCal: A Prior-Posterior Collaborative Calibration Framework for Robust Remote Sensing Image-Text Retrieval

Abstract

Remote sensing image-text retrieval (RSITR) aims to match remote sensing images with their corresponding natural language descriptions. Existing methods primarily focus on learning a discriminative cross-modal space, while paying insufficient attention to the collaborative calibration of image-text correspondences at the levels of prior supervision and posterior scores. Such calibration can facilitate both the learning of this space and the subsequent refinement of retrieval scores. To address this limitation, we propose PCal, a prior-posterior collaborative calibration framework for robust RSITR. Specifically, we first propose a Prior Semantic Calibration Module (PSCM), which purifies supervision through prior semantic calibration to obtain highly reliable supervisory signals. These signals are then used to train our proposed Manifold Residual Adapter (MRA) and Posterior Score Calibration Flow (PSCF). MRA incorporates lightweight manifold residual adapters into the representation pathways to construct a stable discriminative score space. PSCF takes the resulting matching scores as its initial state and learns a continuous residual velocity field to progressively optimize retrieval scores. Extensive experiments on multiple RSITR datasets demonstrate that PCal consistently outperforms existing state-of-the-art methods.

Primary area:
foundation or frontier models, including LLMs
Keywords:
Remote Sensing Image-Text Retrieval

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_EjViOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes a prior-posterior collaborative calibration framework for remote sensing image-text retrieval (RSITR). The central motivation is that reliable image-text matching requires not only discriminative cross-modal representations but also reliable training correspondences and appropriately refined retrieval scores. The method is evaluated on RSICD, RSITMD, and NWPU for bidirectional retrieval. The reported results show consistent improvements over existing methods in the standard retrieval benchmarks and substantial robustness under severe synthetic correspondence noise. However, the current experiments do not sufficiently establish whether the proposed calibration mechanisms, particularly the manifold-specific adaptation and flow-based refinement, are necessary to obtain these improvements.

Critical strengths
  1. Strong and extensive empirical results across retrieval and robustness settings. The paper reports consistent improvements on three established RSITR benchmarks in both retrieval directions. The gains are particularly notable on RSICD and RSITMD. The robustness experiments further demonstrate substantial advantages under severe synthetic correspondence noise.
  2. A coherent framework connecting supervision reliability and retrieval-score refinement. The paper explicitly considers reliability at different stages of the retrieval pipeline rather than treating noisy supervision and inaccurate retrieval ranking as unrelated problems.
Critical weaknesses
  1. The reported improvements do not isolate the contribution of the proposed calibration framework under a matched experimental setup.
  2. The robustness claims depend heavily on synthetic corruption and clean validation supervision. Randomly shuffled captions may exhibit more obvious semantic inconsistencies than naturally occurring annotation errors. Real correspondence noise can involve partially incorrect descriptions, missing attributes, subtle spatial-relation errors, or mismatches between semantically similar scenes. It is unclear whether PSCM can reliably detect these harder cases.
  3. The manuscript also does not directly measure the precision of the filtered correspondences or quantify how many valid pairs are incorrectly discarded.
Questions for authors
  1. Are the improvements attributable to the proposed MRA and PSCF formulations under matched baselines?
  2. How reliable is PSCM's correspondence filtering, and how dependent is it on clean validation supervision?
  3. What is the specific contribution of flow-based calibration to the final retrieval ranking?
  4. How stable are the reported improvements across random seeds?
6 comments
AuthorsOpenReview ↗

Response to Reviewer EjVi

Thank you for asking us to isolate the module contributions under matched conditions, quantify both noise removal and the loss of valid pairs, and assess stability across random seeds. We took each concern seriously and devoted considerable effort to running additional experiments and analyses, which we present below. We will incorporate these findings and any clarifications from the discussion into the revised manuscript. We welcome further discussion. If the additional evidence addresses your concerns, we would be grateful if you would consider raising your rating.

1. Response to Weakness 1 / Question 1

Summary

(1) Parameter-matched comparisons for MRA (Table R1). To isolate the contribution of the MRA design, we compared it with standard adapters. MRA achieves higher retrieval accuracy than Houlsby Adapter and LoRA at comparable parameter budgets.

(2) Reranking comparisons for PSCF (Table R2). To assess the contribution of flow-based score calibration, we compared PSCF with simpler rerankers. With the same candidates and parameter budget, PSCF outperforms both alternatives and improves retrieval over the original ITM scores.

Detailed Response

(1) Parameter-matched comparisons for MRA

We compared MRA, Houlsby Adapter, and LoRA at comparable parameter budgets. In Table R1, MRA achieves an mR of 40.63, compared with 39.86 for Houlsby Adapter and 39.16 for LoRA.

Table R1. Comparison of MRA with parameter-matched LoRA and Houlsby Adapter

MethodParameters (M)I2T R@1I2T R@5I2T R@10T2I R@1T2I R@5T2I R@10mRRSum
MRA15.02024.8944.1057.0017.1142.7657.9040.63243.76
Houlsby Adapter15.01223.0643.7356.2716.9441.6157.5539.86239.16
LoRA15.10422.3243.0956.3617.0740.0456.1039.16234.98

(2) Reranking comparisons for PSCF

We fix MRA, the input features, and the top-128 candidates. Each of the three learned scorers adds 0.035716M parameters. In Table R2, PSCF improves mR from 40.73 with uncalibrated ITM to 41.75. The conventional MLP reranker and the direct residual predictor achieve 38.65 and 37.94, respectively, both below the uncalibrated ITM baseline.

Table R2. PSCF versus simpler rerankers with fixed candidates and matched parameter budgets

Scoring methodAdditional parameters (M)I2T R@1I2T R@5I2T R@10T2I R@1T2I R@5T2I R@10mRRSum
ITM (uncalibrated)025.1644.6557.2716.9442.6557.6840.73244.35
Conventional MLP reranker0.03571623.3343.0956.9115.0039.4154.1338.65231.87
Direct residual predictor0.03571622.8741.0855.9015.1338.9653.6737.94227.61
PSCF0.03571625.7145.1157.8218.4644.1759.1941.75250.46

2. Response to Weakness 2

Summary

(1) Cross-dataset noise filtering (Table R3). To test whether PSCM generalizes across datasets, we added a cross-dataset evaluation. Under the same random text-shuffling protocol, PSCM transfers from RSICD to NWPU and RSITMD with filtering precision and recall close to those obtained with within-dataset fitting.

(2) Within-scene noise filtering and downstream retrieval (Tables R4–R5). To test whether PSCM mitigates the effect of fine-grained mismatches on retrieval, we added within-scene caption swap experiments. The results show that PSCM improves downstream retrieval by conservatively removing a subset of hard-to-detect mismatches. They also suggest that some within-scene mismatches can retain useful scene semantics.

(3) Filtering naturally occurring caption-quality issues (Table R6). To evaluate PSCM on naturally occurring caption-quality issues, we used human ratings from RS5M. PSCM identifies a subset of these issues and lowers their prevalence among the retained pairs.

Detailed Response

(1) Cross-dataset noise filtering

We fit PSCM on RSICD and apply it to NWPU and RSITMD to detect randomly mismatched pairs. In Table R3, cross-dataset filtering precision and recall are close to the corresponding within-dataset results, supporting transfer under the same random text-shuffling protocol.

Table R3. Cross-dataset noise filtering with PSCM

Fitting dataset → Filtering datasetNoise settingFiltering precisionNoise recallPost-filtering noise ratio
RSICD → RSICDNoise Ratio=90%93.90%92.31%60.03%
RSICD → NWPUNoise Ratio=90%94.39%94.28%50.91%
NWPU → NWPUNoise Ratio=90%93.99%93.84%54.66%
RSICD → RSITMDNoise Ratio=90%93.92%93.82%55.07%
RSITMD → RSITMDNoise Ratio=90%93.52%93.35%58.85%

(2) Within-scene noise filtering and downstream retrieval

At Noise Ratio=90% on RSICD, we swap captions within scene categories to simulate mismatches that preserve scene semantics while potentially introducing incorrect details. We then compare retrieval with and without filtering. In Tables R4–R5, PSCM removes a subset of mismatches with 91.91% precision and retains 79.94% of clean pairs, improving downstream mR from 14.90 to 19.06.

Table R4. Filtering random and within-scene mismatches at Noise Ratio=90%

SettingNoise settingFiltering precisionNoise recallPost-filtering noise ratio
Random mismatchesNoise Ratio=90%93.90%92.31%60.03%
Within-scene mismatchesNoise Ratio=90%91.91%25.33%89.37%

Table R5. Retrieval with and without PSCM filtering under within-scene mismatches at Noise Ratio=90%

SettingI2T R@1I2T R@5I2T R@10T2I R@1T2I R@5T2I R@10mRRSum
With PSCM denoising6.4017.8430.105.7820.0934.1619.06114.38
Without denoising4.0312.8122.694.1015.8529.9214.9089.39

(3) Filtering naturally occurring caption-quality issues

Using the official RS5M human ratings, we flag a pair when any rating is ≤ 3 and supplement the PSCM input with four Qwen-generated captions. In Table R6, PSCM achieves 63.64% precision and 28.46% recall, reducing the proportion of flagged pairs from 14.19% before filtering to 10.84% among retained pairs.

Table R6. Caption-quality filtering against official RS5M human ratings

Labeling criterionFlagged pairs before filteringFiltering precisionNoise recallFlagged pairs after filtering
Any rating ≤ 3 counts as noise14.19%63.64%28.46%10.84%

3. Response to Weakness 3 / Question 2

Summary

(1) Filtering across noise ratios (Table R7). To assess filtering across noise ratios, we report filtering precision, noise recall, and the post-filtering noise ratio. PSCM lowers the noise ratio among retained pairs across the reported settings from Noise Ratio=10% to 90%.

(2) Size of the clean validation set (Table R8). To assess PSCM's reliance on clean validation data, we varied the size of the clean validation set. PSCM remains effective when fitted on a small subset.

Detailed Response

(1) Filtering across noise ratios

We fit PSCM on the clean RSICD validation set and evaluate at Noise Ratio=10%–90%, using the noise labels of the target training set solely for evaluation. In Table R7, PSCM lowers the noise ratio among retained pairs in every setting.

Table R7. Filtering performance on RSICD at Noise Ratio=10%–90%

Noise settingFiltering precisionNoise recallPost-filtering noise ratio
Noise Ratio=10%74.59%71.41%3.16%
Noise Ratio=20%78.77%76.75%5.78%
Noise Ratio=30%82.59%80.81%8.15%
Noise Ratio=40%84.17%83.68%10.84%
Noise Ratio=50%85.78%84.65%15.15%
Noise Ratio=60%86.82%86.27%20.40%
Noise Ratio=70%88.84%87.89%27.58%
Noise Ratio=80%90.50%90.71%37.50%
Noise Ratio=90%93.90%92.31%60.03%

(2) Size of the clean validation set

We refit PSCM using 100%, 50%, 25%, 10%, 5%, and 1% of the clean RSICD validation set. At Noise Ratio=90%, the precision and recall values in Table R8 yield F1 scores of approximately 93.10%, 93.23%, and 91.52% for the full set, the 5% subset, and the 1% subset, respectively. Relative to the full set, F1 increases by about 0.13 percentage points with 5% of the data and decreases by about 1.57 percentage points with 1%. PSCM thus maintains a high filtering F1 with substantially less clean validation data in this setting.

Table R8. Sensitivity to the size of the clean RSICD validation set at Noise Ratio=90%

Validation set retainedFiltering precisionNoise recallPost-filtering noise ratio
100%93.90%92.31%60.03%
50%94.08%91.59%61.12%
25%94.08%91.77%60.67%
10%93.72%93.06%58.72%
5%93.97%92.50%59.18%
1%95.18%88.14%64.07%

4. Response to Question 3

To quantify the contribution of PSCF to final retrieval rankings, we analyze changes in the ranks of ground-truth matches for both retrieval directions on RSICD. PSCF applies posterior calibration to the discriminative ITM scores SITMS^{ITM}, moving ground-truth matches up the ranking. In Table R9, ground-truth matches move up or retain their rank for 93.05% of 1,093 I2T queries and 95.99% of 5,465 T2I queries. Among the queries whose matches move up, the average gains are 3.16 and 2.53 ranks, respectively. This analysis provides direct ranking-level evidence for the improvements in Recall@1, Recall@5, and Recall@10 with the same candidate sets.

Table R9. Effect of PSCF on bidirectional retrieval ranks on RSICD

Retrieval directionTotal queriesRank improved or unchanged (%)Mean rank gain among improved queries
I2T1,09393.053.16
T2I5,46595.992.53
Reviewer_dzTgOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes P2Cal, a prior-posterior collaborative calibration framework for robust remote sensing image-text retrieval. The method consists of three components: PSCM, which uses VLM-generated descriptions and synthetic correspondence corruption to identify unreliable training pairs; MRA, which performs lightweight sample-conditioned orthogonal adaptation to improve the discriminative embedding space; and PSCF, which learns a continuous residual velocity field to refine retrieval scores after initial ITM matching. Experiments on RSICD, RSITMD, and NWPU show substantial improvements over prior methods, together with evaluations under synthetic correspondence noise, cross-dataset transfer, visual degradation, and expanded retrieval galleries.

Critical strengths
  1. The prior/posterior connection is conceptually coherent. A major strength is that the paper does not treat supervision purification and score refinement as two independent tricks. PSCM produces the purified supervision used by both MRA and PSCF, creating a relatively coherent pipeline from correspondence reliability → representation learning → score calibration. This is a meaningful conceptual organization of the problem.

  2. The empirical evaluation is unusually broad. The paper evaluates three datasets, both I2T and T2I retrieval, clean and highly corrupted correspondence settings up to 90% noise, cross-dataset transfer, scene-level behavior, visual degradations, and enlarged retrieval galleries. P2Cal also obtains very large gains over prior methods on the standard benchmarks.

  3. MRA has a technically structured design rather than being a purely heuristic adapter. The sample-conditioned skew-symmetric transformation and its orthogonality property provide a principled constraint in the bottleneck space. The paper explicitly shows that the transformation preserves within-sample pairwise inner products before the nonlinear/up-projection stage. This is a technically interesting aspect of the method.

Critical weaknesses
  1. The novelty is not sufficiently isolated from the combination of existing techniques. P2Cal combines several established ingredients, including VLM-based semantic filtering, parameter-efficient adaptation, hard-negative score correction, and flow matching. However, the ablations only remove entire modules and do not establish that the specific designs of MRA and PSCF are necessary. In particular, matched comparisons with standard adapters or simpler reranking/residual alternatives are missing. Thus, the current results show that these components help, but not that the proposed formulations themselves provide the gains.

  2. The validity of PSCM's noise calibration is not fully established for real annotation noise. PSCM is trained and calibrated using validation pairs corrupted by random text shuffling, and the calibration mapping is evaluated on the same synthetic corruption scheme. However, the paper does not report validation against a manually verified set of real annotation errors. Consequently, the experiments establish robustness to the proposed synthetic corruption protocol, but provide limited evidence that PSCM can accurately identify real-world noisy correspondences.

  3. The additional Qwen2.5-VL-7B semantic teacher creates a substantial attribution and fairness issue. PSCM relies on Qwen2.5-VL-7B-generated descriptions, making it unclear how much of the reported gain comes from the proposed calibration mechanism versus the additional semantic supervision. A no-VLM or alternative-teacher ablation would clarify this contribution.

  4. The top-128 ITC candidate bottleneck limits what PSCF can actually correct. PSCF is applied only to the top-128 candidates selected by ITC. Therefore, it cannot recover a ground-truth match that is excluded during candidate selection. The current evaluation does not clearly separate improvements in candidate recall from improvements in candidate reranking. This is particularly relevant because the appendix itself notes that some failures are difficult to attribute to candidate exclusion versus later ranking errors.

Questions for authors

1.Can you compare PSCM against substantially simpler noise filtering methods, such as thresholding the semantic-consistency score, loss-based filtering, or bootstrapping, under the same filtering budget? More importantly, can you evaluate PSCM on a genuinely held-out or manually verified set of real annotation errors rather than only synthetic text shuffling?

2.How much of the final improvement comes specifically from Qwen2.5-VL-7B? Could you provide ablations using no generated descriptions, a weaker VLM, or alternative semantic signals while keeping the rest of PSCM unchanged?

3.Could you compare MRA against standard LoRA/Houlsby adapters under a matched parameter budget, and compare PSCF against a simple direct residual predictor or conventional reranker using the same input features? This would clarify whether the proposed parameterizations themselves are necessary.

4.what percentage of final retrieval failures are caused by the ground-truth item being excluded before PSCF? Reporting the improvement from MRA, ITC candidate selection, and PSCF separately would make the causal contribution of each stage much clearer.

5.For the expanded-gallery experiments, could all methods be evaluated using a single unified preprocessing and numerical pipeline? It would also be useful to report training/inference cost, parameter counts, GPU hours, and latency relative to the strongest baselines.

7 comments
AuthorsOpenReview ↗

Response to Reviewer dzTg

Thank you for identifying gaps in our matched comparisons, the evaluation of PSCM beyond random text shuffling, the attribution of gains to the semantic teacher, and the analysis of the top-128 candidate bottleneck and computational cost. We took each concern seriously and devoted considerable effort to running additional experiments and analyses, which we present below. We will incorporate these findings and any clarifications from the discussion into the revised manuscript. We welcome further discussion. If the additional evidence addresses your concerns, we would be grateful if you would consider raising your rating.

1. Response to Weakness 1 / Question 3

Summary

(1) Parameter-matched comparisons for MRA (Table R1). To isolate the contribution of the MRA design, we compared it with standard adapters. MRA achieves higher retrieval accuracy than Houlsby Adapter and LoRA at comparable parameter budgets.

(2) Reranking comparisons for PSCF (Table R2). To assess the contribution of flow-based score calibration, we compared PSCF with simpler rerankers. With the same candidates and parameter budget, PSCF outperforms both alternatives and improves retrieval over the original ITM scores.

Detailed Response

(1) Parameter-matched comparisons for MRA

We compared MRA, Houlsby Adapter, and LoRA at comparable parameter budgets. In Table R1, MRA achieves an mR of 40.63, compared with 39.86 for Houlsby Adapter and 39.16 for LoRA.

Table R1. Comparison of MRA with parameter-matched LoRA and Houlsby Adapter

MethodParameters (M)I2T R@1I2T R@5I2T R@10T2I R@1T2I R@5T2I R@10mRRSum
MRA15.02024.8944.1057.0017.1142.7657.9040.63243.76
Houlsby Adapter15.01223.0643.7356.2716.9441.6157.5539.86239.16
LoRA15.10422.3243.0956.3617.0740.0456.1039.16234.98

(2) Reranking comparisons for PSCF

We fixed MRA, the input features, and the top-128 candidates. Each of the three learned scorers adds 0.035716M parameters, and we fuse scores with weights of 0.1 for ITM and 0.9 for the calibrated score. In Table R2, PSCF improves mR from 40.73 with uncalibrated ITM to 41.75. The conventional MLP reranker and the direct residual predictor achieve 38.65 and 37.94, respectively.

Table R2. PSCF versus simpler rerankers with fixed candidates and matched parameter budgets

Scoring methodAdditional parameters (M)I2T R@1I2T R@5I2T R@10T2I R@1T2I R@5T2I R@10mRRSum
ITM (uncalibrated)025.1644.6557.2716.9442.6557.6840.73244.35
Conventional MLP reranker0.03571623.3343.0956.9115.0039.4154.1338.65231.87
Direct residual predictor0.03571622.8741.0855.9015.1338.9653.6737.94227.61
PSCF0.03571625.7145.1157.8218.4644.1759.1941.75250.46

2. Response to Weakness 2 / Question 1

Summary

(1) Cross-dataset noise filtering (Table R3). To test whether PSCM generalizes across datasets, we added a cross-dataset evaluation. Under the same random text-shuffling protocol, PSCM transfers from RSICD to NWPU and RSITMD with filtering precision and recall close to those obtained with within-dataset fitting.

(2) Within-scene noise filtering and downstream retrieval (Tables R4–R5). To test whether PSCM mitigates the effect of fine-grained mismatches on retrieval, we added within-scene caption swap experiments. The results show that PSCM improves downstream retrieval by conservatively removing a subset of hard-to-detect mismatches. They also suggest that some within-scene mismatches can retain useful scene semantics.

(3) Filtering naturally occurring caption-quality issues (Table R6). To evaluate PSCM on naturally occurring caption-quality issues, we used human ratings from RS5M. PSCM identifies a subset of these issues and lowers their prevalence among the retained pairs.

(4) Alternative noise-handling strategies (Tables R7–R8). To assess PSCM against other noise-handling strategies, we added comparative experiments. The results show that PSCM’s adaptive noise-ratio estimation helps preserve retrieval accuracy on clean data while maintaining robustness under severe correspondence noise.

Detailed Response

(1) Cross-dataset noise filtering

We fit PSCM on RSICD and apply it to NWPU and RSITMD to detect randomly mismatched pairs. In Table R3, cross-dataset filtering precision and recall are close to the corresponding within-dataset results, supporting transfer under the same random text-shuffling protocol.

Table R3. Cross-dataset noise filtering with PSCM

Fitting dataset → Filtering datasetNoise settingFiltering precisionNoise recallPost-filtering noise ratio
RSICD → RSICDNoise Ratio=90%93.90%92.31%60.03%
RSICD → NWPUNoise Ratio=90%94.39%94.28%50.91%
NWPU → NWPUNoise Ratio=90%93.99%93.84%54.66%
RSICD → RSITMDNoise Ratio=90%93.92%93.82%55.07%
RSITMD → RSITMDNoise Ratio=90%93.52%93.35%58.85%

(2) Within-scene noise filtering and downstream retrieval

At Noise Ratio=90% on RSICD, we swap captions within scene categories to simulate mismatches that preserve scene semantics while potentially introducing incorrect details. We then compare retrieval with and without filtering. In Tables R4–R5, PSCM removes a subset of mismatches with 91.91% precision and retains 79.94% of clean pairs, improving downstream mR from 14.90 to 19.06.

Table R4. Filtering random and within-scene mismatches at Noise Ratio=90%

SettingNoise settingFiltering precisionNoise recallPost-filtering noise ratio
Random mismatchesNoise Ratio=90%93.90%92.31%60.03%
Within-scene mismatchesNoise Ratio=90%91.91%25.33%89.37%

Table R5. Retrieval with and without PSCM filtering under within-scene mismatches at Noise Ratio=90%

SettingI2T R@1I2T R@5I2T R@10T2I R@1T2I R@5T2I R@10mRRSum
With PSCM denoising6.4017.8430.105.7820.0934.1619.06114.38
Without denoising4.0312.8122.694.1015.8529.9214.9089.39

(3) Filtering naturally occurring caption-quality issues

Using the official RS5M human ratings, we flag a pair when any rating is ≤ 3 and supplement the PSCM input with four Qwen-generated captions. In Table R6, PSCM achieves 63.64% precision and 28.46% recall, reducing the proportion of flagged pairs from 14.19% before filtering to 10.84% among retained pairs.

Table R6. Caption-quality filtering against official RS5M human ratings

Labeling criterionFlagged pairs before filteringFiltering precisionNoise recallFlagged pairs after filtering
Any rating ≤ 3 counts as noise14.19%63.64%28.46%10.84%

(4) Alternative noise-handling strategies

At Noise Ratio=0% and Noise Ratio=90%, we compare the full P2CalP^2Cal, filtering out a fixed 50% of the training pairs based on PSCM scores, and adapting the RRSITR loss while retaining all pairs. Tables R7–R8 show that the full P2CalP^2Cal outperforms the RRSITR loss adaptation in both settings. Fixed-budget filtering based on PSCM scores achieves the highest mR at Noise Ratio=90%, but its filtering budget cannot adapt to the actual noise level. At Noise Ratio=0%, it discards many valid image-text pairs and reduces retrieval accuracy.

Table R7. Retrieval performance of P2CalP^2Cal and alternative noise-handling strategies at Noise Ratio=0%

MethodI2T R@1I2T R@5I2T R@10T2I R@1T2I R@5T2I R@10mRRSum
P2CalP^2Cal25.7145.1157.8218.4644.1759.1941.75250.46
PSCM scores with fixed 50% filtering19.7639.2552.2416.3239.3054.8936.96221.77
RRSITR loss adaptation (all pairs retained)21.5043.6455.9917.4043.0057.4039.82238.93

Table R8. Retrieval performance of P2CalP^2Cal and alternative noise-handling strategies at Noise Ratio=90%

MethodI2T R@1I2T R@5I2T R@10T2I R@1T2I R@5T2I R@10mRRSum
P2CalP^2Cal14.3632.2042.0912.2032.9047.9430.28181.69
PSCM scores with fixed 50% filtering16.9332.5745.4712.5035.4448.1831.85191.09
RRSITR loss adaptation (all pairs retained)11.8928.2741.1710.3927.9242.6027.04162.25

3. Response to Weakness 3 / Question 2

Summary

(1) Contribution of the generated-caption prior to downstream retrieval (Table R9). To quantify the contribution of generated captions, we ablated the generated-caption prior. The results show that the Qwen-generated prior complements within-image caption consistency and improves downstream retrieval.

(2) Filtering random mismatches (Table R10). To assess how much filtering depends on the Qwen2.5-VL-7B prior, we compared alternative semantic priors. Under random mismatches, the lighter BLIP-base achieves results close to Qwen2.5-VL-7B. Scene-category labels perform best among the variants tested and require category annotations.

(3) Filtering within-scene mismatches (Table R11). To assess the role of the semantic prior in detecting fine-grained mismatches, we compared the variants under within-scene caption swaps. The variants yield similar filtering results on these harder mismatches.

(4) Filtering naturally occurring caption-quality issues in RS5M (Table R12). To assess the priors on naturally occurring caption-quality issues, we compared them on RS5M. Qwen2.5-VL-7B outperforms BLIP-base and the variant without an independent prior.

Detailed Response

(1) Contribution of the generated-caption prior to downstream retrieval

At Noise Ratio=90%, we remove the generated captions and retain only within-image caption consistency, keeping all other PSCM settings fixed. In Table R9, removing the generated-caption prior lowers mR from 30.28 to 28.75, corresponding to a 1.53-percentage-point gain from generated captions in this setting.

Table R9. Effect of removing the generated-caption prior on downstream retrieval at Noise Ratio=90%

MethodI2T R@1I2T R@5I2T R@10T2I R@1T2I R@5T2I R@10mRRSum
P2CalP^2Cal14.3632.2042.0912.2032.9047.9430.28181.69
PSCM without the generated-caption prior (within-image caption consistency only)14.0029.7339.6210.9832.0646.0928.75172.48

(2) Filtering random mismatches

At Noise Ratio=90%, we compare Qwen2.5-VL-7B, BLIP-base, no generated-caption prior, and scene-category labels, keeping the rest of PSCM fixed. In Table R10, BLIP-base performs close to Qwen. Scene-category labels perform best under random mismatches when such annotations are available.

Table R10. Filtering performance of PSCM with different semantic priors at Noise Ratio=90%

Fitting dataset → Filtering datasetSemantic priorFiltering precisionNoise recallPost-filtering noise ratio
RSICD → RSICDQwen2.5-VL-7B93.90%92.31%60.03%
RSICD → RSICDNo generated-caption prior (within-image caption consistency only)92.12%90.80%73.33%
RSICD → RSICDBLIP-base93.31%92.01%63.90%
RSICD → RSICDScene-category labels96.29%95.06%39.87%
NWPU → NWPUQwen2.5-VL-7B93.99%93.84%54.66%
NWPU → NWPUNo generated-caption prior (within-image caption consistency only)91.58%91.43%76.04%
NWPU → NWPUBLIP-base92.75%92.61%65.63%
NWPU → NWPUScene-category labels97.79%97.64%20.94%
RSITMD → RSITMDQwen2.5-VL-7B93.52%93.35%58.85%
RSITMD → RSITMDNo generated-caption prior (within-image caption consistency only)92.19%92.02%70.66%
RSITMD → RSITMDBLIP-base93.04%92.87%63.15%
RSITMD → RSITMDScene-category labels95.01%94.83%45.73%
Reviewer_PLHhOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper tackles prior noise in training annotations and posterior miscalibration of retrieval scores in RSITR, and proposes P2Cal with three modules: PSCM (Qwen2.5-VL-based auxiliary descriptions with a logistic noise estimator), MRA (lightweight rank-2 orthogonal adapters in a frozen BLIP backbone), and PSCF (flow-matching score calibration). Extensive experiments on RSICD, RSITMD, and NWPU show consistent gains over 10 baselines, with robustness validated under synthetic noise (10%–90%), cross-dataset transfer, visual degradation, and enlarged galleries.

Critical strengths
  1. Large and consistent gains on main results (Table 1): The method ranks first on all six Recall metrics across the three datasets, e.g., RSICD I2T R@1 reaches 25.71% (second-best McRSA: 16.74%, a +8.97-point margin) and RSITMD I2T R@1 reaches 30.09% (second-best: 26.77%). Moreover, the clear advantage over PE (13.08%), which also adopts a BLIP backbone, suggests the gains do not stem merely from backbone upgrades.

  2. Experimental breadth far exceeds the common standard of this field: Beyond the main table, the evaluation includes full metrics on nine levels of synthetic noise (Tables 8–10: 3 datasets × 9 ratios × 11 methods), cross-dataset zero-shot transfer (Table 4), visual degradation robustness (Tables 17–19), an RS5M extended gallery (Table 20), scene-level fine-grained analysis (Tables 11–16), and qualitative retrieval cases (Figures 4–15), providing sufficient evidence for reproducibility.

Critical weaknesses
  1. Circularity between noise evaluation and the noise model trained for PSCM. PSCM's estimator and calibration mapping are both fitted on "text-shuffling" synthetic noise (A.1, A.4.1), and all robustness experiments (Fig. 3, Tables 8–10) adopt the same shuffling protocol (following RRSITR). P2Cal's mR barely drops under 10%–50% noise (RSICD: 41.78→40.97); such a near-flat curve is precisely the signature of a filter matched to the evaluation noise distribution. The paper never validates PSCM under out-of-distribution noise (e.g., in-scene label swaps—semantically plausible yet genuinely erroneous noise in real RSITR annotations), and its detection precision/recall on real noise is entirely unmeasured. This is the most significant scientific gap behind the "robust" claim: existing experiments offer no basis for inferring that this advantage transfers to real annotation noise.

  2. Mismatch between contribution attribution and the framework narrative. The clean-data ablation (Table 2) shows that MRA-only already attains mR 40.63, adding PSCM yields only +0.10 (40.73), and the full model reaches 41.75; i.e., the "prior–posterior collaborative calibration" highlighted in the title contributes roughly +1.1 mR in total on clean data, with the dominant gain stemming from backbone adaptation. While the paper is titled and structured around "collaborative calibration," the evidence for this collaboration derives mainly from synthetic-noise experiments sharing the same distribution as the estimator (see Point 1), so the narrative strength is disproportionate to the empirical contribution.

Questions for authors
  1. Inference overhead is entirely unanalyzed. ITM requires a cross-modal encoding forward pass over the top-128 candidates per query, and PSCF further stacks L_f = 3 velocity-field evaluations—a well-known high-cost inference pattern. The paper cites CFITR, which advertises efficiency as its selling point, yet never compares against it, nor reports any latency/throughput/memory figures. For a retrieval task, a SOTA table without efficiency analysis is incomplete, and reviewers cannot weigh the accuracy–cost trade-off.

  2. A core line of related work is missing. Noisy correspondence learning has been systematically studied—e.g., Bi-CLS (CVPR'22), NCR (CVPR'23, which partitions clean/noisy regions via a predictor with weighted contrastive learning and curriculum training), deep evidential learning (MM'22), and UGNCL (SIGIR'24). PSCM's "predict noise probability, then filter or down-weight accordingly" paradigm is closely aligned with this NCR line of work.

5 comments
AuthorsOpenReview ↗

Response to Reviewer PLHh

Thank you for your careful review and for asking us to test PSCM beyond random text shuffling, clarify the contributions of the calibration modules, analyze the accuracy–cost trade-off, and discuss related work on noisy correspondence learning. We took each concern seriously and devoted considerable effort to running additional experiments and analyses, which we present below. We will incorporate these findings and any clarifications from the discussion into the revised manuscript. We welcome further discussion. If the additional evidence addresses your concerns, we would be grateful if you would consider raising your rating.

1. Response to Weakness 1

Summary

(1) Cross-dataset noise filtering (Table R1). To test whether PSCM generalizes across datasets, we added a cross-dataset evaluation. Under the same random text-shuffling protocol, PSCM transfers from RSICD to NWPU and RSITMD with filtering precision and recall close to those obtained with within-dataset fitting.

(2) Within-scene noise filtering and downstream retrieval (Tables R2–R3). To test whether PSCM mitigates the effect of fine-grained mismatches on retrieval, we added within-scene caption swap experiments. The results show that PSCM improves downstream retrieval by conservatively removing a subset of hard-to-detect mismatches. They also suggest that some within-scene mismatches can retain useful scene semantics.

(3) Filtering naturally occurring caption-quality issues (Table R4). To evaluate PSCM on naturally occurring caption-quality issues, we used human ratings from RS5M. PSCM identifies a subset of these issues and lowers their prevalence among the retained pairs.

Detailed Response

(1) Cross-dataset noise filtering

We fit PSCM on RSICD and apply it to NWPU and RSITMD to detect randomly mismatched pairs. In Table R1, cross-dataset filtering precision and recall are close to the corresponding within-dataset results, supporting transfer under the same random text-shuffling protocol.

Table R1. Cross-dataset noise filtering with PSCM

Fitting dataset → Filtering datasetNoise settingFiltering precisionNoise recallPost-filtering noise ratio
RSICD → RSICDNoise Ratio=90%93.90%92.31%60.03%
RSICD → NWPUNoise Ratio=90%94.39%94.28%50.91%
NWPU → NWPUNoise Ratio=90%93.99%93.84%54.66%
RSICD → RSITMDNoise Ratio=90%93.92%93.82%55.07%
RSITMD → RSITMDNoise Ratio=90%93.52%93.35%58.85%

(2) Within-scene noise filtering and downstream retrieval

On RSICD at Noise Ratio=90%, we swap captions within scene categories and compare downstream retrieval with and without filtering. In Tables R2–R3, PSCM achieves 91.91% filtering precision, and removing a subset of fine-grained mismatches improves downstream mR by 4.16 percentage points.

Table R2. Filtering random and within-scene mismatches at Noise Ratio=90%

SettingNoise settingFiltering precisionNoise recallPost-filtering noise ratio
Random mismatchesNoise Ratio=90%93.90%92.31%60.03%
Within-scene mismatchesNoise Ratio=90%91.91%25.33%89.37%

Table R3. Retrieval with and without PSCM filtering under within-scene mismatches at Noise Ratio=90%

SettingI2T R@1I2T R@5I2T R@10T2I R@1T2I R@5T2I R@10mRRSum
With PSCM denoising6.4017.8430.105.7820.0934.1619.06114.38
Without denoising4.0312.8122.694.1015.8529.9214.9089.39

(3) Filtering naturally occurring caption-quality issues

Using the official RS5M human ratings, we flag a pair when any rating is ≤ 3 and supplement the PSCM input with four Qwen-generated captions. In Table R4, PSCM achieves 63.64% precision and 28.46% recall, reducing the proportion of flagged pairs from 14.19% before filtering to 10.84% among retained pairs.

Table R4. Caption-quality filtering against official RS5M human ratings

Labeling criterionFlagged pairs before filteringFiltering precisionNoise recallFlagged pairs after filtering
Any rating ≤ 3 counts as noise14.19%63.64%28.46%10.84%

2. Response to Weakness 2

Summary

(1) Attributing gains to domain adaptation and MRA (Table R5). To distinguish the gains from general domain adaptation from those specific to the MRA design, we added a comparison with a standard adapter. On RSICD at Noise Ratio=0%, MRA improves retrieval over Houlsby Adapter at a comparable parameter budget.

(2) Framework narrative and module contributions (Table R5 and Tables 2–3 of the submission). We agree that the submission could have explained the contribution of the calibration modules more clearly. To clarify the role of each module, we analyzed the incremental gains at Noise Ratio=0% and Noise Ratio=90%. PSCF further improves ranking with few additional parameters, while PSCM delivers its largest gains at Noise Ratio=90%. These findings support the framework's design of purifying training supervision before calibrating retrieval scores.

Detailed Response

(1) Attributing gains to domain adaptation and MRA

We compare MRA and Houlsby Adapter at comparable parameter budgets on RSICD at Noise Ratio=0%. In Table R5, MRA achieves an mR of 40.63 versus 39.86 for Houlsby Adapter, a gain of 0.77 percentage points.

Table R5. Comparison of MRA with a parameter-matched Houlsby Adapter

MethodParameters (M)I2T R@1I2T R@5I2T R@10T2I R@1T2I R@5T2I R@10mRRSum
MRA15.02024.8944.1057.0017.1142.7657.9040.63243.76
Houlsby Adapter15.01223.0643.7356.2716.9441.6157.5539.86239.16

(2) Framework narrative and module contributions

We agree that the submission could have more clearly separated the gains from general domain adaptation, the MRA design, and the two calibration modules. We added the parameter-matched comparison in Table R5 and use Tables 2–3 of the submission to explain the calibration modules' contributions under the two noise settings.

On RSICD at Noise Ratio=0%, adding PSCM raises mR from 40.63 to 40.73. Adding PSCF, which introduces only 0.035716M parameters, further raises mR to 41.75 (+1.02 percentage points). At Noise Ratio=90%, PSCM raises mR from 6.77 to 29.04 (+22.27 percentage points), and PSCF further raises it to 30.28 (+1.24 percentage points). PSCM thus contributes most at Noise Ratio=90%, and PSCF improves ranking in both settings with little parameter overhead. These complementary gains support the design of first purifying training supervision and then calibrating retrieval scores.

3. Response to Question 1

Summary

(1) Inference-time breakdown (Table R6). To identify the main inference costs and quantify the overhead of PSCF, we measured inference time by stage. ITC and ITM account for most of the runtime, and PSCF adds little overhead.

(2) Scope of the CFITR comparison. Limitations in the public CFITR implementation made it difficult to reproduce its results reliably under a matched setup. To give a more complete picture of the accuracy–cost trade-off, we compare P2CalP^2Cal with all baselines evaluated in our study in terms of retrieval accuracy and computational cost.

(3) Accuracy and computational cost (Tables R7–R9). To assess the accuracy–cost trade-off, we report results for the original and expanded galleries. P2CalP^2Cal maintains its retrieval accuracy advantage in both settings. PSCM filtering reduces training cost at Noise Ratio=90%, and expanding the gallery adds little to the amortized query latency.

Detailed Response

(1) Inference-time breakdown

We measure ITC candidate retrieval, cross-modal ITM for the top-128 candidates, and the three PSCF calibration steps separately. In Table R6, PSCF adds 35,716 trainable parameters and takes about 0.03% of the ITM runtime. ITC and ITM account for most of the inference time.

Table R6. Inference-time breakdown for P2CalP^2Cal

GalleryITC (h)ITM (h)Three-step PSCF calibration (h)PSCF time as a share of ITM time
RSICD0.152550.150800.000045850.03%

(2) Scope of the CFITR comparison

In the public CFITR implementation, the image and text hash buffers HiH_i and HtH_t are initialized to zero. Updating the binary targets at the end of each training epoch requires these buffers, and we could not identify an operation that updates them during training in the execution path we inspected. The code also instantiates LFEM, and its call in the image-encoding branch is commented out. Addressing these issues would require assumptions about the intended implementation, so we could not reliably reproduce the reported CFITR results.

(3) Accuracy and computational cost

On RSICD at Noise Ratio=0% and Noise Ratio=90%, we compare retrieval accuracy, training and inference time, query latency, throughput, and peak CPU and GPU memory. We compute amortized query latency by dividing total inference time by the number of queries in both directions, comprising 1,093 I2T image queries and 5,465 T2I text queries. In Tables R7–R8, P2CalP^2Cal achieves mR scores of 41.75 and 30.28, ahead of the baselines in both settings. We also evaluate the expanded gallery at Noise Ratio=90%. In Table R9, P2CalP^2Cal achieves an mR of 20.85, leading the next-best method by 15.37 percentage points, while amortized query latency rises from 165.01 ms to 182.20 ms (+10.4%). With its higher computational cost, P2CalP^2Cal retains a substantial accuracy advantage at a subsecond amortized query latency in this evaluation.

Table R7. Retrieval accuracy, training time, and inference cost on RSICD at Noise Ratio=0%

MethodmR (%) ↑Training time (h) ↓Total inference time (h) ↓Amortized query latency (ms/query) ↓Throughput (query/s) ↑GPU Memory (GiB) ↓CPU Memory (GiB) ↓
AMFMN16.731.250.01297.06141.608.6115.45
GaLR16.050.990.01055.74174.198.7212.84
PE29.982.190.00643.51285.148.152.19
PIR22.451.730.00291.62618.862.936.48
FGVLA32.180.480.00372.01498.002.466.71
LAPS23.373.870.021611.8484.491.975.20
CUP24.340.750.00754.09244.302.853.03
EAPA29.773.460.00311.69590.931.552.28
McRSA34.131.290.00231.26792.680.701.93
RRSITR30.590.610.00402.22450.113.213.26
P2CalP^2Cal41.7515.190.3034166.556.003.9511.49

Comments on the paper

3 comments
Reviewer_EjViOpenReview ↗

Thank you for the author's reply. I will update my rating as "Weak acceptance".

The author conducted a lot of experiments and addressed my doubts, so I will revise my opinion to "3: Weak acceptance".

Thank you for reviewing our response and additional experiments and for raising your score. This is truly encouraging to us! We sincerely appreciate your recognition and valuable feedback. Additionally, you may need to edit the Rating field in your original review to update your score. We apologize for the extra step and sincerely appreciate your help and support. We wish you all the best!

Dear Reviewers PLHh and dzTg,

The author–reviewer discussion closes today (Nov 18, AoE). The authors posted responses with new experiments. Please reply briefly with which of your concerns are resolved and which are not, and update your rating if your assessment has changed.

Thank you, Reviewer EjVi, for already engaging.

AC

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.