- Rating
- 3(+1)
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper proposes a prior-posterior collaborative calibration framework for remote sensing image-text retrieval (RSITR). The central motivation is that reliable image-text matching requires not only discriminative cross-modal representations but also reliable training correspondences and appropriately refined retrieval scores. The method is evaluated on RSICD, RSITMD, and NWPU for bidirectional retrieval. The reported results show consistent improvements over existing methods in the standard retrieval benchmarks and substantial robustness under severe synthetic correspondence noise. However, the current experiments do not sufficiently establish whether the proposed calibration mechanisms, particularly the manifold-specific adaptation and flow-based refinement, are necessary to obtain these improvements.
Critical strengths
- Strong and extensive empirical results across retrieval and robustness settings. The paper reports consistent improvements on three established RSITR benchmarks in both retrieval directions. The gains are particularly notable on RSICD and RSITMD. The robustness experiments further demonstrate substantial advantages under severe synthetic correspondence noise.
- A coherent framework connecting supervision reliability and retrieval-score refinement. The paper explicitly considers reliability at different stages of the retrieval pipeline rather than treating noisy supervision and inaccurate retrieval ranking as unrelated problems.
Critical weaknesses
- The reported improvements do not isolate the contribution of the proposed calibration framework under a matched experimental setup.
- The robustness claims depend heavily on synthetic corruption and clean validation supervision. Randomly shuffled captions may exhibit more obvious semantic inconsistencies than naturally occurring annotation errors. Real correspondence noise can involve partially incorrect descriptions, missing attributes, subtle spatial-relation errors, or mismatches between semantically similar scenes. It is unclear whether PSCM can reliably detect these harder cases.
- The manuscript also does not directly measure the precision of the filtered correspondences or quantify how many valid pairs are incorrectly discarded.
Questions for authors
- Are the improvements attributable to the proposed MRA and PSCF formulations under matched baselines?
- How reliable is PSCM's correspondence filtering, and how dependent is it on clean validation supervision?
- What is the specific contribution of flow-based calibration to the final retrieval ranking?
- How stable are the reported improvements across random seeds?
6 comments
2. Response to Weakness 2
Summary
(1) Cross-dataset noise filtering (Table R3). To test whether PSCM generalizes across datasets, we added a cross-dataset evaluation. Under the same random text-shuffling protocol, PSCM transfers from RSICD to NWPU and RSITMD with filtering precision and recall close to those obtained with within-dataset fitting.
(2) Within-scene noise filtering and downstream retrieval (Tables R4–R5). To test whether PSCM mitigates the effect of fine-grained mismatches on retrieval, we added within-scene caption swap experiments. The results show that PSCM improves downstream retrieval by conservatively removing a subset of hard-to-detect mismatches. They also suggest that some within-scene mismatches can retain useful scene semantics.
(3) Filtering naturally occurring caption-quality issues (Table R6). To evaluate PSCM on naturally occurring caption-quality issues, we used human ratings from RS5M. PSCM identifies a subset of these issues and lowers their prevalence among the retained pairs.
Detailed Response
(1) Cross-dataset noise filtering
We fit PSCM on RSICD and apply it to NWPU and RSITMD to detect randomly mismatched pairs. In Table R3, cross-dataset filtering precision and recall are close to the corresponding within-dataset results, supporting transfer under the same random text-shuffling protocol.
Table R3. Cross-dataset noise filtering with PSCM
| Fitting dataset → Filtering dataset | Noise setting | Filtering precision | Noise recall | Post-filtering noise ratio |
|---|---|---|---|---|
| RSICD → RSICD | Noise Ratio=90% | 93.90% | 92.31% | 60.03% |
| RSICD → NWPU | Noise Ratio=90% | 94.39% | 94.28% | 50.91% |
| NWPU → NWPU | Noise Ratio=90% | 93.99% | 93.84% | 54.66% |
| RSICD → RSITMD | Noise Ratio=90% | 93.92% | 93.82% | 55.07% |
| RSITMD → RSITMD | Noise Ratio=90% | 93.52% | 93.35% | 58.85% |
(2) Within-scene noise filtering and downstream retrieval
At Noise Ratio=90% on RSICD, we swap captions within scene categories to simulate mismatches that preserve scene semantics while potentially introducing incorrect details. We then compare retrieval with and without filtering. In Tables R4–R5, PSCM removes a subset of mismatches with 91.91% precision and retains 79.94% of clean pairs, improving downstream mR from 14.90 to 19.06.
Table R4. Filtering random and within-scene mismatches at Noise Ratio=90%
| Setting | Noise setting | Filtering precision | Noise recall | Post-filtering noise ratio |
|---|---|---|---|---|
| Random mismatches | Noise Ratio=90% | 93.90% | 92.31% | 60.03% |
| Within-scene mismatches | Noise Ratio=90% | 91.91% | 25.33% | 89.37% |
Table R5. Retrieval with and without PSCM filtering under within-scene mismatches at Noise Ratio=90%
| Setting | I2T R@1 | I2T R@5 | I2T R@10 | T2I R@1 | T2I R@5 | T2I R@10 | mR | RSum |
|---|---|---|---|---|---|---|---|---|
| With PSCM denoising | 6.40 | 17.84 | 30.10 | 5.78 | 20.09 | 34.16 | 19.06 | 114.38 |
| Without denoising | 4.03 | 12.81 | 22.69 | 4.10 | 15.85 | 29.92 | 14.90 | 89.39 |
(3) Filtering naturally occurring caption-quality issues
Using the official RS5M human ratings, we flag a pair when any rating is ≤ 3 and supplement the PSCM input with four Qwen-generated captions. In Table R6, PSCM achieves 63.64% precision and 28.46% recall, reducing the proportion of flagged pairs from 14.19% before filtering to 10.84% among retained pairs.
Table R6. Caption-quality filtering against official RS5M human ratings
| Labeling criterion | Flagged pairs before filtering | Filtering precision | Noise recall | Flagged pairs after filtering |
|---|---|---|---|---|
| Any rating ≤ 3 counts as noise | 14.19% | 63.64% | 28.46% | 10.84% |
3. Response to Weakness 3 / Question 2
Summary
(1) Filtering across noise ratios (Table R7). To assess filtering across noise ratios, we report filtering precision, noise recall, and the post-filtering noise ratio. PSCM lowers the noise ratio among retained pairs across the reported settings from Noise Ratio=10% to 90%.
(2) Size of the clean validation set (Table R8). To assess PSCM's reliance on clean validation data, we varied the size of the clean validation set. PSCM remains effective when fitted on a small subset.
Detailed Response
(1) Filtering across noise ratios
We fit PSCM on the clean RSICD validation set and evaluate at Noise Ratio=10%–90%, using the noise labels of the target training set solely for evaluation. In Table R7, PSCM lowers the noise ratio among retained pairs in every setting.
Table R7. Filtering performance on RSICD at Noise Ratio=10%–90%
| Noise setting | Filtering precision | Noise recall | Post-filtering noise ratio |
|---|---|---|---|
| Noise Ratio=10% | 74.59% | 71.41% | 3.16% |
| Noise Ratio=20% | 78.77% | 76.75% | 5.78% |
| Noise Ratio=30% | 82.59% | 80.81% | 8.15% |
| Noise Ratio=40% | 84.17% | 83.68% | 10.84% |
| Noise Ratio=50% | 85.78% | 84.65% | 15.15% |
| Noise Ratio=60% | 86.82% | 86.27% | 20.40% |
| Noise Ratio=70% | 88.84% | 87.89% | 27.58% |
| Noise Ratio=80% | 90.50% | 90.71% | 37.50% |
| Noise Ratio=90% | 93.90% | 92.31% | 60.03% |
(2) Size of the clean validation set
We refit PSCM using 100%, 50%, 25%, 10%, 5%, and 1% of the clean RSICD validation set. At Noise Ratio=90%, the precision and recall values in Table R8 yield F1 scores of approximately 93.10%, 93.23%, and 91.52% for the full set, the 5% subset, and the 1% subset, respectively. Relative to the full set, F1 increases by about 0.13 percentage points with 5% of the data and decreases by about 1.57 percentage points with 1%. PSCM thus maintains a high filtering F1 with substantially less clean validation data in this setting.
Table R8. Sensitivity to the size of the clean RSICD validation set at Noise Ratio=90%
| Validation set retained | Filtering precision | Noise recall | Post-filtering noise ratio |
|---|---|---|---|
| 100% | 93.90% | 92.31% | 60.03% |
| 50% | 94.08% | 91.59% | 61.12% |
| 25% | 94.08% | 91.77% | 60.67% |
| 10% | 93.72% | 93.06% | 58.72% |
| 5% | 93.97% | 92.50% | 59.18% |
| 1% | 95.18% | 88.14% | 64.07% |
4. Response to Question 3
To quantify the contribution of PSCF to final retrieval rankings, we analyze changes in the ranks of ground-truth matches for both retrieval directions on RSICD. PSCF applies posterior calibration to the discriminative ITM scores , moving ground-truth matches up the ranking. In Table R9, ground-truth matches move up or retain their rank for 93.05% of 1,093 I2T queries and 95.99% of 5,465 T2I queries. Among the queries whose matches move up, the average gains are 3.16 and 2.53 ranks, respectively. This analysis provides direct ranking-level evidence for the improvements in Recall@1, Recall@5, and Recall@10 with the same candidate sets.
Table R9. Effect of PSCF on bidirectional retrieval ranks on RSICD
| Retrieval direction | Total queries | Rank improved or unchanged (%) | Mean rank gain among improved queries |
|---|---|---|---|
| I2T | 1,093 | 93.05 | 3.16 |
| T2I | 5,465 | 95.99 | 2.53 |
Response to Reviewer EjVi
Thank you for asking us to isolate the module contributions under matched conditions, quantify both noise removal and the loss of valid pairs, and assess stability across random seeds. We took each concern seriously and devoted considerable effort to running additional experiments and analyses, which we present below. We will incorporate these findings and any clarifications from the discussion into the revised manuscript. We welcome further discussion. If the additional evidence addresses your concerns, we would be grateful if you would consider raising your rating.
1. Response to Weakness 1 / Question 1
Summary
(1) Parameter-matched comparisons for MRA (Table R1). To isolate the contribution of the MRA design, we compared it with standard adapters. MRA achieves higher retrieval accuracy than Houlsby Adapter and LoRA at comparable parameter budgets.
(2) Reranking comparisons for PSCF (Table R2). To assess the contribution of flow-based score calibration, we compared PSCF with simpler rerankers. With the same candidates and parameter budget, PSCF outperforms both alternatives and improves retrieval over the original ITM scores.
Detailed Response
(1) Parameter-matched comparisons for MRA
We compared MRA, Houlsby Adapter, and LoRA at comparable parameter budgets. In Table R1, MRA achieves an mR of 40.63, compared with 39.86 for Houlsby Adapter and 39.16 for LoRA.
Table R1. Comparison of MRA with parameter-matched LoRA and Houlsby Adapter
(2) Reranking comparisons for PSCF
We fix MRA, the input features, and the top-128 candidates. Each of the three learned scorers adds 0.035716M parameters. In Table R2, PSCF improves mR from 40.73 with uncalibrated ITM to 41.75. The conventional MLP reranker and the direct residual predictor achieve 38.65 and 37.94, respectively, both below the uncalibrated ITM baseline.
Table R2. PSCF versus simpler rerankers with fixed candidates and matched parameter budgets