- Rating
- 2
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
The paper investigates robust text–3D scene retrieval under test-time distribution shifts. It introduces M3SP, a benchmark consisting of 14 types of 3D perturbations at five severity levels, and proposes STAR-3D, a test-time adaptation framework that combines history-aware matching score calibration, centered-residual reliability routing, confidence-guided entropy minimization, and frozen-source representation anchoring. Experiments on multiple datasets and retrieval settings demonstrate improvements over existing test-time adaptation baselines. However, several concerns require further clarification.
Critical strengths
-
The paper addresses a practical problem in Text–3D Scene Retrieval and provides a benchmark with diverse 3D perturbations.
-
The experiments cover multiple datasets and evaluation settings, with overall improvements over existing TTA methods.
Critical weaknesses
-
The novelty is limited. Several key components are similar to those in the exsiting method HAT-VTR. The motivation for the modifications is not sufficiently explained.
-
The ablation studies are insufficient to establish the contributions of individual components. Direct comparisons with existing alternatives, such as HSM and simpler reliability measures, are missing.
Questions for authors
(1)The overall technical framework closely resembles HAT-VTR (ICLR 2026), particularly in history-based score calibration and confidence-guided adaptation. STAR-3D mainly introduces an alternative calibration method, a centered-residual reliability criterion, and a source-anchoring objective. However, the motivation for these modifications and their advantages over the corresponding mechanisms in HAT-VTR are not sufficiently justified. (2)The proposed MACS objective closely resembles the noise-robust adaptation loss in HAT-VTR, as both employ entropy-based confidence weighting with an adaptive threshold. The main difference appears to be the additional CRSI-based sample selection. The authors should clarify the methodological novelty of MACS and distinguish its contribution from the existing confidence-weighted entropy minimization strategy. (3)CRSI measures the consistency of centered cross-modal embeddings, but its relationship to the correctness of pseudo-matches remains unclear. Eq. (7) also reduces to a negative inner product of centered residuals, raising questions about its advantage over simpler similarity measures. The authors should evaluate pseudo-match precision under different corruption levels and compare CRSI with cosine similarity and entropy-based selection. Controlled component substitution experiments with HAT-VTR would further clarify the independent contribution of CRSI-based routing. (4)Table 5 shows that TSM improves retrieval performance, but does not establish the advantage of Sinkhorn calibration over the Hubness Suppression Memory (HSM) used in HAT-VTR. A direct comparison replacing Sinkhorn calibration with HSM under the same adaptation framework would help determine whether the proposed calibration strategy offers additional benefits. (5)Table 5 shows a substantial performance drop when final calibration (FC) is removed (54.537% to 44.623% R@1). Since FC operates on the completed query–gallery score matrix, it requires access to future queries in an online retrieval setting. The authors should clarify the evaluation protocol and report results under strictly online calibration to distinguish the contribution of test-time adaptation from global post-hoc calibration.
3 comments
[W3] CRSI measures the consistency of centered cross-modal embeddings, but its relationship to the correctness of pseudo-matches remains unclear. Eq. (7) also reduces to a negative inner product of centered residuals, raising questions about its advantage over simpler similarity measures. The authors should evaluate pseudo-match precision under different corruption levels and compare CRSI with cosine similarity and entropy-based selection. Controlled component substitution experiments with HAT-VTR would further clarify the independent contribution of CRSI-based routing.
Thank you for this insightful comment. We would first like to clarify that Eq. (7) does not reduce to a negative inner product. It uses three unsquared Euclidean norms to measure the discrepancy between batch-centered query and candidate residuals. To directly evaluate its reliability, we compared CRSI against entropy-based selection and cosine similarity using identical pre-update pseudo-pairs under matched selection coverage across three severity levels of OCR and Gaussian corruptions.
Each entry reports AUROC / selection precision (%).
| Direction | Corruption | Severity | CRSI | Entropy | Cosine |
|---|---|---|---|---|---|
| t2s | OCR | S1 | 0.6615 / 43.28 | 0.5915 / 36.47 | 0.6565 / 42.94 |
| t2s | OCR | S3 | 0.6629 / 35.58 | 0.5966 / 29.76 | 0.6547 / 34.85 |
| t2s | OCR | S7 | 0.6700 / 25.93 | 0.6122 / 21.53 | 0.6553 / 23.79 |
| s2t | Gaussian | S1 | 0.6069 / 65.83 | 0.5165 / 57.91 | 0.6104 / 63.81 |
| s2t | Gaussian | S3 | 0.6233 / 68.35 | 0.5402 / 61.11 | 0.6176 / 66.68 |
| s2t | Gaussian | S5 | 0.6365 / 67.34 | 0.5236 / 57.91 | 0.6307 / 65.49 |
Across all six experimental settings, CRSI achieves superior performance over the entropy-based baseline in both AUROC and selection precision. Compared with cosine similarity, CRSI yields higher selection precision at all Gaussian corruption severity levels and exhibits comparable performance under OCR corruption. These results demonstrate that CRSI captures complementary structural reliability signals beyond vanilla prediction confidence, and shows stronger robustness especially when 3D features are corrupted. We will clarify the mathematical formulation of Eq. (7) and incorporate these supplementary analyses into the revised manuscript.
[W4] Table 5 shows that TSM improves retrieval performance, but does not establish the advantage of Sinkhorn calibration over the Hubness Suppression Memory (HSM) used in HAT-VTR. A direct comparison replacing Sinkhorn calibration with HSM under the same adaptation framework would help determine whether the proposed calibration strategy offers additional benefits.
Thank you for this suggestion. We conducted an additional experiment in which Sinkhorn was replaced with HSM. The evaluation covers all 29 corruption conditions in ScanRefer, comprising 15 text perturbations in t2s and 14 3D perturbations in s2t, with three random seeds.
| Direction | Calibration | R@1 | R@5 | R@10 | Top-20 Share ↓ |
|---|---|---|---|---|---|
| t2s | None | 19.291 | 42.266 | 53.644 | 12.892 |
| t2s | HSM | 19.507 | 42.361 | 53.666 | 9.727 |
| t2s | Sinkhorn | 21.397 | 44.95 | 55.97 | 5.053 |
| s2t | None | 43.585 | 70.629 | 76.508 | 14.696 |
| s2t | HSM | 49.102 | 73.508 | 78.344 | 11.299 |
| s2t | Sinkhorn | 56.858 | 76.495 | 80.57 | 9.453 |
Compared with HSM, Sinkhorn improves R@1 by 1.89 pp in t2s and 7.76 pp in s2t, with positive 95% scene-paired confidence intervals in both directions. Notably, it achieves higher R@1 on all 29 corruption conditions. It also reduces Top-20 gallery assignment share by 4.67 pp and 1.85 pp, respectively, while improving pre-CRSI pseudo-pair accuracy by 1.36 pp and 5.03 pp. These results demonstrate that Sinkhorn provides more effective gallery-level matching calibration than HSM, leading to more reliable provisional correspondences and improved retrieval performance.
[W5] Table 5 shows a substantial performance drop when final calibration (FC) is removed (54.537% to 44.623% R@1). Since FC operates on the completed query–gallery score matrix, it requires access to future queries in an online retrieval setting. The authors should clarify the evaluation protocol and report results under strictly online calibration to distinguish the contribution of test-time adaptation from global post-hoc calibration.
Thank you for your comments. We clarify that our framework supports two distinct calibration protocols.
Causal loop FC performs Sinkhorn calibration after each adaptation batch using only the current batch and up to 32 historical batches of raw scores. The calibrated results are emitted before the next batch is processed and are never revised. The first batch uses native scores. In contrast, global FC operates on the complete score matrix after the entire query stream has been processed and is therefore an offline transductive protocol.
Across all 29 ScanRefer corruption conditions and three random seeds, loop FC improves STAR-3D over its native output by 1.25 pp in t2s and 7.97 pp in s2t R@1. The corresponding results are:
| Direction | Native R@1 | Loop FC R@1 | Global FC R@1 |
|---|---|---|---|
| t2s | 19.17 | 20.42 | 21.4 |
| s2t | 43.72 | 51.69 | 56.82 |
Loop FC therefore enables batch-wise online calibration without accessing future queries, whereas global FC requires the complete query stream and is retained only as an offline reference. We will explicitly distinguish these two protocols in the revised manuscript and report loop FC as the strictly online calibration setting.
Thank you for your comments. HAT-VTR primarily addresses gallery hubness in video–text retrieval, whereas text–3D scene retrieval introduces additional challenges: incomplete or distorted scans may obscure geometric cues required for reliable cross-modal matching, and shared encoder updates can shift the representations of uncertain queries. These differences motivate us to reconsider both correspondence reliability and representation stability within the TTA process. Rather than directly adopting existing mechanisms, STAR-3D introduces history-aware Sinkhorn calibration to mitigate gallery-level matching bias, CRSI to assess structural consistency between query and candidate representations, and frozen-source anchoring to prevent unreliable queries from drifting during adaptation.
With the same causal output calibration on ScanRefer, STAR-3D achieves a higher mean R@1 than HAT-VTR in both directions, with a larger gain in scene-to-text retrieval. We will revise the manuscript to better explain the motivation behind these designs and clarify their distinctions from the corresponding mechanisms in HAT-VTR.
Thank you for your comments. We acknowledge that MACS shares the confidence-weighted entropy minimization formulation with HAT-VTR; our contribution does not lie in reformulating the entropy loss itself, but in improving sample reliability through CRSI-based structural screening. To validate this distinction, we conducted a matched-budget comparison against entropy-based selection using identical candidates and model states.
CRSI consistently achieves higher pseudo-match precision in both retrieval directions, demonstrating that structural consistency provides complementary reliability signals beyond prediction confidence. This improves the quality of queries admitted to adaptation and clarifies the role of CRSI in MACS.