acceptodds
Under review as a conference paper at ICLR 2027

Robust Test-Time Text–3D Scene Retrieval: Benchmark and Reliability-Routing

Abstract

Deployed Text–3D Scene Retrieval (T3SR) systems receive unlabeled queries that deviate from the training distribution: typos and paraphrases on the text side, and occlusion, sensor noise, and partial scans on the 3D side. Test-time adaptation (TTA) can help address these distribution shifts, but its effectiveness depends on the reliability of self-generated cross-modal correspondences. Two coupled challenges arise: gallery-level matching biases can undermine correspondence reliability despite high prediction confidence, while shared query-encoder updates can induce representation drift in uncertain queries even when they are excluded from the adaptation objective. To systematically study these challenges, we introduce M3SP, a benchmark comprising 14 multi-level 3D perturbation types, each instantiated at five severity levels across four datasets. We further propose **STAR-3D** (**S**tructure-guided **T**est-time **A**daptation for Text–3D Scene **R**etrieval). STAR-3D first calibrates query–gallery matching scores through history-aware Sinkhorn normalization, then assesses provisional correspondences by comparing query and candidate representations in a centered residual space. Structural consistency and a memory-adaptive confidence gate jointly determine which queries drive confidence sharpening, while the remaining queries are regularized toward their frozen-source representations. Extensive experiments across datasets demonstrate that STAR-3D outperforms competitive TTA baselines overall under query shift and query–gallery shift. Code is available at [https://anonymous.4open.science/r/STAR-3D-C486](https://anonymous.4open.science/r/STAR-3D-C486).

Primary area:
unsupervised, self-supervised, semi-supervised, and supervised representation learning
Keywords:
Text-3D Scene Retrieval, Test-time Adaptation

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_YjZHOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper investigates robust text–3D scene retrieval under test-time distribution shifts. It introduces M3SP, a benchmark consisting of 14 types of 3D perturbations at five severity levels, and proposes STAR-3D, a test-time adaptation framework that combines history-aware matching score calibration, centered-residual reliability routing, confidence-guided entropy minimization, and frozen-source representation anchoring. Experiments on multiple datasets and retrieval settings demonstrate improvements over existing test-time adaptation baselines. However, several concerns require further clarification.

Critical strengths
  1. The paper addresses a practical problem in Text–3D Scene Retrieval and provides a benchmark with diverse 3D perturbations.

  2. The experiments cover multiple datasets and evaluation settings, with overall improvements over existing TTA methods.

Critical weaknesses
  1. The novelty is limited. Several key components are similar to those in the exsiting method HAT-VTR. The motivation for the modifications is not sufficiently explained.

  2. The ablation studies are insufficient to establish the contributions of individual components. Direct comparisons with existing alternatives, such as HSM and simpler reliability measures, are missing.

Questions for authors

(1)The overall technical framework closely resembles HAT-VTR (ICLR 2026), particularly in history-based score calibration and confidence-guided adaptation. STAR-3D mainly introduces an alternative calibration method, a centered-residual reliability criterion, and a source-anchoring objective. However, the motivation for these modifications and their advantages over the corresponding mechanisms in HAT-VTR are not sufficiently justified. (2)The proposed MACS objective closely resembles the noise-robust adaptation loss in HAT-VTR, as both employ entropy-based confidence weighting with an adaptive threshold. The main difference appears to be the additional CRSI-based sample selection. The authors should clarify the methodological novelty of MACS and distinguish its contribution from the existing confidence-weighted entropy minimization strategy. (3)CRSI measures the consistency of centered cross-modal embeddings, but its relationship to the correctness of pseudo-matches remains unclear. Eq. (7) also reduces to a negative inner product of centered residuals, raising questions about its advantage over simpler similarity measures. The authors should evaluate pseudo-match precision under different corruption levels and compare CRSI with cosine similarity and entropy-based selection. Controlled component substitution experiments with HAT-VTR would further clarify the independent contribution of CRSI-based routing. (4)Table 5 shows that TSM improves retrieval performance, but does not establish the advantage of Sinkhorn calibration over the Hubness Suppression Memory (HSM) used in HAT-VTR. A direct comparison replacing Sinkhorn calibration with HSM under the same adaptation framework would help determine whether the proposed calibration strategy offers additional benefits. (5)Table 5 shows a substantial performance drop when final calibration (FC) is removed (54.537% to 44.623% R@1). Since FC operates on the completed query–gallery score matrix, it requires access to future queries in an online retrieval setting. The authors should clarify the evaluation protocol and report results under strictly online calibration to distinguish the contribution of test-time adaptation from global post-hoc calibration.

3 comments

[W1] The overall technical framework closely resembles HAT-VTR (ICLR 2026), particularly in history-based score calibration and confidence-guided adaptation. STAR-3D mainly introduces an alternative calibration method, a centered-residual reliability criterion, and a source-anchoring objective. However, the motivation for these modifications and their advantages over the corresponding mechanisms in HAT-VTR are not sufficiently justified.

Thank you for your comments. HAT-VTR primarily addresses gallery hubness in video–text retrieval, whereas text–3D scene retrieval introduces additional challenges: incomplete or distorted scans may obscure geometric cues required for reliable cross-modal matching, and shared encoder updates can shift the representations of uncertain queries. These differences motivate us to reconsider both correspondence reliability and representation stability within the TTA process. Rather than directly adopting existing mechanisms, STAR-3D introduces history-aware Sinkhorn calibration to mitigate gallery-level matching bias, CRSI to assess structural consistency between query and candidate representations, and frozen-source anchoring to prevent unreliable queries from drifting during adaptation.

With the same causal output calibration on ScanRefer, STAR-3D achieves a higher mean R@1 than HAT-VTR in both directions, with a larger gain in scene-to-text retrieval. We will revise the manuscript to better explain the motivation behind these designs and clarify their distinctions from the corresponding mechanisms in HAT-VTR.

[W2] The proposed MACS objective closely resembles the noise-robust adaptation loss in HAT-VTR, as both employ entropy-based confidence weighting with an adaptive threshold. The main difference appears to be the additional CRSI-based sample selection. The authors should clarify the methodological novelty of MACS and distinguish its contribution from the existing confidence-weighted entropy minimization strategy.

Thank you for your comments. We acknowledge that MACS shares the confidence-weighted entropy minimization formulation with HAT-VTR; our contribution does not lie in reformulating the entropy loss itself, but in improving sample reliability through CRSI-based structural screening. To validate this distinction, we conducted a matched-budget comparison against entropy-based selection using identical candidates and model states.

DirectionEntropy-basedCRSIImprovement
t2s25.24%28.93%+3.69 pp
s2t53.01%61.47%+8.47 pp

CRSI consistently achieves higher pseudo-match precision in both retrieval directions, demonstrating that structural consistency provides complementary reliability signals beyond prediction confidence. This improves the quality of queries admitted to adaptation and clarifies the role of CRSI in MACS.

[W3] CRSI measures the consistency of centered cross-modal embeddings, but its relationship to the correctness of pseudo-matches remains unclear. Eq. (7) also reduces to a negative inner product of centered residuals, raising questions about its advantage over simpler similarity measures. The authors should evaluate pseudo-match precision under different corruption levels and compare CRSI with cosine similarity and entropy-based selection. Controlled component substitution experiments with HAT-VTR would further clarify the independent contribution of CRSI-based routing.

Thank you for this insightful comment. We would first like to clarify that Eq. (7) does not reduce to a negative inner product. It uses three unsquared Euclidean norms to measure the discrepancy between batch-centered query and candidate residuals. To directly evaluate its reliability, we compared CRSI against entropy-based selection and cosine similarity using identical pre-update pseudo-pairs under matched selection coverage across three severity levels of OCR and Gaussian corruptions.

Each entry reports AUROC / selection precision (%).

DirectionCorruptionSeverityCRSIEntropyCosine
t2sOCRS10.6615 / 43.280.5915 / 36.470.6565 / 42.94
t2sOCRS30.6629 / 35.580.5966 / 29.760.6547 / 34.85
t2sOCRS70.6700 / 25.930.6122 / 21.530.6553 / 23.79
s2tGaussianS10.6069 / 65.830.5165 / 57.910.6104 / 63.81
s2tGaussianS30.6233 / 68.350.5402 / 61.110.6176 / 66.68
s2tGaussianS50.6365 / 67.340.5236 / 57.910.6307 / 65.49

Across all six experimental settings, CRSI achieves superior performance over the entropy-based baseline in both AUROC and selection precision. Compared with cosine similarity, CRSI yields higher selection precision at all Gaussian corruption severity levels and exhibits comparable performance under OCR corruption. These results demonstrate that CRSI captures complementary structural reliability signals beyond vanilla prediction confidence, and shows stronger robustness especially when 3D features are corrupted. We will clarify the mathematical formulation of Eq. (7) and incorporate these supplementary analyses into the revised manuscript.

[W4] Table 5 shows that TSM improves retrieval performance, but does not establish the advantage of Sinkhorn calibration over the Hubness Suppression Memory (HSM) used in HAT-VTR. A direct comparison replacing Sinkhorn calibration with HSM under the same adaptation framework would help determine whether the proposed calibration strategy offers additional benefits.

Thank you for this suggestion. We conducted an additional experiment in which Sinkhorn was replaced with HSM. The evaluation covers all 29 corruption conditions in ScanRefer, comprising 15 text perturbations in t2s and 14 3D perturbations in s2t, with three random seeds.

DirectionCalibrationR@1R@5R@10Top-20 Share ↓
t2sNone19.29142.26653.64412.892
t2sHSM19.50742.36153.6669.727
t2sSinkhorn21.39744.9555.975.053
s2tNone43.58570.62976.50814.696
s2tHSM49.10273.50878.34411.299
s2tSinkhorn56.85876.49580.579.453

Compared with HSM, Sinkhorn improves R@1 by 1.89 pp in t2s and 7.76 pp in s2t, with positive 95% scene-paired confidence intervals in both directions. Notably, it achieves higher R@1 on all 29 corruption conditions. It also reduces Top-20 gallery assignment share by 4.67 pp and 1.85 pp, respectively, while improving pre-CRSI pseudo-pair accuracy by 1.36 pp and 5.03 pp. These results demonstrate that Sinkhorn provides more effective gallery-level matching calibration than HSM, leading to more reliable provisional correspondences and improved retrieval performance.

[W5] Table 5 shows a substantial performance drop when final calibration (FC) is removed (54.537% to 44.623% R@1). Since FC operates on the completed query–gallery score matrix, it requires access to future queries in an online retrieval setting. The authors should clarify the evaluation protocol and report results under strictly online calibration to distinguish the contribution of test-time adaptation from global post-hoc calibration.

Thank you for your comments. We clarify that our framework supports two distinct calibration protocols.

Causal loop FC performs Sinkhorn calibration after each adaptation batch using only the current batch and up to 32 historical batches of raw scores. The calibrated results are emitted before the next batch is processed and are never revised. The first batch uses native scores. In contrast, global FC operates on the complete score matrix after the entire query stream has been processed and is therefore an offline transductive protocol.

Across all 29 ScanRefer corruption conditions and three random seeds, loop FC improves STAR-3D over its native output by 1.25 pp in t2s and 7.97 pp in s2t R@1. The corresponding results are:

DirectionNative R@1Loop FC R@1Global FC R@1
t2s19.1720.4221.4
s2t43.7251.6956.82

Loop FC therefore enables batch-wise online calibration without accessing future queries, whereas global FC requires the complete query stream and is retained only as an offline reference. We will explicitly distinguish these two protocols in the revised manuscript and report loop FC as the strictly online calibration setting.

Reviewer_5Uy7OpenReview ↗
Rating
2
Confidence
1
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper addresses robust text–3D scene retrieval under test-time distribution shifts, aiming to mitigate unreliable cross-modal correspondences and representation drift during test-time adaptation. The authors introduce M3SP, a benchmark comprising 14 types of 3D scene perturbations at five severity levels, and propose STAR-3D, a test-time adaptation framework consisting of three main components: (1) Matching Score Calibration, which employs historical matching statistics and Sinkhorn normalization to reduce gallery-level matching biases; (2) Reliability Routing, which uses centered residual consistency and prediction confidence to identify reliable query–gallery correspondences; and (3) Reliability-guided Adaptation, which applies entropy minimization to reliable queries while preserving unreliable query representations through frozen-source anchoring. Experiments across multiple datasets and retrieval backbones demonstrate improved robustness under query shifts and query–gallery shifts, outperforming existing test-time adaptation methods in most evaluated settings.

Critical strengths
  • The paper addresses the robustness of text–3D scene retrieval under test-time distribution shifts, which is a practically important yet relatively underexplored problem. Improving retrieval reliability under noisy or incomplete 3D observations is valuable for real-world applications.
  • The authors identify two meaningful challenges in test-time adaptation: unreliable pseudo-correspondences and representation drift caused by shared encoder updates. In particular, the observation that excluding unreliable queries from the adaptation objective does not prevent their representations from changing is well-motivated. The proposed strategy of adapting reliable queries while preserving uncertain representations provides a reasonable direction for addressing these challenges.
  • The paper introduces a systematic benchmark covering multiple types and severity levels of 3D perturbations. Experiments across multiple datasets, retrieval backbones, and distribution-shift settings demonstrate promising improvements over existing test-time adaptation methods.
Critical weaknesses
  • The proposed M3SP benchmark is constructed by applying manually designed synthetic perturbations to existing ScanNet-based datasets, rather than collecting real-world degraded 3D observations. Although these perturbations cover various noise and corruption types, they may not fully reflect the complexity of real sensor failures and environmental variations.
  • The proposed Sinkhorn calibration aims to alleviate gallery-level matching biases by encouraging more balanced matching scores. However, the ground-truth query–gallery correspondence distribution is not necessarily uniform. For example, a 3D scene may correspond to substantially more valid text descriptions than another scene, making frequent matches inherently reasonable. Enforcing balanced matching scores could therefore suppress valid correspondences. The paper does not sufficiently investigate this potential issue or evaluate the method under different levels of correspondence imbalance.
  • According to Table 5, final Sinkhorn calibration contributes substantially to the reported improvements. Specifically, applying final calibration alone increases R@1 from 40.119% to 48.030%, while removing it from the complete framework reduces R@1 from 54.537% to 44.623%. These results suggest that a considerable portion of the performance improvement originates from post-hoc score calibration rather than query encoder adaptation. A more detailed analysis is needed to clarify the respective contributions of the proposed adaptation mechanisms and final calibration.
Questions for authors

Please see the Weaknesses

2 comments

[W1] The proposed M3SP benchmark is constructed by applying manually designed synthetic perturbations to existing ScanNet-based datasets, rather than collecting real-world degraded 3D observations. Although these perturbations cover various noise and corruption types, they may not fully reflect the complexity of real sensor failures and environmental variations.

Thank you for raising this concern. We agree that synthetic perturbations cannot fully reproduce the complexity of real-world sensor degradation and environmental changes. However, our goal in constructing M3SP is not to replace real-world degraded scans, but to establish a controlled benchmark for evaluating retrieval robustness under different types of 3D distribution shifts. Specifically, we introduce 14 manually designed perturbation types at the point, object, and scene levels, with five severity levels for each, applied to real scans from four ScanNet-based retrieval datasets. This design allows us to preserve the original scene content while introducing controlled variations, making it possible to isolate the effects of different corruption types and compare model robustness under consistent conditions.

Compared with collecting naturally degraded scans, this controlled setting also makes it easier to reproduce individual corruption conditions and examine how retrieval performance changes as perturbation severity increases. We believe this provides a useful complement to real-world evaluations, rather than a substitute for them. We will further clarify the design objectives and scope of M3SP in the revised manuscript.

[W2] The proposed Sinkhorn calibration aims to alleviate gallery-level matching biases by encouraging more balanced matching scores. However, the ground-truth query–gallery correspondence distribution is not necessarily uniform. For example, a 3D scene may correspond to substantially more valid text descriptions than another scene, making frequent matches inherently reasonable. Enforcing balanced matching scores could therefore suppress valid correspondences. The paper does not sufficiently investigate this potential issue or evaluate the method under different levels of correspondence imbalance.

Thank you for highlighting that query–gallery correspondence frequencies need not be uniform. We conducted an additional controlled imbalance study in the ScanRefer/CrossOver t2s setting, varying the Head:Medium:Tail query-exposure ratio from 1:1:1 to 2:1.5:1 and 8:4:1. We compared None, HSM, and Sinkhorn across 15 text perturbations and three query-order seeds, with query identities, frequencies, orders, and B32 batches matched across methods.

All values are percentages; Δ denotes Sinkhorn minus None in percentage points.

ratioOverall R@1 (None / HSM / Sinkhorn)Δ HeadΔ MediumΔ Tail
1:1:119.231 / 19.448 / 21.3481.662.182.28
2:1.5:119.460 / 19.648 / 21.3561.001.942.72
8:4:119.676 / 19.679 / 20.4271.041.843.57

Sinkhorn achieves the highest mean overall R@1 under all three distributions, with gains of +2.12, +1.90, and +0.75 points, respectively. Notably, all 15 text perturbations show positive overall gains. Under severe imbalance, Sinkhorn also improves Head R@1 by 1.04 points and Tail R@1 by 3.57 points. These results suggest that Sinkhorn calibration remains beneficial under the tested correspondence imbalances, without systematically disadvantaging frequently matched scenes. We will clarify the scope of our calibration claims in the revision.

[W3] According to Table 5, final Sinkhorn calibration contributes substantially to the reported improvements. Specifically, applying final calibration alone increases R@1 from 40.119% to 48.030%, while removing it from the complete framework reduces R@1 from 54.537% to 44.623%. These results suggest that a considerable portion of the performance improvement originates from post-hoc score calibration rather than query encoder adaptation. A more detailed analysis is needed to clarify the respective contributions of the proposed adaptation mechanisms and final calibration.

Thank you for raising this concern. We agree that the original ablation did not sufficiently separate the contribution of test-time adaptation from that of final calibration. We therefore conducted a controlled comparison by applying the same Sinkhorn output calibration to all methods while preserving their respective adaptation procedures. The evaluation covers all 29 ScanRefer corruption conditions and three random seeds.

R@1 (%), averaged over corruption conditions and three random seeds.

DirectionMethodNative R@1Loop FC R@1
t2sTCR19.820.26
t2sHAT-VTR19.0217.63
t2sHAT-VTR pre-HSM19.0320.39
t2sSTAR-3D19.1720.42
s2tTCR41.8750.16
s2tHAT-VTR44.0345.43
s2tHAT-VTR pre-HSM40.5648.99
s2tSTAR-3D43.7251.69

Applying the same loop FC to each method’s native frozen-source output consistently improves performance, verifying that this calibration step yields independent gains orthogonal to adaptation. Under this unified calibration protocol, STAR-3D still preserves performance gains over baselines. Critically, with the same loop FC, STAR-3D outperforms vanilla HAT-VTR by +2.79 pp on t2s and +6.26 pp on s2t. The HAT-VTR pre-HSM control further isolates HAT-VTR’s built-in hubness suppression from the shared Sinkhorn output calibration. Notably, after stripping its native hubness suppression (HAT-VTR pre-HSM), HAT-VTR obtains 20.39 (t2s) and 48.99 (s2t) under the same loop FC, which is on par with STAR-3D on t2s and lower than STAR-3D on s2t.

These results collectively demonstrate that STAR-3D possesses genuine adaptation benefits beyond the shared calibration operation, while the Sinkhorn calibration is a general-purpose component that can boost other retrieval methods as well. We will revise the ablation discussion to clearly disentangle these contributions and explicitly state this unified calibration protocol in the manuscript.

Reviewer_Qp3MOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

This paper studies text–3D scene retrieval under distribution shifts. Test-time adaptation relies on self-generated cross-modal matches, which can concentrate incorrect assignments on a few gallery items. Shared-encoder updates can also cause representation drift in queries excluded from adaptation.

The paper introduces M3SP, a benchmark with 14 point-, object-, and scene-level 3D corruption types at five severity levels over ScanRefer, SceneDepict, Nr3D, and Sr3D, together with text perturbations. STAR-3D calibrates query–gallery scores using Temporal Similarity Memory (TSM), selects adaptation queries through Centered-Residual Structural Inconsistency (CRSI) and entropy, updates the model through confidence-weighted entropy sharpening, and anchors the remaining queries to frozen-source representations. Experiments use CrossOver and Mosaic3D under query shift, cross-dataset query–gallery shift, and zero-shot adaptation, in both scene-to-text and text-to-scene directions.

Critical strengths
  1. M3SP provides a structured robustness protocol for retrieval. Its 14 3D corruptions are organized by spatial scale, severity levels and parameters are documented in Tables 8–9, and evaluation covers four datasets and both retrieval directions. This is the most reusable part of the submission.

  2. The paper quantifies both failure modes and provides broad evaluation. In Figure 1, corruption increases the Top-20 share of gallery assignments from 8.3% to 18.3%; among the most confident raw predictions, 62.3% of t2s and 43.4% of s2t Top-1 matches are incorrect. Shared-encoder updates also cause excluded queries to drift. These results motivate calibration, routing, and anchoring. Evaluation covers two backbones, multiple shift regimes, component ablations, and runtime. The reported 29.65 ms/query is close to HAT-VTR’s 28.39 ms/query.

Critical weaknesses
  1. Calibration gains are not sufficiently separated from adaptation and routing gains. In Table 5, R@1 increases from 40.119 to 48.030 with FC alone, 44.623 with adaptation alone, and 54.537 with both. These gains interact and cannot be interpreted as additive contributions; adaptation alone also trails HAT-VTR’s 45.45 in Table 1. Methods need to be compared under identical FC settings, distinguishing online streaming results from final calibrated rankings. CRSI ablations and routing controls also do not directly establish whether it identifies correct matches. Admission precision and coverage should be evaluated against entropy-, margin-, and distance-based routing. Reduced drift in Figure 1(d) alone does not establish improved retrieval accuracy.

  2. Result stability, implementation details, and consistency between prose and tables remain unresolved. Appendix C.1 reports three paired runs for selected learning-rate comparisons, but the main tables lack repeated-run uncertainty, leaving the stability of margins such as 1.47 points over TCR in Table 2 unclear. The values of λ_E, λ_A, and N_tta, baseline tuning, and hyperparameter-selection data also need to be specified.

Some prose claims disagree with the tables: the introduction attributes 23.55 and a 1.28-point gain to CrossOver t2s, whereas Table 1 assigns them to Mosaic3D s2t; Appendix C.3.2 describes some STAR-3D wins in Tables 11, 12, and 15 as baseline wins. Deployment evidence is limited to synthetic corruptions of ScanNet-derived indoor scenes, without evaluation of real acquisition failures, long-stream drift, or large galleries.

Questions for authors
  1. Matched controls: Please apply the same final Sinkhorn calibration to all baselines and report STAR-3D with and without FC, including end-to-end timing, under a declared streaming-TTA or offline-batched-retrieval protocol. Does online-only adaptation outperform HAT-VTR and TCR?

  2. Stability and implementation: Please report three-seed means, standard deviations, and paired tests for the methods in Tables 1, 2, and 4, and assess sensitivity to query order. Are the t2s and zero-shot margins stable? Please also specify λ_E, λ_A, N_tta, baseline tuning, and hyperparameter-selection data.

  3. Routing validation: Please directly evaluate CRSI against match correctness, for example through AUC, admission precision, and coverage, and compare it with entropy-, margin-, and distance-based routing. Does CRSI provide additional signal, and how does batch size affect it?

  4. Numerical consistency: Please check the attribution of 23.55 and the 1.28-point gain in the introduction, and the descriptions of wins and losses for Tables 11, 12, and 15 in Appendix C.3.2. Are these prose errors or mislabeled results?

3 comments

[W1] Matched controls: Please apply the same final Sinkhorn calibration to all baselines and report STAR-3D with and without FC, including end-to-end timing, under a declared streaming-TTA or offline-batched-retrieval protocol. Does online-only adaptation outperform HAT-VTR and TCR?

Thank you for highlighting the need to disentangle adaptation gains from calibration gains. We conducted a controlled comparison on ScanRefer, covering all 29 corruption conditions and three random seeds. We distinguish native outputs, causal loop FC, and full-stream global FC. All methods use the same B32 query streams, and the same Sinkhorn operator (temperature 0.05, five iterations) is applied for output calibration.

R@1 (%), averaged over corruption conditions and three random seeds.

DirectionMethodNativeLoop FCGlobal FC
t2sTCR19.820.2620.13
t2sHAT-VTR19.0217.6318.94
t2sSTAR-3D19.1720.4221.4
s2tTCR41.8750.1654.94
s2tHAT-VTR44.0345.4346.9
s2tSTAR-3D43.7251.6956.82

In the native setting without FC, our STAR-3D obtains 19.17 (t2s) and 43.72 (s2t): it beats HAT-VTR on t2s and beats TCR on s2t, demonstrating standalone adaptation gains independent of calibration. With causal loop FC, STAR-3D outperforms TCR by 0.16 pp on t2s and 1.53 pp on s2t. With global FC, STAR-3D surpasses both baselines on both directions. This distinguishes adaptation gains from calibration gains, instead of crediting all improvements to TTA.

End-to-end efficiency. We further measure the time to first and last output on a single NVIDIA GeForce RTX 4090 GPU:

Each entry reports first-output / last-output latency.

DirectionNativeLoop FCGlobal FC
t2s0.16 / 36.76 s0.16 / 37.07 s36.75 / 36.75 s
s2t1.41 / 32.55 s1.39 / 32.58 s32.56 / 32.56 s

Loop FC introduces only 0.31 s and 0.03 s additional end-to-end latency in t2s and s2t, respectively, while enabling immediate batch-wise outputs. In contrast, global FC delays the first output until the entire query stream has been processed.

We will revise the manuscript to explicitly separate native TTA, causal online calibration, and offline global calibration, and clarify their respective performance and computational contributions.

[W2] Stability and implementation: Please report three-seed means, standard deviations, and paired tests for the methods in Tables 1, 2, and 4, and assess sensitivity to query order. Are the t2s and zero-shot margins stable? Please also specify λ_E, λ_A, N_tta, baseline tuning, and hyperparameter-selection data.

Thank you for raising these concerns. We conducted evaluations with three random seeds for Tables 1, 2, and 4, resetting the model, optimizer, and online memory for each run while keeping all hyperparameters fixed. We report mean ± standard deviation and perform scene-paired bootstrap and sign-flip tests, with Holm correction.

Query-order stability. The R@1 results show consistent gains in both retrieval directions under query shifts:

SettingBackboneSTAR-3D R@1Best BaselineMargin
Table 1 (s2t)CrossOver56.858 ± 0.198HAT-VTR+11.847
Table 1 (s2t)Mosaic3D23.630 ± 0.006HAT-VTR+1.937
Table 2 (t2s)CrossOver21.397 ± 0.013TCR+1.447
Table 2 (t2s)Mosaic3D11.802 ± 0.025TCR+1.968

STAR-3D wins in all three random seeds in these four settings.

Zero-shot stability. We further examine the margins against the strongest native baseline in each setting:

DatasetDirectionMean MarginWins
ScanRefert2s+1.910 pp3/3
ScanRefers2t+1.739 pp3/3
SceneDepictt2s+1.078 pp3/3
SceneDepicts2t+0.924 pp3/3

The results show our method achieves steady improvements across both directions for both zero-shot retrieval directions.

Implementation and hyperparameter selection. For STAR-3D, we set λEλ_{E} = λAλ_{A} = 1 and adopt AdamW with a batch size of 32 and weight decay of 10−410^{-4} throughout all evaluations. The TTA update steps N_tta are set to 3 for all base retrieval models and datasets. Baselines retain their method-specific optimizers, batch sizes, update steps, and other prescribed settings, rather than being forced into a shared optimization configuration. For hyperparameter selection, the historical CrossOver sweep used labeled R@1 on corrupted ScanRefer and SceneDepict streams, with random seed 42 for candidate selection.

[w3] Routing validation: Please directly evaluate CRSI against match correctness, for example through AUC, admission precision, and coverage, and compare it with entropy-, margin-, and distance-based routing. Does CRSI provide additional signal, and how does batch size affect it?

Thank you for this suggestion. We evaluated CRSI against entropy-, margin-, and distance-based routing on identical pre-update Sinkhorn Top-1 pairs across all 29 ScanRefer–CrossOver corruption conditions and three query orders. Correctness was determined by scene identity, with the same admission budget for all criteria.

Routingt2s AUROCt2s Precisions2t AUROCs2t Precision
Entropy0.583325.24%0.52753.01%
Margin0.609626.00%0.558556.33%
Candidate L20.639129.38%0.599760.35%
CRSI0.633628.93%0.615161.47%

CRSI improves admission precision over entropy by 3.69 pp in t2s and 8.46 pp in s2t, with scene-paired 95% CIs of [+2.92, +4.49] and [+5.16, +11.55] pp, respectively. While margin performs better in t2s, CRSI provides competitive structural reliability assessment, particularly in s2t.

To examine whether CRSI provides information beyond entropy, we further stratified candidates into four entropy quartiles and compared the high- and low-CRSI halves within each quartile. High-CRSI candidates consistently exhibit higher match precision, with gains of 8.42–10.58 pp in t2s and 7.77–19.21 pp in s2t; all eight scene-paired 95% CIs are above zero. This confirms that CRSI provides complementary ranking information among candidates with similar entropy.

Batch-size sensitivity. Using fixed source checkpoints, candidate identities, and query orders, we varied batch-size from 8 to 64 under matched admission coverage.

Batch SizeOCR S7 AUROC / PrecisionGaussian S5 AUROC / Precision
80.6573 / 19.98%0.5823 / 51.81%
160.6746 / 21.51%0.6062 / 55.62%
320.6797 / 22.67%0.6095 / 56.76%
640.6857 / 23.22%0.6110 / 55.24%

CRSI maintains discriminative capability across batch sizes, while admission precision exhibits corruption-dependent sensitivity. We will incorporate these analyses and clarify the corresponding evaluation protocols in the revision.

[W4] Numerical consistency: Please check the attribution of 23.55 and the 1.28-point gain in the introduction, and the descriptions of wins and losses for Tables 11, 12, and 15 in Appendix C.3.2. Are these prose errors or mislabeled results?

Thank you for carefully identifying these inconsistencies. We sincerely apologize for the inaccurate descriptions in the Introduction and Appendix C.3.2. These are writing and attribution errors introduced during manuscript revision, not errors in the experimental tables. Specifically, the reported value of 23.55 and the corresponding 1.28-point gain were assigned to an incorrect backbone and direction in the Introduction. In addition, several win or loss statements for Tables 11, 12, and 15 were not updated to match the revised numerical results.

We have cross-checked these tables against our experimental records and corrected all affected textual descriptions. The numerical values within the tables remain unchanged. We appreciate the reviewer’s feedback, which helps us improve the accuracy and consistency of our manuscript.

[W5] Deployment evidence is limited to synthetic corruptions of ScanNet-derived indoor scenes, without evaluation of real acquisition failures, long-stream drift, or large galleries.

Thank you for raising this concern. M3SP is designed as a controlled and reproducible benchmark for studying the robustness of text–3D scene retrieval, rather than an exhaustive simulation of real-world acquisition failures. Specifically, it applies 14 manually designed perturbation types at the point, object, and scene levels, each with five severity levels, to real ScanNet-derived scenes across four retrieval datasets. By retaining the underlying real-scene content while controlling the type and severity of perturbations, M3SP allows us to systematically investigate how different 3D distribution shifts affect retrieval performance.

We acknowledge that real-world acquisition failures, extended-stream drift, and large-scale gallery deployment involve additional factors beyond the current benchmark. We will clarify the scope of M3SP, and for future work, we plan to extend our evaluation to larger datasets and assess STAR-3D on longer query streams and larger retrieval galleries.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.