acceptodds
Under review as a conference paper at ICLR 2027

Target-Centric Frequency Distillation: Learning Spectrally Aligned Representations for RGB-T Visual Grounding

Abstract

RGB-Thermal (RGB-T) visual grounding localizes a text-referred object in paired RGB and infrared (IR) images. However, RGB-centric vision-language models provide limited IR semantic modeling, hindering the integration of IR cues into query-conditioned representations. To address this, we propose Target-Centric Frequency Distillation (TCFDistill), a framework for learning spectrally aligned representations for robust grounding. Our key insight is that query-relevant RGB–IR structures exhibit distinct frequency characteristics: low frequencies capture slowly varying, coarse-scale spatial structures, while high frequencies reflect fine-grained local variations. Specifically, we introduce Dual-Frequency Spectral Distillation (DFSD), which refines IR representations through structural knowledge transfer from a pretrained RGB–IR teacher model. DFSD aligns query-conditioned low-frequency target-affinity relations and high-frequency RGB–IR correspondences at matched positions and within local neighborhoods. Furthermore, Spectral Alignment-Guided Representation Modulation (SARM) uses local cross-modal frequency consistency to modulate shared structures and complementary responses between RGB and refined IR representations, producing spatially organized frequency tokens for grounding. Experiments on RefFLIR, RefMFD, and RefMFAD demonstrate state-of-the-art performance, surpassing RGBT-VGNet by 2.79% on RefMFD Test-B and 4.54% on RefMFAD Test-C. All code will be released upon acceptance.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
RGBT, RGB-T Visual Grounding, Frequency, Knowledge Distillation

Reviews

Among:
Shape:
1234this paper, Top 15%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.00 over 3 reviews, above 85% of the others.
Reviewer_PvMcOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes Target-Centric Frequency Distillation (TCFDistill) for RGB-T Visual Grounding. The method consists of Dual-Frequency Spectral Distillation (DFSD) to transfer structural relations from an RGB-IR teacher model, and Spectral Alignment-Guided Representation Modulation (SARM) to fuse cross-modal features using local frequency consistency. The approach achieves state-of-the-art results on three benchmarks.

Critical strengths
  1. The topic is well-motivated, and addressing the limited IR semantic modeling in RGB-centric VLMs is highly relevant.
  2. The proposed DFSD is an elegant alternative to direct feature matching.
  3. The paper is well-written, and the experimental evaluation is comprehensive.
Critical weaknesses
  1. The paper broadly assumes that low frequencies capture coarse-scale structures, while high frequencies reflect fine-grained variations and applies this uniformly to both RGB and IR images. However, RGB and IR have fundamentally different imaging physics. The paper fails to analyze or discuss their distinct frequency distributions.
  2. In the SARM module, the authors use a consistency map to weight the RGB and refined IR spectral streams. However, there is no theoretical analysis of why and how they are complementary in the frequency domain. Without understanding their modality-specific frequency traits, the design of SARM appears additional learnable module.
  3. Real-world RGB-T systems frequently encounter modality missing. The proposed method mathematically relies on the simultaneous presence of both modalities to compute cross-modal frequency correspondence and consistency, lacking analysis of this complex scenario.
  4. While the authors claim to address the limited IR semantic modeling of VLMs via TCFDistill, this approach merely acts as a statistical shortcut. It relies heavily on the structural priors of a specific RGB-IR pretrained teacher (UNIV) rather than genuinely modeling the unique thermal semantics of IR images in the frequency domain.
  5. The table formats are inconsistent throughout the manuscript and appendix. Unlike standard grounding benchmarks (e.g., RefCOCO), the paper fails to report an Average metric across the 15 evaluation splits, forcing readers to manually aggregate the results to assess comprehensive performance. For example, in Table 8, the average score for LF-only is ~73.7%, HF-only is ~74.2%, and Full DFSD is ~74.6%. More critically, Full DFSD fails to achieve the best performance in nearly one-third of the splits, and on the Val sets of all three datasets, the Full model performs worse than at least one single-frequency variant. This indicates potential negative transfer between frequency bands.
Questions for authors

See weaknesses.

8 comments

We sincerely appreciate the careful and constructive suggestions from Reviewer PvMc. We will explain the major concerns point-by-point as follows:

Q1. Thanks for highlighting this issue. Fig. 1(b) illustrates the spatial content of the low- and high-frequency components of RGB and IR images. To further characterize the cross-modal differences, we have added Fig. 1(c), presenting normalized two-dimensional Fourier spectra and radial RGB–IR spectral differences in dB. The comparison shows relatively stronger low-frequency responses in IR and stronger high-frequency responses in RGB.

We further provide additional examples in Fig. 6. The left panels visualize paired RGB–IR images and their two-dimensional spectra. In the campus example, RGB spectral responses extend farther toward the high-frequency periphery, whereas IR responses are more concentrated around the low-frequency center. The right panels present radial spectral differences for a separate set of image pairs, illustrating how relative RGB–IR responses vary across frequency bands. Together, these observations clarify the modality-specific frequency characteristics and motivate separate modeling of cross-modal structural relations across spatial scales.

We have revised the Introduction to explain this connection. Appendix A.4, “Image Spectral Analysis,” details the computation, and Appendix B.1, “RGB–IR Spatial-Frequency Distributions,” and Fig. 6 provide the additional examples and analysis.

The corresponding revisions will be highlighted in red at Lines 80–82, 89–98, 787–826, and 882–898 of the manuscript.

Q2. Thanks for raising this concern. As discussed in Q1, Fig. 1(b) illustrates the spatial content of different frequency components, while the added Fig. 1(c) shows modality-specific frequency distributions. Related studies also exploit RGB–IR frequency complementarity through wavelet fusion (WaveMamba [1]), frequency-driven feature decomposition (FD²-Net [2]), and dual-frequency enhancement (DCFusion [3]). These observations and prior studies motivate modeling shared structures and modality-specific responses in the frequency domain.

We have strengthened this rationale at the beginning of Section 3.3. SARM measures cross-modal agreement at corresponding spatial locations and local DCT frequencies using channel-wise cosine similarity. The consistency weight emphasizes RGB components shared with IR, while its complementary weight emphasizes discrepant responses from the DFSD-refined IR stream. These responses offer potentially complementary information, whose relevance to localization is learned through subsequent modulation under the grounding objective. This explains the two-stream weighting and its local frequency-dependent operation.

Moreover, the ablations with the same number of parameters in Appendix D.3 and Table 10 show that full SARM exceeds the spatial-neighborhood and IR-only variants by 0.98 and 1.23 percentage points, respectively, averaged over the 12 test splits. These comparisons demonstrate the benefit of local cross-modal spectral interaction beyond increasing model capacity. We have revised the Introduction and Section 3.3 to clarify the motivation and mechanism.

[1] WaveMamba: Wavelet-Driven Mamba Fusion for RGB-Infrared Object Detection, ICCV 2025.

[2] FD²-Net: Frequency-Driven Feature Decomposition Network for Infrared-Visible Object Detection, AAAI 2025.

[3] DCFusion: A Dual-Frequency Cross-Enhanced Fusion Network for Infrared and Visible Image Fusion, IEEE TIM 2023.

The corresponding revisions will be highlighted in red at Lines 89–98, 299–310, 972–981, and 1002–1017 of the manuscript.

Q3. Thanks for raising this concern. Our cross-modal correspondence and consistency modeling assumes paired RGB–IR observations. We have added missing-modality results and analysis across all three datasets in Appendix D.10, “Sensitivity to Missing Modalities,” and Table 20. Without retraining, we simulate modality absence by replacing the RGB or IR input with zeros after normalization.

On RefFLIR and RefM³FD Test, the model achieves 66.83% and 67.43% accuracy with IR missing, compared with 58.13% and 63.64% with RGB missing, respectively. Across all three datasets, Test-A accuracy is higher with IR missing, indicating greater dependence on RGB under normal illumination. On Test-B, IR becomes more important on RefFLIR and RefM³FD; on RefMFAD, accuracy remains higher with IR missing (61.99%) than with RGB missing (57.22%). Under missing inputs, Test-C accuracy ranges from 36.41% to 47.29%. These results quantify sensitivity to modality absence across datasets and test subsets and clarify the practical limitations of the paired-input design.

Results are [email protected] (%).

DatasetInput conditionValTestTest-ATest-BTest-C
RefFLIRRGB + IR (reference)77.1375.0091.8869.8455.57
RefFLIRMissing IR65.1366.8389.6152.9741.99
RefFLIRMissing RGB62.6658.1375.1561.4136.41
RefM³FDRGB + IR (reference)77.3877.3395.4583.5458.87
RefM³FDMissing IR65.4867.4393.3467.7342.69
RefM³FDMissing RGB55.3663.6478.8173.9544.81
RefMFADRGB + IR (reference)72.0570.4091.6467.3355.72
RefMFADMissing IR65.9265.0891.2561.9947.29
RefMFADMissing RGB60.6758.9881.8857.2242.59

Full-input rows reproduce the main-table results for reference.

The corresponding revisions will be highlighted in red at Lines 1242–1248 and 1279–1289 of the manuscript.

Q4. Thanks for raising this concern. Our claim concerns improving query-relevant IR representations for grounding: text-conditioned targets and grounding supervision provide semantic guidance, while frequency decomposition organizes structural transfer across spatial scales.

Regarding the “statistical shortcut” concern, Appendix D.6, “Additional Analysis of Query Guidance,” and Table 16 report that spatially shuffling LF-TAD query weights, while preserving their distribution and retaining the same reinjection gate, reduces Avg.12 from 74.38% to 73.20%. This control demonstrates the importance of the correspondence between query weights and image locations, beyond the weight distribution alone.

Regarding dependence on UNIV, Appendix D.7, “Additional Analysis of Teacher Selection,” and Table 17 report Avg.12 scores of 74.38%, 73.63%, and 73.17% with UNIV, M-SpecGene, and InfMAE, respectively. These results show that the framework can use different teachers, while teacher selection affects transfer effectiveness. Our contribution is to transfer query-relevant structural knowledge across frequency bands, with the query-weight shuffling control assessing the spatial relevance of this guidance.

The corresponding revisions will be highlighted in red at Lines 1198–1204 and 1208–1220 of the manuscript.

Q5. Thanks for highlighting the need for consistent presentation and explicit aggregate metrics. We have standardized the presentation of the multi-dataset ablation tables and added Avg.12 and Avg.15 to report aggregate performance. As defined in Appendix A.1, these are unweighted means over the 12 test columns and all 15 evaluation columns, including Val, respectively. For the dual-frequency ablation in Appendix D.2 and Table 9, the results are:

VariantAvg.12 (%)Avg.15 (%)
LF-only73.1873.65
HF-only73.7474.20
LF + HF, MSE73.4273.79
Full DFSD74.3874.61

Regarding potential negative transfer, under the same SmoothL1 objective, Full outperforms both single-frequency variants on the overall Test and Test-C splits of all three datasets. Among these SmoothL1 variants, Full achieves the highest accuracy on 11 of the 12 test columns, sharing the best result with HF-only on RefMFAD Test-B. The exception is RefFLIR Test-B, with 69.84% versus 70.15% for HF-only. The RefM³FD Test-A comparison against the MSE variant changes the regression loss while retaining both frequency objectives, so it does not isolate the effect of combining frequency bands.

The lower validation scores indicate that improvements vary across splits, but these single-run comparisons alone do not establish systematic interference between the frequency objectives. Considering aggregate scores together with results on different test subsets provides a more comprehensive assessment of overall performance. Full DFSD’s highest Avg.12 and Avg.15, together with its improvements on the overall Test splits, demonstrate the overall benefit of joint frequency supervision.

The corresponding revisions will be highlighted in red at Lines 737–740 and 930–937 of the manuscript.

Reviewer_H37QOpenReview ↗
Rating
4(+1)
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper addresses RGB-Thermal (RGB-T) visual grounding, i.e., localizing a text-referred object in paired RGB and infrared (IR) images. The starting observation is that grounding backbones are RGB-pretrained VLMs (CLIP), whose IR representations are weakly structured and poorly aligned with text. The proposed framework, TCFDistill, injects this teacher structure through two components built on the HiVG grounding model with frozen CLIP ViT-B/16: (1) DFSD (Dual-Frequency Spectral Distillation), which distills relations (not feature values) from the frozen teacher into a low-rank student "IR refinement adapter", (2) SARM, which extracts 3×3 local DCT coefficients of projected RGB and refined-IR features, computes channel-wise cross-modal consistency per frequency, and modulates the two spectral streams (consistent part from RGB, discrepant part from IR) before IDCT reconstruction, yielding extra "frequency tokens" concatenated into the grounding transformer. Experiments on the three RGBT-GroundBench datasets (RefFLIR, RefM3FD, RefMFAD) report state-of-the-art results, outperforming the strongest prior method RGBT-VGNet.

Critical strengths
  1. Well-motivated core design. The decision to transfer relations rather than match features is the right call given the heterogeneous spaces involved (UNIV vs. CLIP), and the paper validates this choice against direct and frequency-decomposed feature regression (Tables 2, 11).
  2. The authors conducted extensive ablation experiments, and the results are consistent with their claims: Gains are consistent across all 15 splits and are largest exactly where IR should help most (Test-B night scenes: avg +1.98%; Test-C small targets: avg +3.64%; up to +4.54% on RefMFAD Test-C).
Critical weaknesses
  1. Generalization evaluation problem. All three datasets come from a single benchmark (RGBT-GroundBench) and share the same driving/pedestrian domain; the only direct RGB-T grounding competitor is RGBT-VGNet. There is no cross-domain evaluation (e.g., training on one dataset and testing on another, or any other RGB-T task), so generalization of both the method and its hyperparameter choices (c = 0.5, K = 3, τ = 0.07, loss weights) is unknown.

  2. Single-run results; no variance or significance analysis. Several decisive ablation deltas are small: SmoothL1 vs. MSE +0.31%, the pure KD effect (D† vs. full) +0.68%. For 120-epoch grounding training with ~2k-instance test subsets, seed-to-seed variance is plausibly of this order. Without multi-seed runs or error bars, the fine-grained conclusions in section 4.3 (e.g., that the specific frequency assignment matters) rest on differences that may not be statistically meaningful.

  3. Inconsistent training-protocol comparison. TCFDistill uses a new 120-epoch single-stage protocol, while Table 1 compares against published baseline numbers trained with their original schedules (e.g., HiVG's three-stage adaptation). The authors' own retrained baseline (Table 7) shows the protocol change alone is worth a lot on some splits: on RefFLIR Test, published MV-HiVG-B is 69.20, the retrained same-protocol baseline is 72.14 (+2.94), and the full model 75.00; on RefMFAD Val/Test, published MV-HiVG-B is 67.07/67.04 vs. the retrained baseline 70.78/69.13. Thus, of the +5.80 over MV-HiVG-B on RefFLIR Test, roughly half is attributable to the training protocol rather than DFSD/SARM.

Questions for authors

See weaknesses above.

4 comments

We sincerely appreciate the careful and constructive suggestions from Reviewer H37Q. We will explain the major concerns point-by-point as follows:

Q1. Thanks for raising this concern. RGBT-GroundBench integrates three distinct source datasets: FLIR covers daytime and nighttime driving scenes; M3FD includes roads, campuses, and waterfronts; and MFAD covers urban environments such as tunnels, parks, and residential areas.

Beyond separate evaluation on each dataset, we have added six cross-dataset transfer evaluations, directly testing each source-trained model on the other two datasets. Checkpoints are selected using the source validation set, and source preprocessing is retained, with no target-specific fine-tuning or hyperparameter adjustment. The transfer checkpoints use c = 0.5, K = 3, τ = 0.07, and an HF local-loss weight of 0.5, held fixed across the transfer directions.

For example, the model trained on RefFLIR achieves 61.10% and 51.60% Test accuracy on RefM³FD and RefMFAD, respectively, with corresponding Avg.4 scores of 63.50% and 53.41%. Across all six directions, Test-A accuracy ranges from 77.44% to 87.82%, while Test-C accuracy ranges from 28.50% to 36.20%, showing that small-target transfer remains challenging. These results provide evidence of grounding capability across RGB-T data sources without target adaptation.

We also include RGBT-GroundBench’s published zero-shot results from RGB-pretrained models without RGB-T training for reference. The transfer evaluations, zero-shot references, and accompanying analysis are provided in Appendix D.14, “Cross-Dataset Generalization,” and Table 24.

Cross-dataset results are [email protected] (%) for seed 13. Avg.4 averages Test, Test-A, Test-B, and Test-C on the target dataset.

Training sourceEvaluation targetTestAvg.4
RefFLIRRefM³FD61.1063.50
RefFLIRRefMFAD51.6053.41
RefM³FDRefFLIR58.8957.62
RefM³FDRefMFAD49.3651.01
RefMFADRefFLIR55.6153.84
RefMFADRefM³FD59.3261.61

The corresponding revisions will be highlighted in red at Lines 1383–1393 and 1404–1428 of the manuscript.

Q2. Thanks for raising this concern. We have added multi-seed evaluations on RefM³FD, reporting mean ± sample SD for the full model and the relevant ablations.

Compared with the full model trained without KD, TCFDistill improves Test accuracy from 76.21 ± 0.61% to 77.24 ± 0.12%, and Avg.4 from 77.79% to 78.71%. For the regression loss, SmoothL1 increases Avg.4 from 77.64% to 78.71% compared with MSE, while exhibiting lower SD on all four test subsets. For frequency assignment, the proposed configuration achieves 78.71% Avg.4, compared with 77.60% for Full-spectrum and 77.34% for Swapped.

The full model achieves higher mean accuracy on all four test subsets than these variants. These repeated evaluations demonstrate improvements in mean accuracy from distillation, the regression loss, and frequency assignment while explicitly quantifying training variability. We have added the results and accompanying analysis to Appendix D.13, “Multi-seed Evaluation,” and Table 23.

Results on RefM³FD are [email protected] (%, mean ± sample SD). Avg.4 averages the mean accuracies across the four test subsets.

VariantTestTest-ATest-BTest-CAvg.4
Full w/o KD76.21 ± 0.6195.14 ± 0.6682.58 ± 0.4257.24 ± 0.6777.79
Full with MSE76.09 ± 1.2395.39 ± 0.3482.47 ± 1.4056.61 ± 2.2577.64
Full-spectrum76.08 ± 0.8095.29 ± 0.2182.39 ± 0.3556.64 ± 1.5577.60
Swapped75.77 ± 1.0895.16 ± 0.6682.11 ± 1.5356.31 ± 1.4877.34
Full TCFDistill77.24 ± 0.1295.69 ± 0.2683.21 ± 0.4358.71 ± 0.2478.71

The corresponding revisions will be highlighted in red at Lines 1348–1379 of the manuscript.

Q3. Thanks for highlighting this concern. We evaluated TCFDistill following HiVG’s multi-stage training strategy, using the same COCO-pretrained HiVG checkpoint for initialization. TCFDistill achieves 72.51% Avg.12, compared with 70.10% for published MV-HiVG-B. The complete results and analysis are provided in Appendix D.12 and Table 22. Furthermore, RGBT-VGNet also uses single-stage training. Under our default single-stage setting, TCFDistill achieves 74.38% Avg.12, exceeding RGBT-VGNet’s 72.00%.

To assess the contribution of SARM and DFSD under a fixed training protocol, we compare against the retrained HiVG baseline in Table 1 and Appendix D.1. TCFDistill improves Avg.12 from 72.28% to 74.38% and RefFLIR Test accuracy from 72.14% to 75.00%, yielding controlled gains of 2.10 and 2.86 percentage points, respectively. The larger 5.80-point difference from published MV-HiVG-B additionally reflects changes in training settings. The initialization and training protocols are described in Section 4.1 and Appendix A.1, with the controlled gains discussed in Section 4.2.

[email protected] (%). Avg.12 averages the 12 test columns across the three datasets.

MethodTraining protocolRefFLIR TestRefM³FD TestRefMFAD TestAvg.12
MV-HiVG-B (published)Original multi-stage69.2072.3567.0470.10
TCFDistillHiVG multi-stage73.7275.2667.6272.51
RGBT-VGNet (published)Single-stage72.4374.8967.5872.00
HiVG baseline (retrained)120-epoch single-stage72.1475.1369.1372.28
TCFDistill120-epoch single-stage75.0077.3370.4074.38

The corresponding revisions will be highlighted in red at Lines 400–401, 423–426, 730–735, 954–958, 1308–1314, and 1335–1344 of the manuscript.

Thank the authors for the detailed additional experiments, and the authors' response addresses my primary concern, and I will raise my score accordingly.

We sincerely thank Reviewer H37Q for the positive feedback. We are glad that our additional experiments and clarifications have addressed your primary concern.

If Reviewer H37Q has any further questions or concerns, please feel free to raise them before the rebuttal period concludes. We will carefully consider and thoroughly address any additional concerns.

Once again, we sincerely appreciate your valuable time and effort, responsible and thorough review, and constructive suggestions. Thank you.

Reviewer_SPisOpenReview ↗
Rating
3
Confidence
4
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

This paper proposes Target-Centric Frequency Distillation (TCFDistill) for RGB–thermal visual grounding. The method builds on HiVG and introduces two components. Dual-Frequency Spectral Distillation (DFSD) transfers structural relations from a pretrained RGB–IR teacher to an IR refinement adapter: low-frequency supervision aligns query-conditioned target-affinity maps, while high-frequency supervision aligns cross-modal correspondence scores within target regions. Spectral Alignment-Guided Representation Modulation (SARM) performs local DCT-based fusion using cross-modal frequency consistency. Experiments on RefFLIR, RefM³FD, and RefMFAD show consistent improvements over the reported baselines, particularly on small-target subsets.

Critical strengths
  1. Relevant problem and coherent design. The paper addresses the limited thermal representation capability of RGB-centric vision–language models. Transferring relations instead of directly matching heterogeneous teacher and student features is well motivated. The combination of query-conditioned target affinities and local cross-modal correspondences provides a task-specific formulation of frequency-aware distillation.

  2. Consistent benchmark improvements. TCFDistill outperforms RGBT-VGNet across all 12 reported test settings. Overall Test accuracy improves by 2.57, 2.44, and 2.82 percentage points on RefFLIR, RefM³FD, and RefMFAD, respectively. The improvements on Test-C are particularly encouraging.

  3. Substantial ablation coverage. The supplementary material includes several useful controls beyond component removal: the full architecture without distillation, parameter-matched spatial processing in place of DCT, full-spectrum and swapped-frequency objectives, shuffled query weights, and alternative teachers. These experiments strengthen the evidence that the proposed design choices contribute to performance.

Critical weaknesses
  1. The statistical reliability of the distillation-specific gains is unclear. Table 7 appropriately retains the entire architecture while disabling the distillation loss. Under this control, enabling DFSD improves overall Test accuracy by 0.79, 0.52, and 0.19 percentage points on the three datasets, with an average improvement of approximately 0.68 points across the 12 test settings. Since distillation is a central contribution, its relatively modest incremental gains should be distinguished from the larger gains of the complete architecture.

  2. The comparison against the strongest baseline does not fully isolate methodological improvements from training changes. Appendix A.1 states that the proposed method uses a single-stage 120-epoch schedule instead of the original staged HiVG schedule. Retraining HiVG under the new protocol is a useful control, but an equivalent comparison is not reported for RGBT-VGNet. This matters because, on RefM³FD, the retrained HiVG baseline reaches 75.13 Test accuracy, compared with 72.35 for the published MV-HiVG-B result and 74.89 for RGBT-VGNet. The internal ablations support the proposed modules, but the attribution of the improvement over the strongest external baseline remains less clear.

  3. The empirical distinction from existing frequency-aware distillation methods could be stronger. FreeKD already explores frequency-selective relational transfer, while FD-CMKD studies separate treatment of low- and high-frequency information in cross-modal distillation. It seems the distinctive contribution here is the target-conditioned relation formulation.

Questions for authors
  1. Can you report results over multiple training seeds for the full model, the full architecture without KD, and the principal frequency/spatial controls? Are the distillation-specific improvements statistically reliable?

  2. Can RGBT-VGNet be evaluated under the same initialization and training protocol? Please also clarify the training history of the initial HiVG RGB-T checkpoint and the resulting cumulative training budget.

  3. Does DFSD retain its advantage when feature-regression baselines receive equivalent query weighting and target-region masking? How does it compare with adapted versions of FreeKD [1] or FD-CMKD [2] under the same teacher and student setting?

[1] FreeKD: Knowledge Distillation via Semantic Frequency Prompt. CVPR 2024.

[2] Distilling Cross-Modal Knowledge via Feature Disentanglement. AAAI 2026.

3 comments

We sincerely appreciate the careful and constructive suggestions from Reviewer SPis. We will explain the major concerns point-by-point as follows:

Q1. Thanks for raising this concern. We report multi-seed results for Full TCFDistill, the full architecture without KD, LF-only, HF-only, and Spatial SARM on RefM³FD, as mean ± sample SD. Compared with the full architecture without KD, TCFDistill improves Test accuracy from 76.21 ± 0.61% to 77.24 ± 0.12%, with gains of 0.55, 0.63, and 1.47 percentage points on Test-A, Test-B, and Test-C, respectively. Avg.4 increases from 77.79% to 78.71%, also exceeding LF-only (77.72%), HF-only (78.23%), and Spatial SARM (77.84%). These results show consistent improvements in mean accuracy from distillation across the four test subsets while quantifying training variability. We have added the comparison and analysis to Appendix D.13, “Multi-seed Evaluation,” and Table 23.

[email protected] (%) on RefM³FD. Test-split scores are mean ± sample SD; Avg.4 averages the four test-split means.

VariantTestTest-ATest-BTest-CAvg.4
Full w/o KD76.21 ± 0.6195.14 ± 0.6682.58 ± 0.4257.24 ± 0.6777.79
LF-only76.12 ± 0.2495.51 ± 0.3782.43 ± 0.1656.81 ± 0.2777.72
HF-only76.77 ± 0.2995.29 ± 0.2682.59 ± 0.4558.26 ± 0.4278.23
Spatial SARM76.36 ± 0.2495.18 ± 0.5582.48 ± 0.2657.32 ± 0.1177.84
Full TCFDistill77.24 ± 0.1295.69 ± 0.2683.21 ± 0.4358.71 ± 0.2478.71

Q2. Thanks for highlighting this concern. Following the initialization protocol of RGBT-GroundBench, both TCFDistill and RGBT-VGNet initialize from single-modality RGB grounding checkpoints pretrained on COCO, and both use single-stage training for RGB-T grounding. TCFDistill starts from the COCO-pretrained HiVG checkpoint and is trained for 120 epochs on each RGB-T dataset. These 120 epochs refer to RGB-T training after initialization. We have clarified this sequence in Section 4.1 and Appendix A.1. The RGBT-VGNet results in Table 1 are its published results.

To assess the contribution of our components under a fixed training protocol, we have added the retrained HiVG baseline to Table 1. On RefM³FD Test, TCFDistill achieves 77.33%, compared with 75.13% for this baseline, a 2.20-point improvement under the same 120-epoch protocol. Across the 12 test splits, accuracy increases from 72.28% to 74.38%. These controlled comparisons assess the contribution of SARM and DFSD, with the complete analysis in Appendix D.1.

[email protected] (%) under the same 120-epoch training protocol. Avg.12 averages Test, Test-A, Test-B, and Test-C across all three datasets.

MethodRefFLIR TestRefM³FD TestRefMFAD TestAvg.12
HiVG baseline (retrained)72.1475.1369.1372.28
TCFDistill75.0077.3370.4074.38
Improvement (percentage points)+2.86+2.20+1.27+2.10

The corresponding revisions will be highlighted in red at Lines 365–373, 400–401, 709–713, 730–735, and 957–958 of the manuscript.

Q3. Thanks for raising this concern. DFSD retains its advantage over feature regression using the same frequency decomposition, query weighting, and target-region masking. On RefM³FD, it improves Test accuracy from 75.93% to 77.33%, while mean accuracy across the four test subsets increases from 77.40% to 78.80%.

We further compare against adapted FreeKD and FD-CMKD, following their original distillation formulations under the same UNIV teacher and student architecture. DFSD achieves 78.80% mean test accuracy, exceeding FreeKD (78.20%) and FD-CMKD (76.94%) by 0.60 and 1.86 percentage points, respectively. These comparisons demonstrate the effectiveness of DFSD’s frequency-specific relation modeling. We have added the comparison and analysis to Section 4.3, “Comparison with Alternative Distillation Objectives,” and Table 7. The current comparison covers RefM³FD; more complete evaluations across all three datasets will be included in the revised manuscript.

[email protected] (%) on RefM³FD. Avg.4 averages the four test subsets. Frequency feature regression uses the same query weighting and target-region masking as DFSD.

MethodTestTest-ATest-BTest-CAvg.4
Frequency feature regression75.9395.2182.3656.1177.40
FreeKD76.8895.3282.3658.2378.20
FD-CMKD75.5395.3881.5455.3076.94
DFSD (Ours)77.3395.4583.5458.8778.80

The corresponding revisions will be highlighted in red at Lines 481–493 and 509–512 of the manuscript.

The authors have largely addressed my concerns, and I have no further questions. I will maintain my positive rating and lean toward acceptance.

We are grateful to Reviewer SPis for the positive assessment and continued support for acceptance. We are pleased that our additional experiments and clarifications have addressed your concerns.

If Reviewer SPis has any further questions or concerns, please feel free to raise them before the rebuttal period concludes. We will carefully consider and thoroughly address any additional concerns.

We sincerely thank you for your responsible and thorough review, constructive suggestions, and the time devoted to evaluating our work. Your feedback has helped us strengthen the experimental evidence and improve the clarity of the manuscript.

Comments on the paper

2 comments

Dear Reviewers, Area Chairs, Senior Area Chairs, and Program Chairs,

We sincerely thank you for your valuable and thoughtful comments. We are pleased that our work has been recognized by the reviewers for its relevant motivation and well-founded relation-based design, its consistent benchmark improvements, and its extensive ablation coverage.

The reviewers’ main concerns relate to the explanation of RGB–IR frequency characteristics and SARM’s design, as well as experimental validation and comparison settings. In response, we have strengthened the analysis of modality-specific RGB–IR frequency characteristics through additional spectral visualizations (Introduction and Appendix B.1) and clarified SARM’s design rationale and local spectral interaction mechanism, supported by relevant literature and controlled ablations (Section 3.3 and Appendix D.3). We have also expanded the experimental evaluation with multi-seed results (Appendix D.13), additional training-protocol and distillation comparisons (Sections 4.1–4.3 and Appendix D.12), cross-dataset transfer evaluations (Appendix D.14), and missing-modality analyses (Appendix D.10).

To address these concerns, we have provided detailed point-by-point explanations in each reviewer’s response box and included the additional experimental results in the rebuttal PDF. Furthermore, the manuscript has been carefully revised according to the reviewers’ suggestions, with the corresponding changes highlighted in red.

The following are our detailed responses. We greatly appreciate the constructive suggestions that have helped improve the quality of our paper.

Best regards,

The Authors.

Dear Reviewers PvMc, H37Q, and SPis, Area Chairs, Program Chairs, and Senior Program Committee Members,

As the rebuttal period is drawing to a close. We would like to summarize our revisions and the follow-up discussion for the committee’s final assessment.

We are pleased that our responses received positive feedback from Reviewers H37Q and SPis. Reviewer H37Q raised their rating from Weak acceptance (3) to Clear acceptance (4). Reviewer SPis stated that we had largely addressed their concerns, that they had no further questions, and that they would maintain their positive rating and lean toward acceptance. Based on this feedback, we believe that our revisions have addressed most of the concerns raised during the review process.

As for Reviewer PvMc, in November 15 follow-up, Reviewer PvMc recognized our efforts to address their concerns through detailed explanations and experiments and requested RGB+RGB and IR+IR inference comparisons. We provided the results later that day (Table 21, Appendix D.10). The reviewer has raised no further questions or concerns since then. Given our point-by-point responses and the additional results, we believe we have addressed most of the concerns they raised.

Thank you again to all reviewers and committee members for the time and effort you have devoted to evaluating our work. Your constructive suggestions have helped us strengthen the experiments and improve the manuscript.

Best regards,
The Authors.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.