acceptodds
Under review as a conference paper at ICLR 2027

PSDFusion: Prior-Guided Feature Specialization with Cross-Paradigm Distillation for Infrared-Visible Image Fusion

Abstract

Infrared-visible image fusion aims to integrate thermal saliency with visible details while preserving representations useful for downstream perception. Existing prior-guided methods often inject semantic priors into shared multimodal features and transfer teacher knowledge mainly through Euclidean feature matching, which may interfere with low-level visual representation learning and restrict heterogeneous knowledge transfer. We propose PSDFusion, a Prior-guided Specialization and Distillation framework that separates visual specialization from semantic conditioning and enriches student representations across complementary geometries. Specifically, PSDFusion first constructs local-detail and context-preserving representations through asymmetric transformations and cross-gating, followed by text-guided semantic conditioning. It further combines Euclidean and hyperbolic-aware student representations before aligning their channel dependencies with multi-level teacher features in a unified correlation space. Experiments on four infrared-visible fusion benchmarks and two downstream perception tasks against 12 recent baselines demonstrate competitive fusion quality, cross-dataset generalization, and strong task utility. These results support feature specialization and cross-paradigm distillation as complementary mechanisms for balancing visual fidelity and semantic representation learning. The code is publicly available at https://anonymous.4open.science/r/PSDFusion-0CC1.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
Multimodal Representation Learning, Feature Specialization, Knowledge Distillation, Hyperbolic Representation Learning, Infrared-Visible Image Fusion

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_vQwtOpenReview ↗
Rating
4(+1)
Confidence
4
Informativeness
3
Presentation
3
Soundness
3
Read review
Paper summary

This paper proposes PSDFusion, a prior-guided framework for infrared-visible image fusion. The method introduces feature specialization to construct complementary local-detail and context-preserving representations before semantic conditioning. It further employs cross-paradigm distillation to integrate Euclidean and hyperbolic-aware representations under multi-level teacher supervision. Experiments on four fusion benchmarks and two downstream perception tasks demonstrate competitive fusion quality and generalization performance.

Critical strengths
  1. Well-motivated framework. The paper addresses meaningful limitations in semantic-guided fusion and knowledge distillation. The integration of feature specialization and cross-paradigm distillation provides a coherent approach to improving multimodal representation learning.
  2. Comprehensive empirical validation. The method is evaluated against 12 representative baselines across four benchmarks, with additional semantic segmentation and object detection experiments. The strong results on MSRS and competitive cross-dataset performance support the practical value of the proposed framework.
Critical weaknesses
  1. Limited explanation of geometric benefits. Although the cross-paradigm design is interesting, the specific advantage of the hyperbolic-aware pathway over conventional Euclidean refinement could be explained more clearly, particularly since spectral interactions are performed in the tangent space.
  2. Trade-offs in the ablation results. Table 3 shows that removing PFS or CPD improves certain metrics, particularly MI and IFC. Further clarification of these trade-offs would strengthen the justification for combining the two components.
Questions for authors
  1. Could the authors clarify the specific role of the hyperbolic-aware representation, especially when spectral operations are performed in the Euclidean tangent space?
  2. How do the authors interpret the higher MI and IFC values achieved by some ablated variants in Table 3? Does the full model primarily emphasize structural and edge preservation?
  3. Could the authors clarify whether the reported inference time includes text encoding and confirm the role of the teacher network during inference?
  4. Could the authors clarify why three distillation levels are selected and how this choice affects fusion performance?
  5. Could the authors explain why scene-level captions outperform generic prompts on most structural metrics, as shown in Table 10?
6 comments
AuthorsOpenReview ↗

Response to Q1: Role of hyperbolic-aware representation.

We thank the reviewer for the careful evaluation and constructive feedback. We address these concerns below.

The tangent space provides Euclidean coordinates for FFT, but this does not mean the entire hyperbolic-aware pathway reduces to Euclidean spectral processing. Beyond coordinate mappings, this pathway incorporates hyperbolic-aware nonlinear refinement and teacher-conditioned modulation to construct an alternative student representation before distillation. The two representations are subsequently integrated and aligned through Euclidean feature correlations (Eqs. 16–18), rather than explicit hyperbolic metric supervision. Importantly, Table 3's Eul-only and Hyp-only replace E/H with E/E and H/H, respectively, rather than removing one branch. The mixed E/H configuration achieves higher AG, ED, and Qcb than both controls, supporting the value of complementary refinement. We will clarify these operations and ablation settings in the revision.

AuthorsOpenReview ↗

Response to Q2: Interpretation of ablation trade-offs.

We agree that removing PFS or CPD improves MI and IFC, reflecting a trade-off rather than uniform gains. These information-based metrics characterize different properties from structural and edge preservation. Our objective (Eq. 20) jointly optimizes intensity fidelity, gradient preservation, and multi-level correlation distillation, without directly maximizing MI or IFC. Accordingly, the full model places greater emphasis on structural fidelity, achieving higher AG (5.883) and ED (52.00) than w/o PFS (5.604/49.89) and w/o CPD (5.649/50.10), while maintaining competitive Qab/f and Qcb. We recognize the higher MI/IFC of ablated variants as valid advantages on those metrics, rather than evidence of inferior fusion. We will clarify these metric-dependent trade-offs instead of claiming uniform component improvements.

AuthorsOpenReview ↗

Response to Q3: Inference protocol.

The reported inference time (0.4067 s/image, Table 12) measures the fusion network's forward pass using precomputed text embeddings, excluding online BLIP-2 encoding. Text-conditioned interaction remains active through these embeddings (Eq. 10). The frozen MiT-B3 teacher is used exclusively during training for multi-level correlation distillation (Eqs. 2 and 18) and is completely removed during inference, together with the distillation branch. We will explicitly clarify this timing scope and the teacher's training-only role in the revision.

Reviewer_AoLiOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

This paper proposes PSDFusion for infrared-visible image fusion. The method first applies Prior-guided Feature Specialization (PFS), which separates a multimodal representation into local-detail and context-preserving pathways before text-based semantic conditioning, then applies Cross-Paradigm Distillation (CPD), combining Euclidean and hyperbolic-aware student representations before correlation-based multi-level supervision from a frozen SegFormer teacher. The paper evaluates PSDFusion against twelve fusion methods on MSRS, LLVIP, FMB, and M3FD, and reports fusion metrics as well as downstream semantic-segmentation and object-detection results.

Critical strengths
  1. The paper targets a practically relevant tension in infrared-visible fusion: retaining source-image texture and thermal saliency while improving task-relevant semantic content. Separating the specialization and conditioning stages is a reasonable design hypothesis, rather than simply applying text priors directly to an initially entangled fusion feature.
  2. Figure 1 provides a useful high-level account of the intended pipeline. In particular, it distinguishes the learnable fusion pathway from the frozen teacher pathway and makes clear that text conditioning occurs after PFS, while CPD is used for training-time teacher supervision. This separation helps readers understand the intended division of labor between visual specialization, semantic conditioning, and distillation.
  3. The experimental scope is broader than a single in-domain image-fusion evaluation. Tables 1 and 2 include four datasets and report direct transfer from MSRS to LLVIP, FMB, and M3FD without stated fine-tuning. On MSRS, the method is strongest on all six reported fusion metrics. On the cross-dataset benchmarks, it remains competitive, particularly in MI and IFC on LLVIP and M3FD, and in Qab/f and Qcb on FMB.
Critical weaknesses
  1. The paper overstates the interpretation of the “hyperbolic-aware” pathway. The method maps features to a Poincaré ball, performs Fourier-domain manipulation in the tangent space, maps back to Euclidean space in Equation 16, and finally applies the distillation loss in a Euclidean correlation space. No hyperbolic distance, geodesic objective, hierarchy measure, or analysis of learned radial structure is used in the supervision. Consequently, the current evidence supports that a particular nonlinear reparameterization can help, but not the stronger claim that heterogeneous teacher knowledge is being transferred through meaningfully complementary geometries. The Euclidean-only and hyperbolic-only rows in Table 3 are useful, but they do not isolate whether the benefit comes from hyperbolic geometry, additional nonlinear transforms, increased feature capacity after concatenation, or teacher-conditioned spectral gating.
  2. The ablation results do not uniformly support the narrative that each proposed component improves the method. In Table 3, removing PFS increases MI from 4.671 to 4.834 and IFC from 4.838 to 6.005; removing CPD also increases IFC to 5.510 and MI to 4.703. The full system improves AG, ED, and Qcb, but the evidence is a metric trade-off, not a uniform component benefit. The paper acknowledges some of this, but its claims that the ablations “verify complementary contributions” are too strong without explaining why substantially higher IFC from the PFS-removed model should be considered inferior. A more convincing ablation would include clearly specified variants for: ordinary Euclidean correlation distillation only, spectral teacher gating only, hyperbolic reparameterization without added concatenated capacity, and PFS with no text conditioning.
  3. The textual-conditioning protocol creates a potentially important fairness confound. On MSRS, the method uses paired scene-level descriptions from VLFDataset, while on the transfer datasets it uses generic modality prompts. Table 10 shows that scene-level captions change performance materially. The paper does not state whether competing prompt-based methods receive the same scene-level captions, whether captions are automatically generated or human-authored, whether they are available at deployment, or whether the captions encode semantic information not available to non-text baselines. This is especially relevant for the strong MSRS result in Table 1, where PSDFusion is best on every metric. A fair comparison should either provide all text-guided baselines with the same caption information or include a primary comparison in which PSDFusion is restricted to a common fixed prompt.
Questions for authors
  1. Is the teacher used only during training and completely removed at inference? Please report the teacher architecture, pretrained weights, source training data, and whether any teacher pretraining data or labels overlap with MSRS/FMB downstream segmentation evaluation.
  2. What is the exact definition of NormNorm in Equation 5? Which dimensions are normalized or aggregated, and how does the operation capture global spatial dependencies rather than only applying channel projections?
  3. Could the authors add controls that isolate: (i) ordinary correlation distillation without teacher-driven spectral gating, (ii) Fourier teacher gating without hyperbolic mapping, (iii) a capacity-matched nonlinear Euclidean branch, and (iv) the proposed hyperbolic branch? These are needed to identify what aspect of CPD is responsible for the reported gains.
9 comments
AuthorsOpenReview ↗

Response to W1 and Q3: Hyperbolic-aware refinement and controlled ablations.

We have clarified in the revised Sec. 3.3 that "hyperbolic-aware" refers to a Poincaré-parameterized, teacher-conditioned representation refinement pathway, rather than explicit hyperbolic metric learning. Specifically, Eqs. (12–14) map student and teacher features through the Poincaré ball and tangent space, where teacher-guided spectral modulation is performed. The refined features are subsequently integrated in Euclidean space (Eqs. 16–17) before correlation distillation (Eq. 18). Thus, CPD does not impose hyperbolic distance, geodesic, or hierarchical supervision.

To further examine whether the improvements arise from spectral gating, nonlinear transformations, or additional representation capacity, we report three controlled variants in Appendix Table 13. Correlation Only retains Euclidean correlation distillation without spectral gating. Spectral Only applies teacher-guided Fourier modulation without hyperbolic mapping. Nonlinear Euclidean employs a capacity-matched nonlinear Euclidean counterpart. Full E/H outperforms all three controls across six fusion metrics. The existing E/E and H/H variants (Table 3) additionally preserve the dual-path structure while changing the representation configurations. Together, these comparisons support the effectiveness of the complete E/H refinement design, without attributing its gains solely to hyperbolic geometry.

Table R1. Controlled ablation results of CPD on MSRS.

SettingAGMIQab/fIFCEDQcb
Correlation Only5.3444.650‾\underline{4.650}0.694‾\underline{0.694}4.539‾\underline{4.539}50.270.572‾\underline{0.572}
Spectral Only5.3654.5270.6924.50550.28‾\underline{50.28}0.571
Nonlinear Euclidean5.385‾\underline{5.385}4.5560.694‾\underline{0.694}4.52750.210.571
Full CPD (E/H)5.8834.6710.7114.83852.000.590
AuthorsOpenReview ↗

Response to W2: Ablation trade-offs and PFS without textual conditioning.

We have further clarified in the revised Sec. 4.3 and Conclusion that PFS and CPD exhibit metric-dependent contributions rather than uniform improvements. As reported in Table 3, removing either component improves MI and IFC, whereas Full achieves higher AG and ED, indicating stronger gradient and edge preservation. These results reflect different evaluation objectives rather than inconsistent ablation evidence. Specifically, Eq. (20) constrains intensity, gradients, and feature correlations without directly optimizing MI or IFC. To further assess textual conditioning independently of visual specialization, we introduce PFS w/o Text in Appendix Table 14, which retains visual specialization and CPD while removing the text-conditioned interaction in Eq. (10). As shown below, Full outperforms this variant across all six metrics, demonstrating the additional contribution of textual guidance. These findings support a more precise interpretation of the component effects and their metric-dependent trade-offs.

Table R2. Additional ablation of textual conditioning on MSRS.

SettingAGMIQab/fIFCEDQcb
PFS w/o Text5.3454.6360.6974.56450.260.573
Full5.8834.6710.7114.83852.000.590
AuthorsOpenReview ↗

Response to W3: Textual conditioning and evaluation fairness.

As specified in Sec. 4.1, PSDFusion uses paired scene-level descriptions from VLFDataset (Zhao et al., 2024) on MSRS, whereas LLVIP, FMB, and M3FD use fixed modality prompts because scene-level annotations are unavailable. We have further clarified in the revised manuscript that the original prompt-based baselines were not provided with identical scene-level captions. To establish a controlled comparison, we additionally evaluate FILM using the same paired captions as PSDFusion on MSRS. As shown below, PSDFusion outperforms FILM across all six metrics under identical textual inputs. The VLFDataset captions were automatically generated using ChatGPT rather than manually authored. Their use at deployment requires prior availability or external generation, and we recognize that they provide instance-specific semantics unavailable to image-only baselines. Importantly, Appendix Tables 9–10 already evaluate textual conditions without scene-specific annotations. With a single fixed task-oriented prompt, PSDFusion exceeds the strongest per-metric baselines in Table 1 on five of six metrics (MI, Qab/f, IFC, ED, and Qcb). We have further emphasized these protocol distinctions and annotation-free results in the revised manuscript.

Table R3. MSRS comparison using identical paired scene-level captions.

MethodAGMIQab/fIFCEDQcb
FILM4.7923.5310.5742.57844.060.459
PSDFusion5.8834.6710.7114.83852.000.590
Reviewer_AmCCOpenReview ↗
Rating
2
Confidence
4(+1)
Informativeness
2
Presentation
2
Soundness
1(−1)
Read review
Paper summary

PSDFusion is a prior-guided infrared–visible image fusion (IVIF) framework with two main stages. First, Prior-Guided Feature Specialization (PFS) transforms multimodal features into complementary local-detail and context-preserving pathways before injecting textual semantics via a pretrained BLIP-2 encoder and cross-modal attention. Second, Cross-Paradigm Distillation (CPD) transfers multi-level knowledge from a frozen SegFormer teacher into the fusion student using two parallel representation views: a Euclidean view that preserves native spatial structure, and a hyperbolic-aware view that maps features to a Poincaré ball, projects them to the tangent space, performs teacher-gated spectral interaction, and maps them back. The two views are concatenated, residually fused, and distilled in a correlation space. The overall loss combines intensity fidelity, gradient fidelity, and multi-level distillation.

Critical strengths
  1. The entry point of the problem is targeted. Instead of simply "adding textual priors", the paper does not stop at the level of "introducing another textual prior". Instead, it points out two more specific problems: what kind of visual representation is more suitable for the role of external priors; and what kind of geometric representation is more effective for the transfer of multi-level teacher knowledge.
  2. The design of PFS is reasonable: PFS transforms multimodal features into local detail-preserving paths and context-preserving paths before semantic conditions, and then allows text semantics to act on specialized representations through cross-modal attention. This design is more detailed than "directly concatenating text features to visual features" and is a more convincing part of the paper.
  3. The idea of Euclidean hyperbolic dual-path distillation is innovative. After splicing the two paths, the residuals are fused and distillation is performed in the relevant space. Introducing hyperbolic geometry into IVIF distillation brings novelty to the topic selection and provides a thought of "complementary multi-geometric representations".
  4. The experiment covers a wide range, and the main experiment is convincing. The training is conducted on MSRS, and cross-dataset testing is performed on LLVIP, FMB, and M3FD, with reasonable settings; compared with 12 representative methods, the coverage is extensive; all six indicators on MSRS are optimal, indicating strong performance in the main experiment;
  5. The paper writing and chart organization are relatively clear: the overall framework diagram, PFS/CPD sub-diagram, qualitative comparison diagram, indicator curve diagram, and downstream task diagram are all relatively complete, allowing readers to quickly understand the methodological process and experimental setup.
Critical weaknesses

1.Scene-level text exhibits strong prior dependencies, which significantly degrade when crossing domains. The paper utilizes scene-level captions from VLFDataset on MSRS, but LLVIP, FMB, and M3FD lack scene annotations and can only rely on fixed prompts. In essence, when crossing datasets, text priors essentially degenerate into "modal identity prompts," with minimal semantic content. The paper fails to adequately demonstrate the semantic gains achievable by the method in real-world unannotated scenarios.

2.The necessity and mechanism of hyperbolic paths are insufficiently demonstrated. After mapping student features to the sphere and then projecting them to the tangent space, theoretically, they are the original features; the same applies to teacher features. If two TDGs share parameters, hyperbolic paths and Euclidean paths are almost mathematically equivalent, and the curvature c will be cancelled out before the tangent space FFT. Therefore, the actual differences in hyperbolic paths are more likely to come from: additional parameters brought by non-shared TDGs; and the regularization/integration effects of independent gating.

3.It is not stated whether SegFormer has been trained on infrared data. The paper only states "a pretrained SegFormer is adopted as a frozen teacher", without specifying the source of the pretraining data, nor does it mention infrared fine-tuning. Based on the public pretraining settings of SegFormer, it is highly likely that it was pretrained on RGB natural images, without infrared training. The paper does not discuss the impact of this on the reliability of distillation, nor does it analyze whether the teacher's response to the infrared-dominated regions in the fused image is reliable, and whether teacher errors will be distilled into the student.

4.The preconceived rationality assumptions of the teacher are not fully demonstrated. The paper implicitly assumes that the multi-level features of SegFormer are suitable as the distillation target for IVIF, but the main argument is: choose SegFormer; replacing it with DeepLabV3+, UPerNet, or BiSeNetV2 results in performance degradation; therefore, SegFormer features are "effective". More precisely, the paper proves that "among the four candidate teachers, SegFormer has the best performance on MSRS", rather than "SegFormer features are indeed the features required by IVIF".

5.The hyperbolic path lacks efficiency and numerical stability analysis. CPD requires mapping to the Poincaré ball, logarithmic mapping, FFT, gating, exponential mapping, and then remapping back to Euclidean space, making the process more complex than pure Euclidean distillation. However, the paper does not report:

Questions for authors
  1. What is the mathematical necessity of hyperbolic paths? Is the mapping cancelled in the tangent space?
  2. Has the SegFormer teacher been trained on infrared data? How is the rationality of the teacher's prior knowledge for IVIF demonstrated?
  3. When scene-level text priors degenerate into fixed modal cues across domains, how can the semantic gain of the method be ensured?
  4. What are the efficiency, numerical stability, and training stability of hyperbolic paths?
  5. How is semantic information balanced with pixel information? Is there an explicit mechanism?
  6. Is the innovativeness of the technology portfolio sufficient? Is the prior use of text overly simplistic?
12 comments
AuthorsOpenReview ↗

Response to W1 and Q3: Cross-domain textual priors and semantic gains.

We thank the reviewer for the careful evaluation and constructive feedback. We address these concerns below.

As described in Secs. 3.2–3.3, PFS specializes visual features before semantic conditioning (Eqs. 4–10), while CPD transfers multi-level correlations from a pretrained teacher (Eq. 18). Together, they promote task-relevant representations without requiring target-domain scene captions. To verify the additional semantic benefit of coarse textual guidance, we have included a fixed-prompt versus empty-text comparison in Appendix Table 11, using the same MSRS-trained model.

Downstream TaskEmptyFixedGain
FMB mIoU (%) ↑52.3754.18+1.81
LLVIP Precision (%) ↑93.7594.42+0.67
M3FD Precision (%) ↑85.6286.56+0.94
These controlled downstream improvements provide direct evidence of additional task-relevant benefits from fixed textual prompts without scene annotations. We will further emphasize this mechanism and evidence in the revised manuscript.
AuthorsOpenReview ↗

Response to W2 and Q1: Hyperbolic pathway and mapping equivalence

We have further clarified in Sec. 3.3 that the implemented hyperbolic-aware pathway is not simply equivalent to a duplicated Euclidean branch. Although ideal exponential and logarithmic maps are inverse before tangent-space FFT, the actual Poincaré mapping involves ball projection and curvature-dependent gradient scaling, (1−c∥z∥22)2/4(1-c\|\mathbf{z}\|_2^2)^2/4, during backpropagation. Therefore, ideal forward cancellation does not eliminate curvature-dependent training behavior. We have also clarified that the two pathways employ separately instantiated TDGs without parameter sharing.

To distinguish these effects from additional gating or model capacity, Table 3 already includes dual-path E/E and H/H controls. These variants replace the mixed E/H configuration with two Euclidean or two hyperbolic-aware branches, respectively, while retaining independent TDGs and residual integration. Notably, E/H and E/E contain 9,169,932 and 9,169,936 distillation-module parameters, respectively. Nevertheless, E/H achieves higher AG, ED, and Qcb than both controls, supporting the effectiveness of heterogeneous refinement beyond additional gating branches or parameter capacity. We will further emphasize the implemented mapping mechanism and controlled comparisons in the revised manuscript.

AuthorsOpenReview ↗

Response to W3 and Q2: Teacher pretraining and IR reliability

We have clarified in Sec. 4.1 that the frozen MiT-B3 encoder is initialized with ImageNet-1K classification-pretrained weights, without any additional infrared-domain adaptation or segmentation-task fine-tuning, and processes fused images (Eq. 2). CPD transfers feature correlations, not predictions (Eq. 18), while intensity/gradient losses preserve source fidelity (Eqs. 19–20). Appendix B now includes a quantitative IR-preservation evaluation on MSRS test images using fixed pedestrian-salient masks. Infrared-source preservation in pedestrian-salient regions.

MethodIR Intensity Error ↓IR Gradient Error ↓
w/o CPD0.081020.01937
Full CPD0.055470.01625
Reduction31.5%16.1%
Full CPD reduces IR intensity and gradient errors by 31.5% and 16.1% versus w/o CPD, supporting improved infrared-source preservation despite RGB-only teacher pretraining. We will further emphasize this evidence in the revised manuscript.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.