acceptodds
Withdrawn from ICLR 2027

Thermal3R: Physics-Induced Feed-forward Framework for Thermal 3D Reconstruction

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Thermal infrared (TIR) imaging enables reliable all-weather perception, yet generalizable 3D reconstruction from TIR remains underdeveloped. Existing feed-forward multi-view models designed for the visible spectrum fail in the thermal domain due to three physical hurdles: depth stretching from nonlinear emission, texture loss from heat conduction, and radiative distortion from atmospheric attenuation. To foster research in thermal 3D reconstruction, we introduce Largethermal3D, a comprehensive and fully-annotated benchmark for thermal 3D reconstruction, and Thermal3R, a physics-guided framework integrating radiation, conduction, and transmission priors. Specifically, we propose: (i) Physics-Aware Radiation Sensitivity Attention Mechanism (PAM) to handle nonlinear sensor response; (ii) Thermal Gradient Consistency (TGC) to sharpen geometric boundaries attenuated by heat conduction; and (iii) Attenuation-Aware Calibration (AAC) to correct atmospheric biases. Experiments show that Thermal3R achieves competitive performance, outperforming fine-tuned RGB baselines across all tasks. This underscores the superiority of physical priors over simple data-driven fine-tuning

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
Thermal 3D Reconstruction, Physics-Guided, Largethermal3D Benchmark

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_K7rcOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper proposes Thermal3R, a physics-guided feed-forward framework for thermal infrared 3D reconstruction based on VGGT. It introduces three key modules: Physics-Aware Radiometric Sensitivity Attention (PAM), Thermal Gradient Consistency (TGC), and Attenuation-Aware Correction (AAC). Additionally, the authors construct LargeThermal3D, a dataset comprising 90 thermal infrared scenes, with annotations including depth maps and camera parameters.

Critical strengths
  1. Generalizable feed-forward 3D reconstruction from thermal infrared images is a research direction of significant value that remains relatively underexplored.

  2. The construction of the dataset represents a meaningful contribution.

  3. Thermal3R outperforms the fine-tuned VGGT baseline across multiple reported 3D reconstruction tasks.

Critical weaknesses

1. PAM introduces a temperature-weighting mechanism based on thermal radiometric sensitivity, but the relationship between its theoretical derivation and practical implementation remains unclear. The paper approximates the absolute radiometric sensitivity as Sabs(T)∝T3.8S_{\mathrm{abs}}(T)\propto T^{3.8}, while Equation (3) derives a logarithmic radiometric sensitivity proportional to 1/T21/T^2. However, the weighting term 1/T~21/\tilde{T}^2 adopted in Equation (4) does not directly follow from the inverse of the absolute radiometric sensitivity. Although the authors empirically select this weighting formulation through the experiments in Appendix A.7, its physical justification remains insufficiently explained.

2. AAC derives a depth correction formula based on the Beer–Lambert law. However, this derivation implicitly assumes a deterministic relationship between the initial depth estimates and the radiative attenuation model, which has not been adequately justified for depth values directly predicted by a neural network. Furthermore, according to Equation (14), when the observed intensity is lower than the reference intensity and the attenuation coefficient is positive, the depth correction term becomes negative. This appears inconsistent with the explanation in the main text that additional attenuation should lead to an increase in the estimated depth.

3. TGC improves geometric reconstruction by enforcing consistency between the gradients of normalized thermal images and depth maps. However, its formulation closely resembles existing gradient-consistency regularization methods and does not explicitly model heat conduction or thermal diffusion processes. More importantly, Equation (10) not only encourages the spatial alignment of thermal and depth edges but also tends to enforce consistency in their gradient directions. Consequently, even when geometric boundaries are correctly localized, a substantial penalty may arise if the thermal intensity gradients and depth gradients point in opposite directions.

Questions for authors
  1. Could the authors explicitly clarify whether the 1/T~21/\tilde{T}^2 weighting in PAM corresponds to absolute radiometric sensitivity, logarithmic radiometric sensitivity, or an empirically designed approximation? If its purpose is to compensate for absolute radiometric sensitivity, could the authors further explain the quantitative relationship between this weighting term and Equation (2)?

  2. Could the authors explain how the radiometrically inferred depth implicit in Equation (13) is related to the initial depth directly predicted by the neural network? Furthermore, when the observed radiance is lower than the reference radiance, Equation (14) yields a negative depth correction, which appears inconsistent with the physical explanation in the main text that additional attenuation should increase the estimated depth. Please clarify the definitions of the relevant variables and the direction of the depth correction.

  3. For scenarios where thermal image gradients and ground-truth depth gradients point in opposite directions, or where thermal intensity edges exist without corresponding depth discontinuities, could the authors explain how Equation (10) avoids imposing incorrect constraints on geometrically accurate structures and provide corresponding quantitative validation?

Reviewer_oygPOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper introduces Thermal3R, a VGGT-based framework for feed forward 3D reconstruction from thermal infrared imagery, as well as Largethermal3D, a comprehensive 3D thermal reconstruction dataset. Thermal3R proposes three components: Physics-Aware Radiation Sensitivity Attention (PAM), Thermal Gradient Consistency (TGC), and Attenuation-Aware Calibration (AAC), motivated by thermal radiation, heat conduction, and atmospheric attenuation, respectively. Experiments evaluate depth, point-map, and camera-pose estimation, reporting improvements over several RGB-pretrained reconstruction models and a thermal-fine-tuned VGGT baseline.

Critical strengths

The paper addresses a relevant problem and evaluates depth, point maps, and camera poses within a unified framework. The method is backed up with appropriate physics, properly tailoring the method for the thermal infrared domain, as well as being first to properly address sensor nonlinearity. The reported gains demonstrate benefits beyond simply applying an RGB-pretrained model to thermal images. Component ablations provide additional evidence that the proposed additions affect performance.

Critical weaknesses

1. Physically unjustified thermal depth gradient consistency in TGC

TGC directly matches normalized, signed thermal-intensity and depth gradients in equations 9 and 10, although thermal contrast does not determine depth ordering. For example, a hot foreground object against a cooler background produces opposite intensity and depth gradients at a valid boundary, causing the loss to penalize correct geometry. Normalization preserves this polarity mismatch, and the claimed ability of the multiview transformer to reject misleading thermal edges does not explain how conflicting supervision is avoided. This is consequential because the paper presents TGC as a physically grounded mechanism for recovering geometric boundaries. A comparison with a polarity-invariant boundary regularizer, including temperature-contrast reversals at fixed geometry, is needed to substantiate that interpretation.

2. Missing thermal baselines

Although VGGT* provides a useful thermal-fine-tuned baseline, the evaluation omits existing thermal feed-forward reconstruction methods. This includes TEPR 1^1, SEAR 2^2 (it reports thermal-only reconstruction results in its appendix), VGGT-IR introduced in PhysIR-Splat 3^3. The monocular depth evaluation should also include baseline thermal estimator such as Shin et al.4^4 or the thermal-only configuration of Align-and-Fuse 5^5. These methods should be compared under the same input and training settings. Without these comparisons, the SOTA and 'first thermal adaptation' claims are not convincing. With these results, I would be willing to revise my score.

Reference:

1^1 Li et al. TEPR: Thermal Edge-Prompted Geometry Learning for Multi-View 3D Reconstruction. IJCV, 2026.

2^2 Skorokhodov et al. SEAR: Simple and Efficient Adaptation of Visual Geometric Transformers for RGB+Thermal 3D Reconstruction. 2026.

3^3 Shin, Park, and Kweon. Deep Depth Estimation From Thermal Image. CVPR, 2023.

4^4 Gao et al. PhysIR-Splat: Physically Consistent Thermal Infrared Radiative Transfer in 3D Gaussian Splatting. CVPR, 2026. Introduces VGGT-IR.

5^5 Shin, Lee, and Oh. Bridging Spectral-wise and Multi-spectral Depth Estimation via Geometry-guided Contrastive Learning. 2025. Introduces Align-and-Fuse, supporting single-modality and multispectral depth estimation.

Questions for authors
  1. In PAM, how does the1/\tilde{T}^2 weighting follow from the stated inverse-sensitivity model, which suggests 1/T^3.8? Does it outperform a generic learned attention module with the same parameter count and depth supervision?
  2. In TGC, how does the loss handle valid boundaries where thermal intensity and depth gradients have opposite signs? Have you compared it with a polarity-invariant boundary loss?
  3. In AAC, equation 15 decreases depth when the observed intensity is lower than the predicted intensity, whereas the accompanying explanation says depth should increase. Which reflects the implementation? What evidence distinguishes the learned attenuation field from a generic depth-correction mechanism?
  4. Does the monocular depth evaluation use exactly one image? If so, how is AAC’s reference radiance computed without the multiview fusion described in section 3.4?
Reviewer_Wb1ZOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

Thermal3R addresses feed-forward 3D reconstruction from thermal images by adapting VGGT with three physics-inspired components. PAM predicts a simulated temperature map and uses an inverse-square weighting to modulate depth-head features. TGC aligns normalized thermal-image and depth gradients, while AAC learns a per-pixel attenuation field for radiance-based depth correction. The image encoder and attention blocks are frozen, and the prediction heads are trained with geometric supervision and the additional proposed losses.

The paper also introduces Largethermal3D, combining 90 indoor, outdoor, and UAV scenes with camera and geometry annotations, split into 76 training and 14 test scenes. Evaluation covers monocular depth, point-map, and camera-pose estimation. Compared with fine-tuned VGGT, the method reduces AbsRel from 0.6618 to 0.5324 on laser-scanned scenes and improves the reported point-map and pose metrics. Component ablations examine PAM, TGC, and AAC, with further analyses of temperature weighting and attenuation in the appendix.

Critical strengths
  1. Feed-forward thermal 3D reconstruction is a worthwhile problem, and the consolidated benchmark provides useful support for further research.
  2. Evaluation covers three geometric tasks, and the component ablations show improvements over the fine-tuned VGGT baseline.
Critical weaknesses
  1. The gains are not clearly attributable to physical priors. PAM and AAC introduce additional depth supervision as well as model capacity (Eqs. 7–8 and 16–17). Table 5 does not separate these effects from the proposed physical formulations through controls with matched losses and capacity.
  2. The physical interpretation needs stronger justification. In Eq. 3, 1/T² is proportional to log-radiance sensitivity, yet Eq. 4 uses it as an inverse-sensitivity proxy. AAC omits atmospheric path radiance, and the sign of the correction in Eq. 14 contradicts the following explanation. These issues weaken the claimed physical grounding.
  3. TGC assumes compatible thermal and depth gradient directions. The signed-gradient penalty in Eq. 10 can penalize a valid depth boundary when its thermal polarity is reversed. Thermal edges also need not correspond to geometric discontinuities.
  4. Evaluation uncertainty remains substantial. Only 14 scenes are held out, and no separate validation split or repeated-run uncertainty is reported. The exponent selection in Appendix A.7 therefore needs clarification before its results can support a generalization claim.
Questions for authors
  1. Separating physical priors from supervision and capacity. Can you compare Thermal3R with VGGT trained using the same additional depth losses, and with similarly sized generic feature-modulation and depth-residual heads? Please keep trainable layers, data, and optimization budgets matched. These controls would clarify whether the gains require the proposed physical formulations.
  2. PAM and TGC assumptions. How is the inverse-square weighting justified given the distinction between sensitivity and inverse sensitivity in Eqs. 3–4, and what units or calibration define the simulated temperature? For TGC, how are geometrically correct boundaries with reversed thermal polarity handled? A polarity-invariant gradient comparison would help assess this assumption.
  3. AAC implementation and interpretation. How is the predicted radiance in Eq. 15 constructed at inference, particularly for a single input image, and does it use only predicted geometry? Please reconcile Eq. 14 with its verbal explanation and specify the attenuation field's constraints and the conditions under which neglecting atmospheric path radiance is justified.
  4. Model selection and evaluation stability. Was the exponent in Appendix A.7 selected using a validation set separate from the 14 test scenes? Please describe how scenes from the same source or capture sequence are separated, and provide repeated-run variability or scene-level uncertainty for the key comparison with fine-tuned VGGT.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.