acceptodds
Withdrawn from ICLR 2027

AnyThermalSplat: Scaling Pose-Free Feed-Forward Thermal Scene Reconstruction

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Thermal 3D reconstruction typically relies on known cameras and per-scene optimization, while the scarcity of multi-view thermal data hinders scalable feed-forward reconstruction. We present AnyThermalSplat, a pose-free feed-forward framework that predicts renderable thermal Gaussian scenes without external camera parameters or test-time scene optimization. It adapts RGB-pretrained geometric priors through low-rank updates and residual feature bridges, with a frozen RGB teacher providing geometric supervision alongside thermal reconstruction and held-out novel-view objectives; inference requires thermal images only. To expand supervision, we introduce ThermalVerse, a multi-domain RGB–thermal dataset, and a geometry-grounded thermalization pipeline that fits shared thermal appearance on fixed RGB-reconstructed geometry. Together with curated public data, these resources form a 500-scene training pool. On in-domain scenes, AnyThermalSplat achieves 26.76 and 28.48 dB PSNR with 6 and 12 context views, respectively, outperforming our matched thermally adapted AnySplat-IR baseline by 2.84 and 1.77 dB. This baseline is trained on the same thermal data under the same adaptation protocol, providing a controlled comparison beyond directly applying the original RGB model. We also conduct a controlled scaling study on nested 100–500-scene subsets under a fixed update budget and find substantial gains in both in-domain and out-of-domain reconstruction as the training set grows from 100 to 300 scenes. Beyond this regime, out-of-domain performance largely saturates, while in-domain reconstruction retains an overall benefit from larger training sets. We will release the data and code, along with model checkpoints.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
Thermal novel-view synthesis, 3D Gaussian splatting, Feed-forward reconstruction, Pose-free reconstruction, Cross-modal distillation, Data scaling

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_3PB1OpenReview ↗
Rating
3
Confidence
4
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

The paper starts from unposed and uncalibrated multi-view thermal infrared images and directly predicts camera parameters, depth, and renderable thermal 3D Gaussians in a feed-forward manner. It addresses a well-defined and practically valuable problem. In addition, the authors introduce ThermalVerse, a dataset comprising 209 scenes and 26,071 paired RGB–thermal images. Overall, I consider both the proposed method and the dataset to be valuable contributions.

Critical strengths
  1. The problem setting is relevant and addresses a relatively less explored direction. Prior thermal NeRF / Gaussian Splatting methods typically rely on known camera parameters or per-scene optimization, whereas this work considers a feed-forward setting without these assumptions.

  2. The paper also provides a useful data contribution. ThermalVerse contains 209 scenes across 30 content categories, with approximately 26K paired RGB–thermal images.

  3. The experimental evaluation is fairly comprehensive, and the reported results provide reasonable empirical support for the proposed method.

Critical weaknesses
  1. “Matched AnySplat-IR” is not a strictly architecture-matched baseline. Although the two methods share the same training data and evaluation protocol, they differ in the backbone, encoder, teacher model, and LoRA placement. I suggest clarifying this distinction more explicitly in the main text.

  2. The scaling experiments use a fixed training budget of 7,000 steps. As the dataset size increases, the average exposure per scene consequently decreases. The results may therefore be more appropriately interpreted as fixed-compute scaling, rather than as a general scaling law.

  3. The main model uses LoRA on both the attention and FFN modules, whereas the scaling and ablation experiments use attention-only LoRA. It would be helpful to briefly explain the motivation for using different configurations and to clarify that these results are not obtained under exactly the same model setting.

Questions for authors
  1. Could the authors clarify more explicitly which settings are matched in “Matched AnySplat-IR” and which architectural components still differ between the two methods?

  2. How do the authors view the effect of fixing the training budget at 7,000 steps on the scaling results? Would it be more appropriate to characterize these experiments as fixed-compute scaling?

  3. Why does the main model use attention+FFN LoRA, while the scaling and ablation experiments use attention-only LoRA? How should the results under these two configurations be interpreted in relation to each other?

Reviewer_fZ9SOpenReview ↗
Rating
1
Confidence
4
Informativeness
2
Presentation
3
Soundness
1
Read review
Paper summary

The paper adapts an RGB-pretrained reconstruction model to predict thermal Gaussian scenes from unposed thermal images. It combines VGGT-pi, a AnySplat decoder, LoRA, RGB-teacher supervision, and novel-view rendering losses. It also introduces ThermalVerse and a synthetic data pipeline based on F-VITA translations fitted onto RGB geometry. Evaluation covers reconstruction quality, runtime, and training-data scaling.

Critical strengths
  • Thermal-only reconstruction without per-scene optimization is useful. The reported reconstruction time of 1.15 seconds demonstrates a practical benefit.
  • The in-domain improvements over the thermally trained AnySplat-IR baseline are meaningful. Including a thermally trained baseline makes this comparison more informative than comparisons against RGB-pretrained models alone.
Critical weaknesses

1. Lacking substantial methodological contribution beyond adapting existing RGB methods to thermal inputs

The paper combines a existing geometry backbone and gaussian decoder with standard LoRA, residual adapters, geometric distillation, and rendering supervision. Section 3.2 explicitly follows AnySplat. There is no clear new mechanism addressing thermal image formation or RGB-TIR misalignment. Adding held-out rendering supervision in section 3.3 differs from AnySplat’s original training, but is already standard in RGB methods such as pixelSplat$^1$. The remaining changes are a newer VGGT-pi backbone, standard LoRA, residual bottleneck adapters, and RGB-to-TIR teacher supervision. LoRA adaptation of VGGT to RGB-TIR geometry already appears in SEAR$^2$. Changing the input modality does not itself make these established mechanisms novel, and the paper does not identify a substantive thermal-specific solution to justify the adaptation.

The results do not resolve this concern. Out-of-domain PSNR differences against AnySplat-IR are incremental. SSIM and LPIPS improve, but reconstruction quality superiority is not established as vanilla 3DGS has higher PSNR and better LPIPS in both dense out-of-domain settings. The most recent SOTA model PhysIR-Splat (CVPR 2026) 3^3 is discussed but not evaluated.

2. The dataset is stated as the 2nd main contribution, but its supervision quality and construction are insufficiently validated.

Equation 3 compares gaussian-rendered depth with teacher depth at the same pixel location, using the confidence-masked MSE. However, appendix B states that the RGB-TIR pairs have only coarse registration, without pixel-accurate correspondence. Cross-modal registration and image matching are particularly difficult for RGB-TIR cases and something various RGB-TIR dataset/model papers have struggled with because their textures and boundaries can differ, yet accurate correspondence is critical for this supervision. If the pixels show different surfaces, this loss compares depths that should not be matched. Teacher confidence does not fix this, as appendix A.3 explicitly acknowledges. The paper reports neither registration error nor experiments showing how much misalignment the method can tolerate.

The synthetic data is also insufficiently validated. Recent RGB-to-TIR translation models such as TherA$^4$ highlights F-VITA’s poor temporal consistency, which can create conflicting supervision for the same surface across views. Shared field fitting reduces inconsistency but does not guarantee physically plausible thermal appearance. Table 12 shows improvement over independent translation, but neither measures residual inconsistency nor establishes whether synthetic data improve over real-only training. Appendix also admits missing uniform records of appearance-fitting iterations and loss weights, making dataset construction difficult to reproduce.

Reference

1^1 Charatan, David, et al. "pixelsplat: 3d gaussian splats from image pairs for scalable generalizable 3d reconstruction." 2024 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). IEEE, 2024.

2^2 Skorokhodov, Vsevolod, et al. "SEAR: Simple and Efficient A daptation of Visual Geometric Transformers for Unpaired RGB-Thermal 3D Reconstruction." European Conference on Computer Vision. Cham: Springer Nature Switzerland, 2026.

3^3 Gao, Jingyuan, et al. "PhysIR-Splat: Physically Consistent Thermal Infrared Radiative Transfer in 3D Gaussian Splatting." Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2026.

4^4 Lee, Dong-Guw, et al. "TherA: Thermal-aware visual-language prompting for controllable RGB-to-thermal infrared translation." Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2026.

Questions for authors
  1. How accurate is the RGB-TIR registration, and how sensitive is equation 3 to misalignment?
  2. Does adding synthetic data improve over real-only training? How much crossview inconsistency remains after shared field fitting?
  3. What is the thermal-specific methodological novelty beyond AnySplat and SEAR? Can you isolate its benefit while keeping the backbone and RGB teacher identical across comparisons?

I would reconsider my score if the authors can demonstrate a substantive thermal-specific contribution through controlled ablations, quantify robustness to RGB-TIR misalignment, and show that synthetic augmentation improves over real-only training. These concerns require supporting evidence, not just clarification of the wording.

Reviewer_zeAbOpenReview ↗
Rating
1
Confidence
2
Informativeness
1
Presentation
2
Soundness
2
Read review
Paper summary

The paper proposes a method for feed forward thermal scene prediction. Predecessor methods utilise Gaussian Splat / NeRF based optimisation. While this sounds simple at first, fine-tuning a recent feed forward model to achieve this task the authors propose two problems:

  • Modality gap: Feedforward models predict RGB scenes, the authors tease a special adaption.
  • Data scarcity: The authors propose anew dataset, they call ThermalVerse which consists of ~200 scenes.

However, still this claim, feels like a LoRA adapter AnySplat trained on new data. The dataset is a pseudo dataset, and from the experiment details of in and out of distribution results, I would assume that it rather adds harm then good (also see limitations from the authors). The authors show also a simple fine-tune variant Any-Splat IR which performs almost identical.

Critical strengths

It is hard to identify many strength for this paper:

  • The methods or fine-tune show cases achieves better In-domain results.
  • The method achieves faster results for thermal scene reconstruction then optimisation based methods (but is still worse OOD)
Critical weaknesses
  • Task explanation, I'm not profound in the topic of thermal imagery. In your introduction I get a quick glimpse into the world of thermal imagery. However, nowhere is a brief introduction, what makes this imagery special. Is it hard to capture? What is the essence of it? This would have been super helpful at latest in the dataset creation pipeline.

  • From a writing perspective, I would urge the authors to focus on readability, while using AI as a writing assistant is fine, many sections are hardly human parsable. The last section of the abstract is barely readable. The entire paper is cluttered with compound words. The experiment section paragraphs, are stacked with numbers without clear interpretation.

  • Key contribution. It is very hard to identify the key contribution of this paper from a methodological standpoint. The key contributions listed in the introduction, show a very broad statement and I expected more detail in the method section. However, the lines blur in the method section and the authors never clearly state what is their contribution (e.g. in the paper and key contribution, it sounds like they provided a solution for unposed reconstruction in this domain. However, this is a feature of AnySplat). Also see Questions for authors.

  • Table 1 is badly analysed in the experiment section. And it raises many questions. To me it looks like the method only performs in distribution better. Everything else is worse then optimisation based methods or almost on par with the simple fine-tune.

Questions for authors

The core research contribution for me is badly described, what is the methodological novelty in this paper? For section 3.1. you say "Four residual bridges accommodate the adapted features at the decoder input levels:" What does this mean?

Why is this needed? No intuition is described. Let me understand in your method section, why a VGGT-Ω teacher is necessary, and what it contributes to what is its job in the pipeline? "To guide thermal reconstruction with geometric priors pretrained with RGB, we use a frozen VGGT-Ω teacher (Wang et al., 2026a) that processes"

Where is the difference to AnySplat? Could you name clearly what is your contribution?

Why is AnySlat-IR not listed in Table 2?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.