acceptodds
Under review as a conference paper at ICLR 2027

Thermal Sequence Reconstruction under Coupled Motion Blur and Rolling-Shutter Readout

Abstract

Dynamic uncooled infrared imaging is affected by the coupled effects of thermal persistence and rolling-shutter (RS) readout, making fast scene dynamics difficult to recover. The non-instantaneous response of microbolometer pixels introduces an exponentially decaying temporal history into each observation, while row-dependent readout couples the resulting temporal blur with geometric distortion. Existing sequence reconstruction methods are designed for visible-light imaging, whereas infrared restoration methods generally target one-to-one recovery and do not jointly model these effects. In this work, we formulate thermal sequence reconstruction through a unified spatiotemporal observation model that captures the coupling between microbolometer thermal persistence and RS readout. Based on this formulation, we develop a physics-guided framework that reconstructs sharp scene states at target times from consecutive degraded observations and maps them to the target global-shutter (GS) geometry. During training, the reconstructed RS states are passed through the forward thermal response model to enforce physical reblur consistency. We further construct a physics-based synthetic benchmark and collect a real-world dataset using two uncooled infrared cameras. Experiments demonstrate improved performance over existing motion-restoration methods and favorable cross-sensor generalization on real uncooled infrared observations.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
Infrared image debluring, Sequence Reconstruction

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_HNQnOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
2
Soundness
3
Read review
Paper summary

The authors address the degradation caused by the combination of thermal persistence and rolling-shutter readout in uncooled infrared imaging. They define a spatio-temporal observation model that accounts for both phenomena and propose a framework that reconstructs sharp RS states at multiple time points from three consecutive thermal observations, subsequently rectifying them to global-shutter geometry. Additionally, they employ a Physical Reblur Loss to enforce consistency with the thermal response model. The authors construct a physics-based synthetic benchmark and collect separate real-world datasets using two uncooled infrared cameras, reporting performance improvements over existing reconstruction methods.

Critical strengths
  • New synthetic benchmark was developed that simultaneously accounts for thermal persistence and RS distortion. By providing sharp RS/GS ground truth, it enables quantitative evaluation of reconstruction performance across multiple time points. The authors also collected real-world datasets using two different uncooled infrared cameras.
Critical weaknesses
  • Generalization across physical conditions has not been sufficiently verified. The quantitative evaluation uses a single configuration with a thermal time constant of 8 ms and an RS readout duration of 32 ms, and the test set is limited to four scenes. Furthermore, the evaluations on two real cameras are qualitative due to the lack of sharp GS ground truth. Therefore, it remains unclear whether the performance is maintained across sensors with different thermal responses or readout speeds.

  • The temporal and training protocols of the baselines are insufficiently explained. Although the authors state that all methods were retrained on the proposed dataset, the training schemes for the A+B pipelines, intermediate supervision, number of input frames, and handling of target query times remain unclear

Questions for authors
  • TRNeRF already models thermal persistence and RS distortion. Although it uses scene-specific optimization, could the authors clarify whether a comparison with TRNeRF, potentially adapted for temporal reconstruction, is feasible? If not, please explain why. Also, Appendix D reports only single-query inference time. Please provide peak GPU memory and total inference time for all six queries.

  • LPIPS typically uses features pretrained on RGB images, whereas thermal images are generally single-channel. Please clarify the LPIPS backbone and preprocessing used in the evaluation. Is there an established protocol or prior evidence that RGB-based LPIPS correlates with perceptual quality in thermal image restoration?

  • The method predicts only six of the 21 thermal history states, using GT for the remaining 15 during training. Beyond the 78% thermal-weight coverage, what justifies K=6? Please provide any existing K-sensitivity results, including K=21, and discuss the trade-off between reconstruction quality and computational cost.

  • Table 2 shows the benefit of Physical Reblur Loss, but does not isolate the contribution of the physical thermal model. Since the remaining 15 states use GT, the loss reduces to the L1 norm of an exponentially weighted sum of prediction errors over the six predicted states. How does it compare with uniform weights, mismatched thermal time constants, or weighted per-frame loss? Has this loss been evaluated on other reconstruction architectures?

10 comments

W1: Generalization across physical conditions and sensors.

Response. We thank the reviewer for highlighting this limitation. We agree that the original evaluation did not provide sufficient evidence for generalization across different physical imaging conditions and real sensors. In the revised manuscript, we strengthen the evaluation from three complementary aspects.

First, we add a controlled sensitivity study for the physical parameters. The model is trained at tau_th = 8 ms and T_r = 32 ms, and is directly evaluated on newly synthesized test sets with tau_th in {8, 10, 12} ms and T_r in {26, 32, 38} ms. We vary one parameter at a time while keeping the other fixed at its training value, and use the same trained model for all settings without retraining or fine-tuning. The results, added to Appendix B, Table 6 and Fig. 12 in the manuscript. We also reproduced the table below:

SettingAll Query TimesFinal Query
PSNR ↑SSIM ↑LPIPS ↓PSNR ↑SSIM ↑LPIPS ↓
tau_th = 8 ms*28.74280.93340.099128.75650.93260.1006
tau_th = 10 ms28.81480.93320.109528.74290.93220.1116
tau_th = 12 ms28.72900.93140.119128.61650.93050.1214
T_r = 26 ms28.47930.93950.096329.31120.94000.0972
T_r = 32 ms*28.74280.93340.099128.75650.93260.1006
T_r = 38 ms28.79930.93350.104128.75550.93250.1058

Here, * denotes the training setting. The reconstruction quality remains stable across these parameter variations, indicating that the model is not tightly restricted to the exact thermal-response and readout parameters used during training.

Second, regarding the limited number of test scenes, we clarify that the four reported scenes correspond to four top-level physical environments rather than four individual video sequences. Within these environments, the test set contains 21 independent sequences captured from different viewpoints and at different times, involving different moving targets, motion speeds, and motion patterns, including camera motion, object motion, and simultaneous camera-and-object motion. Therefore, although the benchmark contains four top-level environments, each environment covers multiple distinct motion and acquisition conditions. We nevertheless acknowledge that the number of top-level environments remains limited. To further verify that the reported average improvement is not dominated by a particular subset, we additionally provide per-scene results in Appendix E, Table 9.

Third, we substantially strengthen the real-world evaluation. The two real cameras operate under different physical conditions: VIM384GS-USL uses tau_th = 9 ms and T_r = 28 ms, whereas HIKMICRO HM-TM06-LF/ANT uses tau_th = 8 ms and T_r = 32 ms. The same model trained on the synthetic configuration is directly applied to both cameras without sensor-specific retraining or fine-tuning. Qualitative cross-sensor results are provided in Fig. 7 for VIM and Fig. 16 for HIKMICRO.

In addition, we add no-reference quantitative evaluation on both real-world datasets in Appendix H, Table 11, providing complementary assessment from infrared information content, frame-level perceptual quality, and video-level technical quality. Specifically, EN (Entropy) [1] is widely used in infrared image enhancement to characterize the information richness preserved in restored thermal images. CLIP-IQA+ [2] provides a no-reference assessment of per-frame perceptual quality; the original CLIP-IQA study explicitly evaluates fine-grained attributes such as sharpness and further validates the metric on real-world deblurring data, making it suitable for assessing the clarity and perceptual quality of the reconstructed frames. DOVER [3] provides a complementary video-level assessment from both aesthetic and technical perspectives, where the technical-quality branch is designed to capture degradations such as blur, artifacts, noise, and picture clearness. For CLIP-IQA+ and EN, we evaluate each reconstructed state independently and average the scores over the six states of each observation. For DOVER, the six reconstructed states are treated jointly as a short video sequence, thereby providing a sequence-level assessment of the reconstructed video quality. The results are reproduced below:

MethodVIM384GS-USLHIKMICRO HM-TM06-LF/ANT
CLIP-IQA+ ↑EN ↑DOVER ↑CLIP-IQA+ ↑EN ↑DOVER ↑
PMBNet0.15956.15040.01160.23886.14420.0266
DeMFI+JAMNet0.16166.21390.01200.21836.17000.0199
DeMFI+DFRSC0.16726.21840.01210.21996.16970.0198
BiT+JAMNet0.14146.10900.01120.20446.14600.0290
BiT+DFRSC0.14596.10220.01090.21146.15020.0293
BSST+UniVR0.14966.23060.01310.21746.19850.0373
BSST+LBCNet0.13956.25120.01080.21766.20720.0307
DST-Plus+UniVR0.15726.22220.01170.24416.16560.0296
DST-Plus+LBCNet0.15346.24120.00940.25376.16710.0264
PPFN+UniVR0.14876.12290.01160.19676.12130.0309
PPFN+LBCNet0.13796.14680.00930.20376.12510.0265
ECMR+UniVR0.15266.11760.01150.21206.14700.0258
ECMR+LBCNet0.14816.13860.00970.21886.14960.0220
Ours0.19656.29710.01370.26756.15900.0471

These real-world results provide complementary quantitative evidence that the proposed model remains effective across sensors with different thermal responses, readout speeds, resolutions, and imaging characteristics.

[1] Legal-Ayala H, et al. Entropy and contrast enhancement of infrared thermal images using the multiscale top-hat transform[J]. Entropy, 2019, 21(3): 244.

[2] Exploring clip for assessing the look and feel of images[C]//Proceedings of the AAAI conference on artificial intelligence. 2023, 37(2): 2555-2563.

[3] Exploring video quality assessment on user generated contents from aesthetic and technical perspectives[C]//2023 IEEE/CVF International Conference on Computer Vision (ICCV). IEEE, 2023: 20087-20097.

W2: Training and temporal protocols of the compared baselines.

Response. We thank the reviewer for pointing out that the baseline training and temporal protocols were not sufficiently specified in the original manuscript. We have added a dedicated description in Appendix A.5 of the revised manuscript and clarify the complete protocol below.

All competing methods are retrained on our synthetic training set using their official implementations and original supervision strategies. PMBNet follows its original end-to-end training protocol, whereas all A+B pipelines are trained stage-wise, with explicit supervision for each component rather than jointly optimizing the two stages.

We preserve the native input configuration of each method whenever the number of input frames is fixed by its original architecture and official implementation, since forcing a common input length would require architectural modifications and deviate from the intended setting of the baseline. Specifically, PMBNet uses 2 input frames, BiT uses 3, DeMFI uses 4, JAMNet uses 2, DFRSC uses 3, PPFN and ECMR use 1, and UniVR and LBCNet use 2. For BSST and DST-Plus, whose input length is configurable, we use 3 consecutive frames, matching the three-frame input setting of our method.

The supervision and handling of target query times depend on the type of pipeline:

Pipeline typeStage AStage BTraining supervisionHandling of target query times
Sequence reconstruction + RSCDeMFI / BiTJAMNet / DFRSCStage A is supervised with temporally aligned sharp RS states; Stage B is independently supervised with the corresponding sharp GS targetsStage A reconstructs the queried RS states, and Stage B performs RS-to-GS correction at the same temporal positions
Deblurring + RS interpolationBSST / DST-Plus / PPFN / ECMRUniVR / LBCNetStage A is supervised to recover the most recent sharp RS state; Stage B is independently supervised using temporally corresponding sharp RS endpoints and intermediate GS targetsStage A provides the restored endpoint frames, and Stage B generates the required intermediate GS states at the same query times
End-to-endPMBNet--Original end-to-end supervisionDirectly produces the requested reconstructed outputs

For all methods, evaluation is performed at the same target times used by our method. Thus, although the compared methods have different native temporal architectures, their final outputs are evaluated against the same time-aligned GS references under an identical metric protocol.

These details have now been added to Appendix A.5, “Detailed Training Protocols of Compared Methods,” to make the comparison and reproduction protocol explicit.

Q1: Feasibility of comparison with TRNeRF and inference efficiency.

Response. We thank the reviewer for pointing out TRNeRF, which is indeed the closest prior work in terms of jointly modeling microbolometer thermal persistence and rolling-shutter readout. However, TRNeRF and our method solve substantially different inverse problems, which makes a direct adaptation non-trivial.

TRNeRF is a scene-specific neural scene reconstruction method. Given a sequence of degraded thermal images and a known camera trajectory, it incorporates thermal persistence and rolling-shutter readout into the NeRF rendering process, optimizes a scene-level radiance field, and then renders restored GS images from the learned scene representation. In contrast, our method is a feed-forward sequence reconstruction framework that directly takes three degraded thermal observations and reconstructs multiple sharp temporal states associated with the current observation, without constructing a 3D scene representation.

A key practical difference is the dependence on camera poses. TRNeRF assumes known per-frame camera poses and interpolates them into a continuous trajectory to determine the time-varying rays used for thermal-response integration and rolling-shutter rendering. TRNeRF itself reports that COLMAP fails on fast-motion thermal images due to motion blur and rolling-shutter distortion; it therefore estimates poses using additional monochrome cameras and transfers them to the thermal cameras through calibrated extrinsics. We observe the same difficulty on our real sequences: directly applying COLMAP to the degraded thermal observations does not recover a reliable trajectory. Our method instead operates directly on the thermal observations and requires neither camera poses nor auxiliary cameras, cross-sensor calibration, or scene-specific optimization.

Adapting TRNeRF to our temporal reconstruction setting would therefore require more than changing its output time. Its original formulation represents a scene through a radiance field and uses time primarily through the interpolated camera trajectory. To reconstruct multiple evolving scene states as required in our setting, particularly in the presence of independently moving objects, one would need to introduce a dynamic/time-conditioned scene representation in addition to removing or replacing the external pose requirement. These changes would substantially alter the original TRNeRF formulation rather than constitute a faithful baseline adaptation. We therefore regard TRNeRF as the closest physics-based related work, but not as a directly comparable baseline for our pose-free, feed-forward thermal sequence reconstruction setting.

We also thank the reviewer for requesting a more complete efficiency analysis. We have expanded the original single-query evaluation and now report single-query runtime, total runtime for all six reconstructed queries, and peak GPU memory for all compared methods under the same hardware and evaluation setting. The revised results are provided in Appendix F, Table 10 and reproduced below:

MethodSingle-Query Time (s)Six-Query Time (s)Peak Memory (MiB)
PMBNet0.30911.47671124
DeMFI+JAMNet2.49752.72342524
DeMFI+DFRSC0.52132.82355199
BiT+JAMNet0.42181.4411758
BiT+DFRSC0.45681.56205982
BSST+UniVR1.32901.39911288
BSST+LBCNet1.38711.74302628
DST-Plus+UniVR0.19300.26261708
DST-Plus+LBCNet0.25080.60401916
PPFN+UniVR0.54200.62043886
PPFN+LBCNet0.58990.94713894
ECMR+UniVR0.12400.1938672
ECMR+LBCNet0.18030.54501896
Ours0.49791.66831954

Our method requires 0.4979 s for a single query and 1.6683 s for all six queries, with a peak GPU memory usage of 1954 MiB. The increase from single-query to six-query runtime differs across methods because their temporal-generation strategies are different. In particular, several deblurring + RS-interpolation pipelines perform the restoration stage once and subsequently generate multiple intermediate states through interpolation, resulting in a smaller additional cost for multi-query inference. We therefore report these measurements primarily as an efficiency reference rather than a strict architectural complexity ranking.

Reviewer_sVAsOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes a framework combining time-conditioned reconstruction, motion-guided geometric rectification, and forward imaging consistency to address thermal sequence reconstruction under coupled microbolometer thermal persistence and rolling-shutter readout. Experiments on synthetic and real-world infrared data demonstrate promising performance.

Critical strengths
  • The paper addresses a meaningful and relatively underexplored problem in infrared imaging. The motivation is clear and well presented.
  • The unified observation model provides a reasonable formulation for jointly considering thermal persistence and rolling-shutter distortion.
  • The experimental results are strong, with comparisons against multiple baselines and qualitative evaluations on real infrared cameras.
Critical weaknesses
  • The contribution seems incremental. The proposed architecture combines several established techniques, e.g., optical-flow alignment, gated fusion, state-space modeling, wavelet refinement, and flow-based geometric rectification.
  • While the ablations provide individual components, comparisons with independently modeled thermal blur and rolling-shutter degradation are necessary.
  • Since the reblur process uses GT states for part of the thermal history, it is unclear how much additional benefit this constraint provides beyond direct reconstruction supervision.
  • The synthetic experiments use fixed thermal-response and rolling-shutter parameters. Evaluating different thermal time constants and readout durations would strengthen the generalization claims.
  • The real-world evaluation is primarily qualitative, additional quantitative analysis of geometric correction or temporal consistency would offer stronger evidence of practical effectiveness.
Questions for authors

Please refer to the Weaknesses.

9 comments

W1: Incremental architectural contribution.

Response. We thank the reviewer for this important comment. We agree that our framework draws on several established technical ideas, including optical-flow estimation, state-space modeling, wavelet processing, and flow-based rectification. However, these techniques are not directly transplanted as independent off-the-shelf modules. Their formulations and interactions are specifically adapted to the coupled thermal-persistence and rolling-shutter reconstruction problem.

These adaptations are driven by the specific failure modes of motion-degraded uncooled thermal observations. First, optical flow estimated from blurred thermal frames can be unreliable; therefore, instead of directly aggregating warped neighboring features, we introduce a center-dominant Temporal Confidence-Controlled Fusion module that injects neighboring information only through confidence-gated residual paths. Second, our geometric rectification is not a direct reuse of a conventional RSC module. We deliberately preserve the pre-alignment features to retain the native RS geometry and condition correspondence estimation jointly on row-time offsets and bidirectional motion cues. Third, state-space modeling and wavelet refinement are adopted as task-oriented choices for long-range propagation and detail recovery, rather than as standalone contributions.

The effectiveness of these task-specific designs is also supported by the ablation studies in the original manuscript. Removing the motion key, row-time condition, explicit rectification, or feature fusion consistently degrades geometric recovery, while removing flow guidance or the confidence-controlled fusion also reduces reconstruction quality. Likewise, replacing the proposed reconstruction design with alternative scanning or refinement strategies leads to inferior results. These controlled comparisons indicate that the performance gain does not arise from simply stacking generic components, but from adapting and organizing them for the coupled thermal-RS setting.

More fundamentally, the main novelty of this work lies at the problem-formulation and task-specific system-design level. We formulate thermal sequence reconstruction under the coupled degradation of microbolometer thermal persistence and row-dependent rolling-shutter readout. This formulation determines the overall reconstruction strategy: temporally resolved sharp states are first recovered on the RS grid, geometric rectification is then performed in the latent space to obtain GS states, and the reconstructed RS states are linked back to the degraded observation through the forward thermal-response model. Thus, temporal reconstruction, geometric correction, and physical consistency are not treated as independent restoration modules, but are organized according to the coupled image-formation process.

We further strengthened this point in the revised manuscript with two additional controlled experiments. A decoupled two-stage variant, in which our temporal reconstruction stage is followed by an independent DFRSC correction stage, performs consistently worse than the proposed coupled design (Appendix C, Table 8 and Fig. 14). In addition, incorporating the proposed physical reblur constraint into the independent PMBNet architecture also improves its reconstruction performance (Appendix B, Table 5 and Fig. 11), showing that the physical constraint is not merely specific to our own backbone.

Finally, the benchmark itself is also an important part of the contribution, providing temporally aligned sharp RS and GS supervision for quantitative evaluation of sequence reconstruction under coupled thermal persistence and rolling-shutter degradation, for which dedicated benchmarks remain scarce.

Accordingly, our contribution is not the isolated use of mature techniques, but the physical formulation, task-specific adaptation, and system organization that determine how these techniques should interact for thermal sequence reconstruction.

W2: Comparison with independently modeled thermal blur and rolling-shutter degradation.

Response. We thank the reviewer for this important suggestion. We agree that the original presentation did not sufficiently emphasize the comparisons against independently staged restoration pipelines.

In fact, except for PMBNet, all competing pipelines in our main comparison follow a decoupled strategy. As reported in Table 1 and Fig. 4, DeMFI and BiT first reconstruct sharp RS-domain sequences and are subsequently combined with independently trained JAMNet or DFRSC for RS-to-GS correction. Similarly, BSST, DST-Plus, PPFN, and ECMR first perform restoration/deblurring, after which UniVR or LBCNet independently generates the corresponding GS temporal states. Each component is trained separately with its own supervision, and we have further clarified these stage-wise training protocols in Appendix A.5 of the revised manuscript. These decoupled pipelines are also evaluated on the real-world data in Fig. 7 (VIM384GS-USL), Fig. 16 (HIKMICRO), and Table 11.

To provide a more controlled comparison, we have further added a dedicated decoupled two-stage experiment to the revised manuscript, reported in Appendix C, Table 8 and Fig. 14. Our geometric rectification branch cannot be directly separated as an independent RSC network because it operates on the time-conditioned latent representation produced by the preceding reconstruction stage. We therefore first decode our Stage-1 outputs into sharp RS sequences and then use DFRSC as an independent second-stage RSC method. The DFRSC stage follows the same stage-wise training and evaluation strategy used for the decoupled pipelines in the main comparison. In this way, the temporal reconstruction stage is kept unchanged, while RS-to-GS correction is explicitly separated into an independent image-space stage.

The quantitative results from Table 8 are reproduced below:

MethodAll Query TimesFinal Query
PSNR ↑SSIM ↑LPIPS ↓PSNR ↑SSIM ↑LPIPS ↓
Two-Stage28.47240.93190.102828.39780.92930.1076
Ours28.74280.93340.099128.75650.93260.1006

The decoupled two-stage pipeline is consistently inferior across all reported metrics. The visual comparison newly added in Fig. 14 further illustrates this difference on a particularly challenging fast-moving wheel, where severe blur substantially weakens local texture and makes RS correction difficult. Although the two-stage pipeline recovers the overall scene content, its correction of the wheel distortion remains limited, whereas our coupled framework better restores the wheel geometry.

Therefore, the revised evaluation provides two complementary levels of evidence: the main synthetic and real-world comparisons cover a broad range of independently staged restoration+RSC pipelines, while the newly added controlled two-stage experiment keeps our temporal reconstruction stage unchanged and more directly isolates the benefit of coupling temporal reconstruction with geometric rectification.

W3: Benefit of the physical reblur constraint beyond direct reconstruction supervision.

Response. We thank the reviewer for raising this important point. We first clarify the role of the GT states used for the unqueried part of the thermal history. These historical GT states are introduced only to complete the truncated forward thermal-response history. They appear identically in both the synthesized reblurred observation and the corresponding noise-free target, and therefore cancel when the reblur error is computed. Consequently, they do not provide additional supervision to the predicted queried states; the reblur error depends only on the weighted reconstruction errors of the queried states.

The additional benefit of L_phy therefore does not come from these historical GT states. Rather, it provides a different form of supervision from the direct reconstruction loss. The direct RS reconstruction loss constrains each queried state independently at its corresponding time, whereas L_phy constrains how the reconstructed states jointly contribute to the observed thermal image through the physical temporal-response model. This joint formation-level consistency is not explicitly optimized by independent per-state supervision.

Importantly, the original manuscript already contained a direct ablation of this constraint. In the w/o L_phy variant, we remove the physical reblur loss while retaining the direct reconstruction supervision. As reported in Table 2 and Fig. 6, this removal substantially degrades both the quantitative performance and visual reconstruction quality, directly showing that the physical reblur constraint provides additional benefit beyond state-wise reconstruction supervision.

To further isolate where this benefit comes from, we have added a more detailed analysis of the reblur formulation in the revised manuscript (Appendix B, Table 4 and Fig. 10). We consider three alternatives: Unmatched Decay, which uses an incorrect slower thermal decay; Uniform Weight, which assigns equal temporal weights during reblur; and Weighted Frame, which removes reblur and instead applies the same thermal weights independently to the state-wise reconstruction losses.

MethodAll Query TimesFinal Query
PSNR ↑SSIM ↑LPIPS ↓PSNR ↑SSIM ↑LPIPS ↓
Unmatched Decay27.87270.91300.161927.50990.90760.1473
Uniform Weight27.58460.91950.139627.28580.91270.1360
Weighted Frame28.23450.92110.123428.01110.91530.1184
Ours28.74280.93340.099128.75650.93260.1006

The Weighted Frame variant is particularly relevant to this concern because it retains the thermal-response weighting while removing the joint reblur aggregation, and instead applies the weights independently to the state-wise reconstruction losses. Its inferior performance compared with the proposed formulation indicates that temporal weighting alone is insufficient; jointly aggregating the reconstructed states through the forward thermal-response model provides an additional formation-level constraint. The visual comparison in Fig. 10 shows the same trend.

Reviewer_m4esOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper studies sequence reconstruction from uncooled thermal video affected by thermal persistence and rolling-shutter readout. The method first reconstructs sharp states in rolling-shutter coordinates and then maps them to global-shutter geometry. A reblur loss connects the predicted states to the thermal response model during training. Evaluation uses a synthetic benchmark derived from high-speed thermal recordings, with additional qualitative results from two real sensors.

Critical strengths
  • The imaging model is a useful part of the contribution. Thermal persistence differs from the exposure model usually assumed in visible-light deblurring, and the paper accounts for its interaction with row-dependent readout. Separating temporal reconstruction from geometric rectification is a reasonable design for this setting.

  • The comparison is fairly extensive: 13 methods or pipelines are retrained on the proposed data. In Table 1, the method reaches 28.74 dB all-query PSNR, compared with 28.27 dB for the strongest alternative, and achieves the best LPIPS. It does not lead on all-query SSIM, which the paper acknowledges, although it has the best final-query SSIM. The ablations favor the full model on nearly all metrics, and the additional flow-estimator comparison helps assess the dependence on RAFT. The real-camera examples also give some evidence that the method transfers beyond the synthetic data.

Critical weaknesses
  • The physical conditions covered by the evaluation are narrow. Synthetic training and testing use a thermal time constant of 8 ms and a readout duration of 32 ms. For the real cameras, the readout settings and any adjustments to the model are not sufficiently clear. Since the method uses row-time information, this matters when interpreting the cross-sensor results. A description of the settings used for each camera would address much of this concern; a small sensitivity test around the training parameters would further strengthen the evidence.

  • The benchmark leaves some uncertainty about the size and scope of the improvement. The test set contains four scenes, and the source recordings are temporally densified with FLAVR. This is a practical way to construct the benchmark, but the average gain does not show how consistently the method improves across scenes. Per-scene results would be useful here. The six queries span 10 ms, while the interval between the input observations is not stated clearly. Giving that interval and the query timestamps would make the reconstruction task easier to understand and reproduce.

Questions for authors
  • What readout and thermal-response settings are used when processing each real camera, and is any calibration or adaptation performed?
5 comments

W1: Settings statement and sensitivity test.

Response. Thank you for pointing out that the physical conditions in the original evaluation were insufficiently specified. We have revised the manuscript to clarify the sensor-specific settings used in the real-world experiments and have additionally conducted a controlled sensitivity study around the training parameters.

For the synthetic benchmark, the model is trained with a thermal time constant of τth=8\tau_{\mathrm{th}}=8 ms and a rolling-shutter readout duration of Tr=32T_r=32 ms. For the two real cameras, the corresponding physical settings are approximately τth=9\tau_{\mathrm{th}}=9 ms and Tr=28T_r=28 ms for the VIM384GS-USL, and τth=8\tau_{\mathrm{th}}=8 ms and Tr=32T_r=32 ms for the HIKMICRO HM-TM06-LF/ANT. We use the same trained model for both cameras without sensor-specific retraining or fine-tuning. The camera-specific readout duration is used only to construct the row-time condition for geometric rectification, while the network parameters remain unchanged. We have added these camera details to the Real-World Dataset paragraph in Sec. 4 of the revised manuscript.

We also thank the reviewer for suggesting a sensitivity analysis around the training parameters, which provides an important controlled evaluation of robustness to physical-condition mismatch. We additionally synthesize test sets with τth∈{8,10,12}\tau_{\mathrm{th}}\in\{8,10,12\} ms and Tr∈{26,32,38}T_r\in\{26,32,38\} ms. We vary one physical parameter at a time while fixing the other at its training value, and evaluate all settings using the same model trained only at τth=8\tau_{\mathrm{th}}=8 ms and Tr=32T_r=32 ms, without retraining or fine-tuning. The results are reproduced below:

SettingAll Query TimesFinal Query
PSNR ↑SSIM ↑LPIPS ↓PSNR ↑SSIM ↑LPIPS ↓
τth=8\tau_{\mathrm{th}}=8 ms*28.74280.93340.099128.75650.93260.1006
τth=10\tau_{\mathrm{th}}=10 ms28.81480.93320.109528.74290.93220.1116
τth=12\tau_{\mathrm{th}}=12 ms28.72900.93140.119128.61650.93050.1214
Tr=26T_r=26 ms28.47930.93950.096329.31120.94000.0972
Tr=32T_r=32 ms*28.74280.93340.099128.75650.93260.1006
Tr=38T_r=38 ms28.79930.93350.104128.75550.93250.1058

Here, * denotes the training setting. Across these moderate deviations, the reconstruction quality remains stable without substantial degradation, indicating that the model is not tightly restricted to the exact training parameters. We have added this experiment and the corresponding visual comparison to Appendix B, Table 6 and Fig. 12 of the revised manuscript.

In addition to this controlled analysis, the two real cameras provide complementary cross-sensor evidence because they operate under different physical settings. Without any sensor-specific retraining or fine-tuning, the same model is directly evaluated on both the VIM384GS-USL and HIKMICRO HM-TM06-LF/ANT datasets. Since sharp GS ground truth is unavailable for real sequences, we further report complementary no-reference image- and video-quality metrics. CLIP-IQA$^+$ and EN are evaluated frame-wise and averaged over the six reconstructed states, whereas DOVER evaluates the six reconstructed states jointly as a short video sequence. The quantitative results are reproduced below:

MethodVIM384GS-USLHIKMICRO
CLIP-IQA$^+$ ↑EN ↑DOVER ↑CLIP-IQA$^+$ ↑EN ↑DOVER ↑
PMBNet0.15956.15040.01160.23886.14420.0266
DeMFI+JAMNet0.16166.21390.01200.21836.17000.0199
DeMFI+DFRSC0.16726.21840.01210.21996.16970.0198
BiT+JAMNet0.14146.10900.01120.20446.14600.0290
BiT+DFRSC0.14596.10220.01090.21146.15020.0293
BSST+UniVR0.14966.23060.01310.21746.19850.0373
BSST+LBCNet0.13956.25120.01080.21766.20720.0307
DST-Plus+UniVR0.15726.22220.01170.24416.16560.0296
DST-Plus+LBCNet0.15346.24120.00940.25376.16710.0264
PPFN+UniVR0.14876.12290.01160.19676.12130.0309
PPFN+LBCNet0.13796.14680.00930.20376.12510.0265
ECMR+UniVR0.15266.11760.01150.21206.14700.0258
ECMR+LBCNet0.14816.13860.00970.21886.14960.0220
Ours0.19656.29710.01370.26756.15900.0471

These real-world results are reported in Appendix H, Table 11, with qualitative comparisons on both sensors provided in Fig. 7 and Fig. 16. Together with the controlled sensitivity study, they provide complementary evidence that the proposed framework remains effective when the physical imaging conditions deviate from the synthetic training configuration.

W2: Per-scene performance and temporal configuration of the benchmark.

Response. Thank you for raising these concerns regarding both the consistency of the improvement across test scenes and the temporal configuration of the reconstruction task. We have revised the manuscript to address both points explicitly.

First, we agree that reporting only the average performance over the four test scenes does not fully reveal whether the improvement is consistent across different scene contents and motion patterns. We therefore additionally report the quantitative results separately on all four scene-disjoint test subsets. The results are reproduced below:

MethodScene1Scene2Scene3Scene4
PSNR ↑SSIM ↑LPIPS ↓PSNR ↑SSIM ↑LPIPS ↓PSNR ↑SSIM ↑LPIPS ↓PSNR ↑SSIM ↑LPIPS ↓
PMBNet27.51010.88020.221623.87380.96020.103528.22960.94550.092725.84830.90580.1687
DeMFI+JAMNet29.39300.88740.219724.96470.95570.117429.46960.94740.089027.33220.91630.1591
DeMFI+DFRSC30.58980.90240.208525.25680.95590.116029.60370.95130.087727.79320.93200.1476
BiT+JAMNet28.71780.89640.169123.19300.95960.086629.00290.94870.090127.86210.93310.1502
BiT+DFRSC30.69190.93400.144723.95590.96400.087529.27000.95380.084827.69580.94170.1473
BSST+UniVR29.38750.88780.155022.80510.94940.095729.56090.92930.086727.95520.93140.1504
BSST+LBCNet28.29250.86870.158722.28290.95110.114429.35770.93800.087427.81290.91440.1492
DST-Plus+UniVR28.81830.88390.217425.53190.94060.114828.45740.91450.083427.01710.92520.1752
DST-Plus+LBCNet28.57770.87850.215725.43840.94700.120928.64700.92650.090226.65880.91800.1852
PPFN+UniVR27.66350.88500.190823.15750.94430.102729.17380.93470.076426.62620.91550.1518
PPFN+LBCNet26.98560.87300.188723.00430.94880.110028.96520.94500.084526.11830.90500.1526
ECMR+UniVR28.96280.87750.251325.59280.91870.146827.51460.90140.123927.23020.91710.2245
ECMR+LBCNet28.41440.86950.249825.49030.92410.151427.27970.91330.128726.66000.90780.2358
Ours30.71740.90370.133125.64280.95700.073431.32070.95400.056927.89680.93440.1465

The per-scene results show that our method consistently provides strong performance across different scenes rather than relying on a large gain from a particular subset. In particular, it achieves the best PSNR on Scenes 1--3 and the best LPIPS on all four scenes, while remaining competitive in SSIM. We have added these results to Appendix E, Table 9 of the revised manuscript.

Regarding temporal densification, FLAVR is used only to provide sufficiently dense latent motion samples for row-dependent rolling-shutter synthesis. The original cooled infrared sequences are captured at 1,000 fps, corresponding to a 1-ms temporal spacing. Since a 32-ms readout over 512 rows corresponds to approximately 62.6 μ\mus between adjacent rows, we temporally densify the source sequence by 8×8\times to 0.125 ms before row-wise sampling. This operation is distinct from the thermal-response discretization, which uses a 2-ms interval over the truncated thermal history. We have clarified this distinction in Appendix A.1 of the revised manuscript.

We also thank the reviewer for pointing out that the temporal relationship between the input observations and reconstructed queries was not sufficiently explicit. In our benchmark, the rolling-shutter readout duration is Tr=32T_r=32 ms and the inter-frame dead time is Td=10T_d=10 ms, yielding a center-to-center observation interval of Tf=42T_f=42 ms. For each current observation, we reconstruct the six most recent states at {−10,−8,−6,−4,−2,0}\{-10,-8,-6,-4,-2,0\} ms relative to its center-row time. We have clarified the observation interval and query timestamps in Sec. 4 and 5 of the revised manuscript.

Importantly, the 10-ms reconstruction range is not intended to span the 42-ms interval between consecutive observations. Rather, it targets the temporally dominant states contributing to the current observation under the causal thermal response. These six states already account for 78.1% of the normalized truncated thermal-response weight. As further analyzed in Appendix B, Table 7 and Fig. 13, extending the reconstruction to more historical states introduces increasingly weakly observable states and degrades the recovery of the dominant recent states.

Q1: Real-camera settings and sensor-specific adaptation.

Response. Thank you for your question. For the VIM384GS-USL, we use a readout duration of approximately Tr=28T_r=28 ms, while for the HIKMICRO HM-TM06-LF/ANT we use Tr=32T_r=32 ms. These camera-specific readout durations are used to construct the row-time condition for geometric rectification. The corresponding thermal-response constants are approximately τth=9\tau_{\mathrm{th}}=9 ms for VIM and τth=8\tau_{\mathrm{th}}=8 ms for HIK.

Importantly, τth\tau_{\mathrm{th}} is not used as an explicit inference-time input to the network. The model is trained only on the synthetic setting with τth=8\tau_{\mathrm{th}}=8 ms and Tr=32T_r=32 ms, and the same trained checkpoint is directly applied to both real cameras. No sensor-specific retraining, fine-tuning, or thermal-response calibration is performed; only the camera-specific readout duration is supplied for the row-time encoding. We have clarified these settings in the Real-World Dataset paragraph in Sec. 4 of the revised manuscript.

The controlled sensitivity results reported in Appendix B, Table 6 and Fig. 12 from the revised manuscript further show that the model remains stable when the thermal time constant and readout duration deviate from the training configuration, which supports this direct cross-sensor deployment setting.

Comments on the paper

1 comment

Dear reviewers,

the authors have now submitted their responses. Please review them and share your thoughts on whether your concerns have been addressed. I encourage you to engage with one another’s assessments, discuss any remaining disagreements, and update your reviews and scores as appropriate.

Thank you. AC

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.