acceptodds
Under review as a conference paper at ICLR 2027

HaRP: High Dynamic Range Photosequencing through Dual Reversed Shutter Scanning

Abstract

The adoption of CMOS sensors in mobile photography is frequently compromised by the rolling shutter (RS) effect, which introduces geometric distortions and motion artifacts. Particularly, recent rolling shutter with global reset (RSGR) mode, while mitigating some RS issues, also incurs major limitations, including reduced capture speed and compressed dynamic range. To address these problems, we propose a novel dual reversed scanning setup utilizing both RSGR and inverted RSGR views. This solution not only handles the inherent flaws of RSGR by synchronizing complementary exposures to balance the dynamic range across the frames but also introduces an effective method for HDR photosequencing under highly dynamic scenes. Our proposed network first accommodates row-wise complementarity and manages visual shifts by row-adaptive feature alignment. Subsequently, the hallucination module, built upon a correlation-guided mix-attention block, integrates the mutually reinforced features to recover missing details. In addition, we construct a coaxial imaging system to collect a real-world dataset, enabling robust training and evaluation beyond numerical simulation. Experimental results demonstrate the twofold benefits of our solution in mitigating RSGR limitations and advancing HDR reconstruction techniques.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
motion degradation, RS correction, high dynamic range, high-speed imaging

Reviews

Among:
Shape:
1234this paper, Top 17%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.75 over 4 reviews, above 83% of the others.
Reviewer_xrV3OpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

CMOS sensors have dominated the consumer market while it has severe limitations, e.g., rolling shutter effect. Although the recent RSGR mode mitigates this issue to some extent, it also incurs two limitations: reduced capture speed and compressed dynamic range. To tackle this problem, this paper introduces a novel dual reversed scanning setup utilizing both RSGR and inverted RSGR views. In addition, the authors also construct a coaxial imaging system to collect a real-world dataset. Experiments verify the effectiveness of the proposed method.

Critical strengths
  1. This paper proposes HaRP, which mitigates two main limitations of RSGR. By combining RSGR and iRSGR, HaRP establishes row-wise exposure complementarity directly at the physical imaging layer. This concept is elegant and fundamentally alleviates the ill-posed nature of RSGR dynamic range compression and motion blur.

  2. The authors constructed a functional coaxial optical platform and capture RSGR-HDR consisting of 90 dynamic sequences with 500 fps HDR ground truth.

  3. The authors adapted representative baselines into dual-view variants, demonstrating significant and consistent quantitative advantages on both real and synthetic benchmarks.

Critical weaknesses
  1. When scanning reaches the sensor's central rows, the exposure durations of the RSGR and iRSGR cameras converge to the exact same value. If high-contrast highlights or deep shadows appear in this central region, both cameras will saturate or underexpose simultaneously. DoesHDR recovery in the center rows rely predominantly on learned neural inpainting rather than physical measurement?

  2. On the synthetic GOPRO-Dual dataset, the proposed method achieves 35.88 dB; however, on the physical RSGR-HDR benchmark, the PSNR plunges to 24.24 dB. Does this indicate potential unmodeled systematic errors in the physical capturing rig?

  3. Sec. 4.1 acknowledges that the network directly outputs tone-mapped LDR images, and evaluation relies entirely on standard LDR metrics (e.g., PSNR, SSIM, LPIPS). Standard linear radiance metrics (HDR-VDP-2, PSNR-PU, SSIM-μ) are omitted. This is also inconsistent with the title of this paper.

Questions for authors
  1. What is the fundamental cause of the 11.6 dB degradation between synthetic GOPRO-Dual (35.88 dB) and real RSGR-HDR (24.24 dB)? Is it sensor noise, beam-splitter light attenuation, or residual parallax that fails to resolve in 3D scenes?

  2. If claiming HDR reconstruction, please report evaluations predicting unclipped linear radiance maps under standard HDR metrics (HDR-VDP-2 / 3, PSNR-PU, etc).

3 comments

Weakness 1 Physical Measurements and Central-row Recovery

The exposure durations become identical only at the exact central row under an ideal static configuration. Identical exposure does not necessarily mean invalid measurement: most pixels remain properly exposed unless the scene contains extreme highlights or shadows. Moreover, our reconstruction does not process this row independently. RFA aggregates row-shaped neighborhoods, allowing the central row to exploit physically measured information from adjacent rows, where exposure complementarity immediately reappears. Motion-induced visual shifts further cause the same scene content to be observed at different row indices and exposure durations.

To evaluate the practical influence of central exposure convergence, we uniformly divide each reconstructed frame and its corresponding GT into Top, Middle, and Bottom regions:

MethodFullTopMiddleBottom
RSGR (Initial)12.30 / 0.5450 / 0.277414.25 / 0.5827 / 0.232613.39 / 0.5423 / 0.268611.48 / 0.5107 / 0.2766
LAN23.66 / 0.7122 / 0.355222.67 / 0.6907 / 0.351223.66 / 0.6902 / 0.348626.04 / 0.7156 / 0.2949
LAN Gain+11.37 / +0.1672 / −0.0778+8.42 / +0.1080 / −0.1186+10.27 / +0.1479 / −0.0799+14.56 / +0.2049 / −0.0183
UniINR20.61 / 0.7898 / 0.232220.13 / 0.7802 / 0.216821.34 / 0.7858 / 0.220122.41 / 0.8078 / 0.2304
UniINR Gain+8.32 / +0.2448 / +0.0451+5.88 / +0.1975 / +0.0158+7.94 / +0.2435 / +0.0485+10.93 / +0.2971 / +0.0462
Ours23.95 / 0.8595 / 0.105324.17 / 0.8750 / 0.088723.87 / 0.8380 / 0.112125.31 / 0.8654 / 0.0903
Ours Gain+11.65 / +0.3145 / +0.1721+9.92 / +0.2923 / +0.1439+10.48 / +0.2957 / +0.1565+13.83 / +0.3547 / +0.1863

Each entry reports PSNR/SSIM/LPIPS. RSGR represents the initial input performance, and Gain denotes the absolute metric improvement over RSGR in the corresponding region.

In practice, the Middle region still achieves substantial gains of 10.48 dB PSNR, 0.2957 SSIM, and 0.1565 LPIPS, demonstrating that central exposure convergence does not cause reconstruction failure or predominant reliance on unconstrained neural inpainting. Nevertheless, if both views are simultaneously saturated or underexposed and no valid neighboring evidence exists, recovery must rely on learned image priors. This is a fundamental limitation shared by all reconstruction methods when the physical measurements contain no valid information.

Weakness 2 PSNR Difference Between Real and Synthetic Data

Absolute PSNR values across different datasets—particularly synthetic and real-world benchmarks—should be interpreted with caution because they reflect different data distributions and levels of difficulty. Accordingly, the gap between 35.88 dB on GOPRO-Dual and 24.24 dB on RSGR-HDR does not necessarily indicate unmodeled systematic errors in the physical capturing rig. We provide the following analyses to clarify this difference.

First, synthetic data cannot fully reproduce real imaging degradations, and previous studies have demonstrated a substantial domain gap between synthetic and real restoration data due to simplified imaging models [1, 2]. In particular, widely used video-based synthesis pipelines, including ours, temporally interpolate relatively low-frame-rate videos before synthesizing the degraded observations. This produces smoother and more regular frame transitions than real capture, while the simplified forward model further omits many physical degradations, substantially reducing the reconstruction difficulty. This is precisely why we construct a dedicated imaging system and collect the real-world RSGR-HDR dataset as one of our main contributions; GOPRO-Dual is used only for supplementary validation.

Second, instead of directly comparing the final PSNR values, we additionally measure the initial input PSNR and calculate the improvement achieved by our method:

DatasetInitial RSGROursPSNR Gain
GOPRO-Dual19.0235.88+16.86
RSGR-HDR12.3024.24+11.94

After accounting for the initial input quality, the difference decreases from 11.64 dB in absolute PSNR to 4.92 dB in performance gain, demonstrating that a substantial part of the apparent gap originates from the different initial difficulty of the two datasets.

Third, directly comparing the same method’s absolute metrics across different datasets is generally not meaningful because PSNR strongly depends on scene content, dynamic range, degradation severity, and ground-truth distribution. For example, our method obtains 23.54 dB on Adobe-Dual and 24.24 dB on LiU-HDRv-Dual in Section B.2, which are comparable to the 24.24 dB obtained on the real RSGR-HDR benchmark, although Adobe-Dual and LiU-HDRv-Dual do not contain errors from our physical capturing rig. This further confirms that the lower PSNR on RSGR-HDR reflects dataset difficulty and distribution rather than unmodeled systematic errors in the physical system.

[1] Rim et al., “Real-World Blur Dataset for Learning and Benchmarking Deblurring Algorithms,” ECCV 2020.
[2] Rim et al., “Realistic Blur Synthesis for Learning Image Deblurring,” ECCV 2022.

Weakness 3 HDR Evaluation Metrics

As explained in lines 375–391, our output design is grounded in the established exposure-fusion paradigm (Mertens et al., 2007; Lin et al., 2023; Chen et al., 2025). Following these works, we directly reconstruct display-ready tone-mapped results rather than first estimating linear radiance maps and then applying tone-mapping operator. As demonstrated by UltraFusion (Chen et al., 2025), this design avoids error accumulation and additional artifacts introduced by the conventional two-stage pipeline.

Moreover, the “linear-HDR metrics” mentioned in our paper are not actually computed in linear space. In fact, the reviewer-suggested PSNR-PU and SSIM-μ\mu are also computed in the tone-mapped LDR domain: the linear radiance maps are first transformed using PU encoding or μ\mu-law tone mapping and then evaluated using standard PSNR or SSIM. Here, “PU” and “μ\mu” simply indicate the specific tone-mapping transformation used. Since our network directly outputs tone-mapped results, we do not need to first reconstruct linear radiance maps and then apply another tone-mapping transformation before computing PSNR and SSIM. Therefore, our PSNR and SSIM evaluation is fundamentally consistent with PSNR-PU and SSIM-μ\mu, as all of them are computed on clipped, tone-mapped representations of the underlying linear radiance maps.

In addition, following UltraFusion, we report TMQI (Tables 1 and 5), an important metric widely used in HDR imaging. Therefore, our evaluation using PSNR, SSIM, and TMQI is fully consistent with the HDR objective and title of this paper. To further compare with conventional HDR reconstruction methods, we additionally report both HDR-VDP-2 and HDR-VDP-3 below.

MethodHDR-VDP-2 ↑\uparrowHDR-VDP-3 ↑\uparrow
LAN24.76355.7439
HDRFlow24.33155.2198
Dai et al.38.33656.9527
LRHDR23.37075.6970
DeAltHDR17.65065.2260
Ours57.25988.2791

Our method achieves the best performance on both HDR-VDP-2 and HDR-VDP-3, further demonstrating its HDR reconstruction quality in terms of perceptual fidelity and detail preservation across a wide dynamic range.

 

Questions

Q.1 Please refer to our response to Weakness 2.

Q.2 Please refer to our response to Weakness 3.

Reviewer_8PXbOpenReview ↗
Rating
3
Confidence
3
Informativeness
3
Presentation
3
Soundness
2
Read review
Paper summary

This paper introduces a new pipeline named HaRP for HDR photosequencing under a new dual-reversed RSGR capture setting. It also presents RSGR-HDR, a real-world dataset for training and evaluating this task.

Critical strengths
  1. The dual reversed capture setup is an interesting way to obtain complementary exposures within a shared time window, and the authors support it with a real multi-camera dataset.
  2. The paper evaluates single-frame reconstruction and multi-frame synthesis on real and synthetic data, with same-input adapted baselines and component ablations.
Critical weaknesses
  1. Narrow deployment setting. The method requires synchronized, calibrated cameras with opposite global-reset readout directions, and the experiments do not establish how well it transfers beyond this specialized 640 × 480 capture.
  2. Central row exposure degeneracy. Around the image center, the two views have nearly identical exposure times, so they may saturate together and provide no complementary evidence for HDR recovery; the paper lacks a row-wise evaluation of this limitation.
  3. Capture rate claim is unclear. Both input cameras still operate at 25 fps with a 20 ms dead time per cycle, so the work demonstrates temporal reconstruction from each pair rather than an increase in physical acquisition rate; the reported 0.598 s inference time also exceeds the 40 ms capture period.
  4. Limited Comparisons. The HDR-specific baselines stop at HDRFlow (2024); the paper should discuss more recent methods, such as DeAltHDR (ICLR 2026), LRHDR (CVPR 2026), and compare with them where their different input settings can be fairly adapted.
Questions for authors
  1. How does the proposed setup overcome the claimed capture-rate limitation when both input sensors remain synchronized at 25 fps with 20 ms of dead time per cycle?
4 comments

Weakness 1 Generalization Beyond the Prototype Capture Setting

The synchronized RSGR–iRSGR pair is an acquisition requirement of our dual-reversed setting, but the network itself is not restricted to the 640×480640\times480 resolution of our prototype. As described in Appendix A.2.2, GOPRO-Dual uses 512×512512\times512 inputs generated from independent 1280×7201280\times720 videos. More importantly, Appendix B.2 evaluates the same pretrained model without any finetuning or domain adaptation on two third-party testsets, Adobe-Dual and LiU-HDRv-Dual. Their corresponding synthetic settings are summarized below:

Synthetic SettingAdobe-DualLiU-HDRv-Dual
Exp. per Row2∼24.92\sim24.9 ms2∼48.82\sim48.8 ms
Delay per Row≈65.1 μ\approx65.1\ \mus≈65.1 μ\approx65.1\ \mus
Exp. per Frame≈24.9\approx24.9 ms≈48.8\approx48.8 ms
Deadtime00 ms00 ms
Equivalent Frame Rate≈40\approx40 fps≈21\approx21 fps
Resolution640×352640\times3521280×7201280\times720

These datasets contain unseen scene contents, motion patterns, illumination conditions, synthesis parameters, and challenging under- and overexposed regions. Our method achieves the best overall performance on both datasets.

Furthermore, Table 4 evaluates spatial misalignments and shows no significant performance degradation, demonstrating robustness to imperfect camera calibration. These experiments establish generalization across input resolutions, scene distributions, motions, illumination conditions, and calibration errors, although our current real-world prototype captures at 640×480640\times480. As discussed in our response to Weakness 1 of Reviewer 2, the dual-reversed setup could also potentially be extended to a more compact stereo-camera configuration.

 

Weakness 2 Region-wise Evaluation of Central Exposure Degeneracy

We agree that the nominal exposure complementarity is weakest around the central rows. However, the actual reconstruction gain cannot be determined by the row index alone, since it is jointly affected by scene illumination, motion, saturation, sensor noise, and motion-induced visual shifts. To directly evaluate this issue, we uniformly divide each reconstructed frame and its corresponding GT into Top, Middle, and Bottom regions and evaluate them separately:

MethodFullTopMiddleBottom
RSGR (Initial)12.30 / 0.5450 / 0.277414.25 / 0.5827 / 0.232613.39 / 0.5423 / 0.268611.48 / 0.5107 / 0.2766
LAN23.66 / 0.7122 / 0.355222.67 / 0.6907 / 0.351223.66 / 0.6902 / 0.348626.04 / 0.7156 / 0.2949
LAN Gain+11.37 / +0.1672 / −0.0778+8.42 / +0.1080 / −0.1186+10.27 / +0.1479 / −0.0799+14.56 / +0.2049 / −0.0183
UniINR20.61 / 0.7898 / 0.232220.13 / 0.7802 / 0.216821.34 / 0.7858 / 0.220122.41 / 0.8078 / 0.2304
UniINR Gain+8.32 / +0.2448 / +0.0451+5.88 / +0.1975 / +0.0158+7.94 / +0.2435 / +0.0485+10.93 / +0.2971 / +0.0462
Ours23.95 / 0.8595 / 0.105324.17 / 0.8750 / 0.088723.87 / 0.8380 / 0.112125.31 / 0.8654 / 0.0903
Ours Gain+11.65 / +0.3145 / +0.1721+9.92 / +0.2923 / +0.1439+10.48 / +0.2957 / +0.1565+13.83 / +0.3547 / +0.1863

Each entry reports PSNR/SSIM/LPIPS. RSGR represents the initial input performance, and Gain denotes the absolute metric improvement over RSGR in the corresponding region.

Despite the weaker nominal exposure complementarity, the Middle region still achieves substantial gains of 10.48 dB PSNR, 0.2957 SSIM, and 0.1565 LPIPS. The results confirm that, in practice, performance gains do not strictly follow row-wise exposure-time complementarity due to complex scene-dependent factors. Nevertheless, simultaneous saturation of both views remains a fundamental limitation shared by all reconstruction methods, rather than being specific to ours, because no valid observations are available for recovering the missing information.

Weakness 3 Clarification of Offline Reconstruction and Temporal Rate

Our method is designed for offline computational reconstruction rather than real-time in-camera processing. The synchronized RSGR–iRSGR pairs are first captured and stored at 25 fps, after which each pair is processed to reconstruct 3, 5, or 9 latent HDR frames within its exposure window. Therefore, the reported 0.598 s inference time is not required to fit within the 40 ms capture period and does not affect the physical acquisition process.

Our “high-speed” claim refers to the increased temporal resolution of the reconstructed HDR sequence, consistent with related rolling-shutter temporal super-resolution (e.g., RSSR), blur decomposition (e.g., AfB, DeMFI, and BiT), and video frame interpolation methods (e.g., RIFE), which reconstruct intermediate latent frames from low-frame-rate observations rather than increasing the physical frame rate of the input cameras.

 

Weakness 4 Comparison with Recent Methods

We have added comparisons with five recent methods: SelfRSSplat (TCSVT 2025), Dai et al. (TOG 2025), TSANet (CVPR 2026), LRHDR (CVPR 2026), and DeAltHDR (ICLR 2026). All methods are evaluated using the same ground truths and evaluation metrics. For LRHDR and DeAltHDR, we generate their required alternating-exposure inputs from the same HDR source following our synthesis protocol. For Dai et al., we synthesize the optically randomized GRR measurements according to its imaging model. Since LRHDR, DeAltHDR, and Dai et al. reconstruct only a single HDR frame rather than a latent HDR sequence, they are evaluated only under the HDR reconstruction setting.

MethodInputHDR ReconstructionPhotosequencing ×3\times3Photosequencing ×5\times5Photosequencing ×9\times9
SelfRSSplat2·RG12.60 / 0.6007 / 0.2535 / 0.761012.60 / 0.6031 / 0.2523 / 0.762712.61 / 0.6040 / 0.2517 / 0.763312.61 / 0.6045 / 0.2513 / 0.7637
TSANetRG-E21.45 / 0.7905 / 0.1109 / 0.833421.37 / 0.7871 / 0.1130 / 0.831021.38 / 0.7877 / 0.1125 / 0.831521.39 / 0.7878 / 0.1122 / 0.8318
LRHDR3·LS24.03 / 0.7100 / 0.3231 / 0.8004– / – / – / –– / – / – / –– / – / – / –
DeAltHDR3·LS23.65 / 0.6610 / 0.3928 / 0.7441– / – / – / –– / – / – / –– / – / – / –
Dai et al.GRR + OR23.75 / 0.7918 / 0.2402 / 0.7875– / – / – / –– / – / – / –– / – / – / –
OursRG-iRG24.24 / 0.8679 / 0.0994 / 0.854323.84 / 0.8562 / 0.1071 / 0.848023.91 / 0.8584 / 0.1058 / 0.848723.95 / 0.8595 / 0.1053 / 0.8490

Each entry reports PSNR/SSIM/LPIPS/TMQI. OR denotes optical randomization. Since the codes of Dai et al., DeAltHDR, and TSANet are unavailable, we reimplemented them following their papers.

Our method consistently outperforms these recent approaches while uniquely supporting both HDR reconstruction and multi-frame HDR photosequencing.

 

Questions

Q.1 Please refer to our response to Weakness 3.

Reviewer_8PXbOpenReview ↗

Response to the Author's Rebuttal

I thank the authors for their detailed rebuttal, particularly the explanation and experiment for the central exposure degeneracy.

All my concerns have been addressed, and I will maintain my positive rating. Please incorporate the rebuttal content into the revised manuscript.

Thank you for taking the time to review our rebuttal and for maintaining your positive rating. We greatly appreciate your constructive feedback and will incorporate the additional explanations and experimental results into the revised manuscript.

Reviewer_rNX2OpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
2
Soundness
3
Read review
Paper summary

This paper proposes HaRP, a framework for high dynamic range (HDR) video reconstruction using dual reversed rolling shutter cameras with global reset (RSGR). By exploiting the complementary exposure patterns of two cameras scanning in opposite directions, the method aims to address the limitations of conventional RSGR imaging. The authors develop a network with row-adaptive feature alignment and correlation-guided mix-attention to recover sharp HDR video frames. They also build a four-camera imaging system and collect a real-world dataset for training and evaluation. Extensive experiments on both real and synthetic datasets demonstrate the effectiveness of the proposed approach.

Critical strengths
  1. The motivation is clear and well-grounded in the physical characteristics of RSGR cameras. The authors identify the issue of uneven exposure caused by row-wise scanning and propose a dual reversed scanning setup to exploit complementary exposure information.
  2. The authors build a real-world imaging system consisting of four cameras and collect a dataset with corresponding high-speed HDR ground truth.
  3. The experimental evaluation is comprehensive, covering comparisons with various rolling shutter correction, video interpolation, deblurring, and HDR reconstruction methods.
  4. The extensive experimental results demonstrate the effectiveness of the proposed approach.
Critical weaknesses
  1. The proposed approach requires two RSGR cameras to address the limitations of a single RSGR camera. The authors should further discuss whether the performance improvements justify the additional hardware cost.
  2. The computational cost of the proposed model is relatively high compared with some existing methods.
  3. The novelty of the proposed network architecture is relatively limited.
Questions for authors

Please see the weaknesses.

4 comments

Weakness 1 Justification of the Dual-Camera Setup

First, compared with single-camera methods using temporally adjacent frames, our setup introduces additional hardware but delivers substantial performance gains. In Table 1, our method achieves 24.24/0.8679/0.0994, compared with 21.39/0.7796/0.1397 for JAMNet, the strongest single-camera baseline. More importantly, controlled comparisons using the same network architectures isolate the benefit of the dual-reversed setup:

NetworkConsecutive RSGRDual-reversed RSGRPSNR Gain
RIFE / RIFE$_m$19.80 / 0.7542 / 0.191122.37 / 0.8445 / 0.1262+2.57 dB
IFED / IFED$_m$12.30 / 0.4635 / 0.272321.79 / 0.8166 / 0.1072+9.49 dB
BiT / BiT$_m$18.60 / 0.7437 / 0.178322.51 / 0.8341 / 0.1279+3.91 dB

Each entry reports PSNR/SSIM/LPIPS for HDR reconstruction. The single-camera setting uses temporally adjacent RSGR frames, whereas the dual-reversed setting uses a synchronized RSGR–iRSGR pair.

These substantial improvements demonstrate that the additional hardware provides a worthwhile performance gain for high-speed HDR reconstruction in dynamic scenes.

Second, multi-sensor acquisition is well established in related imaging research, including Albl et al. (2020), Zhong et al. (2022), Ji et al. (2023, 2024, 2026), and Kronander et al. (2014). Several competing methods—EBFI, EvUnroll, EFI, and UniINR—additionally require a specialized event camera and cross-modal synchronization and calibration. We also include the recent event-assisted method TSANet (CVPR 2026) in our newly added experiments below. Our method still achieves clear performance advantages over these multi-sensor solutions, even when compared with more complex event-assisted systems. The comparison is summarized below:

MethodAdditional ModalityPSNRSSIMLPIPS
EBFIEvents18.250.75180.1464
EvUnrollEvents14.390.39290.3285
EFIEvents18.610.80230.1664
UniINREvents20.750.80080.2173
TSANetEvents21.450.79050.1109
OursiRSGR24.240.86790.0994

Finally, although our current prototype uses a beam splitter to ensure precise spatial alignment, the dual-reversed setup is not intrinsically restricted to this coaxial configuration. It could potentially be extended to a compact stereo-camera module using two conventional RSGR sensors with opposite readout directions. Stereo calibration and rectification may then remove the need for a beam splitter, offering a possible path toward lower system cost, a smaller optical footprint, and reduced light loss.

Weakness 2 Computational Cost and Efficiency

Our current model is not specifically optimized for efficiency. However, the reported 0.598 s corresponds to jointly reconstructing a nine-frame HDR video clip at 640×480640\times480, rather than a single frame. Conventional HDR reconstruction methods must be repeatedly applied to generate multiple frames, whereas our model processes the entire sequence in a unified manner.

We further analyze the computation introduced by our key components:

VariantTime (s)Params (M)FLOPs (G)PSNR / SSIM / LPIPS
Single-scale RFA0.54434.08178.222.79 / 0.8383 / 0.1214
STB instead of RCMB0.49933.44106.322.48 / 0.8372 / 0.1208
Ours0.59835.05183.123.95 / 0.8595 / 0.1053

The multi-scale RFA introduces only limited additional computation over its single-scale counterpart, while RCMB provides more effective cross-view interaction than the standard STB. Therefore, the additional computation directly contributes to reconstruction performance rather than constituting redundant overhead. Overall, our method provides a favorable trade-off between computational cost and reconstruction quality.

 

Weakness 3 Novelty of the Proposed Framework

The proposed architecture is specifically designed for the row-wise exposure complementarity of dual-reversed RSGR inputs. RFA introduces multi-scale row-shaped attention to accommodate motion-induced visual shifts, RCMB incorporates local correspondence priors through correlation-guided cross-attention, and the tri-pivot temporal loss constrains intermediate- and long-range structural consistency. Table 2 validates their effectiveness: row-adaptive alignment improves PSNR from 22.79 to 23.95 over single-scale alignment, replacing RCMB with a standard Swin Transformer block reduces it to 22.48, and removing temporal supervision reduces it to 23.37.

More broadly, the network architecture is only one of our three main contributions. We (1) introduce a dual-reversed RSGR configuration for high-speed HDR photosequencing, (2) construct a dedicated imaging system and a real-world dataset containing synchronized RSGR–iRSGR observations and high-speed HDR ground truths, and (3) develop a task-specific reconstruction network. Therefore, the overall novelty lies in the joint innovation of the imaging configuration, dataset construction, and network design.

Thanks for the rebuttal. Most of my concerns are resolved. I would keep my positive score.

Thank you for taking the time to carefully review our rebuttal. We are glad that it has resolved most of your concerns, and we sincerely appreciate your constructive comments and continued positive assessment.

Reviewer_DmQvOpenReview ↗
Rating
3(+1)
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes a novel setting for simultaneous rolling-shutter compensation and high dynamic range (HDR) imaging in videos (i.e. photo-sequencing). To that end, the paper uses rolling shutter with global reset (RSGR) cameras, which have a defined and synchronized reset of all sensor rows; the rolling shutter effect in such cameras comes from differing exposure times of the different sensor rows. While RGSR cameras have some benefits for rolling shutter compensation, they also have various drawbacks, e.g. unevenly exposed sensor rows. The paper turns this disadvantage into an advantage by combining an RSGR frame with an inverse RSGR frame in which the exposure times increase from the bottom to the top of the sensor. The benefit is that each row (at least outside of the middle of the sensor) now has two different exposure times, which allows for HDR imaging. To work in this setting, the paper introduces a dataset with simultaneously captured global shutter frames with two different exposures from which a reference HDR image is computed to allow for supervised learning. To reduce rolling shutter artifacts and reconstruct an HDR frame, the paper uses a two-stage approach, where a multi-scale row-adaptive feature alignment mechanism is used to compensate for the rolling shutter and a "mix-attention" mechanism is used for the HDR reconstruction. The paper claims that this allows to "hallucinate" content, including HDR content in the middle of the sensor, where the RSGR and inverse RSGR frames have the same exposure time and thus no inherent HDR information. Experiments on both the proposed dataset and a synthetic dataset show clear improvements over a number of older baselines, which include rolling shutter compensation and HDR reconstruction, as well as a few combination baselines.

Critical strengths
  • The observation that rolling shutter cameras with a global reset (RSGR) can be combined in two different row orderings to create an HRD video is both novel and interesting. Doing this, the paper cleverly turns a disadvantage of RSGR cameras into an advantage. It may not be clear yet how practical this setting is, given that the prototype needs two carefully aligned sensors and a beam splitter, but that is an acceptable trade-off to me for now.
  • The paper contributes a novel dataset for HRD photo sequencing, where the two RSGR cameras are combined with two synchronized global shutter cameras to yield an HRD video baseline. The dataset allows for both supervised learning and evaluation.
  • The results of the approach are both qualitatively and quantitatively appealing (however, some caveats below).
Critical weaknesses
  • The row-adaptive feature alignment mechanism is poorly described and not fully motivated:
    • It did not really become clear what is meant by "row size", why the row size is subtracted from the number of rows to yield the feature size (after division by the stride) and why the row size and stride are exponentiated by the scale k.
    • What the "scale levels" k represent and why we need them is not clearly explained.
    • Which branches are referred to in l. 241? This is the first time a branch is mentioned.
    • Aligned features leading to robust alignment (paraphrased from l. 239f) is a circular argument.
    • It was unclear how the uniform partitioning of the channel dimension according to the scale levels works and why it is used.
    • Why are two different kinds of fusion being used and added in Eq. 5? What is the motivation?
  • The mix-attention block is also a bit obscure:
    • I do not follow the argument that a correlation volume as used here is structurally so different from a cross-attention between the two frames. Why the framing with a correlation volume rather than a cross-attention?
    • The correlation volume has a different tensor shape than the feature map. How can the correlation volume be projected to queries with the size of the feature map? Are the 3x3 blocks concatenated or how does this work?
    • Why are self-attention and correlation volume-based attention additively combined in Eq. 6?
    • What is an RBMC?
  • The tri-pivot temporal loss also "falls a bit from the sky":
    • Why use the three pivot frames? Why use difference maps?
    • How is H defined?
    • Why measure the difference maps in YCrCb space and the reconstruction error in RGB space rather than both in the same space?
  • The experiments are extensive but are still somewhat limited in scope:
    • Almost all the baselines are from 2023 or earlier. A few 2024 papers are also compared to. Is there really no more recent work that can/should be compared to? If there is not, is this topic then really interesting?
    • A comparison to Xiang Dai et al. ACM TOG 2025 seems to be missing (citation as well; this is clearly relevant here).
    • What are the event streams in l. 347? More detail is needed.
    • The ablation compares the mix-attention block only to a Swin Transformer. This is not a good baseline, since we are missing a combination of standard self and cross-attention. The fusion from Eq. 5 is not ablated at all, neither are other choices in both modules (adding self and "cross" attention, choice of pivot frames, color space, etc.). It is also surprising that dense alignment is worse than no alignment at all. Why is this?
    • No citation is provided for the GOPRO-dual data and not details are given in the main paper.
    • There is no evaluation of the dynamic range gain as a function of the row index. The gain is likely to be largest at the top and bottom of each frame and smaller in the middle. But this is not evaluated.
  • One "unintended" consequence of the setup is that the global shutter cameras run at a must shorter exposure time than the RSGR cameras, which means that the approach not only performs HDR reconstruction and rolling shutter correction, but also learns from remove motion blur. This can be seen as beneficial, but it may also disturb the perceptual quality of the video since certain types of motion blur are often used intentionally by the videographer. As far as I could see, this limitation is not clearly discussed in the paper.

Lesser concerns:

  • The term “hallucination module” is perhaps not the best choice. First, hallucination is generally not considered a good thing. Moreover, it implies that some generative model is used, but this is not the case. The approach remains fully discriminative. Why not simply call it a “HDR reconstruction module”?
  • Figure 1b is a bit misleading. It talks about short and long exposure for the RSGR respective iRGSR frames, but whether an exposure is long or short depends on the row of the image not which camera it comes from.
Questions for authors

I like what the paper is trying to do, but as pointed out above, a number of unclear aspects remain. I would appreciate if the authors could clarify the following:

  • Can you explain more clearly how the row-adaptive feature alignment mechanism works and why it is designed as is? See above for more detailed points and questions.
  • Can you explain the design choice between the mix attention and how this differs from using cross-attention? Does it really outperform a more standard cross-attention? Swin is not a good baseline.
  • Can you motivate the tri-pivot loss more clearly?
  • Are there really no more recent baselines that can be compared to? If not, why is this research relevant if the community is seemingly not interested in this setting anymore?
  • Can you analyze what the dynamic range gain looks like as a function of the row index? Is there any dynamic range gain in the middle of the frame?

If the authors can clarify these concerns, I would consider raising my score. (NB: I have raised my score based on the author rebuttal, see discussion message).

6 comments

Weakness 1

As discussed in lines 210–224, our calibrated beam-splitter setup establishes row-to-row correspondence between the simultaneously captured RSGR and iRSGR views (Fig. 1(c)). This motivates the use of row-shaped attention windows to align and fuse features across the two views. Each window has a width equal to the feature-map width WW and a height of rr rows, which we call the “row size.” In dynamic scenes, corresponding rows have different exposure intervals in the two views, so motion can cause visual shifts of varying magnitudes. A single-scale row-wise window (Fig. 4(b)) uses only one value of rr, limiting its ability to accommodate these shifts. We therefore propose a row-adaptive strategy that uses windows of different heights (Fig. 4(c)). In the example in line 224, the three window heights are r=[3,4,5]\boldsymbol{r}=[3,4,5], with corresponding vertical strides s=[2,3,4]\boldsymbol{s}=[2,3,4], where ss denotes the step size when sliding each window vertically along the feature map.

 

1.1 Clarification of Row Size, Scale Index, and Window Partitioning

The three row sizes correspond to three scale levels. Here, kk is the scale-level index (line 228) rather than an exponent. Specifically, rkr^k with k=1,2,3k=1,2,3 denotes the three row sizes r1=3r^1=3, r2=4r^2=4, and r3=5r^3=5, respectively. For a feature map of height HH, a window of height rkr^k sliding vertically with stride sks^k has n=⌊H−rksk⌋+1n=\left\lfloor\frac{H-r^k}{s^k}\right\rfloor+1 valid positions. This is because the last valid window must start no lower than row H−rkH-r^k. Therefore, at scale kk, the feature map is unfolded into nn row-shaped attention windows, yielding a windowed feature of size n×rk×W×C/3n \times r^k \times W \times C/3.

 

1.2 Meaning and Motivation of the Scale Levels

Each scale level kk represents a row-shaped attention window with a specific height rkr^k and vertical stride sks^k. Multiple scale levels are needed because motion in dynamic scenes produces visual shifts of different magnitudes. Windows of different heights accommodate different shift ranges, and their aligned features are subsequently fused adaptively.

 

1.3 Clarification of “Each Branch” in Line 241

The RSGR and iRSGR views are separately processed by the Row-adaptive Feature Alignment module to acquire complementary information from each other. We call these two processing paths the RSGR and iRSGR branches, introduced in line 226. Therefore, “each branch” in line 241 refers to these two branches, in which the multi-scale features are weighted and fused.

 

1.4 Clarification of “Robust Alignment”

Sorry for the confusing expression. By “robust alignments,” we mean robust aggregation. The complete sentence will be revised as follows: “The aligned features under all scales naturally retain motions with different magnitudes, which can be dynamically fused for robust aggregation.”

 

1.5 Clarification of Uniform Channel Partitioning

In Eq. (4), δk\delta_k uniformly partitions the CC channels of F~tw\widetilde{F}_t^w into three C/3C/3-channel groups, one for each scale. The uniform partition is used because every scale-specific feature F~tk\widetilde{F}_t^k has C/3C/3 channels, so its corresponding weight αk\alpha_k must have the same dimension for channel-wise weighting. The softmax across the three groups enables adaptive fusion of the three scales.

 

1.6 Clarification of the Residual Connection in Eq. (5)

Eq. (5) is a residual connection rather than the addition of two different fusion mechanisms. Specifically, the first term serves as the shortcut, while the second term provides an adaptive residual refinement. This design preserves the directly fused multi-scale features while incorporating scale-adaptive information. Moreover, residual connections are widely used in deep networks to preserve the original information, facilitate gradient propagation, and ease network optimization.

Weakness 2

2.1 Correlation-guided Cross-attention

As stated in lines 261–264, the correlation volume guides rather than replaces cross-attention. Our setup establishes strict row-wise correspondence between the two views, while the short exposure time limits visual shifts to a relatively local range. We therefore incorporate this prior by generating the query QcQ_c from a cost volume computed within a proper search range. In contrast, conventional cross-attention directly projects the query from the feature map without explicitly modeling such local correspondences. Our correlation-guided query thus emphasizes information within the relevant matching range and reduces interference from abundant irrelevant information. This is why we adopt correlation-guided cross-attention instead of conventional cross-attention. We further provide a comparison between these two designs below.

MethodPSNRSSIMLPIPS
Conventional Cross-Attention23.480.85390.1201
Correlation-guided Cross-Attention23.950.85950.1053

 

2.2 Clarification of the Correlation-volume Projection

For features of size B×C×H×WB\times C\times H\times W, the 3×33\times3 search produces nine correlation responses per spatial location, forming a volume of size B×9×H×WB\times9\times H\times W. The bidirectional volumes are concatenated channel-wise and projected by a 3×33\times3 convolution to B×C×H×WB\times C\times H\times W. Thus, only the correlation responses—not the 3×33\times3 feature blocks—are concatenated.

 

2.3 Motivation for the Additive Combination in Eq. (6)

Self-attention captures intra-view information, while correlation-guided cross-attention transfers complementary information from the other view. Since their outputs have the same shape, element-wise addition efficiently combines them without increasing the feature dimension. As shown in the additional ablation provided in our response to Comment 4.4, concatenation provides no performance advantage but introduces additional computation.

 

2.4 Clarification of RCMB

We assume that “RBMC” refers to RCMB, which is defined in lines 90–92 as the Residual Correlation-guided Mix-attention Block. An RCMB consists of multiple CMB layers with a residual connection and serves as the basic block of the hallucination encoder–decoder shown in Fig. 3(a).

 

Weakness 3

3.1 Motivation for Three Pivots and Difference Maps

Difference-map supervision is widely used for temporal consistency [1, 2]. Building on this formulation, we make two design choices: using the first, middle, and last frames to constrain intermediate- and long-range consistency, and computing luminance-based difference maps to emphasize structural consistency while reducing interference from color variations. As shown by the pivot-frame ablation in our response to Comment 4.4, three pivots provide a favorable trade-off between reconstruction accuracy and computational cost.

[1] Chung and Cho, “LAN-HDR: Luminance-based Alignment Network for High Dynamic Range Video Reconstruction,” ICCV 2023.
[2] Xu et al., “Frame Difference-Based Temporal Loss for Video Stylization,” 2021.

 

3.2 Definition of HH

As defined in lines 203–205, HH denotes the reconstructed high-speed HDR sequence. However, since HH is also used for the feature-map height in the RFA section, this notation may cause ambiguity. We will therefore replace HH with G^\hat{G} consistently in lines 203–205 and Eqs. (8)–(9), while GG denotes the corresponding ground-truth sequence.

 

3.3 Motivation for Using Different Color Spaces

As explained in lines 291–295, temporal discontinuities mainly arise from incorrectly reconstructed edges and geometric structures. We therefore compute the temporal loss on the Y channel to emphasize luminance and structural consistency while reducing interference from color variations, whereas the RGB reconstruction loss preserves complete color and appearance information. We have also evaluated the same-space alternative in Table 2: calculating both losses in sRGB (V6, R+TR) achieves 23.52/0.8525/0.1204, compared with 23.95/0.8595/0.1053 using our Y-channel temporal loss (Ours, R+TY).

Weakness 4

4.1 Comparison with Recent SOTA Methods

We thank the reviewer for the suggestion. We have added comparisons with four recent SOTA methods: SelfRSSplat (TCSVT 2025), TSANet (CVPR 2026), LRHDR (CVPR 2026), and DeAltHDR (ICLR 2026), covering rolling-shutter temporal super-resolution, event-assisted blur-to-video decomposition, and HDR video reconstruction. All methods are evaluated following the same protocol as Table 5. The results are reported as PSNR/SSIM/LPIPS/TMQI.

MethodInputHDR ReconstructionPhotosequencing ×3\times3Photosequencing ×5\times5Photosequencing ×9\times9
SelfRSSplat2·RG17.60 / 0.6007 / 0.2535 / 0.761017.60 / 0.6031 / 0.2523 / 0.762717.61 / 0.6040 / 0.2517 / 0.763317.61 / 0.6045 / 0.2513 / 0.7637
TSANetRG-E21.45 / 0.7905 / 0.1109 / 0.833421.37 / 0.7871 / 0.1130 / 0.831021.38 / 0.7877 / 0.1125 / 0.831521.39 / 0.7878 / 0.1122 / 0.8318
LRHDR3·LS24.03 / 0.7100 / 0.3231 / 0.8004– / – / – / –– / – / – / –– / – / – / –
DeAltHDR3·LS23.65 / 0.6610 / 0.3928 / 0.7441– / – / – / –– / – / – / –– / – / – / –
OursRG-iRG24.24 / 0.8679 / 0.0994 / 0.854323.84 / 0.8562 / 0.1071 / 0.848023.91 / 0.8584 / 0.1058 / 0.848723.95 / 0.8595 / 0.1053 / 0.8490

Note: Since the codes of DeAltHDR and TSANet are unavailable, we reimplemented them following their papers.

 

4.2 Comparison with Dai et al.

Dai et al. and HaRP use different imaging setups. Dai et al. require a GRR sensor with random optical permutation and reconstruct a static HDR image using TV optimization, whereas HaRP uses real RSGR–iRSGR pairs for dynamic HDR reconstruction. Since RSGR-HDR does not contain the optically randomized GRR measurements or calibration required by Dai et al., direct comparison on real captures is infeasible. We therefore synthesize their required input from the RSGR-HDR test set. Following the settings of Dai et al., we use a fixed ideal random permutation, an 8-bit sensor, Gaussian noise with σ=0.01, saturation erasure, and anisotropic YCbCr-TV reconstruction. For each sample, the ten HS HDR frames within the 20 ms exposure window are integrated using row-dependent GRR exposures from 2 to 20 ms, followed by clipping and quantization. The result is evaluated against the same ×1 target as HaRP. The global gain and TV weight are selected only on the validation set and fixed during testing (g=0.9403, τ=0.15). The quantitative results are reported in Table X. Since no public code or data are available, we reproduce the method according to its published forward model, sensor settings, and reconstruction objective.

MethodPSNRSSIMLPIPSTMQI
Dai et al.23.750.79180.24020.7875
Ours24.240.86790.09940.8543

 

4.3 Clarification of Event Streams

Event streams are sequences of asynchronous brightness-change events captured by event cameras. As described in Appendix A.2.2 (lines 908–913), we synthesize them from high-frame-rate videos using the event simulator of Gehrig et al. (2020). They are used only by the event-assisted baselines.

 

4.4 More Ablation Study

The color-space choice has already been ablated in Table 2: V6 (R+TR) computes the temporal loss in sRGB, while Ours (R+TY) uses the Y channel of YCrCb, with the results discussed in lines 421–424.

Our mix-attention block combines self-attention (SA) and correlation-guided cross-attention (CGCA), whereas the Swin Transformer block in V4 contains only self-attention. We further add three variants: CGCA-only, which retains only correlation-guided cross-attention; SA+CA, which combines standard self- and cross-attention; and MA-Concat, which concatenates the self-attention and CGCA features instead of adding them. For the pivot-frame selection, we additionally evaluate Pivot-2 and Pivot-5, which uniformly sample two and five frames, respectively, against our current Pivot-3 setting.

VariantPSNRSSIMLPIPS
CGCA-only22.920.82930.1366
SA+CA23.480.85390.1201
MA-Concat23.860.85780.1158
Pivot-223.790.85760.1139
Pivot-524.010.85890.1114
Ours23.950.85950.1053

Unlike conventional temporally adjacent frames, the calibrated RSGR and iRSGR views exhibit strict row-wise correspondence. Dense attention ignores this acquisition prior and aggregates features from all spatial locations, introducing many irrelevant matches and substantial interference. It can therefore perform worse than directly concatenating the two-view features without explicit alignment. This also motivates our row-adaptive alignment, which restricts feature matching to row-shaped regions while accommodating visual shifts.

Comments on the paper

1 comment
AuthorsOpenReview ↗

Summary of Rebuttal Discussion and Updates

Dear Area Chair and Reviewers,

This submission initially received ratings of 2, 3, 3, and 2 from Reviewers DmQv, rNX2, 8PXb, and xrV3, respectively. We sincerely thank the Area Chair and all reviewers for their time and effort, particularly the four reviewers for their constructive suggestions and timely follow-up responses during the rebuttal discussion. Their feedback indicates that our responses and additional experiments have addressed or substantially clarified their principal concerns.

For clarity, we summarize the initial ratings, the reviewers’ follow-up assessments, and the currently recorded ratings below:

ReviewerInitial RatingFollow-up AssessmentCurrent Rating
DmQv2Explicitly stated that the rating would be raised3
rNX23Confirmed maintaining the positive rating3
8PXb3Confirmed maintaining the positive rating3
xrV32Explicitly stated that the rating would be raised2

Reviewer DmQv has already updated the rating from 2 to 3. Reviewer xrV3 explicitly stated, “I will raise my score”; however, as of this summary, the intended increase has not yet been reflected in the rating field, which still shows the initial rating of 2. We respectfully highlight this distinction so that the reviewers’ follow-up assessments and the currently recorded ratings can be accurately considered in the final evaluation.

1. Reviewer DmQv: Methodological Clarifications and Experimental Completeness

  • We clarified the row-adaptive feature alignment, the residual connection in Eq. (5), the relationship between CGCA and conventional cross-attention, and the tri-pivot temporal loss. We will also improve the notation and restate the meanings of key variables.
  • We added recent comparisons, attention/fusion/pivot ablations, and region-wise dynamic-range evaluation. We also clarified that simulated events are used only to provide the required inputs for event-assisted baselines.
  • The reviewer acknowledged that the rebuttal and additional experiments clarified many concerns and raised the rating from 2 to 3.

2. Reviewer rNX2: Hardware–Performance Trade-off, Efficiency, and Novelty

  • Controlled comparisons using identical networks showed that the dual-camera setup improves PSNR by 2.57, 9.49, and 3.91 dB over the corresponding single-camera settings. We also compared with event-assisted systems and discussed a potential compact stereo-camera extension.
  • We clarified that the reported runtime covers an entire nine-frame HDR sequence and provided component-level efficiency analyses. We further emphasized our joint contributions in the imaging configuration, real-world dataset, and task-specific network.
  • The reviewer stated that most concerns were resolved and confirmed the decision to maintain the positive rating of 3.

3. Reviewer 8PXb: Generalization, Central-Row Evaluation, and Temporal Rate

  • Zero-shot evaluations on Adobe-Dual and LiU-HDRv-Dual demonstrated generalization across different resolutions and capture settings, while additional experiments verified robustness to spatial misalignment.
  • Region-wise evaluation showed that the Middle region still achieves gains of 10.48 dB PSNR, 0.2957 SSIM, and 0.1565 LPIPS, despite its weaker nominal exposure complementarity.
  • We clarified that HaRP performs offline temporal reconstruction rather than real-time in-camera processing and added five recent methods.
  • The reviewer explicitly stated that all concerns had been addressed and confirmed the decision to maintain the positive rating of 3.

4. Reviewer xrV3: Central-Row Recovery, Real–Synthetic Gap, and HDR Evaluation

  • We clarified that central-row reconstruction exploits physical observations from adjacent complementary rows. Region-wise evaluation confirms that exposure convergence does not cause reconstruction failure, while simultaneously invalid measurements remain a limitation shared by all reconstruction methods.
  • We analyzed the different initial difficulties of GOPRO-Dual and RSGR-HDR. Their PSNR gains are 16.86 dB and 11.94 dB, respectively, indicating that the absolute PSNR difference cannot be directly attributed to errors in the physical capture system.
  • We explained the exposure-fusion basis of our display-ready output and added HDR-VDP-2 and HDR-VDP-3 evaluations. Our method achieves the best results of 57.2598 and 8.2791, respectively.
  • The reviewer stated that the concerns were almost addressed and intended to raise the rating. As of this summary, the recorded rating remains unchanged at the initial score of 2.

Overall, the rebuttal has strengthened the paper through clearer explanations, recent comparisons, comprehensive ablations, region-wise analysis, and HDR-specific evaluations. We again thank the Area Chair and reviewers for their constructive feedback and careful reassessment.

Sincerely,

The Authors

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.