acceptodds
Withdrawn from ICLR 2027

Scene-Preserving Counterfactual Diffusion for Low-Light Video Restoration

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Multi-frame low-light restoration typically operates over predefined candidate temporal dependencies, yet relevant degradation factors and propagation paths may vary across targets as imaging conditions change. We formulate restoration as a scene-preserving counterfactual problem, asking how the target would appear if relevant degradation conditions shifted toward normal light while the factual scene remained fixed. We propose Scene-Preserving Counterfactual Diffusion (SPCD), which infers a target-specific intervention structure. Using controlled responses and diffusion-score differences, SPCD selects a minimal set of degradation factors and propagation paths to guide reverse diffusion toward normal light. Experiments on real-world and controlled benchmarks across multiple dynamic regimes demonstrate improved quality. Code will be released after acceptance.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
low-light video restoration, causal reasoning, diffusion models

Reviews

Among:
Shape:
1234this paper, Bottom 25%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.75 over 4 reviews, above 25% of the others.
Reviewer_7W7bOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
3
Read review
Paper summary

This paper proposes Scene-Preserving Counterfactual Diffusion (SPCD) for low-light video restoration. The method formulates restoration as an instance-specific counterfactual problem, aiming to modify relevant degradation factors while preserving the factual scene. SPCD uses diffusion-score differences to identify target-specific intervention factors and sparse temporal propagation structures, which subsequently guide counterfactual reverse diffusion. Experiments on real-world and controlled benchmarks demonstrate competitive restoration performance, particularly under dynamically varying degradation conditions.

Critical strengths
  1. The paper addresses an interesting problem by considering target-specific degradation factors and temporal propagation structures rather than relying solely on predefined temporal dependencies.
  2. The proposed framework integrates scene-preserving diffusion inversion, intervention selection, and sparse propagation structure inference into a coherent restoration pipeline.
  3. The experimental evaluation is relatively comprehensive. In particular, the controlled degradation experiments and cardinality-matched sparsification comparisons provide useful evidence for the effectiveness of the proposed structure-selection mechanism.
Critical weaknesses
  1. Insufficient justification of the causal interpretation. The proposed diffusion-score discrepancies measure the sensitivity of the denoising model to changes in conditioning variables, but such sensitivity does not necessarily correspond to genuine causal effects. Although the paper provides validation under controlled degradation mechanisms, the extent to which the inferred structures capture meaningful causal dependencies in real-world videos remains unclear.

  2. Gap between the theoretical minimal-intervention formulation and its practical implementation. The theoretical objective seeks a minimal intervention subset satisfying a joint response constraint, whereas the implemented method independently thresholds singleton intervention responses. This approximation may fail when degradation factors interact, and the paper does not establish conditions under which the heuristic recovers or approximates the intended minimal intervention set.

  3. Limited theoretical support for the diffusion-score surrogate. The connection between squared noise-prediction differences and KL divergence is established for individual Gaussian reverse transitions with shared variance. However, the method uses only a few score evaluations along a fixed inversion trajectory to approximate divergences between complete counterfactual response distributions. The validity of this approximation for intervention and propagation selection requires further justification.

  4. Missing comparisons with simpler adaptive conditioning mechanisms. The paper includes several useful internal baselines, but it does not directly compare SPCD with a capacity-matched adaptive attention or gating mechanism using the same diffusion backbone. Such a comparison is important to determine whether explicit counterfactual response evaluation offers benefits beyond conventional learned condition selection.

Questions for authors
  1. Under what assumptions does singleton intervention thresholding approximate the joint minimal-intervention objective, particularly when degradation factors interact?
  2. Can the authors explain why removing inversion initialization improves GT-PSNR in Table 4(a), and provide additional evidence that inversion improves scene preservation?
Reviewer_pNYVOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

In this paper, Scene-Preserving Counterfactual Diffusion (SPCD) is proposed for low-light video restoration. The method uses diffusion-score differences for selecting degradation factors and sparse temporal propagation paths. The experiments on real and synthetic datasets show promising results, especially under dynamic degradation conditions.

At this stage, I lean toward a weak reject. If the authors can provide convincing answers to the above questions, I am willing to increase my score.

Critical strengths

The problem studied in this paper is interesting. I think that selecting relevant degradation factors and temporal paths is meaningful for low-light video restoration.

The proposed framework combines counterfactual modeling with diffusion-based restoration. The overall design is reasonable and well structured, I think.

The experiments cover several datasets and degradation conditions. The ablation studies provide useful evidence for the proposed components.

Critical weaknesses

Evidence for causal modeling seems limited. The method relies on learned factor representations and controlled supervision. It is not clear to me whether the selected factors and propagation paths represent actual causal relationships in real videos.

There is a gap between theory and implementation, I am afraid. Equations (6) and (7) define optimization objectives based on distribution distances. However, in the implementation, individual factor screening and diffusion-score differences are used. The connection between these approximations and the theoretical objectives is not sufficiently validated, I think.

I have some concerns about experimental fairness. In Table 2, external baselines are evaluated without fine-tuning on Causal-V-SDSD, while SPCD uses controlled training data. For this reason, it is difficult to isolate the improvement coming from the proposed method.

Validation of scene preservation seems insufficient to me. In Table 4(a), SPCD and Selected + Full achieve the same CI-L of 0.140. Removing inversion initialization also produces a similar result. I am afraid that these findings do not clearly support the claimed improvement in scene consistency.

Questions for authors

I would like to ask the authors how they can verify that the selected degradation factors and propagation paths reflect meaningful causal relationships rather than learned correlations. Could the authors provide more explicit counterfactual validation across different scenes, especially for joint interventions?

Could the authors fine-tune representative baselines on the same controlled training data and compare them with SPCD? I think this would help determine whether the improvements come from the proposed counterfactual modeling or from additional training supervision.

Reviewer_4GaLOpenReview ↗
Rating
1
Confidence
4
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

The paper proposes Scene-Preserving Counterfactual Diffusion for multi-frame low-light video enhancement. The authors first formulate video degradation from a causal perspective and then introduce a latent-code replacement strategy to perform counterfactual restoration. Experiments are conducted on the widely used SDSD dataset for the main comparisons, a real-world dataset for zero-shot evaluation, and a newly constructed synthetic benchmark, Causal-V-SDSD. Although the experimental results are promising, the implementation details are insufficiently documented. Moreover, the code release is still listed as a TODO after acceptance, and the appendix is poorly written, making the method difficult to reproduce.

Critical strengths

Strong quantitative performance.

As shown in Tab. 1, the proposed SPCD achieves state-of-the-art performance on SDSD-Indoor and ranks at least second on all the evaluated datasets. Tab. 2 further demonstrates its superiority over the compared baselines on the newly introduced Causal-V-SDSD benchmark. These quantitative results are promising.

However, the paper provides no visual comparisons on public datasets, making it difficult to qualitatively verify the reported improvements and raising concerns about whether the quantitative gains translate into perceptually better restoration results.

Critical weaknesses

The theoretical contribution is disconnected from the implemented algorithm.

The paper formulates restoration using SCMs, abduction, action, and prediction, whereas the actual implementation relies on learned prototypes, diffusion-score differences, and amortized graph gates. The correspondence between these theoretical objects and their neural approximations remains insufficiently established.

The practical intervention does not match the theoretical SCM intervention.

The paper defines interventions on causal mechanisms, but actually implements them by interpolating latent codes toward normal prototypes:

d~im=(1−η)dim+ηpm.\tilde d_i^m=(1-\eta)d_i^m+\eta p_m.

The authors do not demonstrate that this operation corresponds to a valid causal intervention or changes only the intended degradation factor.

The practical intervention selection does not solve the optimization problem in Eq. (6).

The theory seeks the smallest joint intervention set:

At⋆=arg⁡min⁡A∣A∣,DJS(πt⋆∣Rt(A))≤ρdeg.A_t^\star=\arg\min_A |A|,\quad D_{JS}(\pi_t^\star\\|R_t(A))\leq\rho_{deg}.

However, the implementation selects factors independently using single-factor thresholds. This may miss interactions between degradation factors and does not guarantee the theoretically optimal set.

The diffusion-score discrepancy is not sufficiently justified.

The paper uses a finite number of score comparisons along a fixed inversion trajectory:

dt(C,C′)=1K∑k=1Kwτk∣ϵθ(zτkinv,τk;C)−ϵθ(zτkinv,τk;C′)∣22.d_t(C,C')=\frac{1}{K}\sum_{k=1}^{K}w_{\tau_k} \\|\epsilon_\theta(z_{\tau_k}^{inv},\tau_k;C) -\epsilon_\theta(z_{\tau_k}^{inv},\tau_k;C')\\|_2^2.

It remains unclear whether this surrogate reliably approximates the distribution-level JS/KL divergence defined in the theory.

Fixing scene features does not guarantee scene preservation.

The method keeps the scene feature and factual inversion trajectory fixed, but changes the diffusion condition. This does not necessarily ensure that the generated video preserves the original scene content. The paper lacks sufficient theoretical or experimental evidence for this claim.

The causal claims lack convincing validation.

The paper itself shows that identical observational distributions can lead to different counterfactual responses. Yet, the proposed method learns interventions and propagation graphs from latent representations without ground-truth causal structures or real intervention responses. Therefore, it remains unclear whether the learned graph captures meaningful causal relationships rather than statistical correlations.

Lack of visual comparison.

For a low-level computer vision paper, comprehensive qualitative comparisons are essential for evaluating restoration performance. However, this paper provides only one figure showing results on its own synthetic dataset, which is far from sufficient.

Questions for authors
  1. What are the advantages of using a diffusion-based method for this task? RetinexMCNet and AMNet achieve substantially higher computational efficiency (less than 100 G FLOPs, compared with more than 6 T FLOPs for your method), while also delivering competitive restoration quality. Could you clarify what benefits your diffusion-based approach offers that justify its significantly higher computational cost?

  2. What is the connection between the degradation model and the proposed restoration method? More specifically, how does the causal degradation model guide or contribute to the design and implementation of the restoration network? It would be helpful to clarify how the proposed restoration method explicitly benefits from the degradation modeling.

  3. The visual comparisons are insufficient for a low-level computer vision paper. The paper provides very few visual comparisons, making it difficult to assess the actual restoration quality and the effectiveness of the proposed method. Please include more qualitative results, particularly challenging cases and comparisons with representative competing methods.

  4. Please revise the appendix using formal academic language. The current appendix contains wording that resembles instructions or prompts to an LLM, rather than a properly written academic manuscript. Please rewrite these sections in a clear and professional academic style.

Reviewer_dmckOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper formulates dynamic low-light video restoration as scene-preserving counterfactual generation. SPCD combines factorized degradation representations, normal-prototype replacement, diffusion-response screening, and sparse temporal propagation, with scene conditioning and DDIM inversion anchoring the factual scene. Experiments on SDSD, zero-shot BVI-RLV, and a controlled dynamic benchmark demonstrate restoration improvements, supported by structure-selection ablations and intervention-response diagnostics.

Critical strengths
  1. Meaningful formulation and evaluation design. The paper distinguishes visual plausibility from factual scene preservation and evaluates intervention responses using shared scenes and noise realizations, providing a useful framework beyond standard restoration metrics.

  2. Measurable restoration gains supported by controlled comparisons. SPCD improves over the matched Full-Graph model by 1.41–2.02 dB on the controlled benchmark. Same-checkpoint comparisons also favor selected structures over cardinality-matched random structures, supporting their practical value.

Critical weaknesses
  1. The connection between the formal objectives and the implemented algorithm remains incomplete. Singleton thresholding in Eq. 11 approximates the minimal joint intervention in Eq. 6 without validating the resulting set’s feasibility or minimality. The main text defines probes on the factual inversion trajectory, whereas Appendix B.4 describes pre-inversion, forward-noised observed latents for restoration; B.6 additionally uses clip-averaged sufficiency checks. The reported evidence therefore does not directly validate the claimed target-specific counterfactual selection mechanism.

  2. The key mechanism validation has insufficient coverage. Appendix D.1 states that Figure 5’s explicit-response diagnostic uses only one source video. The correlation of 0.8411 measures factor-score agreement with the model’s own responses; generator agreement is 0.4836, and edge-surrogate agreement is 0.3153. Factor AP is also below Lagged Corr. (0.3952 vs 0.4974). These results provide limited evidence of mechanism validity across scenes, despite the demonstrated restoration gains.

Questions for authors
  1. Implementation clarification: Which probe trajectory, score convention, and threshold does each table use? During restoration, what fraction of individual targets satisfy the propagation tolerance, beyond clip-averaged compliance?

  2. Assessment-relevant validation: Could Figure 5 be extended to independent test sources, reporting source-level uncertainty, label prevalence, and a zero-response baseline? Please also explain why factor AP falls below Lagged Corr.

  3. Assessment-relevant validation: On tractable controlled windows, how does singleton screening compare with exhaustive joint oracle search in feasibility and selected cardinality?

  4. Ablation clarification: Removing inversion in Table 4(a) improves GT-PSNR without worsening CI-L. Is its demonstrated benefit primarily perceptual, rather than improved fidelity or cross-trajectory consistency?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.