acceptodds
Withdrawn from ICLR 2027

Deep Unfolding Network via Tri-Domain Structural Learning for Light Field Image Super-Resolution

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Light field image super-resolution (LFSR) aims to reconstruct high-resolution LF images from low-resolution observations while preserving geometric structures. However, existing model-based and deep learning-based methods cannot simultaneously achieve rigorous mathematical modeling and powerful feature representation. To address this issue, we propose a deep unfolding network via tri-domain structural learning (LF-DUTS). Specifically, we formulate a 4D LF degradation model under the maximum a posteriori framework and unfold its optimization process into an interpretable multi-stage network. We further design a tri-domain structural refinement group to jointly exploit texture, multi-view, and disparity information from the spatial, angular, and epipolar plane image domains. In addition, we develop a cross-stage feature fusion module to enable feature interaction across unfolding stages. Experimental results on multiple benchmark datasets demonstrate that LF-DUTS consistently outperforms state-of-the-art methods, producing sharper textures, and better disparity consistency.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
Light field image super-resolution, Deep unfolding, Tri-domain structural learning, Cross-stage feature fusion

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_vK5qOpenReview ↗
Rating
1
Confidence
4
Informativeness
1
Presentation
2
Soundness
1
Read review
Paper summary

In this paper, the authors propose a deep unfolding network for light field (LF) image super-resolution (SR). The authors apply deep unfolding SR method into LF image SR task, and perform tri-domain structural learning by designing a series of modules including CSFM, TSRG, ACRB, ADRB, and EGRB. Experimental results demonstrate that the proposed network achieves state-of-the-art PSNR and SSIM performance.

Critical strengths
  1. The proposed method achieve a significant performance gains over the existing state-of-the-art LF image SR method.
Critical weaknesses
  1. The novelty of this paper is limited. First, both deep unfolding network (even those for image SR) and tri-domain feature disentangling have been thouroughly investigated by previous works. The proposed design is a simple combination of several existing techniques.

  2. The design of the proposed blocks (i.e., CSFM, TSRG, ACRB, SDRB, EGRB) is complex and independent of the unfolding process. It makes the proposed method like a combination of a series of engineering improvements, rather than a profound finding.

  3. The presentation of this paper need much improvements. For example, what do \italic{X}, LL, XX means in Eq.(2)? What is the relationship between ZZ and zz in Eq.(3)?

  4. In the experiment part, even without EGRB and CSFM, the proposed method can also achieve state-of-the-art SR performance. Similarly, even T=1 (without iterative unfolding operation), the proposed method can also achieve state-of-the-art SR performance. These results reveal that the superior performance of the proposed method does not heavily rely on the core design of this paper (may be the backbone or some tricks work well)

  5. In Table 4, the performance of T=3 is worse than T=2 is also strange. It demontrates that the unfolding operation cannot work well with more iterations.

Questions for authors

See weakness.

Reviewer_wayBOpenReview ↗
Rating
1
Confidence
4
Informativeness
1
Presentation
2
Soundness
1
Read review
Paper summary

This paper proposes LF-DUTS, a deep unfolding framework for light field image super-resolution that combines model-based optimization with learned feature representations. The paper formulates a MAP reconstruction objective incorporating data fidelity, light field priors, and graph Laplacian regularization for geometric consistency. Using half-quadratic splitting (HQS), the optimization process is unfolded into a trainable multi-stage network. For the high-resolution reconstruction subproblem, LFSRPN integrates spatial, angular, and EPI refinement modules to recover fine details and preserve geometric consistency. Cross-stage feature fusion (CSFM) enables information exchange between stages, while a hyperparameter network (HPN) predicts stage-specific optimization parameters. Experiments on five benchmark datasets demonstrate competitive performance for 2× and 4× super-resolution, achieving the best PSNR and SSIM across all five datasets at 4×. The paper also includes component ablations, geometric evaluations, and real-image examples.

Critical strengths

(1) LF-DUTS achieves the best PSNR and SSIM across all five benchmark datasets at 4× SR, while remaining competitive at 2×. Qualitative comparisons and per-view PSNR analyses further demonstrate its reconstruction performance. (2) The ablation studies support the effectiveness of EGRB, CSFM, and PCFN. Additional depth estimation analyses provide evidence of improved geometric consistency, while real-world light field results demonstrate its potential practical applicability.

Critical weaknesses

(1) The optimization unfolding and geometric modeling are not sufficiently justified. Although the reconstruction objective includes graph Laplacian regularization, the graph construction and its connection to EGRB are not clearly established. Moreover, Eqs. (4), (7), and Appendix Eq. (12) appear to contain inconsistencies in the optimization variables, while the update order, parameter relationships, and EPI sampling operations described in the paper do not fully align with the code provided in the supplementary materials. These discrepancies raise concerns about whether the claimed optimization interpretation accurately reflects the implemented network. In addition, some modules implemented in the supplementary code appear closely related to existing designs in DSAN and SMFANet, but their adoption and modifications are not clearly acknowledged. These issues weaken the interpretability of the proposed framework and make it difficult to distinguish its methodological contributions from existing approaches. (2) The reported performance improvements come at a substantial computational cost. According to Table 5, LF-DUTS uses approximately 3.67 times TriFormer’s parameter count and 4.24 times its FLOPs at 4× SR, while improving average PSNR by 0.34 dB. Similar disparities are observed at 2× SR. Furthermore, Table 4 shows that increasing the unfolding stages from one to two nearly doubles the computational cost for a PSNR improvement of 0.26 dB. Since this change simultaneously increases network evaluations, data feedback, and cross-stage feature fusion, the existing ablations do not sufficiently isolate the contribution of the unfolding mechanism. Comparisons under matched computational budgets are needed to establish whether the additional complexity is justified.

Questions for authors

Q1. The optimization variables in Eqs. (4), (7), and Appendix Eq. (12), as well as the actual X/Z update order and final output in the supplementary code, require clarification. A more explicit explanation of the relationship between the HPN parameters and the derived constraint, the graph Laplacian construction and its connection to EGRB, and the actual EPI sampling coordinates corresponding to Fig. 4 would also be helpful. Q2. A comparison between variants with and without data feedback, using the same prior network, number of network calls, and fusion configuration under comparable computational budgets, would better isolate the contribution of unfolding. An additional ablation separating cross-stage feature propagation from self-gating in CSFM would further clarify their respective contributions. Q3. The manuscript could more clearly identify the sources of DSA, DSCNPair, and PCFN in the supplementary code, with appropriate citations to DSAN and SMFANet where applicable. The specific modifications to these modules, as well as any components adapted from M2MT, also need clarification to distinguish the proposed contributions from existing designs.

Reviewer_J23cOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

LF-DUTS is a light-field super-resolution method organized as repeated data and prior updates. Its learned prior combines spatial and angular attention, deformable strip convolutions on horizontal and vertical EPIs, and cross-stage feature fusion. The paper evaluates ×2 and ×4 reconstruction on five standard datasets and includes component ablations, qualitative real-image examples, and depth-estimation results. At ×4, it reports the highest PSNR and SSIM on all five datasets, with average PSNR increasing from 32.77 dB for TriFormer to 33.11 dB.

Critical strengths
  1. The accuracy improvements are supported across several benchmarks. Table 1 reports the best PSNR and SSIM on all five datasets at ×4, including a 0.55 dB gain over the strongest listed competitor on EPFL. The ×2 results and the EPI and depth-estimation examples provide complementary evidence. Although the gains are not uniform across every dataset and scale, the overall reconstruction result is convincing.

  2. The paper tests the main architectural choices with relevant alternatives. In particular, the EPI ablation compares deformable strip convolution with square convolution and non-deformable strip convolution. This is informative because the proposed operation is motivated by disparity-induced slanted EPI structures. Cross-stage fusion and the feed-forward component are also ablated. Domain-specific processing and learned unfolding are established ideas, but the particular integration here constitutes a useful empirical method contribution.

Critical weaknesses
  1. The mathematical description does not yet give a sufficiently precise account of the implemented update. Eq. (2) introduces a graph-Laplacian regularizer, but the graph is not constructed explicitly, and it is unclear whether such an operator is used in the network. EGRB is described as unfolding this constraint, although the presented operation is a learned deformable-convolution module. A geometry-inspired learned prior is a reasonable design; the paper should distinguish it from an explicit graph-regularized update. The blur and downsampling operators, initialization, and use of the stage-specific prior parameter also need concrete definitions. The previous-iterate notation in Eqs. (4), (7), and (12) should be corrected where optimization variables are intended. These are fixable issues, but they matter to reproducing and interpreting the method.

  2. The computational tradeoff is understated. Table 4 shows that the second stage improves PSNR from 32.85 to 33.11 dB while doubling FLOPs from 70.72 G to 141.45 G. Table 5 reports 141.45 G FLOPs for LF-DUTS versus 33.37 G for TriFormer at ×4. The accuracy gain may justify this cost, but the increase should not be described as slight. The stage ablation also establishes the benefit of repeated processing in the complete system, rather than isolating the contribution of the data-consistency update. A comparable two-pass control would help explain that contribution, although the absence of this additional analysis does not invalidate the demonstrated performance of the full method.

Questions for authors
  1. Please provide a concise description of the actual stage update, including the forward operators, initialization, and how the stage-specific prior parameter enters the reconstruction network. Can the authors confirm the intended optimization variables in Eqs. (4), (7), and (12)?

  2. Is a graph Laplacian explicitly constructed or applied anywhere in the implementation, or does EGRB serve as a geometry-inspired learned prior? Please align the mathematical interpretation and interpretability claims with the implemented method.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.