acceptodds
Under review as a conference paper at ICLR 2027

Learning Piecewise-Smooth Image Structure with Implicit Regularization

Abstract

In image regression, coefficient images are often piecewise smooth: approximately constant or smoothly varying within regions, with abrupt changes across boundaries. Explicit penalties such as total variation (TV) exploit this structure for spatially coherent, interpretable estimates but require a nonsmooth fit for every tuning value. To learn it through implicit regularization instead, we decompose the coefficient image into a polynomial background and a component reconstructed from local differences of a chosen order. With the background profiled out, a difference-of-squares factorization encourages sparse differences and a quadratic term keeps them consistent with an image. Gradient descent from a small balanced initialization, with backtracking and validation-based early stopping, fits the factors. Under correlated sub-Gaussian designs, we establish finite-step prediction and coefficient-error bounds. Under bounded variation, conservative steps yield an prediction rate up to logarithmic and geometric factors, without restricted-eigenvalue assumptions. A validation oracle inequality supports checkpoint and order selection at logarithmic cost in the candidate count. The small-initialization gradient-flow limit connects the method to minimum-variation interpolation. Simulations and a sea surface temperature application show competitive accuracy at lower computational cost than TV in the studied settings.

Primary area:
learning theory
Keywords:
Image Regression, Higher-Order Total Variation, Overparameterization, Gradient Descent

Reviews

Among:
Shape:
1234this paper, Top 15%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.00 over 3 reviews, above 85% of the others.
Reviewer_RPYXOpenReview ↗
Rating
3(+1)
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes an implicit regularization method for piecewise-smooth image regression. It combines a local difference-based decomposition with a difference-of-squares parameterization. The paper establishes finite-step prediction and coefficient-error bounds. In addition, the paper demonstrates competitive accuracy and computational efficiency on three-dimensional simulations and a real-data application.

Critical strengths
  1. The paper provides a framework for modeling piecewise-constant image structures through implicit regularization.
  2. The theoretical analysis is comprehensive, including finite-step prediction and coefficient-error bounds.
  3. The experimental results demonstrate that the proposed method outperforms TV regularization in terms of both accuracy and computational efficiency.
Critical weaknesses
  1. The novelty of this paper is limited. The proposed approach mainly combines existing difference-based representations with difference-of-squares implicit regularization.
  2. The experiments are insufficient to demonstrate the benefits of the proposed parameterization method. In particular, the paper lacks comparisons with direct gradient descent and more efficient TV optimization methods.
Questions for authors
  1. What is the main novelty of the proposed method compared with existing implicit regularization methods?

  2. Can the authors provide a comparison with direct gradient descent on the difference variables?

  3. Are the step sizes and number of iterations used in the experiments consistent with the theoretical analysis?

3 comments

Dear Reviewer RPYX,

Thank you once more for your careful review and precise questions. Our response clarifies the novelty relative to existing implicit-regularization methods and adds comparisons with direct gradient descent on the difference variables and a faster matrix-free TV solver, together with a step-size study linking the experiments to the theory (Table 3 and Appendices A.5–A.7).

We would greatly appreciate knowing whether our response has fully addressed your concerns. If any questions remain, we would be glad to continue the discussion before November 18.

Thank you for your reply. These addressed some of my questions, so I’ve raised my rating from 2 to 3.

Thank you very much for your reply and for raising your rating. If any of your questions remain unresolved, please let us know, and we would be glad to address them.

Reviewer_xQCdOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
2
Soundness
3
Read review
Paper summary

This paper focuses on image regression and proposes a gradient descent (GD) method to learn piecewise-smooth image coefficients without using explicit total variation regularization. The authors use a different parameterization to encourage smooth regions while preserving boundaries. They also provide theoretical analysis to support their method under certain assumptions. They test their approach on synthetic 3D image data and a real-world sea surface temperature (SST) dataset and compare it with several baselines. The results show good prediction accuracy and lower computational cost compared to TV regression in their experiments.

Critical strengths
  1. The paper presents an interesting approach with strong theoretical analysis. While it's pretty dense in places, the paper is reasonably well-written.

  2. Specifically, relaxing some of the assumptions/considerations in previous works such as restricted-isometry condition and the fixed-dimensional regime is useful.

Critical weaknesses
  1. The empirical evaluation could be a bit stronger. The authors tested their method on several synthetic datasets (that seem a bit idealistic) but only one real-world dataset (SST). It would be good to see evaluation on more real-world datasets to help us understand the efficacy and practical usefulness of the proposed approach.

  2. The authors mention previous implicit regularization methods, such as Vaškevičius et al. (2019) and Zhao et al. (2022), but they only compare their method with TV, CP, Ridge, and Elastic Net in the experiments.

Questions for authors
  1. Could the authors present a comparison of their method with other implicit regularization approaches, or explain why these methods were not included in the experiments?
3 comments

Dear Reviewer xQCd,

We sincerely appreciate your constructive comments. Our response and revised PDF add a real data analysis on six datasets, real-image-based simulations (Section 5, Tables 5–6), and comparisons with the methods of Vaškevičius et al. (2019) and Zhao et al. (2022) (Table 3 and Appendix A.5).

Could you kindly let us know whether these additions resolve your concerns? We would welcome any further comments before the discussion period closes.

Reviewer_xQCdOpenReview ↗

Thanks for addressing my comments

I would like to thank the authors for addressing my concerns and providing additional results/clarifications. I will maintain my positive view of the paper.

We sincerely thank the reviewer for your time and thoughtful suggestions, which have improved the quality of this paper.

Reviewer_T1gCOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

The paper studies linear regression with image predictors and a piecewise-smooth coefficient image. It separates a polynomial background from a component reconstructed in difference coordinates, then fits a difference-of-squares parameterization with a consistency penalty. Validation selects a stopping point along a single gradient descent trajectory. The analysis gives finite-step prediction bounds, a conservative stopping window, and a separate gradient-flow result. Simulations and a sea surface temperature application show competitive accuracy at lower cost than the tested TV implementation.

Critical strengths
  1. The handling of difference coordinates is technically convincing. The consistency term accounts for coordinates that do not correspond to a valid image, while the polynomial background is eliminated analytically. Under bounded variation and the other stated assumptions, Theorem 2 gives a finite-step prediction bound without a restricted-eigenvalue condition, with explicit geometric and logarithmic factors. The authors are careful about which step-size rules this result covers.

  2. The main GD and TV comparisons use 100 paired seeds, independent validation, and full fitting and selection times. The TV path is warm-started, and the selected solutions are checked using primal-dual gaps. The consistency and initialization ablations test specific parts of the method. These controls give me confidence in the comparisons with the tested TV implementation.

Critical weaknesses

The main omission is a comparison with Huang et al., Split LBI. Split LBI also studies linear regression with sparse transformed coefficients and constructs an early-stopped regularization path through quadratic coupling. This is close to the motivation in Sections 1 to 3, but the paper neither discusses nor evaluates it. The comparisons with TV, CP, ridge, and elastic net in Tables 1 and 5 therefore leave the benefit over an existing iterative regularization approach unclear.

The square-factor dynamics, pseudoinverse reconstruction, and prediction guarantee without a restricted-eigenvalue condition may offer useful advances. I would like to see these differences explained, including the assumptions and targets of the theoretical results. A comparison on one existing 16316^3 or 32332^3 setting would also help: use the same difference operator and data split, then report prediction and coefficient error alongside total fitting and selection time, with matched hardware and validation budgets. A representative case would be sufficient; there is no need to repeat the full experimental grid.

Questions for authors
  1. What differences in inductive bias do you expect between Split LBI's coupled structural coordinates and the proposed consistency projection and square-factor dynamics? Which differences are supported by the analysis?
  2. Do you already have a small comparison with Split LBI under the same data and selection protocol, or is there a specific obstacle to applying it here? Existing accuracy and timing results would help assess the additional benefit of the proposed trajectory.
1 comment

Dear Reviewer T1gC,

Thank you again for your thoughtful review and for pointing us to Split LBI. Our response and revised PDF add a theoretical comparison with Split LBI (Appendix B.2) and a paired experiment on the 16×16×1616\times16\times16 benchmark under the same difference operator, data splits, hardware, and validation checkpoints (Table 3 and Appendix A.4).

We would be grateful to know whether our response fully addresses your questions. If you have further comments, we would be happy to continue the discussion before November 18.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.