acceptodds
Under review as a conference paper at ICLR 2027

Learning to Train Underwater Image Enhancers with Reinforcement-Guided Update Scheduling

Abstract

Underwater images often suffer from color distortion and scattering-related detail degradation, which arise from different physical processes and exhibit distinct optimization dynamics. Existing methods that jointly optimize color correction and detail restoration typically rely on fixed loss weights or predefined update schedules, implicitly treating their relative importance as constant throughout training. However, the two objectives often progress asynchronously, making static coordination unable to accommodate their stage-dependent optimization demands. We propose a learning-to-train framework that formulates update scheduling as a Markov decision process. A PPO-based controller observes the current training state and dynamically selects among alternating, joint, and weighted updates for a cascaded enhancement network. We further construct a frequency-aware state representation from recent reconstruction quality trends and low/high frequency alignment to track the relative progress of global color recovery and local detail restoration. By maximizing cumulative rewards that account for reconstruction improvement, loss reduction, and frequency balance, the controller learns stage-adaptive update policies. Comprehensive qualitative and quantitative evaluations on multiple benchmark datasets, together with extensive ablation studies, demonstrate the effectiveness and robustness of the proposed method.

Primary area:
unsupervised, self-supervised, semi-supervised, and supervised representation learning
Keywords:
Underwater Image Enhancement, Color Correction, Dynamic Update Scheduling, Proximal Policy Optimization

Reviews

Among:
Shape:
1234this paper, Bottom 9%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.50 over 2 reviews, above 9% of the others.
Reviewer_69sDOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes a reinforcement learning framework for underwater image enhancement. The authors claim that global color recovery and local detail restoration need to be learned at different stages, and that reinforcement learning can achieve this. The framework formulates update scheduling as a Markov decision process and uses a PPO-based controller to select among alternating, joint, and weighted updates according to a frequency-aware training state. The method is validated on four underwater benchmarks, and ablation studies show that the proposed reinforcement learning framework is effective.

Critical strengths

Critical Strengths:

The paper brings reinforcement learning into underwater image enhancement for update scheduling. This is a relatively novel angle, since most existing methods still rely on fixed loss weights or predefined training schedules. Framing the coordination of global color recovery and local detail restoration as a sequential decision problem is consequential because it offers a different way to handle their stage-dependent optimization demands.

The empirical study is broad and includes multiple benchmarks, several comparison dimensions, and ablation studies. Although the core motivation is not fully supported by sufficient evidence, the reported metric improvements and ablations provide reasonable support that the proposed framework is effective. The paper is also generally clear and easy to read, which makes its contributions easier to assess.

Critical weaknesses

Critical Weaknesses:

The core motivation is not sufficiently supported. The paper assumes that global color recovery and local detail restoration need to be learned at different stages, but it does not provide cross-model visualizations or statistics showing that this phenomenon is widespread. Moreover, although the authors claim a learning-to-train framework, all experiments are conducted on a single architecture. This is consequential because the stage-dependent motivation and the claimed generality of the framework remain unproven.

The experimental comparison is outdated and limited. Only one compared method is from 2025, and the second-best method is from 2023. This is consequential because the reported improvements do not establish an advantage over recent state-of-the-art methods, which weakens the empirical case for acceptance.

Questions for authors

The comparison includes only one 2025 method, and the second-best baseline is from 2023. Could you clarify the baseline selection criteria and provide comparisons with more recent 2025–2026 methods where available?

The paper claims a learning-to-train framework, but all experiments are conducted on a single architecture. Could you clarify whether the framework is intended to be architecture-agnostic? If so, can you provide results on additional backbones with redefined action spaces?

The central motivation assumes that color recovery and detail restoration progress asynchronously. Could you provide cross-model visualizations or statistics showing that this phenomenon is widespread rather than specific to your architecture?

Reviewer_AjBzOpenReview ↗
Rating
1
Confidence
4
Informativeness
1
Presentation
2
Soundness
1
Read review
Paper summary

The paper treats the update schedule of a two-module underwater image enhancer as a Markov decision process. The enhancer consists of a Global Color Correction (GCC) module followed by a Detail Restoration (DR) module, with frequency-decomposition gated fusion (FDGF) and fixed branch biases (MBB). A PPO controller observes sliding-window histories of PSNR and of a low/high-frequency energy-ratio error. At each decision step it selects one of three predefined update rules: alternating, joint, or weighted joint updates. The reward combines normalized changes in PSNR, training loss, and frequency error. Experiments on LSUI, EUVP, UIEB (paired) and U45 (unpaired) compare against seven methods. Ablations on UIEB study reward terms, fixed strategies, and the FDGF/MBB modules.

Critical strengths

Asking whether training dynamics, and not only architecture, should adapt to sub-objectives that converge at different rates is a reasonable question. Casting module-level update selection as a sequential decision problem is a natural, plug-in idea. The evaluation also goes beyond mean scores (percentile curves, a user study).

Critical weaknesses

1.The evidence for the central contribution (the learned scheduler) is weak. Table 1 mixes a new architecture, the training protocol, and the scheduler. The scheduler's isolated effect appears only in Table 2(b): 23.64 vs. 23.37 dB against the best fixed strategy, on one dataset (UIEB), apparently from a single run with no variance. The reward ablation spans 23.32–23.64 dB, and the module ablation gives 23.62 vs. 23.64 dB. Run-to-run noise with an RL controller on a small test set could easily be this large, so multiple seeds with standard deviations are needed. Key controls are missing: (a) a random policy over the same three actions (or one matching the learned policy's action frequencies), to show that state conditioning matters rather than merely mixing update rules; (b) simple heuristic schedules (e.g., color first then detail, or thresholding dtd_t); and (c) standard adaptive multi-objective balancing methods (GradNorm, uncertainty weighting, DWA, PCGrad). There is no analysis of what the controller learns (action frequencies over training, dependence on state). There is also no measurement in this model supporting the motivating claim of Fig. 1 that color and detail objectives converge asynchronously. Several qualitative effects are attributed to the controller without a controlled comparison, e.g., histogram agreement in Fig. 4 and smoother luminance in Fig. 5. Table 2(b) reports different metrics (PSNR/CCF/UCIQE) from Tables 2(a)/(c) (PSNR/SSIM/PCQI), and CCF is never defined. Please use one metric set throughout. 2.Missing details, baseline protocol, and positioning make the work hard to assess or reproduce. Several method details are absent: decision frequency and episode definition; whether the controller is trained online in the same run or pre-trained and transferred; PPO hyperparameters, ww, and the λ\lambda values; the concrete weights of the "weighted" action and the exact alternation rule; the loss weights λc\lambda_c and λs\lambda_s; and the controller's compute overhead. Most importantly, it is not stated on which data mtm_t and dtd_t are computed. If they are computed on a validation or test split, there is a risk of leakage. The baseline protocol is unclear: were all methods retrained on the same splits, or were released weights used? The gaps on EUVP (+5.4 dB over U-Shape) and LSUI (+2.7 dB) are much larger than is typical in this area. Some numbers also look unusual (HLRP SSIM 0.18–0.23, ICSP PSNR around 11 dB), which raises concerns about inconsistent evaluation settings (value range, color space, resolution). Recent UIE methods (2023–2025, including DCD-UIE, which the paper cites) are not compared. On positioning: RL controllers that learn discrete update schedules over multiple objectives or parameter groups have been studied before, e.g., AutoLoss (Xu et al., ICLR 2019) and Learning to Teach (Fan et al., ICLR 2018), alongside the multi-task loss-balancing literature. These should be discussed and compared. Some current citations do not support the claims they are attached to (Uni-MoE, a multimodal LLM, is cited as an expert-based restoration method; an underwater acoustic ISAC paper is cited for learned optimizer decisions). The framework is presented as a general learning-to-train approach but is only shown on the authors' own GCC–DR network.

Questions for authors

1.Please provide multi-seed mean ± std for Table 2(b), together with the random-policy and adaptive-weighting baselines. 2.On which data are mtm_t, dtd_t and the reward computed? Is any validation or test image used during training? 3.How were the baselines trained and evaluated (splits, retrained vs. released weights, resolution)? 4.What exactly are the three actions, how often are decisions made, and what is the training overhead? How does the method differ from AutoLoss? 5.Please report the params and the running memory/times of your model and the baselines.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.