acceptodds
Desk rejected at ICLR 2027

Look Before You Restore: Learning to Predict Before Acting for Agentic Image Restoration

Desk rejected at ICLR 2027, so left out of the paper lists and the map.

Abstract

Agents for mixed-degradation image restoration usually learn what a tool does to an image only after running it, so failed calls waste computation and harmful outputs are caught late. We introduce PreActIR, an agentic restoration framework that predicts tool consequences before execution. For each candidate tool, a tool-conditioned world model predicts changes in residual degradation, image quality, and fidelity, together with the risk of harm and the uncertainty of the prediction. The agent executes the top-ranked candidate, and an independent verifier commits or rolls back its output before the agent replans from the observed state. PreActIR also audits whether performance is limited by the selector or by the candidate space, and studies targeted tool adaptation as a controlled way to expand that space. In a controlled replay that shares the candidate pool, verifier, and budget and uses reference-annotated degradation states, PreActIR uses 42.5% fewer real tool calls and 63.4% fewer rejected calls than reactive, execute-then-evaluate trial and error, while improving Y-PSNR by 0.92 dB.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
Agentic Image Restoration

Reviews

Among:
Shape:
1234this paper, Top 15%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.00 over 3 reviews, above 85% of the others.
Reviewer_SbTAOpenReview ↗
Rating
3(+1)
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

PreActIR is an agentic image restoration framework that predicts tool consequences before execution. A tool-conditioned model estimates degradation reduction, quality and fidelity changes, benefit and harm probabilities, and uncertainty. The controller ranks candidate tools, executes a selected action, and uses a separately trained verifier to commit or roll back the result. The paper also introduces an offline candidate-frontier audit and examines targeted tool adaptation. Experiments include MiO100, a paired real-world dataset, and controlled out-of-fold ranking and execution-order studies.

Critical strengths
  1. The problem is practically meaningful. Restoration tools can be expensive and may damage images when applied incorrectly. Predicting their effects before execution is a sensible way to reduce wasted calls.

  2. The controlled comparisons provide useful evidence for pre-execution ranking. With the candidate pool, verifier, and budget fixed, predictive ranking substantially reduces tool calls and rejections compared with reactive trial and error.

Critical weaknesses
  1. The deployment-time efficiency gains remain unverified. The headline call savings use reference-annotated degradation states rather than estimated states. Reported timing includes restoration-tool execution only, excluding prediction, verification, and other system overhead.

  2. The benefit of the full world model over simple ranking is limited. Fidelity-head ranking achieves nearly identical call savings and slightly higher PSNR. Two-step lookahead does not improve results, and removing several utility terms leaves top-1 performance unchanged.

  3. The main benchmark gains are difficult to attribute to the proposed predictor. The full system additionally uses post-execution candidate-value models, output ensembles, and an adapted HAT-S tool. Matched full-pipeline ablations are needed to separate their contributions.

  4. Generalization evidence is limited. The mechanism studies use 160 states from only 20 clean-image identities, also used during development. On the real-world set, improved no-reference scores come with lower PSNR than all compared agents.

Overall, the paper establishes that pre-execution ranking can reduce restoration-tool trial and error. However, it does not yet convincingly establish the necessity of the structured world model or demonstrate the headline efficiency gains under realistic estimated states and complete runtime accounting. Additional full-system components also complicate attribution. These gaps currently outweigh the useful empirical analysis and diagnostic contributions.

Questions for authors

See weaknesses.

6 comments
AuthorsOpenReview ↗

Response to Reviewer SbTA (Part 1/3): (W1, W2)

We sincerely thank you for reading our paper and appendices so closely. Your critiques on realistic estimated states, the value of the multi-head world model over a simple fidelity scorer, and full-pipeline attribution are completely fair and pinpoint where our evaluation needed stronger controls. Motivated by your feedback, we ran strictly controlled experiments and share our findings below.

W1. End-to-End Efficiency Under Estimated Degradation States. You rightly noted that Table 4 relied on reference-annotated states and counted only tool execution time. To test realistic deployment efficiency, we ran an end-to-end wall-clock evaluation using estimated degradation states and complete runtime accounting on an RTX 4090.

We evaluated 96 states (12 clean identities) using out-of-fold models, a shared candidate pool, the learned verifier, and budget B=4B=4. Timing spans input reading to output writing, including degradation estimation, world-model prediction, tool loading/execution, verification, and rollback (with controller weights preloaded):

\text{Method} & \text{End-to-End Time (s/image)} & \text{Calls/image} & \text{Y-PSNR (dB)} \\\\ \hline \text{Reactive trial-and-error} & 22.55 & 3.55 & 20.27 \\\\ \text{Degradation-based routing} & 14.54 & 2.36 & 21.80 \\\\ \text{PreActIR (ours)} & \textbf{11.15} & \textbf{1.78} & \textbf{21.93} \\\\ \end{array}$$ Even with full system overhead and state-estimation noise, **PreActIR cuts wall-clock latency by 50.6% vs. trial-and-error and 23.3% vs. routing**, using fewer calls (1.78 vs. 3.55 / 2.36) while reaching higher PSNR (21.93 dB). Because the world model and verifier are lightweight forward passes, their overhead is negligible compared to the heavy restoration tools they avoid calling. **W2. Why Train a Structured World Model If Simple Ranking Works Well?** This is a sharp observation: in Table 4 and Appendix Table 7, ranking by the fidelity head alone is close to the full 7-term score, and two-step lookahead ($H=2$) does not beat single-step ranking. **(1) Multitask supervision significantly improves the predictor itself.** To isolate the training benefit, we ran 3 paired seeds with identical architecture, initialization, data, batch order, and 3,000 updates, varying only whether the model was trained on fidelity alone or on our joint multitask targets. We evaluated both on 200 unseen rain/haze states (100 new source images) under estimated states, 6 shared tools, a frozen verifier, and budget $B=4$: $$\begin{array}{lccccc} \text{Setting} & \text{Calls/image} & \text{Y-PSNR} \uparrow & \text{Y-SSIM} \uparrow & \text{LPIPS} \downarrow & \text{MUSIQ} \uparrow \\\\ \hline \text{Fidelity-only training (scored by fidelity)} & 2.67 & 17.39 & 0.616 & 0.370 & 36.272 \\\\ \text{Multitask training (scored by fidelity only)} & 2.65 & 17.52 & 0.624 & \textbf{0.360} & 37.230 \\\\ \text{Multitask training (scored by fidelity + } p^{\text{ben}}\text{)} & \textbf{2.50} & \textbf{17.60} & \textbf{0.627} & \textbf{0.360} & 37.000 \\\\ \text{Multitask training (full 7-term score)} & 2.53 & 17.57 & 0.626 & \textbf{0.360} & 37.110 \\\\ \end{array}$$ Comparing the first two rows, which use the exact same inference ranking formula ($\Delta\widehat{\text{PSNR}}_Y + 10\Delta\widehat{\text{SSIM}}_Y$), shows that jointly supervising the model on structured action consequences improves every quality metric. Learning *how* a tool alters specific degradations and whether it causes harm regularizes the shared representation and yields more accurate fidelity predictions on new images. **(2) Simplifying the inference score.** Once the multitask model is trained (last three rows), a compact score combining fidelity gain and benefit probability ($p^{\text{ben}}$) reduces calls from 2.65 to 2.50 and selects the exact same output as the full 7-term score in 95.2% of cases (571/600). This also explains Appendix Table 7: under global actions, outside-region damage $d$ is zero by definition and $C(a)$ serves only as a tie-breaking prior, while the side-effect term ($\xi$) drives nearly all remaining differences (20/160 dev states and 31/600 new pairs) with a minor PSNR/MUSIQ trade-off. We completely agree with your critique of over-complicating the inference formula. **In the revised paper, we will foreground multitask consequence learning paired with compact one-step scoring (fidelity + $p^{\text{ben}}$)**, while presenting the extra utility terms and $H=2$ rollout transparently as ablation explorations rather than core complexity.
AuthorsOpenReview ↗

Response to Reviewer SbTA (Part 2/3): (W3)

W3. Disentangling Full-Pipeline Attribution. To separate pre-action world-model (WM) ranking from post-execution selection, output blending (Groups A/B), and adapted HAT-S (Group C) in Table 1, we ran factorized full-pipeline ablations.

(1) Isolating WM's contribution. On 200 new rain–haze states (100 unseen source images, half with ×4\times 4 downsampling), we fixed the estimator, tool pool (without adapted HAT-S), post-execution selector, and blending, varying only the pre-execution selector at our primary two-stage budget B=8B=8:

\text{Pre-Execution Selector} & \text{Calls} & \text{Y-PSNR} \uparrow & \text{Y-SSIM} \uparrow & \text{LPIPS} \downarrow & \text{MUSIQ} \uparrow \\\\ \hline \text{Degradation routing} & 7.85 & 15.74 & 0.5989 & 0.455 & 29.274 \\\\ \text{WM (fidelity + } p^{\text{ben}}\text{)} & \textbf{7.79} & \textbf{16.24} & \textbf{0.6038} & \textbf{0.419} & \textbf{35.654} \\\\ \text{WM (full score)} & \textbf{7.79} & \textbf{16.24} & 0.6036 & \textbf{0.419} & 35.599 \\\\ \end{array}$$ With downstream modules fixed, **WM selection at ~7.8 calls improves Y-PSNR by +0.50 dB, LPIPS by -0.036, and MUSIQ by +6.38** (+0.035 MANIQA, +0.065 CLIP-IQA), showing that better root candidates directly raise the downstream ceiling. **(2) Downstream selection and blending.** Replacing the learned post-execution selector with a fixed training-derived tool-priority rule on the same 200 states ($B=8$) yields: $$\begin{array}{lcccc} \text{Selector (Fixed Downstream)} & \text{Y-PSNR} \uparrow & \text{Y-SSIM} \uparrow & \text{LPIPS} \downarrow & \text{MUSIQ} \uparrow \\\\ \hline \text{Degradation routing} & 15.39 & 0.5840 & 0.535 & 24.667 \\\\ \text{WM (fidelity + } p^{\text{ben}}\text{)} & 15.61 & 0.5943 & 0.522 & 25.837 \\\\ \text{WM (full score)} & \textbf{15.65} & \textbf{0.5956} & \textbf{0.519} & \textbf{25.984} \\\\ \end{array}$$ WM still beats routing on all metrics (+0.26 dB PSNR, +1.32 MUSIQ), while the learned downstream selector adds **+0.35 dB** for routing and **+0.60–0.63 dB** for WM. Separately, on 40 fixed-pool states, we isolated the 3-branch blend (Eq. 28) against the top-weight single learned branch: $$\begin{array}{lcccc} \text{Output Rule} & \text{Y-PSNR} \uparrow & \text{Y-SSIM} \uparrow & \text{LPIPS} \downarrow & \text{MUSIQ} \uparrow \\\\ \hline \text{Fixed priority (no selector/blend)} & 21.07 & 0.6510 & 0.369 & 58.184 \\\\ \text{Single-branch learned selection} & 22.84 & 0.6998 & 0.300 & \textbf{62.978} \\\\ \text{Three-branch blend (Group A/B)} & \textbf{23.04} & \textbf{0.7066} & \textbf{0.296} & 62.616 \\\\ \end{array}$$ Candidate selection drives most of the gain (+1.77 dB), while 3-branch blending adds a modest **+0.20 dB PSNR** (-0.36 MUSIQ). **(3) HAT-S adaptation on Group C.** Across all 400 Group C states under identical fusion: $$\begin{array}{lcccc} \text{Group C SR Configuration} & \text{Y-PSNR} \uparrow & \text{Y-SSIM} \uparrow & \text{LPIPS} \downarrow & \text{MUSIQ} \uparrow \\\\ \hline \text{Without SR branch} & 19.49 & 0.5469 & 0.437 & \textbf{58.426} \\\\ \text{Original HAT-S + fusion} & 19.69 & 0.5639 & 0.462 & 54.676 \\\\ \text{Adapted HAT-S + fusion} & \textbf{21.05} & \textbf{0.6040} & \textbf{0.418} & 54.782 \\\\ \end{array}$$ Replay-adapted HAT-S adds **+1.36 dB** overall (+2.71 dB on the 200 low-resolution states). All three ablations will be added to the revised paper. **(4) Budget sensitivity.** Paired differences (Full WM minus Routing) across $B \in \{4, 8, 12\}$ on the 200-state two-stage pipeline are: $$\begin{array}{lcccc} \text{Total Budget (WM vs. Routing Calls)} & \Delta \text{Y-PSNR} \uparrow & \Delta \text{Y-SSIM} \uparrow & \Delta \text{LPIPS} \downarrow & \Delta \text{MUSIQ} \uparrow \\\\ \hline B = 4 \text{ (3.30 vs. 3.56)} & -0.48 & -0.0316 & +0.035 & +2.120 \\\\ B = 8 \text{ (7.79 vs. 7.85)} & \textbf{+0.51} & \textbf{+0.0047} & \textbf{-0.036} & \textbf{+6.325} \\\\ B = 12 \text{ (11.08 vs. 10.90)} & +0.17 & -0.0118 & -0.019 & +3.402 \\\\ \end{array}$$ In two-stage expansion, $B=4$ leaves too few calls after stage-1 root exploration to finish stage-2 refinement, whereas at $B=12$ even routing eventually exhausts the pool. At a practical mid-sized budget ($B=8$), pre-action ranking focuses calls on promising branches (+0.22 dB PSNR and +9.24 MUSIQ on the $\times 4$ subset). **In short, these factorized ablations cleanly separate each module's role:** pre-action WM ranking drives a +0.51 dB PSNR and +6.33 MUSIQ gain under fixed downstream rules, learned post-execution selection adds +0.60 dB, and blending (+0.20 dB) plus Group C tool adaptation (+1.36 dB) provide orthogonal improvements that we will detail in the revised paper.
AuthorsOpenReview ↗

Response to Reviewer SbTA (Part 3/3): (W4, Summary)

W4. Generalization Beyond the 20 Dev Identities & The Real-World PSNR Trade-Off.

(1) Generalization to 200 unseen source images. We completely understand your concern that the 160 OOF states in Secs. 4.3–4.4 came from 20 clean-image identities. That is why both new mechanism studies in W2 (100 new source images / 200 states) and W3 (another 100 disjoint source images / 200 states, verified pixel-disjoint from both MiO100 dev and W2) were conducted on 200 entirely new source identities drawn from external AiOIR archives. Across both sets, the benefits of multitask world-model supervision (W2) and pre-action tool selection under estimated states (W3) transfer cleanly to unseen images.

(2) Understanding the lower PSNR on the 100 real-world pairs (Table 2). Your question about why PreActIR has lower PSNR (22.64 dB) than other agents on the real-world paired dataset prompted us to inspect the raw input–reference pairs directly. What we found was eye-opening, when we compute the exact same metrics on the unprocessed degraded input images (I0I_0) against the ground-truth references, we get:

\text{Metric on Real‑World 100 Pairs} & \text{Degraded Input } (I_0) & \text{PreActIR Output} \\\\ \hline \text{LPIPS (Full‑Reference)} \downarrow & 0.445 & \textbf{0.370 (-16.7\\%)} \\\\ \text{MUSIQ (No‑Reference)} \uparrow & 35.421 & \textbf{57.330 (+21.91)} \\\\ \text{MANIQA (No‑Reference)} \uparrow & 0.239 & \textbf{0.389 (+0.150)} \\\\ \text{CLIP‑IQA (No‑Reference)} \uparrow & 0.338 & \textbf{0.525 (+0.187)} \\\\ \text{Y‑PSNR (Full‑Reference)} \uparrow & \textbf{24.94} & 22.64 \\\\ \text{Y‑SSIM (Full‑Reference)} \uparrow & \textbf{0.7521} & 0.7310 \\\\ \end{array}$$ Notice a striking phenomenon: the unprocessed degraded input $I_0$ has a Y-PSNR of 24.94 dB, which is higher than every single restoration method in Table 2 (including SEAR at 24.41 dB, MiOIR at 24.00 dB, and AgenticIR at 23.93 dB)! In real-world capture pairs, actively removing haze, low-light darkness, or blur inevitably shifts global luminance/tone and reconstructs sharp textures that deviate pixel-wise from the pseudo-reference; thus, a conservative model that barely modifies the image preserves higher pixel-wise PSNR. Crucially, however, our improvement is **not** limited to no-reference metrics: on **full-reference LPIPS**, PreActIR reduces error by **16.7% (from 0.445 to 0.370)** while substantially boosting visual quality (Fig. 5). We will add the degraded-input baseline row to Table 2 so readers can immediately see this distortion–perception dynamic on the real-world set. --- **Summary of Revisions** Thanks to your feedback, our revised manuscript will be significantly stronger and more transparent: 1. Realistic End-to-End Efficiency (**W1**): We will add the end-to-end wall-clock benchmark under estimated states showing **51% / 23% real latency savings** (11.15s vs. 22.55s / 14.54s) with all prediction and verification overhead included. 2. Streamlined World-Model Narrative (**W2**): We will include the paired training study proving that **structured multitask supervision** improves all 6 restoration metrics over fidelity-only training, while simplifying our default inference score to the compact formulation. 3. Full-Pipeline Ablations & Generalization (**W3 & W4**): We will add the factorized ablations on **200 unseen source identities** isolating pre-action WM selection (+0.51 dB PSNR, +6.33 MUSIQ at $B=8$), post-execution selection, blending, and HAT-S adaptation, alongside the input-baseline analysis for Table 2. We hope these new experiments and clarifications directly address your concerns, and we would be happy to discuss any further questions!
Reviewer_yB1zOpenReview ↗
Rating
2
Confidence
2(−1)
Informativeness
1
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes PreActIR, an agentic image restoration framework that uses a tool-conditioned world model to predict restoration outcomes before execution. An independent verifier accepts or rejects the executed results. The paper also introduces a candidate-frontier audit and targeted tool adaptation to address tool-space limitations. Experiments demonstrate improved restoration quality and reduced tool calls compared to reactive baselines.

Critical strengths
  1. Predicting tool outcomes before execution addresses the computational waste of reactive trial-and-error in agentic image restoration.

  2. The paper evaluates restoration quality, tool efficiency, and harmful actions, with controlled comparisons and a candidate-frontier analysis that helps distinguish selection limitations from tool-space limitations.

Critical weaknesses
  1. In Table 3, simple degradation routing outperforms the proposed world model on several metrics, raising questions about its effectiveness.

  2. The tool-evolution component appears largely independent of the proposed world model and planning framework. It mainly consists of an offline frontier analysis followed by conventional fine-tuning of an existing restoration model. While the analysis is informative, it is unclear how this component contributes to the core methodological novelty beyond introducing a stronger restoration tool.

Questions for authors

NA

7 comments
AuthorsOpenReview ↗

Response to Reviewer yB1z (Part 1/2): (W1)

We sincerely appreciate your thoughtful review and recognition of our predict-before-act motivation and frontier analysis. Your two questions regarding why the world model (WM) trailed simple routing in Table 3 and how tool adaptation fits into our core methodology go straight to the heart of our paper. Below, we address both with new controlled experiments and our revision plan.

W1. Why Routing Looked Strong in Table 3, and How the World Model Excels in Budgeted Restoration?

Your question about Table 3 is spot-on. In Table 3's forced single-step, open-loop setting, degradation routing picks the highest-probability degradation and invokes the tool with the best average training-fold gain for that degradation. When forced to make a single blind guess on step 1, a static training-set average is a safe baseline.

However, static routing has a fundamental limitation as soon as we move from a single blind guess to sequential, budgeted restoration: it is image-blind within a degradation type. It tries tools in the same fixed order regardless of the specific image content, cannot anticipate whether a tool will damage a particular image (whereas WM achieves 0.729 ROC-AUC in detecting harmful actions in Fig. 6a), and wastes budget when its default top choice fails verification (as shown in Appendix Fig. 10, where PreActIR beats routing by +0.95 to +1.58 dB while using 1.0 to 1.2 fewer calls per image).

(1) Matched-budget comparison under estimated states and the original toolbox. To test whether action-conditioned WM predictions outperform degradation routing in realistic deployment without any adapted tools, we evaluated 200 compound rain-haze states from 100 unseen source images. All policies share the exact same estimated degradation states, the original unadapted toolbox (no adapted HAT-S branch), frozen downstream selectors, and a prespecified total budget of B=8B=8 tool calls across the two-stage pipeline:

\text{Pre-Execution Policy (Original Toolbox)} & \text{Calls/image} & \text{Y-PSNR} \uparrow & \text{Y-SSIM} \uparrow & \text{LPIPS} \downarrow & \text{MUSIQ} \uparrow \\\\ \hline \text{Degradation routing} & 7.85 & 15.74 & 0.5989 & 0.455 & 29.274 \\\\ \text{PreActIR World Model (Ours)} & \textbf{7.79} & \textbf{16.24} & \textbf{0.6036} & \textbf{0.419} & \textbf{35.599} \\\\ \end{array}$$ Under the same ~7.8-call budget and estimated states, WM-guided selection outperforms degradation routing across every metric: +0.51 dB PSNR, -0.036 LPIPS, and +6.325 MUSIQ, alongside +0.035 MANIQA and +0.065 CLIP-IQA. Conditioning predictions on both the current image and candidate tool allows the planner to allocate its limited call budget to far more effective root actions. **(2) Disentangling pre-execution tool choice from final output selection.** To verify that this gain comes from the world model's pre-execution choices rather than the learned post-execution selector, we evaluated the exact same generated candidates under both a fixed training-prior output rule and the learned final selector ($B=8$): $$\begin{array}{lccccc} \text{Final Output Selection Rule} & \text{Routing PSNR} & \text{WM PSNR} & \Delta\text{PSNR} \uparrow & \Delta\text{LPIPS} \downarrow & \Delta\text{MUSIQ} \uparrow \\\\ \hline \text{Fixed training-prior rule} & 15.39 & \textbf{15.65} & \textbf{+0.25} & \textbf{-0.016} & \textbf{+1.317} \\\\ \text{Learned post-execution rule} & 15.74 & \textbf{16.24} & \textbf{+0.51} & \textbf{-0.036} & \textbf{+6.325} \\\\ \end{array}$$ WM beats routing under **both** rules (+0.25 dB with the fixed rule and +0.51 dB with the learned rule), confirming a direct benefit from pre-execution prediction and a clear synergy: when the world model feeds stronger candidates into the pipeline, the downstream selector has a much better pool to choose from. **(3) Performance across different call budgets.** Sweeping the total budget across $B \in \\{4, 8, 12\\}$ on the 200 unseen states (WM minus routing) further clarifies how the two-stage planner uses its budget: $$\begin{array}{lcccc} \text{Total Call Budget } B & \Delta\text{Y-PSNR} \uparrow & \Delta\text{Y-SSIM} \uparrow & \Delta\text{LPIPS} \downarrow & \Delta\text{MUSIQ} \uparrow \\\\ \hline B = 4 \text{ (3.30 vs. 3.56 calls)} & -0.48 & -0.0316 & +0.035 & +2.120 \\\\ B = 8 \text{ (7.79 vs. 7.85 calls)} & \textbf{+0.51} & \textbf{+0.0047} & \textbf{-0.036} & \textbf{+6.325} \\\\ B = 12 \text{ (11.08 vs. 10.90 calls)} & +0.17 & -0.0118 & -0.019 & +3.402 \\\\ \end{array}$$ In two-stage tree expansion, $B=4$ is too tight to explore stage-1 roots and still complete stage-2 refinement, whereas at $B=12$ even naive routing eventually stumbles upon good tools by exhausting the candidate pool. At a practical mid-sized budget ($B=8$), pre-execution ranking is decisive for focusing calls on the most promising branches.
AuthorsOpenReview ↗

Response to Reviewer yB1z (Part 2/2): (W2, Summary)

W2. Reframing the Frontier Audit and Tool Adaptation: Diagnostic Role vs. Methodological Novelty

We completely agree with your point here. Fine-tuning HAT-S on external replay pairs is standard supervised adaptation, and presenting the offline audit + tool adaptation as a "Tool-Evolution Loop" alongside the world model in Figures 1 and 2 made it look like we were claiming algorithmic novelty for the fine-tuning step itself.

We included this analysis to answer a diagnostic question that naturally arises when evaluating any tool-selection agent: when restoration quality plateaus on hard compound degradations, is the agent limited by imperfect tool selection, or by the capacity of the frozen toolbox? A stronger restoration backbone alone cannot answer that question, whereas our constrained frontier audit makes both factors quantitatively separable:

(1) Quantifying selection headroom vs. toolbox expansion under a unified objective. On the 400-state Group C benchmark, solving the exact audit program to maximize mean PSNR at a matched MUSIQ floor of 54.780 separates what is already reachable inside the fixed 33-tool pool from what toolbox expansion adds:

\text{Group C Candidate Pool (400 States)} & \text{Total Options} & \text{Attainable Oracle PSNR} \uparrow \\\\ \hline \text{Fixed 33-tool pool + frozen policies} & 22,985 & 21.77 \text{ dB} \\\\ \text{+ Adapted HAT-S direct outputs (200 LR states)} & 23,185 & 22.01 \text{ dB (+0.24 dB)} \\\\ \text{+ Adapted HAT-S and fusion outputs} & 23,585 & \textbf{22.27 dB (+0.50 dB)} \\\\ \end{array}$$ Crucially, at our pre-adaptation policy's MUSIQ level (58.426), the existing frozen pool already supports 21.16 dB versus the policy's 19.49 dB, revealing a +1.66 dB selection gap. Rather than distracting from the world model, this diagnostic proves that improving tool selection within the existing pool remains the largest source of untapped gain (+1.66 dB), while expanding the toolbox adds a complementary +0.50 dB ceiling shift on the low-resolution subset. **(2) Transparently separating tool adaptation from output fusion.** To ensure that the benefit of the stronger SR tool is never conflated with our planning framework, we also ran a $2 \times 2$ ablation on the 200 low-resolution Group C states comparing pretrained vs. adapted HAT-S under both direct output and fixed fusion: $$\begin{array}{llccc} \text{HAT-S Tool Version} & \text{Integration Mode} & \text{Y-PSNR} \uparrow & \text{LPIPS} \downarrow & \text{MUSIQ} \uparrow \\\\ \hline \text{Off-the-shelf Pretrained} & \text{Direct tool output} & 18.52 & 0.554 & 44.912 \\\\ \text{Replay-Adapted} & \text{Direct tool output} & \textbf{23.36} & 0.400 & 45.840 \\\\ \text{Off-the-shelf Pretrained} & \text{Fixed fusion with base candidate} & 19.98 & 0.394 & 64.265 \\\\ \text{Replay-Adapted} & \text{Fixed fusion with base candidate} & 22.69 & \textbf{0.306} & \textbf{64.476} \\\\ \end{array}$$ This cleanly separates the +4.85 dB direct adaptation gain and +2.71 dB fused gain from the fidelity-perception trade-off of output fusion. In the revised manuscript, we will make this division of roles explicit in Sections 3.5 and 4.5: our core algorithmic contribution is pre-execution consequence prediction and planning (validated on the fixed original toolbox in **W1**), while the frontier audit and HAT-S adaptation serve as a quantitative diagnostic that separates selector headroom within a fixed pool (+1.66 dB) from toolbox capacity expansion (+0.50 dB frontier shift; +4.85 dB direct / +2.71 dB fused adaptation gain). --- **Summary.** With the original unadapted toolbox and estimated states, pre-execution world-model prediction improves restoration quality across fidelity, perceptual similarity, and no-reference metrics (+0.51 dB PSNR, -0.036 LPIPS, +6.325 MUSIQ at $B=8$), and its advantage over routing holds under both fixed and learned downstream selection. Complementing this online controller, our frontier audit and $2 \times 2$ tool-adaptation controls cleanly separate selection headroom from toolbox capacity, giving each component a clear, evidence-backed role. We hope this candid clarification and the new matched-budget experiments resolve your concerns, and we welcome any further questions!

Thank you for the rebuttal. The additional experiments address my first concern, but my second concern regarding the limited novelty of the tool-evolution component remains. Overall, I still find the contribution insufficient and will maintain my original score.

Reviewer_iYfPOpenReview ↗
Rating
4
Confidence
4
Informativeness
3
Presentation
3
Soundness
3
Read review
Paper summary

This paper studies a well-known inefficiency of agentic image restoration: in an execute–observe–replan loop, a tool’s effect is only known after it has been run and paid for. The proposed framework, PreActIR, uses a tool-conditioned world model to predict, for every candidate tool, its consequences before execution: degradation change, proxy-quality and Y-channel fidelity change, content damage, benefit/harm/severe-harm probabilities, and predictive uncertainty. The agent executes the top-ranked candidate under a structured utility, and an independent verifier commits or rolls back the result before replanning.

Critical strengths
  1. Well-posed problem with a clear conceptual contribution. The paper turns “predict before acting” into a concrete, testable design for restoration agents. It also explicitly separates selector limitations from tool-space limitations.

  2. Substantial and robust efficiency gains. The reductions in calls and rejections are large and statistically supported. The budget sweep shows that pre-execution ranking reaches a given quality level with fewer real calls than reactive and routing baselines at every budget. This addresses a real cost bottleneck for deploying restoration agents.

  3. Competitive benchmark performance. On MiO100, PreActIR leads on fidelity metrics in Groups A and C and on perceptual metrics in Group B. On the paired real-world set, it achieves the highest MANIQA, CLIP-IQA and MUSIQ.

Critical weaknesses
  1. The net contribution of the learned verifier is unclear. On what appear to be the same states and candidate pool, unconditionally committing the world model’s top-1 action is better in fidelity than the full predict–act–verify loop at B = 4. The two tables should be reconciled, and an “always commit” row added to Table 4.

  2. The system integrates multiple additional components beyond the predict–act–verify loop. The features used by the value model overlap with evaluation metrics. Moreover, only tool-execution time is accounted for, and component ablation studies as well as full end-to-end cost comparisons are missing.

  3. The real-world PSNR results in Table 2 suggest potential limitations in generalization beyond the MiO100 degradation pipeline. The overhead of introducing new tools including transition collection and model retraining has not been discussed. The claim in the conclusion regarding the best overall performance across all three groups may need moderation.

  4. The comparisons and figures could be more complete. Table 2 lacks MAIR and InstructIR rows even though it uses MAIR’s test set. Figs. 4, 5 and 8 omit SEAR and MAIR, are too small to judge differences, and would benefit from zoomed crops.

  5. Some terms are used before their definitions. “OOF” appears in Sec. 3.3 but is defined only in Sec. 4.3. The comparisons in Sec. 4.3 refer to “Direct”, “routing” and “WM-F”, whose definitions are provided in Appendix E.1. Adding brief definitions for each baseline within Sec. 4.3 would make this section self-contained.

Questions for authors

Some terms are used before their definitions. “OOF” appears in Sec. 3.3 but is defined only in Sec. 4.3. The comparisons in Sec. 4.3 refer to “Direct”, “routing” and “WM-F”, whose definitions are provided in Appendix E.1. Adding brief definitions for each baseline within Sec. 4.3 would make this section self-contained.

4 comments
AuthorsOpenReview ↗

Response to Reviewer iYfP (Part 1/3): (W1)

We are deeply grateful for your generous assessment and for recognizing both the conceptual value of predicting tool consequences before acting and the rigor of our efficiency and frontier analyses. Your constructive suggestions on reconciling the verifier tables, isolating downstream components, and improving the self-containedness of our exposition are spot-on and have helped us make the paper much clearer. Below, we address each of your points in detail.

W1. Reconciling Tables 3 and 4, and Quantifying the Verifier's Net Contribution.

Thank you for catching this apparent discrepancy. Looking across Table 3, Table 4, and Appendix Table 5, it is completely understandable why unconditionally committing the world model's (WM's) top-1 action appeared to yield higher PSNR (22.55 dB) than the full predict-act-verify loop at B=4B=4 (22.08 dB).

(1) Reconciling state inputs, candidate pools, and reported metrics. Both tables evaluate the same 160 out-of-fold (OOF) states from 20 clean-image identities, but under two different protocols and reporting conventions:

  • Reporting convention: Table 3 reports the change in PSNR (ΔPSNRY\Delta\text{PSNR}_Y) over the degraded input (whose mean PSNR is 20.03 dB), so the WM's +1.14 dB gain in Table 3 corresponds to a 21.17 dB final PSNR. Table 4 and Appendix Table 5 report final PSNR directly.
  • State inputs and candidate pools: Table 3 tests realistic deployment under estimated degradation states. In contrast, Table 4 (and Table 5) isolates execution ordering under reference-annotated states. We will make these pool and metric definitions explicit in both table captions.

(2) The requested always-commit row on Table 4's annotated-state pool. Holding the frozen WM ranking fixed, Always commit top-1 executes the top-ranked candidate and stops, whereas Verify + retry rejects outputs that fail the frozen acceptance rule and stops at the first accepted candidate within 4 attempts (retaining the 20.03 dB input if none passes):

\text{Acceptance Rule (Table 4 Annotated Pool)} & \text{Calls} & \text{Y-PSNR} \uparrow & \text{Y-SSIM} \uparrow & \text{LPIPS} \downarrow & \text{Harm Rate} \downarrow & \text{Severe Rate} \downarrow \\\\ \hline \text{Always commit top-1} & 1.00 & \textbf{22.55} & \textbf{0.696} & \textbf{0.405} & 15.6 & 5.6 \\\\ \text{Verify + retry } (B=4) & 1.71 & 22.08 & 0.690 & 0.416 & \textbf{14.4} & \textbf{4.4} \\\\ \end{array}$$ When given oracle ground-truth degradation labels, the WM's top-1 choice is already beneficial 84.4% of the time (with only 5.6% severe harm, defined as $\Delta\text{PSNR}_Y \le -1.0$ dB or $\Delta\text{SSIM}_Y \le -0.05$). Because the learned verifier is tuned conservatively, it abstains on 25/160 states (accepting 135/160 and keeping the 20.03 dB input on the rest), which slightly lowers mean PSNR from 22.55 to 22.08 dB in exchange for lower harm (14.4% vs. 15.6%). **(3) The same control on Table 3's realistic estimated-state pool.** Retaining all 2,147 candidates, frozen WM scores, and fold-local verifier thresholds on the 160 estimated states tells a very different story: $$\begin{array}{lcccccc} \text{Acceptance Rule (Table 3 Estimated Pool)} & \text{Calls} & \text{Y-PSNR} \uparrow & \text{Y-SSIM} \uparrow & \text{LPIPS} \downarrow & \text{Harm Rate} \downarrow & \text{Severe Rate} \downarrow \\\\ \hline \text{Always commit top-1} & 1.00 & 21.17 & 0.669 & 0.428 & 42.5 & 21.9 \\\\ \text{Verify only } (B=1) & 1.00 & 21.41 & 0.670 & 0.431 & \textbf{23.8} & \textbf{5.6} \\\\ \text{Verify + retry } (B=4) & 1.81 & \textbf{21.64} & \textbf{0.678} & \textbf{0.425} & 32.5 & 6.9 \\\\ \end{array}$$ When degradation states must be estimated from the image, estimation noise causes unconditional top-1 execution to suffer **42.5% broad harm and 21.9% severe harm (35/160 states)**. Here, the verifier is essential: at $B=1$ (1.00 call), it catches 26 of the 35 severe failures, slashing severe harm from **21.9% to 5.6%** and raising PSNR from 21.17 to 21.41 dB. Allowing retries ($B=4$, 1.81 calls) recovers valid tools for initially rejected states, boosting PSNR to **21.64 dB **, reducing broad harm by -10.0 percentage points, and cutting severe harm by -15.0 percentage points, from 35/160 down to 11/160 states). We will add both Always commit top-1 controls and this discussion to Sec. 4.4.
AuthorsOpenReview ↗

Response to Reviewer iYfP (Part 2/3): (W2, W3)

W2. Component Ablations and End-to-End Runtime.

(1) Metric overlap. The post-execution value model uses deployment-visible NR-IQA scores and never accesses test references or FR metrics at test time, so we evaluate all ablations below on FR metrics alongside NR-IQA.

(2) Component ablations. On 200 rain-haze states (100 unseen IDs, half ×4\times 4 downsampled) at B=8B=8 with the original toolbox (no adapted HAT-S), crossing pre-execution and final selection gives:

\text{Pre-Exec.} & \text{Final} & \text{Calls} & \text{PSNR} \uparrow & \text{SSIM} \uparrow & \text{LPIPS} \downarrow & \text{MUSIQ} \uparrow \\\\ \hline \text{Routing} & \text{Fixed} & 7.85 & 15.39 & 0.584 & 0.535 & 24.67 \\\\ \text{WM} & \text{Fixed} & \textbf{7.79} & \textbf{15.65} & \textbf{0.596} & \textbf{0.519} & \textbf{25.98} \\\\ \hline \text{Routing} & \text{Learned} & 7.85 & 15.74 & 0.599 & 0.455 & 29.27 \\\\ \text{WM} & \text{Learned} & \textbf{7.79} & \textbf{16.24} & \textbf{0.604} & \textbf{0.419} & \textbf{35.60} \\\\ \end{array}$$ WM beats routing under both fixed (+0.25 dB PSNR, -0.016 LPIPS, +1.32 MUSIQ) and learned selection (+0.51 dB PSNR, -0.036 LPIPS, +6.325 MUSIQ); learned selection adds +0.34/+0.60 dB. (At $B=4/12$, WM\$-\$routing $\Delta$PSNR/$\Delta$LPIPS/$\Delta$MUSIQ are -0.48/+0.035/+2.120 and +0.17/-0.019/+3.402.) On 40 fixed-pool states, isolating 3-branch blending (Eq. 28) gives: $$\begin{array}{lcccc} \text{Output Rule} & \text{PSNR} \uparrow & \text{SSIM} \uparrow & \text{LPIPS} \downarrow & \text{MUSIQ} \uparrow \\\\ \hline \text{Fixed priority} & 21.07 & 0.651 & 0.369 & 58.18 \\\\ \text{Learned single} & 22.84 & 0.700 & 0.300 & \textbf{62.98} \\\\ \text{3-branch blend} & \textbf{23.04} & \textbf{0.707} & \textbf{0.296} & 62.62 \\\\ \end{array}$$ Single-branch selection adds **+1.77 dB** and blending adds **+0.21 dB** (-0.362 MUSIQ). On Group C's 200 LR states, adapted HAT-S adds **+2.71 dB** over original HAT-S under identical fusion (+1.36 dB across all 400 states). **(3) End-to-end runtime.** On 96 OOF states (12 IDs, RTX 4090, estimated states, $B=4$), total time from read to write (estimation, WM, tool load/run, verification, rollback) is: $$\begin{array}{lccc} \text{Policy } (B=4) & \text{s/image} & \text{Calls} & \text{PSNR} \uparrow \\\\ \hline \text{Trial-and-error} & 22.55 & 3.55 & 20.27 \\\\ \text{Routing} & 14.54 & 2.36 & 21.80 \\\\ \text{PreActIR} & \textbf{11.15} & \textbf{1.78} & \textbf{21.93} \\\\ \end{array}$$ PreActIR cuts latency by 50.5% vs. trial-and-error and 23.3% vs. routing. Uncached timing on 12 two-stage states at $B=8$: $$\begin{array}{lcccc} \text{Two-Stage } (B=8) & \text{s/image} & \text{Calls} & \text{PSNR} \uparrow & \text{LPIPS} \downarrow \\\\ \hline \text{Routing} & \textbf{58.07} & \textbf{7.58} & 15.97 & 0.432 \\\\ \text{World Model} & 68.95 & 7.92 & \textbf{16.50} & \textbf{0.382} \\\\ \end{array}$$ Two-stage expansion trades ~10.9s of heavier tool calls for +0.53 dB PSNR and -0.050 LPIPS, while the $B=4$ loop cuts latency by 23%–51%. **W3. Real-World Generalization, Tool Cost, and Conclusion.** **(1) Real-world trade-off and unseen-ID transfer.** Scoring the degraded input ($I_0$) on the 100 local real pairs clarifies Table 2: $$\begin{array}{lcc} \text{local100} & \text{Input } (I_0) & \text{PreActIR} \\\\ \hline \text{LPIPS} \downarrow & 0.445 & \textbf{0.370} \\\\ \text{MUSIQ} \uparrow & 35.421 & \textbf{57.33} \\\\ \text{MANIQA} \uparrow & 0.239 & \textbf{0.389} \\\\ \text{CLIP-IQA} \uparrow & 0.338 & \textbf{0.525} \\\\ \text{Y-PSNR} \uparrow & \textbf{25.94} & 22.64 \\\\ \text{Y-SSIM} \uparrow & \textbf{0.7521} & 0.731 \\\\ \end{array}$$ Notably, the unprocessed input $I_0$ has higher Y-PSNR (25.94 dB) than almost every method in Table 2 (SEAR 24.41, AgenticIR 23.93), as removing haze/darkness/blur shifts luminance and sharpens textures away from the pseudo-reference. Crucially, PreActIR cuts full-reference LPIPS by 16.7% (0.445 to 0.370) while boosting MUSIQ by +21.91 and transfers cleanly to the 100 unseen AiOIR IDs in W2.2 (+0.51 dB PSNR, -0.036 LPIPS). **(2) Tool-adaptation cost.** Single-GPU training for adapted HAT-S took 69 min 19 s: $$\begin{array}{lccc} \text{Job} & \text{Pairs} & \text{Updates} & \text{Time} \\\\ \hline \text{Parent replay} & 6,000 & 12,000 & 52\text{m } 01\text{s} \\\\ \text{Adaptation} & 4,000 & 4,000 & 17\text{m } 18\text{s} \\\\ \end{array}$$ Stage 2 starts from the 4,000-step parent checkpoint (no MiO100 images). Fixed fusion on Group C LR states needs zero retraining of the WM/verifier; adding an online selectable tool instead requires logging transitions and refitting the lightweight WM/verifier heads (Sec. 4.5).
AuthorsOpenReview ↗

Response to Reviewer iYfP (Part 3/3): (W4, W5, Summary)

W4. Completing Table 2 and Enlarging Visual Comparisons.

(1) Adding InstructIR and MAIR to Table 2. We will add both published baselines to Table 2 with explicit source and protocol labels:

\text{Method / Source} & \text{PSNR} \uparrow & \text{SSIM} \uparrow & \text{LPIPS} \downarrow & \text{MANIQA} \uparrow & \text{CLIP-IQA} \uparrow & \text{MUSIQ} \uparrow \\\\ \hline \text{InstructIR [S]} & \textbf{26.15} & \textbf{0.764} & 0.424 & 0.256 & 0.341 & 37.82 \\\\ \text{MAIR [M]} & 21.67 & 0.727 & \textbf{0.324} & 0.320 & 0.403 & 55.21 \\\\ \text{PreActIR (Ours)} & 22.64 & 0.731 & 0.370 & \textbf{0.389} & \textbf{0.525} & \textbf{57.33} \\\\ \end{array}$$ Here, `[S]` denotes InstructIR re-evaluated under the Y-channel protocol in SEAR (Table 3), and `[M]` denotes MAIR's original paired results (MAIR Table 3). Notably, PreActIR outperforms MAIR across PSNR (+0.97 dB), SSIM, MANIQA, CLIP-IQA, and MUSIQ. **(2) Larger visual panels and zoomed crops.** In Figs. 4, 5, and 8, we will add SEAR and MAIR visual outputs, enlarge the main-text panels with readable labels, and include matched zoomed-in crops at identical spatial locations to highlight fine textures, edge structure, and residual artifacts (with full-page comparison grids in Appendix D). **W5 & Questions. Defining OOF and Ranking Baselines at First Use.** We will define **OOF** at its first appearance in Sec. 3.3 and place concise definitions for every ranking baseline directly in Sec. 4.3 before Table 3: * **OOF (out-of-fold):** A 5-fold cross-fitting protocol split by clean-image identity (keeping all degraded variants of an identity together), evaluated on held-out identities excluded from fitting and inner validation. * **Degradation routing:** Prioritizes the available degradation target with the highest estimated presence probability, then selects the tool with the largest mean training-fold fidelity gain ($\Delta F_Y$) for that target. * **Direct:** A separately trained image-and-action scorer sharing the WM's inputs, backbone, folds, and training rows, supervised only on fidelity and ranked by $\Delta\widehat{\text{PSNR}}_Y + 10\Delta\widehat{\text{SSIM}}_Y$. * **WM-F:** The trained multitask WM, ranking candidates at inference by its fidelity head alone using the same score as Direct (no retraining). Thus, Direct tests a standalone fidelity predictor, whereas WM-F tests the fidelity readout of the structured multitask WM. --- **Summary.** Incorporating your suggestions, we will revise the paper to: (1) clarify the pool/metric conventions in Tables 3 and 4 and add the matched always-commit verifier controls (**W1**); (2) include the factorized full-pipeline ablations and end-to-end wall-clock runtime tables (**W2**); (3) add the real-world input ($I_0$) analysis, offline tool-adaptation costs, and moderated conclusion in Sec. 5 (**W3**); (4) add InstructIR and MAIR to Table 2 and update Figs. 4, 5, and 8 with SEAR/MAIR visuals and zoomed-in crops (**W4**); and (5) define OOF and all ranking baselines at first use (**W5**). Thank you again for your insightful and supportive review, and we would be happy to discuss any further questions!

The reviewer has satisfactorily addressed my concerns, and I cast a vote for acceptance.

Comments on the paper

1 comment

Dear reviewers,

Thank you for your reviews. The authors have posted their rebuttal. Please read their response and leave a brief comment as soon as possible so the authors have time to follow up before the discussion closes.

Please indicate which of your main concerns the rebuttal addresses, what significant issues remain, and whether your overall recommendation has changed. Even if the rebuttal has not changed your assessment, please reply and briefly explain why, as this is important for the review process. If your assessment has changed, please also update your review and recommendation in OpenReview.

Thank you for your time and for taking part in the discussion.

Best regards, ICLR AC

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.