acceptodds
Withdrawn from ICLR 2027

MAPO-Net: MLLM-guided Artifact Perception and Feedback Optimization Network for HDR Reconstruction

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Despite remarkable advances in dynamic High Dynamic Range (HDR) reconstruction, artifacts such as underexposure, overexposure, ghosting, and motion blur still persist severely when reconstructing HDR images for dynamic scenes. Existing methods mainly rely on implicit feature alignment and lack explicit semantic understanding, making it difficult to robustly identify and restore complex local image degradation. To address this limitation, we propose MAPO-Net, a novel framework that bridges high-level semantic reasoning and low-level image reconstruction. For the first time, we introduce the high-level reasoning capability of Multimodal Large Language Models (MLLMs) into the feedback loop of HDR reconstruction. Our core idea is to convert artifact restoration into a "text-guided detection and penalization" pipeline. Specifically, MAPO-Net consists of two critical stages: Extensive experiments demonstrate that our method significantly improves artifact suppression and detail preservation under dynamic scenes, achieving superior performance on real-world HDR benchmarks. Our framework highlights the effectiveness of integrating MLLM-based reasoning with low-level vision reconstruction tasks. 1.Cross-Modal Artifact Perception: We freeze the MLLM (Qwen3-VL) to act as a diagnostic module that outputs semantic tokens describing various artifacts. A lightweight cross-modal detector then aligns these textual priors with visual features to accurately segment artifact regions. 2.Perception-Driven Feedback Optimization: The predicted artifact masks are converted into adaptive weight maps. Via artifact-aware loss, the base HDR network is compelled to focus on optimizing these defective regions flagged by the MLLM during training. Furthermore, existing public datasets suffer from limited diversity of challenging samples for the aforementioned artifacts. To mitigate this issue, we contribute 2,300 real-world LDR scenes and construct a brand-new HDR dataset comprising these 2,300 hard samples. Extensive experiments conducted on our newly built dataset and existing public benchmarks demonstrate that the proposed MAPO-Net can effectively suppress artifacts that cannot be eliminated by prior approaches, achieving substantial improvements over state-of-the-art methods.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
MLLMs, HDR, Artifact

Reviews

Among:
Shape:
1234this paper, Bottom 2%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.33 over 3 reviews, above 2% of the others.
Reviewer_fs7QOpenReview ↗
Rating
1
Confidence
4
Informativeness
1
Presentation
2
Soundness
1
Read review
Paper summary

This paper proposes MAPO-Net, an MLLM-guided framework for multi-exposure HDR reconstruction in dynamic scenes. By leveraging a frozen multi-modal large language model to generate artifact descriptions, the method introduces a cross-modal detector that combines textual and visual features to localize reconstruction artifacts. The predicted masks are converted into spatial weight maps to guide an artifact-aware reconstruction loss, complemented by an MLLM-based perceptual loss. These auxiliary modules are used only during training. At inference, only the HDR reconstruction backbone is retained. Furthermore, the authors introduce Hard-HDR, a dataset containing 2,300 real-captured scenes with HDR references and artifact annotations. The reported experiments on three datasets and multiple reconstruction backbones show improvements in reconstruction quality and ghosting suppression.

Critical strengths

The proposed Hard-HDR dataset could be a useful resource for the community if publicly released, providing additional real-captured scenes and artifact annotations for training and evaluation.

Critical weaknesses
  1. The manuscript contains numerous substantive citation errors, including references to its central backbone and competing methods. For instance, AFUNet is attributed to Yan et al., TPAMI 2024, whereas the actual work is by Li et al., ICCV 2025. The DiffHDR reference pairs the paper title with an entirely incorrect author list, and Table 6 identifies HDR-Transformer as TPAMI 2024 despite citing its ECCV 2022 paper. Additional author and title mismatches occur in the SUPIR and GPT-4V references. The extent of these errors raises serious concerns about the accuracy of the literature review and the identification of the methods used in the experiments.

  2. The experimental setup is insufficiently specified, making it difficult to assess how the reported comparisons were obtained. The tables also contain unexplained inconsistencies: Table 4 describes an ablation on Hard-HDR, yet its baseline exactly matches all five Kalantari17 baseline values in Tables 1 and Table 5 rather than the Hard-HDR results. Its full-model results also differ from those reported for Hard-HDR in Tables 1 and Table 7. These discrepancies undermine confidence in the reported improvements and ablation conclusions.

Questions for authors

• How do the authors explain the numerous incorrect author–title associations and publication details in the references? Were these entries generated or modified using AI tools, and how were they verified?

• How do the authors explain the discrepancies between Table 4 and Tables 1 and Table 7? In particular, why does the Hard-HDR ablation baseline exactly match the Kalantari17 baseline in Tables 1 and Table 5, and why do the full-model results differ across tables? Please clarify the experimental settings and identify the correct results.

Reviewer_MSTLOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

This paper proposes MAPO-Net, an MLLM-guided framework to reduce ghosting and other motion artifacts for HDR reconstruction in dynamic scenes. MAPO-Net has two main stages. First, it uses a frozen MLLM (Qwen3-VL) and a cross-modal detector (MPA-Detector) to identify artifact regions. Second, it converts the predicted artifact masks into adaptive weight maps. These maps guide the HDR reconstruction network to focus on regions with severe artifacts. The authors also introduce Hard-HDR, a new dataset with 2,300 real-captured LDR-HDR sample groups. Experiments on three datasets show improvements over several existing methods.

Critical strengths
  1. An interesting framework for artifact-aware HDR reconstruction. The paper introduces MLLM-based semantic information into HDR reconstruction. The predicted artifact masks are used to guide the reconstruction loss. This allows the network to pay more attention to regions with severe artifacts.
  2. A new HDR dataset and experiments on different networks. The authors construct Hard-HDR with 2,300 real-captured samples. The dataset includes motion patterns. The authors also test the proposed method on different networks, including AFUNet, AHDR, and SCTNet. These experiments suggest that the proposed method may work with different network architectures.
Critical weaknesses
  1. Limited methodological novelty and unclear contribution of the MLLM. The proposed feedback optimization is mainly based on a spatially weighted reconstruction loss. The MLLM generates semantic descriptions, but artifact localization is still performed by a supervised detector. It is unclear why an MLLM is necessary for this process. In addition, the proposed method does not clearly demonstrate a real iterative feedback mechanism.
  2. Insufficient experimental evidence. The paper does not provide enough comparisons with simpler artifact detection and loss weighting methods. Therefore, the actual contribution of the MLLM remains unclear. Moreover, the results in Table 1 and Table 4 are inconsistent, and the generation of artifact-free HDR ground truth for dynamic scenes is not sufficiently explained.
Questions for authors
  1. What is the main advantage of using an MLLM instead of a conventional artifact detector?

  2. Why are the results in Table 1 and Table 4 inconsistent, especially for AFUNet and MAPO-Net on Hard-HDR?

  3. How is artifact-free HDR ground truth generated for dynamic scenes? Is motion alignment performed before exposure fusion?

Reviewer_jUNDOpenReview ↗
Rating
1
Confidence
4
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

The paper propose MAPO-Net, a novel framework that bridges high-level semantic reasoning with low-level reconstruction by reformulating artifact removal as a text-guided detection and feedback optimization process. MAPO-Net operates in two stages: 1) Cross-Modal Artifact Perception, which leverages a frozen MLLM (Qwen3-VL) and a lightweight detector (MPA-Detector) to generate semantic descriptions and localize artifact regions; and 2) Perception-Driven Feedback Optimization, which converts localized masks into adaptive weight maps to guide an artifact-aware loss, steering the HDR network toward repairing defective regions. However, the manuscript contains clear issues with fabricated references. I am inclined to recommend a desk reject.

Critical strengths
  1. The paper introduces an HDR reconstruction dataset with thousands of paired samples.
  2. As shown by the experimental results, it outperforms existing methods by a significant margin on distortion metrics such as PSNR and SSIM.
Critical weaknesses
  1. There are a large number of false references in this manuscript. For many cited works, the authors, publication venues and titles are inconsistent with one another.
  2. Although PSNR-μ and SSIM-μ have been adopted in many prior papers, they cannot effectively measure deghosting performance. In addition, comparisons with many SOTA methods are missing. The method also raises several counterintuitive concerns. For instance, encoders of MLLMs are typically trained on SDR data; directly encoding HDR content introduces substantial information loss and thus degrades performance.
Questions for authors
  1. Could the authors clarify the authenticity of the references?
  2. How is the information loss mitigated when the MLLM encoder processes HDR images?
  3. Is HDR-VDP2 computed in the HDR domain or the SDR domain? Please also provide the relevant computation parameters.
  4. The masks in Figure 3 appear to segment full human figures rather than detecting ghosting artifacts, which is confusing.
  5. Could the authors report the inference computational overhead?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.