- Rating
- 2
- Confidence
- 4
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 3
Read review
Paper summary
This paper proposes AnyMEF, an observation-guided multi-exposure image fusion framework designed for diverse MEF protocols. It addresses the limited generalization of methods tailored to specific input counts, exposure configurations, and anchor choices. Built upon a pretrained generative model, it introduces an extracted-attention-guided fusion mechanism to inject source structural information into the decoding process. A quality-based token selector retains informative reference regions to reduce computational cost, while a synthetic training pipeline supports flexible exposure configurations and anchor selection. Experiments on several benchmarks evaluate fusion quality.
Critical strengths
- It reformulates MEF as a unified observation-guided enhancement task. It treats different-exposed source images as degraded observations of underlying HDR scenes and seeks an enhanced, display-ready image corresponding to the selected anchor’s content and viewpoint. This formulation accommodates exposure correction and other degradations within a common framework.
- It addresses generalization across multi-exposure fusion settings by proposing a unified framework with three aspects of flexibility: i) allowing an arbitrary number of source images; ii) accommodating arbitrary exposure levels or configurations; and iii) allowing any source image to serve as the anchor, preserving its main scene content while spatially aligning the other source images to it.
- The method combines latent-space generation with source-guided detail refinement. The flow-predicting backbone generates an enhanced latent conditioned on the anchor and references, while the attention-guided refiner injects source structure into the decoder. This design provides a concrete means of combining generative priors with observed image details.
Critical weaknesses
- The discussion and summary of existing methods are insufficient. In Sec. 3.1, the discussion of conventional MEF formulations is largely limited to weighted fusion introduced in 2007. Many subsequent methods, including large-model-based approaches and methods supporting flexible exposure inputs, deserve further consideration. Although some methods accept only two source images, they can process multiple images through sequential fusion. Merely identifying restrictions on input configurations as limitations is insufficient. The authors should review existing work more broadly and explain the specific technical challenges.
- The functional advantages are emphasized more than the technical contributions. Much of the presentation focuses on what the method supports, including flexible input counts, anchor selection, and reference subareas. Beyond the synthetic training pipeline and the use of a pretrained generative model, the main technical additions are EAGF and quality-based token selection. The contribution statements remain fairly general.
- The technical novelty beyond the backbone is limited. EAGF relies on attention produced by FLUX transformer backbone and applies aggregation and normalization to obtain fusion weights. Similarly, the quality-based token selector relies on quality scores derived from degradation differences and luminance characteristics to select reference tokens. These designs seem to be some specific operations. The methodological innovation appears incremental.
- The experimental results have some limitations, and the evaluation is not sufficiently comprehensive. Although the outputs often exhibit clearer textures, some displayed results appear relatively dark. The quantitative evaluation relies heavily on no-reference perceptual metrics. Only one reference metric TMQIs is applied. The computational cost is lower than that of UltraFusion but remains considerably higher than that of other methods.
Questions for authors
- The task formulation appears closely related to HDR imaging. Could the authors clarify its distinction from HDR reconstruction? Multi-exposure images may also contain spatial displacement. Although Section 3.2 describes RAFT-based alignment, how does the method handle residual misalignment? Is further correction implicitly learned in latent space through ground-truth supervision?
- Is the core of EAGF the attention-weighted generation of the fused structure map in Eq. (5)? Are the attention values directly extracted from the FLUX transformer backbone and then aggregated and normalized to produce fusion weights? Please clarify the core innovation and importance of these operations.
- For datasets with ground truth, only TMQIs is reported as a reference-based metric, which provides a limited evaluation. Could the authors include additional appropriate metrics, such as the widely used PSNR, SSIM? Why are ground-truth images not shown in the qualitative comparisons? Appropriately ground-truth visualizations would help assess structural and detail preservation.
3 comments
Reply to Reviewer t2WA (Part 2)
- Output: HDR reconstruction must produce an HDR image in the linear domain, where the intensity of each output pixel is linearly related to the scene radiance. The output of MEF is a tone-mapped LDR image in the sRGB domain. To represent the details of the scene within a limited dynamic range, the relation between its pixel intensities and the scene radiance can be not only nonlinear but even locally non-monotonic: for example, in a night scene, bringing a signboard whose intensities far exceed those of the background down to a range close to the background texture is often preferable in MEF.
In MEF, because of the uncertainty in the input, which tone mapping---in brightness, contrast and color---the output image should adopt is ambiguous. This also affects our choice of evaluation metrics.
W4.2 & Q3.1 PSNR & SSIM.
We recommend reading this response together with Sec. A.3 (revised appendix), which includes Table A.1 and Fig. A.1.
Thank you for this suggestion. We have included PSNR and SSIM results in Table A.1, and we would also like to explain why we did not adopt them as our primary reference metric.
As argued in W4.1 & Q1.1, which tone mapping---in brightness, contrast and color---an MEF output should adopt is ambiguous, because the inputs are uncalibrated and various LDR images may be consistent with the same scene content. Scored against a single reference image, a reference-based metric for MEF should therefore
(1) be insensitive to small deviations in brightness;
(2) be insensitive to small differences in contrast;
(3) be insensitive to small deviations in color;
(4) be sensitive to texture, i.e., to the relative relationships among pixel intensities;
(5) be sensitive to misalignment (e.g., a small translation);
(6) be sensitive to common degradations such as blur and noise.
PSNR fails (1)--(3), since it compares pixel intensities directly. SSIM decomposes into luminance, contrast and structure terms, of which the first two measure agreement in local mean intensity and in local contrast, penalizing legitimate choices. Moreover, to calculate PSNR and SSIM of a tonemapped image, the ground truth HDR image must first be converted into the LDR range using a specific tonemap: the choice of the tonemap strongly influences the metric results.
TMQI-S instead compares the output against the linear HDR reference, which it normalizes internally: its dominant term is the normalized local cross-correlation between the two. Local contrast enters only through a visibility test, so it tolerates smoothly varying brightness, contrast and colour differences that a different tone mapping introduces.
Fig. A.1 illustrates one RealHDRV scene. Taking the standard tone mapping (b) as the reference, we render two other tone mappings (c, d) that differ slightly in brightness---which we regard as equally valid MEF outputs---and three degradations (e, f, g), namely a spatial shift, a slight blur and noise, which we regard as strictly worse. Under PSNR and SSIM the best result is the blurred image (f), at 44.70 dB / 0.9844, while the valid tone mapping (d) drops to 17.26 dB. Under TMQI-S the three valid outputs take the top three ranks (0.8990, 0.8828, 0.8673), and the translation and the noise fall far below (0.3362, 0.7760).
That said, we report PSNR and SSIM on the 2-OE protocol in Table A.1. AnyMEF obtains the best PSNR and the best TMQI-S on both datasets, while UltraFusion obtains the best SSIM:
| Method | RealHDRV PSNR (dB) | RealHDRV SSIM | RealHDRV TMQI-S | DeepHDRVideo PSNR (dB) | DeepHDRVideo SSIM | DeepHDRVideo TMQI-S |
|---|---|---|---|---|---|---|
| MEFLUT | 16.64 | 0.8485 | 0.803 | 17.18 | 0.8701 | 0.722 |
| HSDS-MEF | 16.36 | 0.7907 | 0.805 | 16.49 | 0.7912 | 0.696 |
| TC-MoA | 14.75 | 0.7805 | 0.823 | 15.12 | 0.7862 | 0.714 |
| FreeMEF | 16.51 | 0.8035 | 0.830 | 18.16 | 0.8263 | 0.711 |
| UltraFusion | 17.88 | 0.8653 | 0.838 | 18.54 | 0.8878 | 0.719 |
| AnyMEF (Ours) | 19.77 | 0.8335 | 0.842 | 21.69 | 0.8423 | 0.747 |
W4.3 Computational cost.
We agree that AnyMEF is computationally heavy compared to non-generative methods. We have added a discussion of potential acceleration paths to Sec. A.5.
Reply to Reviewer t2WA (Part 3)
Q1.2. Handling residual misalignment.
We deal with RAFT alignment inaccuracies with the strategies listed below, in which 1, 2 and 4 follow UltraFusion. Specifically, when warping image B to the anchor image A:
-
To cope with exposure differences, we match the histogram of B to that of A before feeding them to RAFT.
-
We compute both the optical flow from A to B, and the flow from B to A. Then we perform a forward-backward consistency check: it measures how far each pixel deviates from its original position after being mapped forward and then backward by the estimated optical flows. We mask out the inconsistent areas (corresponding to occlusions or optical flow inaccuracies) with black.
-
Masked tokens are excluded from token selection: we assign them the lowest quality, so they are never routed to the backbone. Masked pixels also receive zero weight in EAGF structure fusion. Since AnyMEF can take multiple warped images as reference, for each spatial area, the probability that at least in one image it is non-occluded and nicely warped is high.
-
In some cases, parts of the black occlusion masks may still enter the backbone (e.g., tokens that are only partially masked). To make our models robust to them, we add random black occlusion masks to our model inputs when training. The masks are sourced from real occlusion masks of the Vimeo dataset.
With 4, AnyMEF implicitly learns to cope with occlusion masks. However, we did not explicitly train it to correct residual alignment errors. We attribute its robustness to such errors in practice to two factors: (1) most large alignment errors are filtered out by the forward-backward consistency check; and (2) the anchor requires no warping, so the model always has a stable fallback when warping artifacts appear in the references.
We have added these details and examples of train/test warp masks to Sec. A.4.
Q2. Details and innovation of EAGF.
We recommend reading this response together with Sec. A.7 (revised appendix), especially Table A.5 and Fig. A.6.
Yes. The core of EAGF is the attention-weighted fusion of structure maps in Eq. (5), and the attention values are directly extracted from the finetuned FLUX backbone during inference, without any additional learnable parameters.
The innovation of EAGF is not a newly designed module trained for structure fusion. Rather, our key finding is that the quality-aware fusion weights we need already exist in the inference pipeline: to generate its output, the backbone must decide which observation to draw information from at each position, and its cross-image attention records this decision in an interpretable form. This allows us to obtain the fusion weights as a free lunch, at negligible cost, for any number of inputs and any anchor. To the best of our knowledge, using the cross-image attention of a multi-reference generative backbone as per-pixel fusion weights for source structures has not been explored before.
Regarding the interpretability of EAGF, we provide new evidence: a manipulation experiment shows that the extracted attention decreases consistently as reference patches are degraded (blur, noise, over-/under-exposure), i.e., it reflects reference quality.
Q4. Lack of ground-truth images.
Among our benchmarks, MEFB, UltraFusionBench and DeghostingIQA do not provide ground truth images. RealHDRV and DeepHDRVideo provide GT but we forgot to put them in. Thank you for pointing this out; we have replaced Figure 3 with a version including GT.
We thank you again for your detailed feedback, which has helped us improve the presentation of our method considerably. Should any part of our response or of the revised manuscript remain unconvincing, we would appreciate the opportunity to clarify it further.
Reply to Reviewer t2WA (Part 1)
We sincerely thank Reviewer t2WA for recognizing that our formulation "accommodates exposure correction and other degradations within a common framework" and that our design "provides a concrete means of combining generative priors with observed image details". We address questions and concerns below. We have revised the manuscript (revised manuscript) and added many new figures and tables to the appendix (revised appendix); we recommend viewing both alongside this response. Revised text is marked in blue font.
W1.1 Insufficient discussion of existing work.
Thank you for pointing out the gap in our discussion. Recent neural-network-based works do not simply predict fusion weights; they perform explicit or implicit multi-image fusion at the feature level. We have revised Section 3.1 to make this clearer. If we still lack necessary references, we would appreciate it if you could point out the specific works, and we will add them.
W1.2 Processing multiple images through sequential fusion.
We recommend reading this response together with Sec. A.11 (revised appendix), which includes Fig. A.7 and Fig. A.8.
Thank you for raising this. Sequential fusion is indeed a viable way to apply a two-image method to more images, and we agree that it deserves a direct comparison rather than being dismissed as a limitation. However, we find that it degrades in practice: iteratively applying the model leads to error accumulation and distribution drift, and the outputs eventually contain visible artifacts.
We therefore ran this comparison on RealHDRV. Starting from the anchor frame I4,OE, we fused the other seven frames one at a time with MEFLUT, HSDS-MEF, TC-MoA and UltraFusion, where the accumulated output of each step serves as the anchor of the next step. As shown in Fig. A.7, fidelity trends downward for all four methods as more frames are fused, and the no-reference metrics saturate instead of improving. Fig. A.8 shows a qualitative example with UltraFusion, where accumulated color drift is clearly visible.
The reason is that each step feeds a synthesized image back into a model trained on real photographs, so alignment and radiometric errors compound instead of cancelling out. This is why we train AnyMEF to handle multiple images from the start, warping all references to a single anchor frame once and fusing them jointly.
W2 & W3. Lack of technical novelty.
We understand the concern, and we would like to emphasize our conceptual and emperical contributions, before discussing our technical contributions.
Conceptual contributions. Our paper "presents a well-motivated problem reformulation" (Reviewer GjJL), proposing "a practically meaningful unified framework" (Reviewer PDv3).
Emperical contributions. Our paper "directly addresses real deployment constraints" (Reviewer GjJL), and shows "strong empirical performance with substantially improved efficiency" (Reviewer PDv3).
Technical contributions. We believe our technical contribution lies in uncovering properties and capabilities of existing generative models and developing methods to exploit them. For example:
Though we only experimented on multi-exposure fusion, we believe that these techniques may also transfer to other multi-observation low-level tasks such as burst denoising and multi-frame deblurring, allowing them to exploit the rich priors and multi-reference abilities provided by generative models.
W4.1 & Q1.1. Difference between HDR reconstruction and MEF.
HDR reconstruction and MEF are related but distinct tasks. Broadly speaking, HDR reconstruction constrains both its input and its output more strictly, whereas MEF is more permissive on both sides.