acceptodds
Desk rejected at ICLR 2027

AnyMEF: Unified Multi-Exposure Fusion with a Flexible Generative Model

Desk rejected at ICLR 2027, so left out of the paper lists and the map.

13 views

Abstract

Multi-exposure fusion (MEF) combines images captured at different exposure levels for HDR imaging. However, existing MEF methods are typically tailored to specific exposure protocols, limiting their applicability in general scenarios. We formulate MEF as a unified observation-guided enhancement task and present AnyMEF, a generative framework that flexibly supports diverse MEF protocols. Built upon a pretrained generative model with multi-reference capabilities, AnyMEF introduces an extracted-attention-guided fusion mechanism (EAGF) to inject structural information from an arbitrary number of input images. Its synthetic training-data pipeline removes the assumption that the anchor must have a particular exposure, allowing any input frame to be selected as the anchor. In addition, AnyMEF allows adaptively routing image subareas to the transformer backbone, and implements a quality-based token selection strategy to improve efficiency. Experiments on multiple MEF benchmarks show that AnyMEF significantly outperforms existing methods while requiring only 20% as many TFLOPs and 6% as much inference time as UltraFusion.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
HDR Imaging, Generative models, Low level vision, Multi-exposure Fusion

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_t2WAOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

This paper proposes AnyMEF, an observation-guided multi-exposure image fusion framework designed for diverse MEF protocols. It addresses the limited generalization of methods tailored to specific input counts, exposure configurations, and anchor choices. Built upon a pretrained generative model, it introduces an extracted-attention-guided fusion mechanism to inject source structural information into the decoding process. A quality-based token selector retains informative reference regions to reduce computational cost, while a synthetic training pipeline supports flexible exposure configurations and anchor selection. Experiments on several benchmarks evaluate fusion quality.

Critical strengths
  1. It reformulates MEF as a unified observation-guided enhancement task. It treats different-exposed source images as degraded observations of underlying HDR scenes and seeks an enhanced, display-ready image corresponding to the selected anchor’s content and viewpoint. This formulation accommodates exposure correction and other degradations within a common framework.
  2. It addresses generalization across multi-exposure fusion settings by proposing a unified framework with three aspects of flexibility: i) allowing an arbitrary number of source images; ii) accommodating arbitrary exposure levels or configurations; and iii) allowing any source image to serve as the anchor, preserving its main scene content while spatially aligning the other source images to it.
  3. The method combines latent-space generation with source-guided detail refinement. The flow-predicting backbone generates an enhanced latent conditioned on the anchor and references, while the attention-guided refiner injects source structure into the decoder. This design provides a concrete means of combining generative priors with observed image details.
Critical weaknesses
  1. The discussion and summary of existing methods are insufficient. In Sec. 3.1, the discussion of conventional MEF formulations is largely limited to weighted fusion introduced in 2007. Many subsequent methods, including large-model-based approaches and methods supporting flexible exposure inputs, deserve further consideration. Although some methods accept only two source images, they can process multiple images through sequential fusion. Merely identifying restrictions on input configurations as limitations is insufficient. The authors should review existing work more broadly and explain the specific technical challenges.
  2. The functional advantages are emphasized more than the technical contributions. Much of the presentation focuses on what the method supports, including flexible input counts, anchor selection, and reference subareas. Beyond the synthetic training pipeline and the use of a pretrained generative model, the main technical additions are EAGF and quality-based token selection. The contribution statements remain fairly general.
  3. The technical novelty beyond the backbone is limited. EAGF relies on attention produced by FLUX transformer backbone and applies aggregation and normalization to obtain fusion weights. Similarly, the quality-based token selector relies on quality scores derived from degradation differences and luminance characteristics to select reference tokens. These designs seem to be some specific operations. The methodological innovation appears incremental.
  4. The experimental results have some limitations, and the evaluation is not sufficiently comprehensive. Although the outputs often exhibit clearer textures, some displayed results appear relatively dark. The quantitative evaluation relies heavily on no-reference perceptual metrics. Only one reference metric TMQIs is applied. The computational cost is lower than that of UltraFusion but remains considerably higher than that of other methods.
Questions for authors
  1. The task formulation appears closely related to HDR imaging. Could the authors clarify its distinction from HDR reconstruction? Multi-exposure images may also contain spatial displacement. Although Section 3.2 describes RAFT-based alignment, how does the method handle residual misalignment? Is further correction implicitly learned in latent space through ground-truth supervision?
  2. Is the core of EAGF the attention-weighted generation of the fused structure map in Eq. (5)? Are the attention values directly extracted from the FLUX transformer backbone and then aggregated and normalized to produce fusion weights? Please clarify the core innovation and importance of these operations.
  3. For datasets with ground truth, only TMQIs is reported as a reference-based metric, which provides a limited evaluation. Could the authors include additional appropriate metrics, such as the widely used PSNR, SSIM? Why are ground-truth images not shown in the qualitative comparisons? Appropriately ground-truth visualizations would help assess structural and detail preservation.
3 comments
AuthorsOpenReview ↗

Reply to Reviewer t2WA (Part 1)

We sincerely thank Reviewer t2WA for recognizing that our formulation "accommodates exposure correction and other degradations within a common framework" and that our design "provides a concrete means of combining generative priors with observed image details". We address questions and concerns below. We have revised the manuscript (revised manuscript) and added many new figures and tables to the appendix (revised appendix); we recommend viewing both alongside this response. Revised text is marked in blue font.

W1.1 Insufficient discussion of existing work.

Thank you for pointing out the gap in our discussion. Recent neural-network-based works do not simply predict fusion weights; they perform explicit or implicit multi-image fusion at the feature level. We have revised Section 3.1 to make this clearer. If we still lack necessary references, we would appreciate it if you could point out the specific works, and we will add them.

W1.2 Processing multiple images through sequential fusion.

We recommend reading this response together with Sec. A.11 (revised appendix), which includes Fig. A.7 and Fig. A.8.

Thank you for raising this. Sequential fusion is indeed a viable way to apply a two-image method to more images, and we agree that it deserves a direct comparison rather than being dismissed as a limitation. However, we find that it degrades in practice: iteratively applying the model leads to error accumulation and distribution drift, and the outputs eventually contain visible artifacts.

We therefore ran this comparison on RealHDRV. Starting from the anchor frame I4,OEI_{4,\mathrm{OE}}, we fused the other seven frames one at a time with MEFLUT, HSDS-MEF, TC-MoA and UltraFusion, where the accumulated output of each step serves as the anchor of the next step. As shown in Fig. A.7, fidelity trends downward for all four methods as more frames are fused, and the no-reference metrics saturate instead of improving. Fig. A.8 shows a qualitative example with UltraFusion, where accumulated color drift is clearly visible.

The reason is that each step feeds a synthesized image back into a model trained on real photographs, so alignment and radiometric errors compound instead of cancelling out. This is why we train AnyMEF to handle multiple images from the start, warping all references to a single anchor frame once and fusing them jointly.

W2 & W3. Lack of technical novelty.

We understand the concern, and we would like to emphasize our conceptual and emperical contributions, before discussing our technical contributions.

Conceptual contributions. Our paper "presents a well-motivated problem reformulation" (Reviewer GjJL), proposing "a practically meaningful unified framework" (Reviewer PDv3).

Emperical contributions. Our paper "directly addresses real deployment constraints" (Reviewer GjJL), and shows "strong empirical performance with substantially improved efficiency" (Reviewer PDv3).

Technical contributions. We believe our technical contribution lies in uncovering properties and capabilities of existing generative models and developing methods to exploit them. For example:

  • EAGF: We find that attention weights within these models have interpretable meanings. By extracting these weights from the model’s black box, we turn implicit quality judgments in latent space into explicit fusion weights in pixel space.
  • Sparse reference inputs: We find that models pretrained on dense reference images have the potential to accommodate sparse references, and that this capability can be enabled through fine-tuning with random token dropout.

Though we only experimented on multi-exposure fusion, we believe that these techniques may also transfer to other multi-observation low-level tasks such as burst denoising and multi-frame deblurring, allowing them to exploit the rich priors and multi-reference abilities provided by generative models.

W4.1 & Q1.1. Difference between HDR reconstruction and MEF.

HDR reconstruction and MEF are related but distinct tasks. Broadly speaking, HDR reconstruction constrains both its input and its output more strictly, whereas MEF is more permissive on both sides.

  1. Input: To establish the relation between an LDR image and the actual scene radiance, HDR reconstruction usually requires calibrated inputs, such as the exposure time, the ISO and the camera response curve. MEF instead takes uncalibrated images that have been processed by an unknown ISP, whose pixel intensities cannot be mapped back to the irradiance of the scene.
AuthorsOpenReview ↗

Reply to Reviewer t2WA (Part 2)

  1. Output: HDR reconstruction must produce an HDR image in the linear domain, where the intensity of each output pixel is linearly related to the scene radiance. The output of MEF is a tone-mapped LDR image in the sRGB domain. To represent the details of the scene within a limited dynamic range, the relation between its pixel intensities and the scene radiance can be not only nonlinear but even locally non-monotonic: for example, in a night scene, bringing a signboard whose intensities far exceed those of the background down to a range close to the background texture is often preferable in MEF.

In MEF, because of the uncertainty in the input, which tone mapping---in brightness, contrast and color---the output image should adopt is ambiguous. This also affects our choice of evaluation metrics.

W4.2 & Q3.1 PSNR & SSIM.

We recommend reading this response together with Sec. A.3 (revised appendix), which includes Table A.1 and Fig. A.1.

Thank you for this suggestion. We have included PSNR and SSIM results in Table A.1, and we would also like to explain why we did not adopt them as our primary reference metric.

As argued in W4.1 & Q1.1, which tone mapping---in brightness, contrast and color---an MEF output should adopt is ambiguous, because the inputs are uncalibrated and various LDR images may be consistent with the same scene content. Scored against a single reference image, a reference-based metric for MEF should therefore

(1) be insensitive to small deviations in brightness;

(2) be insensitive to small differences in contrast;

(3) be insensitive to small deviations in color;

(4) be sensitive to texture, i.e., to the relative relationships among pixel intensities;

(5) be sensitive to misalignment (e.g., a small translation);

(6) be sensitive to common degradations such as blur and noise.

PSNR fails (1)--(3), since it compares pixel intensities directly. SSIM decomposes into luminance, contrast and structure terms, of which the first two measure agreement in local mean intensity and in local contrast, penalizing legitimate choices. Moreover, to calculate PSNR and SSIM of a tonemapped image, the ground truth HDR image must first be converted into the LDR range using a specific tonemap: the choice of the tonemap strongly influences the metric results.

TMQI-S instead compares the output against the linear HDR reference, which it normalizes internally: its dominant term is the normalized local cross-correlation between the two. Local contrast enters only through a visibility test, so it tolerates smoothly varying brightness, contrast and colour differences that a different tone mapping introduces.

Fig. A.1 illustrates one RealHDRV scene. Taking the standard tone mapping (b) as the reference, we render two other tone mappings (c, d) that differ slightly in brightness---which we regard as equally valid MEF outputs---and three degradations (e, f, g), namely a spatial shift, a slight blur and noise, which we regard as strictly worse. Under PSNR and SSIM the best result is the blurred image (f), at 44.70 dB / 0.9844, while the valid tone mapping (d) drops to 17.26 dB. Under TMQI-S the three valid outputs take the top three ranks (0.8990, 0.8828, 0.8673), and the translation and the noise fall far below (0.3362, 0.7760).

That said, we report PSNR and SSIM on the 2-OE protocol in Table A.1. AnyMEF obtains the best PSNR and the best TMQI-S on both datasets, while UltraFusion obtains the best SSIM:

MethodRealHDRV PSNR (dB)RealHDRV SSIMRealHDRV TMQI-SDeepHDRVideo PSNR (dB)DeepHDRVideo SSIMDeepHDRVideo TMQI-S
MEFLUT16.640.84850.80317.180.87010.722
HSDS-MEF16.360.79070.80516.490.79120.696
TC-MoA14.750.78050.82315.120.78620.714
FreeMEF16.510.80350.83018.160.82630.711
UltraFusion17.880.86530.83818.540.88780.719
AnyMEF (Ours)19.770.83350.84221.690.84230.747

W4.3 Computational cost.

We agree that AnyMEF is computationally heavy compared to non-generative methods. We have added a discussion of potential acceleration paths to Sec. A.5.

AuthorsOpenReview ↗

Reply to Reviewer t2WA (Part 3)

Q1.2. Handling residual misalignment.

We deal with RAFT alignment inaccuracies with the strategies listed below, in which 1, 2 and 4 follow UltraFusion. Specifically, when warping image B to the anchor image A:

  1. To cope with exposure differences, we match the histogram of B to that of A before feeding them to RAFT.

  2. We compute both the optical flow from A to B, and the flow from B to A. Then we perform a forward-backward consistency check: it measures how far each pixel deviates from its original position after being mapped forward and then backward by the estimated optical flows. We mask out the inconsistent areas (corresponding to occlusions or optical flow inaccuracies) with black.

  3. Masked tokens are excluded from token selection: we assign them the lowest quality, so they are never routed to the backbone. Masked pixels also receive zero weight in EAGF structure fusion. Since AnyMEF can take multiple warped images as reference, for each spatial area, the probability that at least in one image it is non-occluded and nicely warped is high.

  4. In some cases, parts of the black occlusion masks may still enter the backbone (e.g., tokens that are only partially masked). To make our models robust to them, we add random black occlusion masks to our model inputs when training. The masks are sourced from real occlusion masks of the Vimeo dataset.

With 4, AnyMEF implicitly learns to cope with occlusion masks. However, we did not explicitly train it to correct residual alignment errors. We attribute its robustness to such errors in practice to two factors: (1) most large alignment errors are filtered out by the forward-backward consistency check; and (2) the anchor requires no warping, so the model always has a stable fallback when warping artifacts appear in the references.

We have added these details and examples of train/test warp masks to Sec. A.4.

Q2. Details and innovation of EAGF.

We recommend reading this response together with Sec. A.7 (revised appendix), especially Table A.5 and Fig. A.6.

Yes. The core of EAGF is the attention-weighted fusion of structure maps in Eq. (5), and the attention values are directly extracted from the finetuned FLUX backbone during inference, without any additional learnable parameters.

The innovation of EAGF is not a newly designed module trained for structure fusion. Rather, our key finding is that the quality-aware fusion weights we need already exist in the inference pipeline: to generate its output, the backbone must decide which observation to draw information from at each position, and its cross-image attention records this decision in an interpretable form. This allows us to obtain the fusion weights as a free lunch, at negligible cost, for any number of inputs and any anchor. To the best of our knowledge, using the cross-image attention of a multi-reference generative backbone as per-pixel fusion weights for source structures has not been explored before.

Regarding the interpretability of EAGF, we provide new evidence: a manipulation experiment shows that the extracted attention decreases consistently as reference patches are degraded (blur, noise, over-/under-exposure), i.e., it reflects reference quality.

Q4. Lack of ground-truth images.

Among our benchmarks, MEFB, UltraFusionBench and DeghostingIQA do not provide ground truth images. RealHDRV and DeepHDRVideo provide GT but we forgot to put them in. Thank you for pointing this out; we have replaced Figure 3 with a version including GT.


We thank you again for your detailed feedback, which has helped us improve the presentation of our method considerably. Should any part of our response or of the revised manuscript remain unconvincing, we would appreciate the opportunity to clarify it further.

Reviewer_PDv3OpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes AnyMEF, a unified generative framework for Multi-Exposure Fusion (MEF) that supports arbitrary numbers of input images, flexible anchor-frame selection, and partial reference observations. The authors reformulate MEF as an observation-guided enhancement task and adapt a pretrained FLUX.2-klein-4B model for one-step image generation. To preserve fine-grained structural details that may be lost through VAE compression, the method introduces Extracted-Attention-Guided Fusion (EAGF), which leverages attention weights from the generative backbone to fuse structural information from multiple reference images and inject it into the VAE decoder. Additionally, a learned quality-based token selection strategy identifies informative reference regions to reduce computational overhead. The authors also develop a synthetic training-data pipeline to support different exposure conditions and anchor configurations. Experiments on five MEF benchmarks demonstrate improvements over existing methods in perceptual quality and structural fidelity metrics. The method also achieves substantially lower computational costs than UltraFusion, requiring approximately 20% of its TFLOPs and 6% of its inference time.

Critical strengths
  1. A practically meaningful unified framework for different MEF protocols.

The paper addresses an important practical limitation of existing multi-exposure fusion methods, which are often designed for specific input configurations and anchor-selection strategies. By reformulating MEF as an observation-guided enhancement task and leveraging the multi-reference capabilities of a pretrained generative model, AnyMEF provides a unified framework that supports varying numbers of input images, flexible anchor selection, and partial reference observations. The experiments across 2-OE, 2-UE, and N-OE protocols provide encouraging evidence of its flexibility, particularly the stable perceptual quality when switching between overexposed and underexposed anchors. This is a meaningful contribution because it improves the applicability of generative MEF methods to more diverse real-world acquisition scenarios.

  1. Strong empirical performance with substantially improved efficiency over existing generative MEF methods.

AnyMEF demonstrates substantial improvements over existing methods across five MEF benchmarks, particularly in perceptual quality metrics such as MUSIQ and HyperIQA, while also achieving competitive or improved structural fidelity measured by TMQI-S. The qualitative results further illustrate its ability to recover visually coherent details from differently exposed observations. Importantly, compared with UltraFusion, another generative MEF method, AnyMEF requires only approximately 20% of the TFLOPs and 6% of the inference time. These results demonstrate the practical effectiveness of the proposed framework and suggest that pretrained generative priors can be utilized for MEF with considerably lower computational costs than previous generative approaches.

Critical weaknesses
  1. Insufficient validation of arbitrary anchor selection and structural fidelity.

Although AnyMEF demonstrates strong performance across different MEF protocols, the experiments do not sufficiently establish whether the generated results faithfully preserve the spatial and temporal content of the selected anchor. In particular, Table 3 evaluates anchor-switching robustness primarily using MUSIQ and HyperIQA, which measure perceptual quality but do not directly assess anchor consistency. For example, in a dynamic scene where an object moves between two frames, a model could achieve similar perceptual scores regardless of the selected anchor while generating the object at an incorrect position. Furthermore, Table 2 shows that increasing the number of reference images from 2-OE to N-OE slightly decreases TMQI-S on both DeepHDRVideo (0.747 to 0.745) and RealHDRV (0.842 to 0.835). These observations suggest that the claimed flexibility has not been fully validated in terms of structural fidelity. Since arbitrary anchor selection is a central contribution of this work, a quantitative evaluation of anchor consistency on dynamic scenes would be important for supporting the main claim.

  1. Insufficient isolation of the contributions of the proposed quality-aware mechanisms.

The paper introduces extracted-attention-guided fusion (EAGF) and learned quality-based token selection as important components of AnyMEF. However, the current ablations do not fully establish the independent effectiveness of these mechanisms. In Appendix Table 7, varying the number of selected reference tokens kk simultaneously changes the amount of information provided to the backbone and the computational budget. Without comparing the proposed quality-based selection against simpler alternatives, such as random or exposure-based selection under the same token budget (k=1k=1), it remains unclear whether the improvements arise from learned quality estimation or primarily from reducing the number of reference tokens. Similarly, the improvements from EAGF over uniform structure fusion are relatively modest in Table 5, and the paper provides limited direct evidence that the extracted attention weights reliably reflect reference quality. These missing controls make it difficult to determine how much the proposed mechanisms contribute to the overall performance. More carefully controlled ablations would strengthen the methodological contribution of the paper.

Questions for authors

Q1. Clarification on the extracted-attention-guided fusion mechanism.

In Eq. (5), the authors define attention weights ap,ia_{p,i} for each input image and use them to fuse the corresponding structure maps. However, Section 3.4 states that only the top-kk reference tokens are retained at each spatial position, with k=1k=1 by default. If the other reference tokens are discarded before entering the Transformer, how are the attention weights for these discarded references obtained? Does EAGF perform structure fusion over all input images or only over the anchor and the selected references? Additionally, could the authors clarify how the original structure maps SiS_i are constructed? These details are important for understanding and reproducing the proposed attention-guided fusion mechanism.

Q2. Clarification on the claimed support for arbitrary input subareas.

One of the main contributions of AnyMEF is its ability to handle arbitrary input subareas. However, the inference pipeline first applies RAFT alignment and VAE encoding to the input images before selecting reference tokens for the Transformer. Could the authors clarify whether "any subarea" refers to selecting arbitrary subsets of tokens from already processed full-frame images, or whether the framework can genuinely accept spatially incomplete reference observations, such as cropped images or irregularly masked regions, without access to the original full frames?

If the latter is supported, how are spatial alignment and quality estimation performed for these partial observations? Has the method been evaluated under such conditions? This distinction would help clarify the extent to which the proposed framework supports arbitrary observation protocols.

5 comments
AuthorsOpenReview ↗

Reply to Reviewer PDv3 (Part 1)

We sincerely thank Reviewer PDv3 for recognizing that our work is "practically meaningful" and shows "strong empirical performance". We address questions and concerns below. We have revised the manuscript (revised manuscript) and added many new figures and tables to the appendix (revised appendix); we recommend viewing both alongside this response. Revised text is marked in blue font.

W1. Insufficient validation of arbitrary anchor selection and structural fidelity.

We recommend reading this response together with Sec. A.6 (revised appendix), including Table A.3, Table A.4, Fig. A.3, Fig. A.4 and Fig. A.5.

Thank you very much for pointing out this gap in our experimental design. We have conducted the corresponding experiments.

Among our five benchmarks (MEFB, UltraFusionBench, RealHDRV, DeepHDRVideo, DeghostingIQA), only RealHDRV provides a ground truth frame for every input frame. Hence we conduct the experiments on RealHDRV.

For each RealHDRV sequence, we provide all 8 images I1,UE,I2,OE,I3,UE,I4,OE,I5,UE,I6,OE,I7,UE,I8,OE{I_{1,UE}, I_{2,OE}, I_{3,UE}, I_{4,OE}, I_{5,UE}, I_{6,OE}, I_{7,UE}, I_{8,OE}} as input, and rotate the anchor over all 8 images -- that is, we use I1,HDR,I2,HDR,...,I8,HDRI_{1,HDR}, I_{2,HDR}, ..., I_{8,HDR} as the ground truth, respectively. We denote the protocol that uses the i-th image as the anchor by N-i; N-4 is identical to the N-OE protocol in the paper.

The results are listed below (also in Table A.3). Although the metrics fluctuate slightly, AnyMEF consistently maintains a high TMQI_S as the anchor rotates. Values in parentheses are differences from N-4.

Protocol (anchor)RealHDRV TMQI_SRealHDRV MUSIQRealHDRV HyperIQA
N-1 (I1,UEI_{1,UE})0.806 (-0.030)75.042 (-0.043)0.694 (-0.011)
N-2 (I2,OEI_{2,OE})0.834 (-0.002)75.057 (-0.028)0.703 (-0.001)
N-3 (I3,UEI_{3,UE})0.812 (-0.024)74.976 (-0.109)0.695 (-0.009)
N-4 (I4,OEI_{4,OE})0.83575.0850.705
N-5 (I5,UEI_{5,UE})0.813 (-0.022)75.041 (-0.044)0.696 (-0.008)
N-6 (I6,OEI_{6,OE})0.834 (-0.002)75.132 (+0.047)0.704 (-0.001)
N-7 (I7,UEI_{7,UE})0.811 (-0.024)74.988 (-0.097)0.694 (-0.011)
N-8 (I8,OEI_{8,OE})0.833 (-0.002)75.239 (+0.155)0.704 (-0.001)

To verify that TMQI_S reflects whether the output is aligned to a specific anchor frame, rather than rewarding a temporally "averaged" result, we plot cross-frame heatmaps (Fig. A.3). In the left heatmap, entry (i, j) is the TMQI_S computed with the tone-mapped ground truth of frame i as the prediction and the HDR ground truth of frame j as the reference. Off-diagonal entries are clearly lower, showing that TMQI_S does not assign high scores to content from a different time instant. In the right heatmap, entry (i, j) is the TMQI_S computed with the AnyMEF output under protocol N-i as the prediction and the HDR ground truth of frame j as the reference. The outputs of AnyMEF are similar to the ground truth at their corresponding anchor time instant, and differ from the ground truth at other time instants.

Notably, in the protocols with OE anchors (N-2, N-4, N-6 and N-8), AnyMEF achieves a slightly higher TMQI_S than the tone-mapped ground truth. This is because TMQI_S measures how well the details in the linear HDR ground truth are presented, while the tone-mapped ground truth is produced with a globally uniform, monotonic tone-mapping curve, which limits its ability to enhance local details.

A qualitative example on RealHDRV is shown in Fig. A.4. A qualitative example with larger misalignment is shown in Fig. A.5: in its three input images, the woman's arm is at different positions, and the outputs of AnyMEF with each of the three images as the anchor correspond to their respective anchor frames. Since this example comes from DeghostingIQA, which provides no ground truth, its structural fidelity cannot be evaluated quantitatively.

AuthorsOpenReview ↗

Reply to Reviewer PDv3 (Part 2)

Regarding the slight TMQI_S drop from 2-OE to N-OE on DeepHDRVideo and RealHDRV, we conjecture that it is because the selected reference tokens become more scattered across frames, which slightly affects performance, while the occluded regions that can benefit from the additional information occupy only a small fraction of the image (~3%). We therefore separately evaluate the regions that are occluded under the 2-OE protocol, which are expected to be improved by the additional inputs of N-OE. As shown below (Table A.4), N-OE brings considerable improvements in these regions.

ProtocolDeepHDRVideo masked TMQI_SDeepHDRVideo masked MUSIQDeepHDRVideo masked HyperIQARealHDRV masked TMQI_SRealHDRV masked MUSIQRealHDRV masked HyperIQA
2-OE0.83856.0360.6940.83558.0380.698
N-OE0.85857.1210.7030.85859.0550.707

W2.1. Ablation to quality selection.

Thank you for suggesting this control. With the same token budget (we keep the anchor complete, and exactly one reference token is retained per position), we compare three token selection policies:

  • Quality: we use the learned quality predictor as described in our paper.
  • Random: a valid candidate (fully occluded tokens are excluded) is drawn uniformly at each position.
  • Illumination: the valid candidate whose warped 16×16 RGB patch has mean closest to 0.5, i.e., the most moderately exposed one, is selected.

We evaluate on RealHDRV, both on the original inputs and on a degraded version described below (also Table A.8). The last column is the average quality predicted by our learned predictor for the tokens each selector actually picks.

InputsSelectorTMQI_SMUSIQHyperIQAPredicted quality of selected tokens
OriginalQuality0.835275.080.70470.9827
OriginalRandom0.833974.800.70200.8857
OriginalIllumination0.833375.210.70690.9702
DegradedQuality0.762258.200.52190.9425
DegradedRandom0.758454.560.49860.7796
DegradedIllumination0.758556.710.51000.8389

Original inputs. Quality is higher than Random on all three metrics. Inspired by your suggestion of an exposure-based selector, we were surprised to find that the much simpler Illumination criterion also works well here: it is slightly higher on the no-reference metrics and slightly lower on TMQI_S. The last column explains why: scored by our learned predictor, the tokens chosen by Illumination are almost as good as those chosen by Quality (0.9702 vs. 0.9827). On these test sets the reference frames differ mainly in exposure, with little blur or noise, so brightness alone already identifies the most useful token in most positions.

Degraded inputs. To verify this explanation, we degrade each of the eight input frames of the same 50 scenes (including the anchor) with one of motion blur (15-pixel linear trajectory, random orientation), Gaussian blur (σ=2) or Gaussian noise (σ=0.05 on [0,1]), chosen at random per frame and applied before alignment; the ground truth is unchanged. Now that the references differ in degradation and not only in exposure, Quality outperforms both alternatives on all three metrics, with clear margins on MUSIQ (+1.49 over Illumination, +3.64 over Random) and HyperIQA. Correspondingly, the tokens picked by Illumination are rated much lower by the predictor (0.8389 vs. 0.9425), since brightness cannot reveal blur or noise. Hence the small gap on the original benchmarks reflects that their reference tokens differ little in usefulness beyond exposure, not that learned quality estimation is unnecessary.

The above results are included in Section A.10 of the revised appendix.

We also note that this comparison is only possible because the selector is decoupled from the backbone: since the backbone is finetuned with random token dropout, any selection policy can be plugged in without retraining. A simple brightness criterion is thus a zero-cost alternative when references differ only in exposure, while the learned predictor remains our default for general inputs. We have stated this in Sec. 3.4 of the manuscript.

AuthorsOpenReview ↗

Reply to Reviewer PDv3 (Part 3)

W2.2. Evidence that the extracted attention reflects reference quality.

We recommend reading this response together with Sec. A.7 (revised appendix), which includes Table A.5 and Fig. A.6.

To establish a causal relationship between reference quality and the extracted attention, we conduct a controlled manipulation experiment. We use the N-OE protocol on RealHDRV and disable quality-based token selection, so that the tokens of all eight input images enter the backbone. From each sequence, we randomly sample one non-anchor frame and five 64×64 regions within it. After the frame is warped to the anchor, we apply one of the following manipulations to the selected regions:

(1) Baseline: no manipulation. (2) Gaussian blur (weak): σ=1, 5×5 kernel, simulating defocus blur. (3) Gaussian blur (strong): σ=2, 9×9 kernel. (4) Gaussian noise (weak): zero-mean i.i.d. Gaussian noise with σ=0.02 (pixel values in [0,1]). (5) Gaussian noise (strong): σ=0.05. (6) Overexposure: all pixels set to white. (7) Underexposure: all pixels set to black.

For each manipulation, we compute the normalized attention weight wp,i=ap,i/Σjap,jw_{p,i} = a_{p,i} / Σ_j a_{p,j} along exactly the same extraction path used by EAGF (Eq. 5; head-averaged attention from output position p to the co-located token of input image i in the last two backbone blocks, normalized over all eight input images including the anchor). We report the attention share, i.e., wp,iw_{p,i} of the manipulated frame averaged over all positions in the manipulated regions, which measures how much attention the manipulated tokens receive relative to the co-located tokens of all other inputs. Results are as follows (also Table A.5):

ManipulationAttention shareRelative reduction (%)
Baseline0.109590.00
Gaussian blur (weak)0.0750731.50
Gaussian blur (strong)0.0432360.55
Gaussian noise (weak)0.0443759.51
Gaussian noise (strong)0.0207681.06
Overexposure0.0052695.20
Underexposure0.0042596.12

Since only the quality of the selected regions is changed while everything else is kept fixed, these results show that the extracted attention responds to reference quality: every manipulation reduces the attention share, and the more severe the degradation, the less attention the degraded tokens receive. Qualitative examples of the manipulated inputs and the resulting attention maps are shown in Fig. A.6.

Q1. Clarification on the extracted-attention-guided fusion mechanism.

Structure fusion is performed only over the anchor and the selected reference subareas. Thank you for pointing out this ambiguity; we have clarified this in the revision.

We have added the definition of the structure maps SiS_i as Eq. 6 in Sec. 3.3 of the revised paper. The structure map is the locally normalized luminance of the an input image, defined in the same way as the structural term of SSIM: absolute brightness is removed by subtracting the local mean, and local contrast scale is removed by dividing by the local standard deviation, leaving only the edges and texture.

Q2. Clarification on the claimed support for arbitrary input subareas.

Thank you for this precise question. In the implementation reported in our paper, it is the former: RAFT alignment and VAE encoding are both applied to full-frame images, and "any subarea" refers to routing the encoded tokens and structure maps of the selected subareas to the transformer backbone and the refiner. These two components account for about 63% of the total TFLOPs under the N-OE protocol when all reference tokens are routed.

That said, with minor modifications, AnyMEF can also accept spatially incomplete reference observations (e.g., cropped images or irregularly masked regions) without access to the full frames. We can place each incomplete observation on a full-size canvas with the missing areas filled in black, and process it as follows:

(1) RAFT alignment: Optical flow is computed on the black-filled images as usual, although it may be less accurate near the missing areas; unreliably aligned pixels are masked by the forward-backward consistency check (Sec. A.4).

(2) VAE encoding: The black-filled images are encoded as usual; the latent tokens in the missing areas are not used afterwards.

(3) Token selection: Using the known missing mask, we discard the reference tokens in the missing areas, so they are never routed to the backbone.

(4) Backbone and refiner: No modification is needed, since the discarded tokens and their structure maps never enter them.

Reviewer_GjJLOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

This paper formulates multi-exposure fusion (MEF) as an observation-guided enhancement task and proposes AnyMEF, built on a FLUX.2 flow-matching backbone fine-tuned with LoRA. Three components address the "unified protocol" goal: (1) an extracted-attention-guided fusion (EAGF) refiner that fuses structure maps from an arbitrary number of inputs using attention maps extracted from late backbone blocks; (2) a synthetic data pipeline with a "synthetic illuminated sign" mechanism to model non-monotonic tone mapping; and (3) a quality-predictor-based top-k token selection scheme for efficiency. The model supports any number of inputs, any anchor frame, and subarea routing.

Critical strengths
  1. This paper presents a well-motivated problem reformulation. Moving from "guided inpainting of an OE anchor with a UE reference" to a protocol-agnostic observation-guided enhancement task is a conceptual contribution, and the formalism (degradation model, anchor choice, non-monotonic tone mapping) is clean.
  2. Support for any input count, any anchor, and token-level subarea routing directly addresses real deployment constraints (e.g., best-focused frame as anchor). The anchor-swap robustness results shown in Table 3 are a convincing demonstration.
Critical weaknesses
  1. While AnyMEF is vastly more efficient than UltraFusion, its absolute runtime (around 1.8 to 5.2 seconds per sample depending on resolution and parallelization) still precludes real-time mobile or edge device deployment. The authors transparently discuss this in Appendix A.1, but further discussion on potential distillation or acceleration paths would be welcome.
  2. The framework relies on RAFT to pre-align reference frames to the anchor. In scenes with extreme exposure differences, heavy occlusions, or large non-rigid motion, optical flow failures can introduce artifacts into the warped latents and structure maps. It would be insightful to discuss how sensitive AnyMEF is to alignment inaccuracies.
  3. The introduction and abstract mention "adaptively routing image subareas to the transformer backbone," but the core methodology focuses heavily on VAE latent token selection. A clearer conceptual distinction between subarea routing and token selection would enhance precision.
  4. As shown In Table 1, the advantage of the proposed method over FreeMEF is that it doesn't need to select middle-exposed image. FreeMEF also support input frames of arbitrary number. What's the advantage of "any frame" anchor selection?
Questions for authors

Please refer to weakness.

1 comment
AuthorsOpenReview ↗

Reply to Reviewer GjJL

We sincerely thank Reviewer GjJL for recognizing our problem reformulation as "well-motivated" and the robustness results as "a convincing demonstration". We address questions and concerns below. We have revised the manuscript (revised manuscript) and added many new figures and tables to the appendix (revised appendix); we recommend viewing both alongside this response. Revised text is marked in blue font.

W1. Potential distillation or acceleration paths.

Thank you for this suggestion. Several acceleration paths are worth exploring:

  1. Backbone compression. Structural pruning and knowledge distillation, as explored in BK-SDM [1] and SANA 1.5 [2], could reduce the cost of each forward pass.

  2. Step distillation. Methods such as DMD [3] accelerate generation by distilling multiple sampling steps into one. Since inference of AnyMEF is already single-step, reducing the sampling-step count alone would not further accelerate it.

  3. Quantization. Low-bit inference methods such as ViDiT-Q [4] and SVDQuant [5] offer promising efficiency improvements.

We have added the above discussion to the end of Section A.5 (Computational cost breakdown).

[1] BK-SDM: A Lightweight, Fast, and Cheap Version of Stable Diffusion. ECCV 2024.

[2] SANA 1.5: Efficient Scaling of Training-Time and Inference-Time Compute in Linear Diffusion Transformer. ICML 2025.

[3] One-step Diffusion with Distribution Matching Distillation. CVPR 2024.

[4] ViDiT-Q: Efficient and Accurate Quantization of Diffusion Transformers for Image and Video Generation. ICLR 2025.

[5] SVDQuant: Absorbing Outliers by Low-Rank Components for 4-Bit Diffusion Models. ICLR 2025.

W2. RAFT alignment inaccuracies.

We deal with RAFT alignment inaccuracies with the strategies listed below, in which 1, 2 and 4 follow UltraFusion. Specifically, when warping reference image B to the anchor image A:

  1. To cope with exposure differences, we match the histogram of B to that of A before feeding them to RAFT.

  2. We compute both the optical flow from A to B, and the flow from B to A. Then we perform a forward-backward consistency check: it measures how far each pixel deviates from its original position after being mapped forward and then backward by the estimated optical flows. We mask out the inconsistent areas (corresponding to occlusions or optical flow inaccuracies) with black.

  3. Masked tokens are excluded from token selection: we assign them the lowest quality, so they are never routed to the backbone. Masked pixels also receive zero weight in EAGF structure fusion. Since AnyMEF can take multiple warped images as reference, for each spatial area, the probability that at least in one image it is non-occluded and nicely warped is high.

  4. In some cases, partially masked tokens may still enter the backbone (e.g., when no better token is available). To make our models robust to them, we add random black occlusion masks to our model inputs when training. The masks are sourced from real occlusion masks of the Vimeo dataset.

We have added these details and examples of train/test warp masks in Appendix A.4 (Alignment details).

W3. Adaptive routing vs. token selection.

Token selection is indeed the main form of adaptive routing in AnyMEF. However, "adaptive routing" also includes routing selected subareas of the structure maps -- calculated from the original image subareas, instead of encoded tokens -- to the decoder. Hence we use this more general term.

W4. Advantage of "any frame" anchor selection.

Imagine this scenario: you have a phone that automatically captures a burst of 3 multi-exposure images when you press "capture", with exposure sequence (1) mid (2) low (3) high. You are photographing a portrait under fireworks. In image 1, the person’s eyes are closed; in image 2, no fireworks are in the sky; and in image 3, the fireworks are bursting while the person’s eyes are perfectly open. Naturally, you would want the fused image to align with image 3.

FreeMEF can take any number of images as input and merge them. However, it assumes that the anchor frame must be an image with middle exposure -- that is, if image 2 or 3 is set to be the anchor, its performance will decline (see the 2-UE rows of Table 3). AnyMEF, however, allows "any frame" anchor selection: any of images 1, 2 and 3 can serve as the anchor without systematic performance loss.


We hope that our responses have addressed your concerns. If anything remains unclear, or if there are further experiments or analyses that you believe would strengthen the paper, please do not hesitate to let us know. We would be glad to provide them during the discussion period. Thank you again for your time and constructive feedback.

Comments on the paper

1 comment

Dear Reviewers,

Thank you again for your valuable feedback. We have posted our responses and hope they address your concerns. Since the discussion period is limited, we would greatly appreciate the opportunity to continue the discussion. Please let us know if anything still needs clarification.

Best regards, The Authors

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.