acceptodds
Under review as a conference paper at ICLR 2027

FMGRPA: Flow Matching with Group-Relative Preference Alignment for Pansharpening

Abstract

Pansharpening aims to fuse a low resolution multispectral (LRMS) image and a high resolution panchromatic (PAN) image to recover a high resolution multispectral (HRMS) image with spectral fidelity and spatial detail. Existing methods usually reconstruct paired HRMS images, but a single objective constrains only overall reference error, making it difficult to distinguish spectral shifts from insufficient spatial detail or express the desired spectral and spatial fusion preference among results with comparable reconstruction errors. To this end, we propose FMGRPA, a two stage pansharpening framework combining LRMS anchored conditional flow matching with group relative preference alignment. Stage I takes upsampled LRMS as the flow source and learns a PAN guided conditional vector field toward the HRMS reference, yielding a base prediction initialized from observed multispectral information. Stage II constructs a task specific candidate group around this prediction for spectral enhancement, spatial enhancement, and balanced fusion, then jointly evaluates the candidates by spectral relation consistency, spatial relation consistency, and reference fidelity. Softmax normalization within each group converts candidate rewards into relative supervision weights that guide refinement toward a better spectral and spatial balance. Furthermore, spatial preference aggregation combines global and local evidence, while temporal preference aggregation integrates relative evaluations across flow time conditions to improve preference stability. Extensive experiments demonstrate competitive performance and validate explicit group relative fusion preferences for generative pansharpening.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
Pansharpening, Remote Sensing, Flow Matching

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_sBnXOpenReview ↗
Rating
3
Confidence
3
Informativeness
1
Presentation
3
Soundness
3
Read review
Paper summary

The authors propose FMGRPA, a two-stage pansharpening framework combining conditional flow matching (CFM) with group relative preference alignment (GRPA). CFM learns a PAN-guided conditional flow from LRMS to HRMS, while GRPA refines the base prediction through structured candidate construction, hybrid rewards, and spatial and temporal preference aggregation. Experiments on WV2, WV3, and GF2 demonstrate competitive performance, with ablation studies validating the effectiveness of the proposed components.

Critical strengths

1.The proposed framework explicitly addresses the spectral and spatial fusion preferences in pansharpening, rather than relying solely on conventional reconstruction objectives.

2.The proposed GRPA is simple and intuitive, constructing spectral, spatial, and balanced candidates by adjusting LRMS low-frequency content and PAN high-frequency details without requiring separate prediction branches.

Critical weaknesses

The necessity of the CFM formulation is not sufficiently justified. Moreover, the paper lacks a direct end-to-end reconstruction baseline without CFM and time conditioning, making it difficult to assess the actual contribution of the CFM formulation.

Questions for authors

1.Could the authors clarify the motivation for predicting the HRMS image and deriving the velocity from it, rather than directly predicting the conditional vector field as in conventional flow matching?

2.What motivates the use of the L1 velocity matching term combined with the SSIM loss in Eq. (3)? Could the authors provide further justification for this objective?

3.Could the authors provide an ablation study using a direct end-to-end reconstruction model without CFM and time conditioning to demonstrate the necessity and effectiveness of the proposed CFM formulation?

4 comments
AuthorsOpenReview ↗

Response to Reviewer sBnX

Thank you for your valuable comments. Your feedback helped us identify where the paper needs further clarification and improvement. We will revise the paper accordingly and respond to each point below. Please let us know whether we have addressed your questions.

Q1. Why predict the HRMS image before computing the velocity?

GRPA needs to compare the spectral and spatial quality of candidates under the same observations. Conventional CFM integration starts from the same observation and traces a single reconstruction trajectory. Scoring intermediate states directly along that trajectory would conflate candidate quality with reconstruction progress. Continuing the integration from those states to the endpoint along the same trajectory would yield the same fused image. To construct and compare candidates, we therefore hold the upsampled LRMS LL fixed and query the model at different time conditions tt, obtaining one complete HRMS image per query.

Given LL, image and velocity predictions can be converted into one another using Eq. (2). The predicted velocity and the LRMS starting point allow one-step image reconstruction, as in OTFM (Cao et al., ICCV 2025, §3.3). Our implementation uses an image parameterization to directly obtain a complete HRMS image X^\hat X at each time condition and then computes the velocity as sθ=X^−Ls_\theta=\hat X-L. JiT (Li and He, CVPR 2026, §4.3) likewise derives the velocity from the predicted image for supervision.

Q2. Why combine the L1 velocity matching loss with the SSIM loss?

The velocity term in Eq. (3) computes the mean absolute difference between the predicted velocity sθs_\theta and the target velocity utu_t, constraining the reconstruction correction at each pixel in each band. When errors are summed pixel by pixel, errors distributed continuously along object boundaries and errors scattered across flat regions may yield similar L1 losses. The former are more likely to blur object contours. With SSIM, the training objective also compares the local luminance, contrast, and structure of the predicted and reference HRMS images, encouraging consistency in edges and textures.

To isolate the contribution of the SSIM loss, we compare two settings for first-stage CFM on GF2. Both use the same NAT backbone, interpolated states, time conditioning, training budget, and evaluation protocol. The ablation sets the weight of the SSIM loss in Eq. (3) to zero and retains the L1 velocity matching loss.

Training lossPSNR ↑SSIM ↑SAM ↓ERGAS ↓
L1 only48.74210.98630.00990.4920
L1 + SSIM48.84460.98780.00980.4862

Adding SSIM improves structural consistency, and the spectral angle remains essentially unchanged.

Q3 and the main concern. What does CFM contribute to first-stage reconstruction?

To assess the contribution of CFM, we designed a comparison on GF2 between first-stage CFM and end-to-end direct reconstruction (Direct). Direct starts from the upsampled LRMS LL, reconstructs HRMS directly under PAN guidance, and uses no time conditioning. CFM learns to reconstruct from interpolated states XtX_t with time conditioning. Both settings use the same NAT backbone, L1 and SSIM losses, training budget, and evaluation protocol.

MethodTraining stateTime conditioningPSNR ↑SSIM ↑SAM ↓ERGAS ↓
DirectLLNo48.62780.98720.01040.5002
CFMXtX_tYes48.84460.98780.00980.4862

In the comparison with the same backbone and training budget, CFM, which is trained on path states and uses time conditioning, outperforms Direct on all four reconstruction metrics.

Thanks for your rebuttal. I would like to further clarify my questions. First, why is L1 used instead of the squared L2 commonly adopted in CFM, which has a theoretical basis for learning the marginal velocity field? Second, since the model directly predicts HRMS and uses only one step for inference, could a direct end-to-end reconstruction model combined with GRPA, potentially using simpler techniques such as dropout or noise perturbation, achieve similar results?

AuthorsOpenReview ↗

Response to Reviewer sBnX (Round 2, Part 1 of 2)

Thank you for carefully reviewing our response and further clarifying these two questions. Below, we provide specific explanations for both questions.

Q1. Why use L1 rather than the squared L2 loss commonly used in standard CFM?

The goal of pansharpening is to reconstruct a high-resolution multispectral (HRMS) image that preserves the spectral information of LRMS while incorporating the spatial details of PAN. We let the network directly predict this image so that each query yields a complete HRMS image that can be used for supervision and candidate construction. During training, SSIM directly compares the local structure of this image with that of the reference HRMS image, constraining the recovery of edges and textures. GRPA uses this image as the basis for constructing spectral, spatial, and balanced candidates from the low-frequency residuals of LRMS and the high-frequency details of PAN, and compares the fusion quality after each correction. Thus, directly outputting HRMS allows the same prediction to serve image-structure supervision, candidate construction, and final reconstruction. The predicted velocity is obtained by subtracting the upsampled LRMS starting point from the predicted HRMS, while the target velocity is obtained by subtracting the same starting point from the reference HRMS. Therefore, the L1 error between the two velocities is exactly equal to the pixelwise absolute error between the predicted and reference HRMS images, allowing velocity supervision to directly constrain the reconstruction error at each pixel in each band.

Our specific motivation for choosing L1 is to control how errors of different magnitudes affect optimization. For squared L2, the gradient magnitude for an individual error term increases with the error magnitude, whereas for L1, it does not. This limits the dominance of a small number of pixels with large errors in the updates. Meanwhile, pixelwise errors cannot fully distinguish the spatial distribution of errors. Similar pixelwise errors may correspond to different degrees of edge blurring and texture loss. We therefore incorporate SSIM to compare the local luminance, contrast, and structure of the reconstructed and reference images, providing complementary supervision for edges and textures.

Standard CFM regresses path velocities using squared L2, whose conditional-mean property provides a theoretical basis for learning the marginal velocity field. We retain conditional probability path sampling and the corresponding velocity targets, and use endpoint parameterization to formulate the training objective as L1 and SSIM tailored to HRMS reconstruction. This choice is motivated by the optimization properties of reconstruction errors, while the conditional-mean result for squared L2 relies on its specific loss assumption.

AuthorsOpenReview ↗

Response to Reviewer sBnX (Round 2, Part 2 of 2)

Q2. Can direct reconstruction combined with GRPA, supplemented by dropout or noise perturbation, achieve similar results?

Thank you for proposing this specific alternative. We consider your proposed combination of direct reconstruction and GRPA to be feasible, and dropout or random noise can also introduce more variation during training. On this basis, our method further uses paired LRMS and HRMS images from the same scene to construct a conditional probability path, so that training inputs at different times contain different amounts of target information. In this way, input variations have an explicit reconstruction direction and degree of interpolation, enabling the model to jointly learn to recover the same HRMS image from different intermediate states. Near the starting point, the model needs to recover more details from states close to LRMS. Near the endpoint, the model needs to complete reconstruction from states containing more HRMS information. Time conditioning informs the model of the current degree of interpolation, and the same network jointly learns these related reconstruction tasks, thereby obtaining more diverse training inputs and reconstruction supervision with the same number of training image pairs.

Dropout introduces variation by randomly masking features, while ordinary noise perturbation introduces variation by randomly changing the input. Neither operation by itself uses the paired HRMS image of the current scene to organize training states along a path from LRMS to HRMS, nor does it provide information about the corresponding degree of interpolation. Our path-based training therefore gives time conditioning an explicit meaning. In the second stage, we fix the same LRMS and PAN observations and separately read out complete HRMS predictions under different time conditions near the start of the path. GRPA constructs and evaluates spectral, spatial, and balanced corrections under each time condition, then aggregates the weights for the same type of correction, reducing the dependence of candidate weights on scoring under a single time condition.

In our response to your previous Question 3, we provided a comparison between CFM and end-to-end direct reconstruction on GF2. Both use the same NAT backbone, L1 and SSIM losses, training budget, and evaluation protocol. Direct reconstructs HRMS from upsampled LRMS with time conditioning disabled. CFM learns reconstruction from time-conditioned interpolated states. This comparison evaluates the reconstruction performance of the two training approaches before adding GRPA.

MethodTraining inputTime conditioningPSNR ↑SSIM ↑SAM ↓ERGAS ↓
DirectUpsampled LRMSNo48.62780.98720.01040.5002
CFMInterpolated state XtX_tYes48.84460.98780.00980.4862

CFM achieves gains over Direct on all four reconstruction metrics.

Reviewer_1AawOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This manuscript introduced FMGRPA, a two-stage pansharpening framework combining conditional flow matching with group-relative preference alignment. At Stage 1, they reconstruct HRMS images from upsampled LRMS under PAN guidance, while Stage 2 constructs spectral, spatial, and balanced alternatives around the base prediction and uses reward-based weights for further training. The proposed method aggregates candidate weights across spatial regions and time conditions and reports competitive reconstruction results on three datasets.

Critical strengths

I'm just happy to review pan-sharpening paper at ML top-tier conference. I just found some strength of this paper as follows:

  • The motivation is relevant to pansharpening. Explicitly considering the balance between spectral fidelity and spatial detail is a reasonable direction. Constructing spectral, spatial, and balanced candidates around a common base prediction also gives the post-training stage a clear task-specific motivation.

  • The proposed method shows competitive results against the reported baselines. In Table 1, FMGRPA achieves the best or comparable result in nine of the twelve metric and dataset combinations. The full-resolution GF2 results are also great, with QNR improving from the previous best of 0.823 to 0.844.

  • The paper includes ablations of its main components. Table 3 examines GRPA and the spatial and temporal aggregation modules, while Table 4 evaluates the effect of removing individual candidates. These results provide useful evidence for the effectiveness of the complete configuration.

Critical weaknesses

There are few concern before publishing at ICLR. I just summarize few concerns as follows:

  • The connection between flow training and the prediction procedure is unclear. Equation (1) shows the training pipeline that trains the model on interpolated states (X_t), whereas Eq. (4) and the temporal aggregation query the same LRMS input (L) at selected time conditions. For nonzero times, (L) generally differs from the corresponding state on the stated training path, and the connection between these direct predictions and the learned flow is not sufficiently explained. I think that the authors should specify the time settings and final inference procedure, and compare against a direct reconstruction model using the same backbone and training budget to establish the benefit of the flow formulation.

  • The preference-directed behavior of the GRPA update needs further validation. Eq. 13 assigns positive weights to all non-base candidates, including those with lower rewards than the current prediction, while the base candidate contributes zero loss. Consequently, the GRPA term can pull the prediction toward lower-reward candidates even when the base receives the highest reward, although the ground-truth reconstruction loss may counteract this effect. Comparisons with uniform weighting and a rule that uses only candidates with higher rewards than the base would help establish the benefit of the proposed group-relative weighting.

  • The evaluation (comparison) is incomplete and does not sufficiently establish the contribution relative to recent methods. Table 1 reports reduced-resolution results on all three datasets, but full-resolution evaluation in Table 2 is limited to GF2, full-resolution comparisons on WV2 and WV3, complementary metrics such as SCC, and evaluation on an unseen dataset without fine-tuning are needed. The most recent baselines date to 2024, leaving out OTFM (ICCV2025) and PAN-Crafter (ICCV 2025), recent CVPR 2026 approaches. I think that the authors should provide these comparisons under matched data and evaluation protocols, and include ground-truth-only fine-tuning with the same Stage 2 inputs and training budget to separate the contribution of GRPA from additional supervised training.

Questions for authors

Why didnt the author compare the other recent method which were published at top-tier venue, such as CVPR2026, ECCV2026, ICCV2025??

And please check the weakness section.

2 comments
AuthorsOpenReview ↗

Response to Reviewer 1Aaw (Part 1 of 2)

Thank you for your valuable comments. Your feedback helped us identify where the paper needs further clarification and improvement. We will revise the paper accordingly and respond to each point below. Please let us know whether we have addressed your questions.

Q1. The connection between flow matching training and inference, and comparison with direct reconstruction

Equation (4) generates the base prediction directly from the upsampled LRMS LL, allowing the second stage to correct spectral deviations and recover missing spatial details that arise at inference. If we used XtX_t to generate candidates, the model could recover details using the ground-truth HRMS information present in the interpolated state. This would mask errors that arise when reconstructing from LRMS and PAN alone, which would in turn affect the candidate ranking. Using LL for candidate construction and fine-tuning focuses preference supervision on reconstruction errors that occur when using the actual observations.

The alignment time tet_e in Eq. (4) is sampled uniformly from [0.02,0.05][0.02,0.05]. We choose this interval near the start of the path because the mean of the path in Eq. (1) remains close to the observation LL at these times. In the second stage, we fine-tune the model with LL and tet_e directly as inputs so that it learns to reconstruct HRMS from the actual observations under these time conditions. Four-point aggregation uses the two endpoints and two equally spaced interior points, namely 0.020.02, 0.030.03, 0.040.04, and 0.050.05. Averaging the weights of each candidate type across the four time points for the same LL can reduce the dependence of the candidate weights on scores from any single time condition. We apply the aggregated weights in the loss for the candidates generated at tet_e.

At inference, we start from LL and perform a single Euler update at t=0t=0. The velocity parameterization sθ=fθ−Ls_\theta=f_\theta-L gives

X^=L+sθ(L,0,C)=fθ(L,0,C).\hat X=L+s_\theta(L,0,C)=f_\theta(L,0,C).

To assess the benefit of first-stage CFM, we compare CFM with direct reconstruction (Direct). Both methods use the same NAT backbone, a training budget of 40k steps, and the same evaluation protocol. Direct uses LL as its training state. CFM uses the interpolated state XtX_t with time conditioning.

MethodTraining stateTime conditioningPSNR ↑SSIM ↑SAM ↓ERGAS ↓
DirectLLNo48.62780.98720.01040.5002
CFMXtX_tYes48.84460.98780.00980.4862

With the same backbone and training budget, CFM achieves better overall reconstruction performance than Direct, as shown in the table, demonstrating the combined benefit of training on path states and using time conditioning.

Q2. The role of lower-reward candidates in group-relative weighting

Once the base prediction is available, the second stage uses candidate weighting to guide the updates.

Pansharpening must balance spectral fidelity and spatial detail, so a candidate with a lower overall reward than the base prediction can still provide a useful correction in one respect. During the extensive candidate validation experiments we conducted early on to select the construction coefficients, we observed that the spatial candidate had slightly lower PSNR than the base prediction and lower spatial distortion. Even when the base prediction receives the highest overall reward, GRPA still assigns smaller weights to the other candidates to retain these complementary corrections. Even if a low-scoring candidate's correction direction conflicts with the reference reconstruction objective, the GT reconstruction loss in our design still pushes the prediction toward the HRMS ground truth, offsetting some of the drift.

To test whether relative reward weighting makes better use of candidate corrections and whether low-scoring candidates should be retained, we set up two comparisons on GF2. Uniform assigns equal weights to all four candidates, including the base prediction, to test the effect of weighting candidate contributions by reward. Within each scoring group, Better-only sets the weights of non-base candidates whose rewards are at or below that of the base prediction to zero, retains the base prediction, and renormalizes the weights. This tests how discarding low-scoring corrections affects reconstruction. All settings start from the same first-stage CFM weights and use the same candidate construction, spatiotemporal aggregation, GT reconstruction loss, training budget, and evaluation protocol.

Part 2 reports the Q2 results and addresses Q3.

AuthorsOpenReview ↗

Response to Reviewer 1Aaw (Part 2 of 2)

Q2 (continued). Candidate-weighting results

The following table compares reduced-resolution reconstruction performance under different weighting rules.

MethodCandidate weightingPSNR ↑SSIM ↑SAM ↓ERGAS ↓
UniformEqual weights within each group48.75280.98750.01110.5062
Better-onlyRetains only correction candidates with higher rewards than the base prediction49.06170.98900.00920.4696
GRPASoft weighting based on relative rewards within each group49.19650.98960.00890.4594

GRPA's improvement over Uniform in the table demonstrates the benefit of weighting candidate contributions by reward. Its improvement over Better-only shows that, under the same GT constraint, retaining low-scoring candidates with soft weights yields better overall reconstruction performance.

Q3. Evaluation coverage and comparisons with recent methods

The comparisons in the preceding two responses focus on training design. We now provide additional evaluations and comparisons for the full method.

  1. Comparison with recent pansharpening methods. In the reduced-resolution evaluations on WV2, WV3, and GF2, we include OTFM (ICCV 2025), PAN-Crafter (ICCV 2025), SRINO (CVPR 2026), and ViPS (ECCV 2026), using the same data splits, band settings, downsampling procedures, and metric implementations. We report the spatial correlation coefficient (SCC) in addition to the four metrics in Table 1 of the main paper.
DatasetMethodPSNR ↑SSIM ↑SAM (rad) ↓ERGAS ↓SCC ↑
WV2OTFM42.29730.97240.02230.92920.9644
PAN-Crafter42.37020.97280.02200.89890.9741
SRINO42.45140.97190.02170.88770.9735
ViPS42.51040.97300.02170.89080.9798
FMGRPA42.67380.97390.02150.87540.9803
WV3OTFM31.21330.93060.07122.98110.9883
PAN-Crafter31.17010.92820.07082.97600.9868
SRINO31.15390.93470.07243.05030.9879
ViPS31.21660.92890.06942.95330.9875
FMGRPA31.23890.93930.07022.94250.9897
GF2OTFM49.11340.98900.00900.45960.9920
PAN-Crafter48.87930.98910.00980.46040.9936
SRINO49.12480.98920.00900.45730.9928
ViPS48.90600.98940.00920.46010.9939
FMGRPA49.19650.98960.00890.45940.9941

SCC measures agreement in spatial detail between the fused output and the HRMS reference image. FMGRPA achieves competitive reconstruction performance on WV2, WV3, and GF2.

  1. Full-resolution evaluation on WV2 and WV3. We report additional full-resolution results for WV2 and WV3 using LRMS/PAN image pairs at their native resolution. We follow the evaluation protocol in Table 2 of the main paper and retain HFIN and FAME-Net as representative baselines from that table.
DatasetMethodDλD_\lambda ↓DsD_s ↓QNR ↑
WV2HFIN0.06000.04930.8937
FAME-Net0.05180.05210.8989
FMGRPA0.04630.04800.9078
WV3HFIN0.03440.04540.9219
FAME-Net0.02960.05040.9213
FMGRPA0.02940.04620.9258

FMGRPA achieves the highest QNR and the lowest spectral distortion in the native-resolution evaluations on both WV2 and WV3.

  1. Evaluation on an unseen dataset. We use QuickBird (QB) from PanCollection as an external test dataset and evaluate on its standard reduced-resolution test split. We train and select models for all methods exclusively on GF2. At test time, we standardize the band order and radiometric normalization, freeze the model weights, and evaluate directly on QB.
MethodPSNR ↑SSIM ↑SAM (rad) ↓ERGAS ↓
HFIN29.420.68370.208012.75
FAME-Net29.670.69670.217012.42
FMGRPA30.060.71520.198011.88

FMGRPA leads on all four metrics without target-domain fine-tuning.

  1. GT-only comparison. On GF2, GT-only and GRPA start from the same CFM checkpoint trained for 40k steps. Each undergoes 20k steps of second-stage training with the same inputs, GT reconstruction loss, and evaluation protocol. GT-only removes only the candidate preference loss.
Second-stage methodPSNR ↑SSIM ↑SAM (rad) ↓ERGAS ↓
GT-only48.90120.98830.00990.4836
GRPA49.19650.98960.00890.4594

Adding GRPA yields better performance on all four metrics than continuing training with GT supervision alone.

Reviewer_nhkPOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

This paper proposes FMGRPA, a two-stage pansharpening framework combining conditional flow matching (CFM) with group-relative preference alignment (GRPA). In the first stage, CFM learns a PAN-guided reconstruction process from upsampled LRMS images to HRMS references. In the second stage, GRPA constructs spectral, spatial, and balanced candidates around the initial prediction and assigns relative preference weights based on spectral consistency, spatial consistency, and reference fidelity. Spatial and temporal preference aggregation further refine these weights. Experiments on WV2, WV3, and GF2 demonstrate competitive performance against existing pansharpening methods.

Critical strengths
  1. The paper explores an interesting direction by introducing group-relative preference alignment into pansharpening, explicitly considering the trade-off between spectral fidelity and spatial detail preservation.
  2. The proposed two-stage framework is reasonably structured, and the candidate construction strategy provides an interpretable way to represent different fusion preferences.
Critical weaknesses
  1. Insufficient justification for conditional flow matching. The necessity of CFM is not convincingly established. According to Eqs. 2-3, the velocity matching term can be algebraically reduced to a standard L1 reconstruction loss, since (s_\theta=f_\theta-L) and (u_t=X-L). Therefore, the practical advantages of flow matching over conventional supervised reconstruction remain unclear. A direct comparison with a non-flow-based reconstruction baseline is necessary.
  2. Insufficient validation of the preference alignment mechanism. The experiments do not adequately establish whether group-relative preference weighting is essential. Comparisons with uniform candidate weights, direct spectral–spatial loss optimization, and conventional second-stage fine-tuning are missing. Consequently, it is difficult to determine whether the improvements arise from relative preference alignment or simply from additional training and regularization.
  3. Questionable candidate construction and reward design. The candidate images are generated through fixed combinations of LRMS low-frequency corrections and PAN high-frequency details, which may not adequately represent the diversity of possible fusion outcomes. Moreover, the hybrid reward relies on manually selected coefficients, while sensitivity analyses of these coefficients and the Softmax temperature are not sufficiently reported. The robustness of the resulting preferences therefore remains uncertain.
  4. Insufficient analysis of optimization stability. The GRPA loss uses detached candidates constructed from the model's own predictions as optimization targets. However, the paper lacks a detailed analysis of how these self-generated targets influence optimization dynamics and convergence. In particular, it remains unclear whether the resulting updates consistently improve reference fidelity or introduce biases associated with the predefined candidate directions.
  5. The paper should provide more details regarding the selection of candidate construction coefficients and reward weights.
  6. The computational overhead introduced by candidate generation, reward computation, and spatial–temporal aggregation should be quantified.
Questions for authors

Please see the weaknesses above.

4 comments
AuthorsOpenReview ↗

Response to Reviewer nhkP (Part 1 of 4)

Thank you for your valuable comments. Your feedback helped us identify where the paper needs further clarification and improvement. We will revise the paper accordingly and respond to each point below. Please let us know whether we have addressed your questions.

Q1. How CFM differs from direct reconstruction

We thank the reviewer for raising this key issue. We agree that, in the first stage of our method, the L1 loss on velocity is equivalent to the L1 loss on HRMS reconstruction.

Along a linear interpolation path, the standard CFM velocity regression objective can also be cast as a reconstruction objective with an appropriate endpoint prediction parameterization. In our method, the key differences between CFM and direct reconstruction lie in the distribution of training states and in time conditioning. We define the conditional path as

\qquad t\sim\mathcal U(0,1),\quad \epsilon\sim\mathcal N(0,I),$$ with the corresponding conditional velocity given by $$u_t=\frac{\mathrm dX_t}{\mathrm dt}=X-L.$$ Given a path state $X_t$, the network is conditioned on time $t$ and the LRMS/PAN inputs and regresses this velocity via $s_\theta=f_\theta-L$. This follows the basic CFM training procedure of sampling from a conditional path and regressing the corresponding target velocity. This design extends reconstruction training from a single observed state to a continuum of interpolated states. Direct reconstruction predicts HRMS from LRMS/PAN inputs. In CFM, the same network reconstructs the same HRMS image from intermediate states that contain varying amounts of information from the target. Because the interpolation varies along the residual $X-L$ between the paired images, these training tasks involve actual differences in the spectral and spatial details of the scene. Time conditioning distinguishes different levels of interpolation, and parameter sharing enables the network to learn features for reconstruction jointly across these related tasks. This design therefore provides richer reconstruction supervision from the same paired sample. The following comparison evaluates its practical benefits. On GF2, we added a comparison between first-stage CFM and direct reconstruction (Direct). Both methods use the NAT backbone and a training budget of 40k steps. The two methods share all other training and evaluation settings, with the training states and time conditioning listed in the table. | Method | Training state | Time conditioning | PSNR ↑ | SSIM ↑ | SAM ↓ | ERGAS ↓ | |---|---|---|---|---|---|---| | Direct | $L$ | No | 48.6278 | 0.9872 | 0.0104 | 0.5002 | | CFM | $X_t$ | Yes | 48.8446 | 0.9878 | 0.0098 | 0.4862 | With the same backbone and training budget, CFM achieves better overall reconstruction results, confirming the contribution of this training design to first-stage reconstruction. Part 2 addresses the role of group-relative weighting in GRPA.
AuthorsOpenReview ↗

Response to Reviewer nhkP (Part 2 of 4)

Q2. The role of group-relative weighting in GRPA

The first-stage CFM provides the base prediction for the second stage. We agree that a more targeted comparison is needed to distinguish the effects of group-relative weighting, additional training, and the added loss constraints.

GRPA assigns supervision weights based on the relative rewards of candidates within the same group, enabling the second stage of training to distinguish the quality of the spectral, spatial, and balanced candidates. The candidates are constructed from the same base prediction. The weights within the group adjust how much each candidate contributes to the update, and the candidate loss is optimized jointly with the GT reconstruction loss. We therefore need to examine the effects of continued training, candidate supervision itself, and relative weighting separately.

We designed the following second-stage comparisons on GF2. All methods start from the same first-stage CFM weights and use identical second-stage inputs, time conditioning, training steps, and other training and evaluation settings. They also retain the same GT reconstruction loss. We use the reduced-resolution evaluation protocol from the ablation experiments in the paper.

GT-only continues training using only the GT reconstruction loss to test whether additional training is sufficient to explain the gains. Uniform uses the same candidate construction and candidate loss coefficient as GRPA and replaces the weights within the group with uniform weights wg=1/Gw_g=1/G, where G=4G=4. Direct loss bypasses candidate construction and adds the spectral and spatial consistency errors from Eqs. (7)–(8) directly to the training loss for the current prediction. The relative weighting of the two terms follows Eq. (9). GRPA uses group-relative weights and retains the other training settings described in the paper.

Reduced-resolution results

MethodSupervision beyond the GT reconstruction lossPSNR ↑SSIM ↑SAM ↓ERGAS ↓
GT-onlyNone48.90120.98830.00990.4836
UniformUniformly weighted candidate loss48.75280.98750.01110.5062
Direct lossDirect optimization of spectral and spatial consistency losses48.85370.98800.00920.4721
FMGRPAGroup-relative weighting of the candidate loss49.19650.98960.00890.4594

With the same training budget, GRPA achieves better overall performance in the reduced-resolution evaluation. The comparison with GT-only shows that GRPA improves on the gains from continued training under GT supervision. The comparison with Uniform shows that group-relative weighting is more effective than using uniform weights for the candidate loss. The comparison with Direct loss shows that candidate comparison and relative weighting are more effective than directly optimizing the spectral and spatial consistency objectives. These results support a contribution from the proposed preference alignment mechanism beyond the gains attributable to additional training and the added loss constraints alone.

Part 3 addresses candidate construction and sensitivity to reward parameters.

AuthorsOpenReview ↗

Response to Reviewer nhkP (Part 3 of 4)

Q3. Rationale for candidate construction and sensitivity to reward parameters

Group-relative weighting depends on candidate construction and reward scoring. We provide the rationale for choosing the candidate construction coefficients and report comparisons on sensitivity to the reward coefficients and Softmax temperature.

When initially selecting the candidate construction coefficients, we ran extensive validation experiments comparing reconstruction quality against the reference, spectral performance, and spatial performance across different candidate constructions and coefficient combinations. The representative validation used 64 reduced-resolution GF2 samples. All candidate groups for a given sample shared the same base prediction. In Eq. (6), α\alpha and β\beta are the coefficients for low-frequency correction and PAN high-frequency detail injection, respectively. The metric DsD_s measures spatial distortion.

The following table compares the base prediction and different combinations of candidate construction coefficients in terms of reconstruction quality relative to the reference, spectral fidelity, and spatial distortion.

Candidateα\alphaβ\betaPSNR ↑SAM ↓DsD_s ↓
base0048.81320.0098620.0513
Spectral candidate0.12048.99460.0093800.0533
Balanced candidate ×10.100.003548.97840.0095500.0489
Balanced candidate ×40.400.014049.05910.0094800.0438
Spatial candidate ×100.0148.84610.0097940.0464
Spatial candidate ×400.0448.77860.0101500.0329

Candidate construction uses two complementary sources of imaging information. The low-frequency LRMS residual corrects spectral deviations, and the high-frequency PAN component adds spatial detail. Both corrections are computed for the current image. The combination of coefficients determines the emphasis of the correction, and the magnitude of each coefficient controls its strength. In the initial validation, the spectral candidate mainly improved SAM, the spatial candidate mainly reduced DsD_s, and the balanced candidate accounted for both reconstruction quality relative to the reference and spatial consistency. Based on the improvements on the target metrics for each candidate type, we selected the spectral candidate ×1, the balanced candidate ×4, and the spatial candidate ×4, and retained the base prediction as a reference. This candidate group captures the trade-offs among prioritizing spectral fidelity, prioritizing spatial quality, and balancing the two in pansharpening. The GF2, WV2, and WV3 experiments use the same candidate construction coefficients. Table 4 of the paper further examines the contribution of each candidate type to the final reconstruction through candidate removal ablations.

The coefficients for the spectral, spatial, and reference fidelity terms in Eq. (9) are 1, 0.8, and 2.5, respectively. We selected several reward weight combinations based on experience and compared their reconstruction performance. In these comparisons, we fix the spectral coefficient at 1 and the Softmax temperature at 0.50, and vary the spatial and reference fidelity coefficients. The following table compares the reconstruction performance of different reward weight combinations.

Spectral coefficientSpatial coefficientReference fidelity coefficientPSNR ↑SSIM ↑SAM ↓ERGAS ↓
10.82.549.19650.98960.00890.4594
10.62.049.12840.98940.00880.4612
10.63.049.17260.98940.00900.4618
11.02.049.11280.98950.00920.4658
11.03.049.15870.98950.00900.4614

In the reward weight comparison, the baseline combination performs best on PSNR, SSIM, and ERGAS. The 1:0.6:2.01:0.6:2.0 combination achieves a slightly lower SAM. The PSNR of each combination differs from that of the baseline by at most 0.084 dB.

We fix the reward coefficients in the temperature comparison. The following table compares reconstruction performance at temperatures of 0.25, 0.50, and 1.00.

Softmax temperaturePSNR ↑SSIM ↑SAM ↓ERGAS ↓
0.2549.14380.98940.00890.4628
0.5049.19650.98960.00890.4594
1.0049.08640.98920.00920.4667

A temperature of 0.50 gives the best overall results. At temperatures of 0.25 and 1.00, PSNR is 0.053 and 0.110 dB lower than the baseline, respectively.

Part 4 addresses optimization stability, coefficient selection, and computational overhead.

AuthorsOpenReview ↗

Response to Reviewer nhkP (Part 4 of 4)

Q4. Effect of self-generated candidates on optimization stability

This is a pertinent question. When designing GRPA, we explicitly considered the directional drift that self-generated candidates could introduce. To address this, we incorporated constraints based on the HRMS ground truth into both candidate scoring and model updates.

For each update, Eq. (13) holds the candidates and their weights fixed and uses the weighted differences between the prediction and the candidates to update the current prediction. We recompute the candidates and their weights from the prediction at each step, so the contribution of each predefined direction to the update depends on the relative scores of the current candidates.

To mitigate drift from the predefined directions over successive updates, Eq. (9) penalizes candidates according to how far they deviate from the HRMS ground truth each time they are scored. With the other reward terms held equal, a candidate with a larger error relative to the reference receives a lower weight within the group. Eq. (14) computes the GT reconstruction loss directly on the current prediction, so candidate corrections remain constrained by the HRMS ground truth for the corresponding sample.

The GF2 comparison in Table 3 shows that the preference-aligned model improves over the first-stage CFM on all full-reference metrics, indicating that the final model achieves higher reference fidelity after incorporating self-generated candidates into the updates.

Q5. Selection of candidate construction coefficients and reward weights

We selected the candidate construction coefficients based on the initial validation. We compared different strengths of low-frequency correction and PAN high-frequency detail injection on the same base predictions. We selected the spectral candidate primarily based on improvements in SAM, evaluated whether the balanced candidate could improve both PSNR and spatial distortion DsD_s, and weighed reductions in DsD_s against the cost in PSNR for the spatial candidate. The final coefficients (α,β)(\alpha,\beta) for the spectral, balanced, and spatial candidates are (0.12,0)(0.12,0), (0.40,0.014)(0.40,0.014), and (0,0.04)(0,0.04), respectively.

We set the reward weights in Eq. (9) based on experience. Using 1 as the coefficient for the spectral consistency term, we set the spatial consistency and reference fidelity coefficients to 0.8 and 2.5, respectively, so that candidate ranking accounts for spectral consistency, spatial consistency with the PAN image, and deviation from the HRMS reference. Our response to Q3 presents the comparisons of candidate construction coefficients and the sensitivity comparisons based on this set of reward weights.

Q6. Computational overhead of candidate construction, reward computation, and spatiotemporal aggregation

On GF2, second-stage training with four-time-point GRPA takes 1,938.7 ms/step, an increase of 126.0% from the 857.6 ms/step for continued training with ground truth alone (GT-only). Peak allocated GPU memory increases from 8.75 GiB to 8.79 GiB.

All settings start from the same CFM checkpoint and use the same batch size and image dimensions (24 images, 128 × 128), FP32 precision, and optimizer settings on the same RTX 4090. After 15 warm-up steps, we measure the mean step time over 100 steps. Timing includes forward passes, backpropagation, optimizer steps, and EMA updates. Data transfer, validation, and saving are outside the timed interval.

Second-stage settingBackbone forward passes per stepMean time (ms/step)Peak allocated GPU memory (GiB)
GT-only1857.68.75
GRPA, one time point, full-image scoring only21,087.48.79
GRPA, one time point, scoring on the full image and two local regions21,131.18.79
GRPA, four time points, scoring on the full image and two local regions51,938.78.79

When spatial aggregation is enabled, local scoring reuses the full-image candidates, and the time per step increases by 43.7 ms. With the same spatial setting, increasing the number of scoring time points from 1 to 4 increases the time per step by 807.5 ms. In separate timing measurements for the components of full GRPA, the 4 forward passes without gradient computation take 606.1 ms/step in total. Candidate construction takes 3.9 ms/step. Full-image and local reward computation (including Softmax) take 267.3 and 192.1 ms/step, respectively, and spatiotemporal weight aggregation and the associated statistics take 14.3 ms/step. The additional overhead comes mainly from backbone forward passes at multiple time points and reward computation.

Candidate construction, reward computation, and spatiotemporal aggregation are performed only during training.

Comments on the paper

10 comments

Dear Authors,

There are fewer than four days left in the author-reviewer discussion period, and you have not yet submitted a rebuttal. If you do not plan to withdraw your paper, please submit your rebuttal as soon as possible. This will give reviewers enough time to read your responses, provide feedback, and discuss the paper with you.

I also have the following concerns and would appreciate your responses:

  1. The role of flow matching. Under the proposed parameterization, the velocity-matching loss in Eq. (3) appears to be equivalent to direct reconstruction of the ground truth. The role of the random noise along the interpolation path is also unclear. Please explain what the flow-matching formulation adds compared with direct reconstruction.

  2. The GRPA objective. GRPA uses manually designed spectral and spatial candidates and heuristic reward functions. The objective appears to be a reward-weighted regression loss toward pseudo-targets that are treated as fixed during optimization. The reward also includes the ground-truth reconstruction error. Please clarify what this objective adds beyond reconstruction and regularization.

  3. The connection to preference alignment. Unlike standard GRPO-style methods, the candidates are deterministic perturbations in predefined spectral and spatial directions, rather than diverse samples generated by the model. GRPA therefore seems closer to adaptive weighting of manually designed spectral and spatial regularization terms. Please explain why it should be viewed as preference alignment and what new idea it introduces.

Overall, please clarify the reasoning behind these design choices and how the method differs conceptually from existing approaches.

Thank you.

Your AC

AuthorsOpenReview ↗

Response to the Area Chair (Part 1 of 2)

Thank you for your valuable comments. Your feedback helped us identify where the paper needs further clarification and improvement. We will revise the paper accordingly and respond to each point below. Please let us know whether we have addressed your questions.

Q1. How CFM differs from direct reconstruction and the role of noise along the path

We thank the Area Chair for raising this key issue. We agree that, in the first stage of our method, the L1 loss on velocity is equivalent to the L1 loss on HRMS reconstruction.

Along a linear interpolation path, the standard CFM velocity regression objective can also be cast as a reconstruction objective with an appropriate endpoint prediction parameterization. In our method, the key differences between CFM and direct reconstruction lie in the distribution of training states and in time conditioning. We define the conditional path as

\qquad t\sim\mathcal U(0,1),\quad \epsilon\sim\mathcal N(0,I),$$ with the corresponding conditional velocity given by $$u_t=\frac{\mathrm dX_t}{\mathrm dt}=X-L.$$ Given a path state $X_t$, the network is conditioned on time $t$ and the LRMS/PAN inputs and regresses this velocity via $s_\theta=f_\theta-L$. This follows the basic CFM training procedure of sampling from a conditional path and regressing the corresponding target velocity. Direct reconstruction takes the same observed state $L$ as input and predicts the HRMS image under PAN guidance. CFM requires the model to reconstruct the same HRMS image from different intermediate states. These states contain varying amounts of information from the ground-truth HRMS image. At smaller values of $t$, the interpolation center is closer to $L$, and the model must recover more spectral and spatial detail. As $t$ increases, the interpolation center moves toward the ground truth $X$, and the residual $(1-t)(X-L)$ to be recovered becomes smaller. We train a single network on these reconstruction tasks at different interpolation levels and use the time condition $t$ to indicate the current level of interpolation. Training samples at different time points update the same set of parameters, allowing the network to reuse spectral and spatial features across these tasks. Thus, with the same number of training image pairs, the model sees more varied inputs and receives reconstruction supervision across different input states. We introduce random noise to expose the model to states near the interpolation path during training, encouraging it to reconstruct stably when the input is perturbed. For a given image pair and a fixed time $t$, $(1-t)L+tX$ is the interpolation center at that time. Each draw of zero-mean Gaussian noise produces a different training input near this center. The LRMS/PAN conditioning inputs and the HRMS ground truth stay the same, allowing the model to learn to recover the spectral and spatial details of the same image from these nearby states. This use of random noise helps reduce the effect of random input fluctuations on the output, supporting stable reconstruction under perturbations. To further evaluate the benefits of CFM over direct reconstruction, we added a first-stage comparison with direct reconstruction (Direct) on GF2. Both methods use the same NAT backbone, L1 and SSIM losses, training budget, and evaluation protocol. | Method | Training state | Time conditioning | PSNR ↑ | SSIM ↑ | SAM ↓ | ERGAS ↓ | |---|---|---|---|---|---|---| | Direct | $L$ | No | 48.6278 | 0.9872 | 0.0104 | 0.5002 | | CFM | $X_t$ | Yes | 48.8446 | 0.9878 | 0.0098 | 0.4862 | With the same backbone and training budget, this comparison shows that our CFM improves reconstruction relative to Direct. Part 2 addresses the GRPA objective and preference alignment.
AuthorsOpenReview ↗

Response to the Area Chair (Part 2 of 2)

Q2. Additional supervision provided by the GRPA objective

The first-stage CFM provides the base prediction for the second stage. Beyond direct ground-truth supervision, GRPA provides spectral, spatial, and balanced candidate corrections to the current prediction and uses each candidate's overall performance to determine how much its correction direction contributes to the update.

We use the low-frequency LRMS residual to construct the spectral candidate, the high-frequency PAN details to construct the spatial candidate, and both corrections to construct the balanced candidate. We score these three candidates together with the base prediction. The scoring accounts for both spectral and spatial consistency and uses the reconstruction error relative to the ground truth to penalize candidates that deviate from the HRMS reference. We convert the scores into relative weights within the group and use them to weight the regression loss between the current prediction and each candidate. Higher-scoring corrections receive greater weight in the candidate loss, giving the corresponding correction directions more influence on the update.

When we add spectral and spatial consistency losses directly, we update the model based on the consistency errors of its current prediction. GRPA adds a comparison of different correction outcomes to determine which correction better balances spectral consistency, spatial consistency, and reference fidelity for the current sample. The within-group scores assign greater weight to corrections that perform better overall, and weighted regression applies those weights to the model update. We recompute the candidates and weights as the prediction changes, allowing the model to adjust the emphasis on spectral and spatial corrections based on the current reconstruction for each sample.

Q3. Why deterministic candidates can be used for preference alignment

These candidate weights reflect relative preferences among fusion results within the same group. GRPA draws on GRPO's approach to evaluating results relative to others in the same group and applies it to the spectral and spatial trade-off in pansharpening. Directly applying adaptive weights to spectral and spatial regularization terms adjusts which constraints the current prediction is encouraged to satisfy more closely. Our GRPA adds a comparison of multiple correction outcomes for the same prediction and uses the comparison to determine which specific corrections to emphasize during training.

Starting from the same base prediction, we construct the spectral candidate using the low-frequency LRMS residual, the spatial candidate using the high-frequency PAN details, and the balanced candidate by combining the two corrections. The base prediction is also included in the comparison as the reference before any correction is applied. The deterministic rules specify which correction directions the candidates cover, and the current prediction and the observations for that sample determine the candidate images themselves.

We evaluate each candidate image for spectral consistency, spatial consistency, and reference fidelity together to assess whether a correction can improve one aspect while accounting for the others. For the current sample, the relative scores within the group express preferences among the different corrections. During training, we use the candidate images as regression targets and use these scores to steer the model's predictions more strongly toward candidates with better overall performance. This step turns within-group preferences into reconstruction supervision, which is what we mean by "alignment" in GRPA. We recompute the candidates and scores as the current prediction changes, so the same construction rules can yield different preferences across samples and allow training to adjust the emphasis on spectral and spatial corrections based on the current fusion result.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.