Ratio-RFE: From Appearance Fitting to Source-Relative Relighting Field Estimation for Image Harmonization

Image harmonization aims to adjust a composite foreground to match the background while preserving its content and structure. Existing methods typically learn the desired target appearance directly, requiring appearance adjustment and content preservation to be handled within the same prediction process. This work instead formulates image harmonization as source-relative relighting field estimation. Motivated by the approximately shared foreground reflectance between the composite and harmonized images, the required source-to-target change is represented as a source-relative log-ratio relighting field, by canceling the shared reflectance. The harmonized foreground can then be obtained from the observed source through analytical reconstruction based on relighting field, without explicitly recovering reflectance or absolute lighting. Based on this formulation, a source-relative Relighting Field Estimation (in the log-Ratio form) based image harmonization framework, termed as Ratio-RFE, is proposed. It is developed with Phong-inspired Ambient-Diffuse-Specular (ADS) representations, where the Ambient, Diffuse and Specular components are estimated separately based on their different characteristics. An ADS-guided dynamic aggregation is further developed, where these representations guide spatially adaptive aggregation for unified relighting field estimation. Moreover, transformation-aligned learning is used to directly constrain the predicted log-ratio relighting field together with the reconstructed appearance. Experiments demonstrate state-of-the-art performance on iHarmony4 and strong cross-dataset generalization to ccHarmony without fine-tuning. Moreover, although trained only at $256\times 256$, Ratio-RFE generalizes directly to $1024\times 1024$ and $2048\times 2048$ inputs without retraining, demonstrating strong cross-resolution generalization.