acceptodds
Withdrawn from ICLR 2027

LightDial:Progression-Conditioned Attention for Continuous Portrait Relighting

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Portrait relighting plays an important role in portrait enhancement and creative editing by reshaping how illumination defines facial structure, subject appearance, and visual atmosphere. Despite recent advances in instruction-based image editing, current models still struggle to produce consistently plausible and visually coherent portrait lighting across diverse effects, while offering limited control over how strongly each effect is expressed. A key challenge is that increasing lighting strength is not a uniform change in edit magnitude: different effects evolve through distinct, spatially structured changes in illumination. In this work, we present LightDial, which formulates lighting strength through a continuous progression variable. This progression-based formulation turns strength control from magnitude scaling into learning structured changes along an ordered lighting progression. By modulating the interactions among text, reference-image, and noisy-output representations according to the requested progression state, and by explicitly constraining response changes between neighboring states during training, LightDial enables smooth and structured evolution of lighting effects while preserving identity and scene content. To provide the effect diversity and ordered progression supervision required by this formulation, we construct PortraitLight-50K, a large-scale portrait relighting dataset covering structurally diverse lighting effects with multi-level progression sequences. Extensive experiments demonstrate high-quality portrait relighting and strong identity preservation, together with smoother and more consistent progression responses than existing continuous-control approaches.

Primary area:
generative models
Keywords:
Diffusion Models, Controllable Generation, Image Editing, Continuous Control, Representation Learning, Flow Matching, Portrait Lighting Editing

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_fkn4OpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
1
Read review
Paper summary

LightDial tackles continuous portrait relighting by modeling lighting strength as an effect-specific ordered progression variable s ∈ [0,1] rather than a generic edit-magnitude scalar. The method introduces (1) TriDial, a parameter-free progression-conditioned relative rotation applied to queries/keys of text, noisy-output, and reference token groups in joint attention; and (2) Strength-Aware Alignment (SAA), a training-only finite-difference regularization that aligns predicted velocity differences of adjacent progression states. To support this, the authors build PortraitLight-50K, a ~50K-image / 7.1K seven-level sequence dataset covering six structurally distinct lighting effects (Hair, Tyndall, Dappled, Rembrandt, Sunset, Slit). Experiments compare LightDial against general-purpose editors, retrained continuous controllers, inference-time controls, and endpoint-conditioned transitions, showing stronger identity preservation, smoother/monotonic progression, and good endpoint quality.

Critical strengths

The core framing is genuinely nice: “strength” in portrait relighting is effect-dependent and spatially structured, not a single scalar magnitude. That motivation is well explained.

TriDial is elegant and cheap, orthogonal group-relative rotations modulate cross-stream attention without scaling query/key norms or adding trained modules; the common-angle ablation correctly exposes that only relative angles matter.

SAA is a sensible inductive bias, it regularizes neighboring-state response changes under matched noise/time, without altering the pointwise flow-matching target.

PortraitLight-50K is a real contribution: multi-source candidate generation + VLM/expert verification + sequence-level ordering is more principled than endpoint-only pairs.

Critical weaknesses

Missing comparison with dedicated SOTA portrait/image relighting methods.​ The paper compares mainly to general-purpose instruction editors (GPT-Image 2, Nano Banana, FLUX, Qwen, Seedream) and generic continuous-control tricks (CFG/LoRA scaling, text-embedding interpolation, GRAG, FreeMorph, Wan). But it does not​ compare against relighting-specific baselines such as IC-Light​ (Zhang et al., ICLR 2025), Total Relighting, SwitchLight, SynthLight, IntrinsicEdit / Latent Intrinsics, or CFT​ beyond a one-line related-work mention.

The dataset is generated by the very models used as baselines.​ Endpoints come from GPT-Image 2 / Nano Banana / FLUX / Qwen, and intermediate candidates partly from Wan 2.1. Even with VLM + expert filtering, the training distribution may inherit generator biases, and some “baselines” are also data sources. The paper acknowledges this, but the risk is that LightDial’s advantage partially reflects fitting the filtered generator space rather than surpassing independent relighting systems.

A single scalar couples many attributes.​ One s controls direction, softness, color temperature, contrast, spatial extent, and intensity at once. The paper is honest about this, but it limits the method’s usefulness for fine artistic/professional control and weakens the “continuous control” claim: it is ordered, not disentangled.

Generalization evidence is thin.​ Cross-backbone results are qualitative; extrapolation beyond s=1 is shown only to demonstrate failure. There is no study on out-of-domain portraits, non-face subjects, group photos, or heavily occluded/extreme-pose cases.

Questions for authors

Why not include IC-Light, SynthLight, Total Relighting, SwitchLight, and CFT​ as direct baselines? Without them, is the claim “strong portrait relighting” evaluated against the right neighborhood?

If IC-Light were given the same six effects / instructions, how would you expect LightDial’s ordered-progression behavior to differ from IC-Light’s consistent-light-transport relighting? Can you add at least one quantitative table vs. IC-Light?

Can you report per-effect structure-accuracy metrics (not just preference share), e.g., does the Rembrandt lit-cheek triangle appear, are Tyndall shafts directional, is Slit lighting narrow-band?

Could LightDial be extended to multi-variable control (direction × warmth × intensity × softness)? If not, what is the intended deployment boundary?

Reviewer_Ts2tOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper presents LightDial, a framework for diverse portrait relighting with continuous strength control. Two components are proposed: (1) TriDial, a progression-conditioned attention mechanism that applies group-specific orthogonal rotations; (2) Strength-Aware Alignment (SAA), a training objective that regularizes the differences between predicted flow-matching velocities at adjacent progression states.

Besides, PortraitLight-50K, a large-scale portrait relighting dataset is constructed. It consists of seven-level progression sequences across six portrait-lighting effects.

Extensive experiments demonstrate high-quality portrait relighting and strong identity preservation over general-purpose image editors, continuous editing controllers, and endpoint-conditioned transition methods

Critical strengths
  • The paper addresses a meaningful problem in instruction-based image editing: continuous control over editing strength.
  • The proposed TriDial mechanism provides a simple and interpretable approach to progression conditioning by modulating the relative attention interactions among text, reference-image, and noisy-output tokens. The mechanism introduces no additional trainable parameters and can be integrated into existing frameworks like Kontext and Qwen-Image-Edit.
  • The paper introduces PortraitLight-50K which offers explicit supervision for intermediate editing states. It provides some insights to the community to build datasets with continuous control.
Critical weaknesses
  • A major concern is that the improvements may be substantially driven by the high-quality multi-level supervision rather than the proposed architectural and optimization components.
  • Insufficient motivation and justification for the TriDial design. It is unclear why this requires group-specific query-key rotations rather than more conventional conditioning mechanisms.
  • The construction of PortraitLight-50K relies heavily on generated intermediate states, but several important implementation details remain underspecified. In particular, the source-endpoint difference-based procedure is described only at a high level. Similarly, the Wan 2.1 endpoint-conditioned transition procedure lacks details about endpoint-conditioning implementation, prompts, sampling configurations, and intermediate-frame selection.
  • The progression labels are assigned uniformly as (s_k=k/6) after the intermediate images have been selected and ordered. However, the authors explicitly acknowledge that these labels do not correspond to perceptually equal changes or a calibrated physical lighting intensity. As a result, equal changes in the control variable may correspond to substantially different perceptual changes across images and lighting effects. This raises questions about the semantic consistency and predictability of the slider control.
Questions for authors
  • Wan2.1 which was released in early 2025, approximately 18 months before this submission, while video generation models have advanced substantially since then. Why was this model chosen for constructing the dataset?
Reviewer_XS1SOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

The paper studies continuous strength control for instruction-based portrait relighting. The strength s∈[0,1] is treated as a position along an ordered, effect-specific "progression".

The method has three parts.

TriDial modifies the attention layers of a LoRA-adapted FLUX.2-klein-9B. Post-RoPE queries and keys are rotated by group-specific orthogonal matrices, with angles proportional to s and multipliers 1:2:3 for text, output, and reference tokens. Cross-group attention logits then change with s, while same-group logits stay fixed.

SAA is a finite-difference regularizer. Two adjacent strength levels share noise and timestep, and the difference of the predicted velocities is trained to match the difference of the target latents.

The dataset, PortraitLight-50K, contains ~7.1K seven-level sequences over six lighting effects. Endpoints come from five existing editors (GPT-Image 2, Nano Banana 2, Qwen-Image-Edit, FLUX.1 Kontext, FLUX.2-klein) and are filtered by MUSIQ/ArcFace/DINO/VLM/expert checks. Intermediate levels are picked from a pool built by a "difference-based construction" plus Wan 2.1 endpoint-conditioned transition frames.

Evaluation includes endpoint quality and preservation (Table 1), a human endpoint-preference study (Table 2), and a progression benchmark with automatic diagnostics (Mono/CV) and human ratings against learned controllers, inference-time controls, and endpoint-conditioned transitions (Table 3). Ablations are in Table 4.

Critical strengths

The main assets here are the problem framing and the dataset. Treating strength as a position along an ordered, effect-specific progression, rather than just a magnitude, is a sensible way to think about interactive portrait editing. A released set of ~7.1K verified seven-level sequences across six spatially distinct effects would be genuinely useful to the community.

TriDial itself is a simple, parameter-free way to inject s. It rotates post-RoPE queries and keys by group-specific angles (Eq. 3–4), which leaves same-group logits fixed and lets cross-group logits vary smoothly with s. No extra parameters, clean closed form.

The SAA ablation is a clean positive result. Removing SAA drops Mono from 0.87 to 0.81 and worsens CV from 0.43 to 0.71, while ArcFace/DINO stay essentially unchanged (Table 4). So the adjacent-level supervision seems to do its job without hurting preservation.

Critical weaknesses

The dataset's "progression" is inherited from existing generators. Endpoints are outputs of five existing editors; intermediates come from Wan 2.1 endpoint-conditioned transitions plus a "difference-based construction" that is never really specified (Sec. 3.2, A.8). Wan 2.1 is also the strongest baseline in Table 3 (Mono 0.84 / CV 0.52 vs. 0.87 / 0.43), so LightDial is essentially distilling an editor-plus-transition pipeline into a LoRA. The share of Wan-derived intermediates, and performance on non-Wan groups, are not reported.

TriDial is not shown to beat standard scalar conditioning. The rotation amounts to a group-specific virtual positional offset from which the LoRA has to infer s. No other way of injecting s is compared under the same backbone, LoRA, data, and SAA — e.g., adding s to the timestep/guidance embedding or to the prompt. The "common angle (1,1,1)" row of Table 4 doesn't fill this gap: by Eq. 4, equal angles leave every logit unchanged, so it is equivalent to removing s, not a scalar-conditioning baseline. Kontinuous Kontext and SliderEdit use different backbones and adapters, so Table 3 can't attribute the gain to relative rotation either.

SAA is a mild regularizer, and its evidence rests on narrow metrics. By Eq. 8 the residual equals e_b − e_a and vanishes once flow matching is fit; it looks closely related to CFT's pairwise velocity supervision and slider consistency terms, with no comparison offered. The ablation gain is measured only by Mono/CV, which summarize order and regularity of illumination deviation, not whether the requested effect is right (App. D.3). A linear source–endpoint blend scores well on these, and since the training targets are transition frames, LightDial's lead is partly expected.

Questions for authors

(Clarification, critical) Can you compare TriDial with s injected as a scalar condition (timestep/guidance embedding or prompt), with backbone, LoRA, data, and SAA held fixed? Was Kontinuous Kontext retrained on the same FLUX.2-klein-9B backbone? (Clarification, critical) How does SAA differ in effect from CFT-style pairwise velocity supervision or slider consistency losses? A direct comparison would help. (Clarification) What fraction of intermediate targets are Wan-derived, and how does LightDial perform on non-Wan groups? (Clarification) What exactly is the "difference-based construction" (pixel/latent interpolation? alpha blending?)? (Clarification) Does the model reconstruct the source at s=0? Please report PSNR/LPIPS against the source. (Clarification) Are the Mono/CV gaps in Table 4 (e.g., 0.81 vs. 0.87) significant over the 200 groups? Please report standard deviations or CIs. (Suggestion) Add a baseline that linearly blends the source with the s=1 output, to calibrate what Mono/CV and the human ratings measure.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.