acceptodds
Under review as a conference paper at ICLR 2027

KINESYS: Test-Time Adaptation for Reference-Based Effect Editing in Images

Abstract

Visual effects are often easier to demonstrate via reference than to describe in words. We study effect-reference editing, where a reference image shows a desired visual effect and language identifies which properties should guide the edit. Existing single-pass editing techniques often fail to make fine-grained edits, resulting in subtle identity distortions or deviations from the reference effect. We introduce KINESYS, a test-time adaptation framework for effect-reference editing that combines reference-aware and source-aware feedback to transfer the desired effect while preserving the identity of the original shot. To ensure KINESYS can produce a diverse set of identity-preserving edits, we develop a robust optimization strategy in which the generative model is fine-tuned across multiple noise samples and optimization runs. Evaluation with VLM judges, source-preservation metrics (LPIPS, TPIPS), and a user study shows that KINESYS outperforms state-of-the-art image editors in source identity preservation; on effect transfer, it remains competitive with Nano Banana 2 and outperforms open-weight editors.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
reference based editing, visual effects

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 2 reviews, above 27% of the others.
Reviewer_jikXOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

KINESYS edits an image by transferring a visual effect (smoke, snow, fire, a boat wake, etc.) from a reference image while keeping the source scene intact. For each request, it fine-tunes LoRA weights on FLUX.2 Klein 4B using two frozen judges: a Gemma-4 VQA judge that compares the edit with the reference, and EditReward, which compares it with the source. It runs 5 independent fine-tuning runs of 3 candidates each, prunes the weaker runs, and picks the final checkpoint by EditReward. On a new 81-request benchmark, it has the best source preservation (LPIPS 0.377), beats the open-weight editors. It comes close to Nano Banana 2 on the Gemini and EditScore judges, and in a 16-edit user study it is preferred for identity preservation and roughly matches Nano Banana 2 on effect transfer.

Critical strengths
  1. The paper tackles a practical problem and shows a clear advantage in identity preservation. Specifying the effect with a reference image plus yes/no questions is a useful interface for VFX-style edits. KINESYS preserves the source better than every baseline, including Nano Banana 2 (LPIPS 0.377 vs. 0.429; user preference 45% vs. 25%).

  2. The evaluation is transparent. The authors flag which metrics are not independent of training, and where a baseline's generator shares a model family with the judge (Nano Banana 2 under the Gemini judge); both are marked in Table 1. They use two seeds for every request and report worst-seed scores in the run ablation (Table 2), and they show that the adapted LoRA generalizes to fresh noise (EditScore 8.91 vs. 8.93).

Critical weaknesses
  1. The gains may come from search and selection rather than from fine-tuning. KINESYS uses far more compute than the single-pass baselines (5 runs of 3 candidates each, with up to 100 LoRA steps per run) and then picks its output by EditReward. Yet the main paper has no compute-matched best-of-M baseline and reports no compute cost (it only notes that fine-tuning adds latency). The paper's own numbers suggest that fine-tuning alone may help little. VQA-only fine-tuning of the same generator (Dual-Process) scores 7.09 on Gemini, below the 7.90 of the unadapted editor on the same cases (Table 1), and a single KINESYS run (K = 1) scores far below five runs in Table 2 (7.31 vs. 9.00, on a 16-request subset for which the paper gives no unadapted score; see Question 4). The novelty over Dual-Process is mainly EditReward and the multi-run search, so this comparison is central.

  2. Fidelity of the transferred effect to the reference is not measured automatically. None of the automated metrics in Table 1 scores it. The Gemini judge sees the reference but scores only perceptual quality and source consistency, EditScore and EditReward never see it, and LPIPS and TPIPS reward edits that change little. EditReward is also part of the training objective and the checkpoint-selection criterion. Apart from qualitative figures, the effect-transfer claim therefore rests on a 16-edit user study. The main paper reports no significance test and says only that it uses two edits per category, not how they were selected. Against Nano Banana 2 that result is close (48.2% vs. 42.1%).

Questions for authors
  1. Please add a compute-matched best-of-M baseline: sample unadapted FLUX.2 Klein and select with the same VQA and EditReward judges. Please also report per-request time and GPU memory.

  2. Which direct editors receive the effect reference image, and how? If some never see it, the comparison of effect transfer is not between equal inputs.

  3. Please report a metric that compares the edit with the reference for all methods. Examples are held-out VQA with a VLM from a family other than Gemma or Gemini, or the ICE-Bench judge. Please also add quantitative ablations of each feedback term, the diversity term and candidate weighting.

  4. What does unadapted FLUX.2 Klein score on the same 16 requests as Table 2? If K = 1 scores below it, what does fine-tuning add beyond selection?

2 comments
AuthorsOpenReview ↗

We address whether adaptation improves on frozen sampling at matched compute, report runtime and GPU memory, and clarify reference-image inputs. We also report reference adherence under Gemini and Qwen3.5 and quantitative component ablations.

Adaptation versus sampling (Q1, Q4; W1)

The next table shows an equal-compute comparison on the same 16 requests used for ablations, testing whether the gains in main-paper Table 2 can be explained by sampling and selection alone. All methods receive the same compute budget on the same hardware: two 45-minute attempts per request, then best-of-two selection by overall quality. Frozen best-of-MM generates and scores candidates without model updates.

PQ/SC: perceptual quality/source consistency. Judge scores: 0-10. Adherence: effect transfer, judged by Gemini. Higher is better except LPIPS; bold/underline: best/second best.

MethodPerceptual qualitySource consistencyReference adherenceLPIPS
Frozen best-of-MM9.1889.2198.6880.433
KINESYS9.4389.4069.0520.362

KINESYS achieves higher mean perceptual quality, source consistency, and reference adherence, with lower LPIPS, than frozen best-of-MM sampling. Under matched compute, adaptation improves effect transfer and source preservation together, beyond sampling and selection alone.

For VFX, we prioritize overall quality and identity preservation: the edit must transfer the requested effect while preserving the source subject and scene composition.

Runtime costs (Q1; W1)

Runtime budgets are user-defined. We use the same 16 requests and best-of-two selection at each budget. Artists can select usable intermediate outputs without waiting for the full budget.

Minutes per attemptPQSCLPIPS
309.4588.0730.445
459.4389.4060.362‾\underline{0.362}
609.448‾\underline{9.448}9.396‾\underline{9.396}0.352

PQ changes little; preservation improves from 30 to 45 minutes. At 60, SC is nearly unchanged and LPIPS decreases further. We host the generator and feedback models on two RTX A6000 GPUs (48 GB each), with peak memory of 24.58 GiB for generation and 44.98 GiB for feedback.

Reference image inputs (Q2)

KINESYS, native FLUX.2 Klein 4B, Dual-Process, Nano Banana 2, Qwen-Image-Edit, OmniGen2, and ACE++ receive the source and effect reference as separate image inputs. Step1X receives a side-by-side composite containing both images. ICEdit receives only the source and is therefore labeled as a source-only baseline.

Reference adherence and Qwen3.5 evaluation (Q3; W2)

Selected baselines on the original 156 shared cases (81 requests), averaged over both seeds. KINESYS leads evaluated open-weight editors in overall quality under both judges and has the lowest LPIPS; adherence remains competitive.

MethodGemini adherenceQwen qualityQwen adherenceLPIPS
Dual-Process7.3727.1018.1000.577
FLUX.2 Klein 4B8.314‾\underline{8.314}7.8768.822‾\underline{8.822}0.579
Step1X-Edit6.9788.2297.7630.449
Nano Banana 28.9368.7589.1020.429‾\underline{0.429}
KINESYS8.3088.580‾\underline{8.580}8.5510.377

For VFX, it is critical to balance adherence with visual quality and source preservation. Figure 1 in the main paper demonstrates this: Dual-Process follows the reference strongly but loses the source building and composition; KINESYS adds puddles while preserving both.

Strong adherence is undesirable when it imports reference content at the expense of the artist's source. Conversely, source similarity alone can favor under-editing. We therefore interpret adherence alongside overall quality and LPIPS, favoring edits that transfer the effect while preserving the source identity and scene composition. Automated scores complement visual and human evaluation.

Component ablations (Q3; W1)

We run ablations on the same 16 requests as above. Overall quality=(PQ+SC)/2.

ConfigurationOverallSCAdherenceLPIPS
No EditReward objective8.8598.6888.6250.398‾\underline{0.398}
No VQA objective8.7819.188‾\underline{9.188}6.5940.337
Uniform candidate weights8.891‾\underline{8.891}8.9068.594‾\underline{8.594}0.476
Full KINESYS9.0169.4388.1250.414

Full KINESYS leads in overall quality and SC. Removing VQA causes a large adherence drop (8.125 to 6.594) despite better LPIPS; source similarity is insufficient without the requested effect being transferred well. Removing EditReward increases adherence but reduces SC, weakening source preservation. Full KINESYS uses EditReward to weight candidates, while uniform weights give all edits equal emphasis. Full KINESYS improves SC and LPIPS at the cost of adherence, balancing effect transfer with preservation of the artist's source.

We want to thank you for the detailed review. We wanted to ask whether any concerns remain after our response. If there are areas you would most want us to update to strengthen the paper, we will prioritize them before the discussion period ends.

Reviewer_GU27OpenReview ↗
Rating
2
Confidence
4
Informativeness
1
Presentation
3
Soundness
2
Read review
Paper summary

This paper studies effect-reference image editing, which aims to transfer a visual effect from a reference image while preserving the source image. The proposed model performs test-time LoRA adaptation using multiple candidate edits. A reference-aware VLM evaluates effect fidelity, while EditReward evaluates instruction following and visual quality. Several techniques in Sec. 3.2 are introduced to stabilize optimization. Experiments on eight effect categories show promising effect transfer and source preservation.

Critical strengths
  1. Effect-reference editing is a meaningful and practical task.
  2. Test-time adaptation is a promising approach for fine-grained image editing.
Critical weaknesses
  1. The novelty is limited. Test-time generation guided by VLMs or reward models has already been widely studied [1][2]. The main contribution appears to be combining VLM feedback with EditReward and introducing several engineering techniques for robust optimization.
  2. The performance is not sufficiently addressed. First, source consistency remains imperfect. In Figure 4, the "blue" color changes in Row 3, the sky color changes in Row 4, and the lighting changes in Row 5. Second, the evaluation covers only eight effect categories, which is insufficient to demonstrate broad generalization.

[1] Directly Fine-Tuning Diffusion Models on Differentiable Rewards.

[2] Dual-process image generation

Questions for authors
  1. How much time is required to produce one final output?
  2. The final result appears sensitive to the intermediate candidates. If the initial candidates are all poor, how does the method prevent optimization from following an incorrect direction?
3 comments
AuthorsOpenReview ↗

We address novelty in our method, runtime in the artist workflow, and sensitivity to poor starting candidates. We also discuss source preservation and effect coverage.

Novelty and contributions (W1)

  1. Joint effect transfer and source preservation. Reviewer Vti5 highlights the useful formulation of competing editing requirements. Reference-aware VQA guides effect transfer; source-aware EditReward limits unwanted source changes, addressing the VFX need to preserve the artist's subject and scene.
  2. Reference-based image-to-image adaptation for VFX. We extend Dual-Process from text-to-image generation to editing an existing source using an effect reference. Reviewer jikX highlights the use of references and yes/no questions for VFX and the advantage in identity preservation.
  3. Optimization tailored to this editing problem. As Reviewer Vti5 notes, the contribution adapts the objective and search procedure to editing with a source and reference. Candidate weighting, diversity regularization, and run selection support the objective by reducing sensitivity to poor starts and early convergence toward similar outputs.

Adaptation versus sampling (W1)

The next table shows an equal-compute comparison on the same 16 requests used for ablations. All methods receive the same compute budget on the same hardware: two 45-minute attempts per request, then best-of-two selection by overall quality. Frozen best-of-MM generates and scores candidates without model updates.

PQ/SC: perceptual quality/source consistency. Judge scores: 0-10. Adherence: effect transfer, judged by Gemini. Higher is better except LPIPS; bold/underline: best/second best.

MethodPerceptual qualitySource consistencyReference adherenceLPIPS
Frozen best-of-MM9.1889.2198.6880.433
KINESYS9.4389.4069.0520.362

KINESYS improves all four metrics over frozen sampling. Under matched compute, adaptation improves effect transfer and source preservation together, beyond sampling and selection alone.

For VFX, we prioritize overall quality and identity preservation: the edit must transfer the requested effect while preserving the source subject and scene composition. Figure 1 in the main paper demonstrates this: Dual-Process follows the reference strongly but loses the source building and composition; KINESYS adds puddles while preserving both.

Component ablations (W1)

We run ablations on the same 16 requests as above. Overall quality=(PQ+SC)/2.

ConfigurationOverallSCAdherenceLPIPS
No EditReward objective8.859‾\underline{8.859}8.6888.6250.398‾\underline{0.398}
No VQA objective8.7819.188‾\underline{9.188}6.5940.337
Full KINESYS9.0169.4388.125‾\underline{8.125}0.414

Full KINESYS leads in overall quality and SC. Removing VQA causes a large adherence drop (8.125 to 6.594) despite better LPIPS; source similarity is insufficient without the requested effect being transferred well. Removing EditReward increases adherence but reduces SC, weakening source preservation. These results support combining the two objectives: VQA guides effect transfer, while EditReward helps preserve the source subject and scene.

Runtime costs (Q1)

Runtime budgets are user-defined. We use the same 16 requests and best-of-two selection at each budget. Artists can select usable intermediate outputs without waiting for the full budget.

Minutes per attemptPQSCLPIPS
309.4588.0730.445
459.4389.4060.362‾\underline{0.362}
609.448‾\underline{9.448}9.396‾\underline{9.396}0.352

PQ changes little; preservation improves from 30 to 45 minutes. At 60, SC is nearly unchanged and LPIPS decreases further.

Poor initial candidates (Q2)

Each attempt starts five independent adaptation runs with three noise seeds each, giving 15 starting candidates. This reduces dependence on any single favorable initialization. Pruning directs compute toward promising runs, while LoRA adaptation can improve weak candidates. In the worst case, the method can be rerun from a fresh set of starting candidates.

Source preservation and unintended changes (W2)

Unintended color and lighting changes remain a limitation. Our priority is to preserve subject identity and scene composition, keeping the edit tied to the artist's source.

Effect coverage (W2)

KINESYS is not restricted by design to the eight evaluated effect categories and requires no effect-specific training data. Its scope depends on the generator's ability to represent the requested effect and the VLM's ability to assess its properties. We also tested movie-effect references, including fire breathing, portals, and magic, and observed similar qualitative results. As Reviewer Vti5 notes, the documented scope of eight effect categories is reasonable for a complete study.

I thank the authors for their detailed response. The rebuttal addresses some of my concerns, but two major ones remain unresolved.

My concern about the novelty remains. Extending Dual-Process from text-to-image generation to editing a source image with an effect reference is, in my view, not a sufficiently substantial technical contribution.

For source consistency, the authors acknowledge that source consistency remains a limitation. For an editing model, however, I regard it as one of the core requirements.

Based on the remaining concerns above, I would like to keep my original score.

We thank you for your feedback.

Regarding your concern about source consistency (W2), we agree that it is a core requirement of editing. In fact, source preservation is the key goal of this paper, and KINESYS outperforms both the evaluated open-weight and proprietary editing systems on source-preservation metrics. Our results show KINESYS preserving subject identity and pose, camera perspective, and scene layout while transferring the requested effect. We observe color or lighting shifts in the 3 mentioned cases. Other examples like dent (Figure 4) and car smoke (Figure 1) demonstrate our method retaining the source's overall color and lighting.

Preserving the original subject and scene is a substantial advantage over edits that change their structure or pose. For example, Figure 1 shows KINESYS retaining the source building and composition, while Dual-Process imports the reference street scene and other baselines alter the building or perspective. KINESYS outperforms all evaluated baselines on both source-preservation metrics, with the lowest LPIPS (0.377) and highest TPIPS (0.731). The blinded user study further supports source preservation as a strength of our method.

Comments on the paper

4 comments
AuthorsOpenReview ↗

We address fixed-budget comparisons with frozen sampling and repeated adaptation, component contributions, and runtime, memory, and baseline setup. We also examine reference adherence and clarify human-study selection.

Adaptation versus sampling (Q1; W1)

The next table shows an equal-compute comparison on the same 16 requests used for ablations. All methods receive the same compute budget on the same hardware: two 45-minute attempts per request, then best-of-two selection by overall quality. Frozen best-of-MM generates and scores candidates without model updates. Repeated adaptation runs KINESYS without pruning: independent 30-step runs with VQA + EditReward and one candidate per run.

PQ/SC: perceptual quality/source consistency. Judge scores: 0-10. Adherence: effect transfer, judged by Gemini. Higher is better except LPIPS; bold/underline: best/second best.

MethodPerceptual qualitySource consistencyReference adherenceLPIPS
Frozen best-of-MM9.1889.2198.688‾\underline{8.688}0.433
Repeated adaptation9.375‾\underline{9.375}9.312‾\underline{9.312}8.3440.328
KINESYS9.4389.4069.0520.362‾\underline{0.362}

KINESYS improves all four metrics over frozen sampling. Against repeated adaptation, adherence improves and PQ/SC rise slightly. Under matched compute, adaptation improves effect transfer and source preservation together, beyond sampling and selection alone.

For VFX, we prioritize overall quality and identity preservation: the edit must transfer the requested effect while preserving the source subject and scene composition.

Component ablations (W1)

We run ablations on the same 16 requests as above. Overall quality=(PQ+SC)/2.

ConfigurationOverallSCAdherenceLPIPS
No EditReward objective8.8598.6888.6250.398‾\underline{0.398}
No VQA objective8.7819.188‾\underline{9.188}6.5940.337
Uniform candidate weights8.891‾\underline{8.891}8.9068.594‾\underline{8.594}0.476
Full KINESYS9.0169.4388.1250.414

Full KINESYS leads in overall quality and SC. Removing VQA causes a large adherence drop (8.125 to 6.594) despite better LPIPS; source similarity is insufficient without the requested effect being transferred well. Removing EditReward increases adherence but reduces SC, weakening source preservation. Full KINESYS uses EditReward to weight candidates, while uniform weights give all edits equal emphasis. Full KINESYS improves SC and LPIPS at the cost of adherence, balancing effect transfer with preservation of the artist's source.

Runtime costs (Q1; W1)

Runtime budgets are user-defined. We use the same 16 requests and best-of-two selection at each budget. Artists can select usable intermediate outputs without waiting for the full budget.

Minutes per attemptPQSCLPIPS
309.4588.0730.445
459.4389.4060.362‾\underline{0.362}
609.448‾\underline{9.448}9.396‾\underline{9.396}0.352

PQ changes little; preservation improves from 30 to 45 minutes. At 60, SC is nearly unchanged and LPIPS decreases further.

Reference adherence and Qwen3.5 evaluation (Q2; W2)

Selected baselines on the original 156 shared cases (81 requests), averaged over both seeds. KINESYS leads evaluated open-weight editors in overall quality under both judges and has the lowest LPIPS; adherence remains competitive.

MethodGemini adherenceQwen qualityQwen adherenceLPIPS
Dual-Process7.3727.1018.1000.577
FLUX.2 Klein 4B8.314‾\underline{8.314}7.8768.822‾\underline{8.822}0.579
Step1X-Edit6.9788.2297.7630.449
Nano Banana 28.9368.7589.1020.429‾\underline{0.429}
KINESYS8.3088.580‾\underline{8.580}8.5510.377

For VFX, it is critical to balance adherence with visual quality and source preservation. Figure 1 in the main paper demonstrates this: Dual-Process follows the reference strongly but loses the source building and composition; KINESYS adds puddles while preserving both.

Implementation and baseline configuration (Q1; W1)

We adapt FLUX.2 Klein 4B using bfloat16, four flow steps, guidance 1.0, rank-16 LoRA, and AdamW (learning rate 5e-5). Five independent runs each use three fixed-noise candidates for up to 100 updates; pruning reduces runs 5 to 3 to 2 at checkpoint labels 15 and 30. VQA/EditReward weights are 1 and 2; diversity weight starts at 1 and decays to zero after 60% of the schedule. Three reference-grounded yes/no questions cover placement, realism, and a defining effect property.

Dual-Process uses the same generator and inputs, VQA-only feedback, one candidate per step, and 200 updates. Two RTX A6000 GPUs (48 GB each) host generation and feedback, with peak memory of 24.58 and 44.98 GiB, respectively.

Human study and request selection (Q2; W2)

The blinded study uses 16 randomly selected requests, two per category.

We want to thank you for the detailed review. We wanted to ask whether any concerns remain after our response. If there are areas you would most want us to update to strengthen the paper, we will prioritize them before the discussion period ends.

I would like to thank the reviewers for their response. Authors have addressed my concerns. I have raised my score accordingly.

AuthorsOpenReview ↗

Discussion Phase Summary

Reviewers appreciated the practical importance of the problem: transferring a visual effect while keeping the source scene intact. They recognized the identity-preservation advantage and appreciated the evaluation’s coverage, with independent judges and human comparisons providing evidence beyond the optimized reward.

Concerns addressed and findings

  • Sampling versus adaptation (Vti5, GU27, jikX). The gains go beyond sampling and selection alone. Under matched compute on 16 requests, KINESYS improved all four reported metrics over frozen sampling and achieved higher mean perceptual quality and reference adherence than repeated adaptation. Details: “Adaptation versus sampling” table.

  • Component contributions (Vti5, GU27, jikX). The objectives support complementary editing requirements. Ablations on the same 16 requests showed that removing the VQA objective weakened effect transfer, while removing the EditReward objective lowered the source-consistency score; full KINESYS achieved the highest mean overall quality. Details: “Component ablations” table.

  • Additional judge (Vti5, jikX). The added Qwen3.5 judge family supports the editing-quality advantage over evaluated open-weight systems. Evaluation on 156 shared cases showed that KINESYS leads these systems in overall quality under Qwen3.5, consistent with the original judge. Details: Qwen3.5 results in the rebuttal evaluation table.

  • Runtime and comparison setup (Vti5, GU27, jikX). Longer budgets mainly improved identity preservation. Tests at 30, 45, and 60 minutes showed improved preservation from 30 to 45 minutes, with little change in perceptual quality. We also reported peak memory and clarified baseline inputs and random selection of human-study requests. Details: “Runtime costs,” “Reference image inputs,” and “Human study and request selection.”

Discussion status and remaining disagreement

After our initial replies, we followed up with all reviewers and heard back from everyone except Reviewer 3 (jikX). Reviewer 1 (Vti5) confirmed that their concerns were addressed and increased their score to 3.

Reviewers differed in their assessment of identity preservation: jikX highlighted a clear advantage, while GU27 regarded it as unresolved. Our later follow-up distinguished color or lighting shifts from loss of subject identity, pose, or scene layout. Identity preservation is a measured strength: KINESYS outperforms all evaluated baselines on both source-similarity metrics, LPIPS and TPIPS (main paper Table 1), and the blinded study favored KINESYS for identity preservation over both comparison systems. For VFX, changing the underlying subject or scene can make an otherwise convincing effect unusable, exactly what KINESYS is designed to mitigate.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.