- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
KINESYS edits an image by transferring a visual effect (smoke, snow, fire, a boat wake, etc.) from a reference image while keeping the source scene intact. For each request, it fine-tunes LoRA weights on FLUX.2 Klein 4B using two frozen judges: a Gemma-4 VQA judge that compares the edit with the reference, and EditReward, which compares it with the source. It runs 5 independent fine-tuning runs of 3 candidates each, prunes the weaker runs, and picks the final checkpoint by EditReward. On a new 81-request benchmark, it has the best source preservation (LPIPS 0.377), beats the open-weight editors. It comes close to Nano Banana 2 on the Gemini and EditScore judges, and in a 16-edit user study it is preferred for identity preservation and roughly matches Nano Banana 2 on effect transfer.
Critical strengths
-
The paper tackles a practical problem and shows a clear advantage in identity preservation. Specifying the effect with a reference image plus yes/no questions is a useful interface for VFX-style edits. KINESYS preserves the source better than every baseline, including Nano Banana 2 (LPIPS 0.377 vs. 0.429; user preference 45% vs. 25%).
-
The evaluation is transparent. The authors flag which metrics are not independent of training, and where a baseline's generator shares a model family with the judge (Nano Banana 2 under the Gemini judge); both are marked in Table 1. They use two seeds for every request and report worst-seed scores in the run ablation (Table 2), and they show that the adapted LoRA generalizes to fresh noise (EditScore 8.91 vs. 8.93).
Critical weaknesses
-
The gains may come from search and selection rather than from fine-tuning. KINESYS uses far more compute than the single-pass baselines (5 runs of 3 candidates each, with up to 100 LoRA steps per run) and then picks its output by EditReward. Yet the main paper has no compute-matched best-of-M baseline and reports no compute cost (it only notes that fine-tuning adds latency). The paper's own numbers suggest that fine-tuning alone may help little. VQA-only fine-tuning of the same generator (Dual-Process) scores 7.09 on Gemini, below the 7.90 of the unadapted editor on the same cases (Table 1), and a single KINESYS run (K = 1) scores far below five runs in Table 2 (7.31 vs. 9.00, on a 16-request subset for which the paper gives no unadapted score; see Question 4). The novelty over Dual-Process is mainly EditReward and the multi-run search, so this comparison is central.
-
Fidelity of the transferred effect to the reference is not measured automatically. None of the automated metrics in Table 1 scores it. The Gemini judge sees the reference but scores only perceptual quality and source consistency, EditScore and EditReward never see it, and LPIPS and TPIPS reward edits that change little. EditReward is also part of the training objective and the checkpoint-selection criterion. Apart from qualitative figures, the effect-transfer claim therefore rests on a 16-edit user study. The main paper reports no significance test and says only that it uses two edits per category, not how they were selected. Against Nano Banana 2 that result is close (48.2% vs. 42.1%).
Questions for authors
-
Please add a compute-matched best-of-M baseline: sample unadapted FLUX.2 Klein and select with the same VQA and EditReward judges. Please also report per-request time and GPU memory.
-
Which direct editors receive the effect reference image, and how? If some never see it, the comparison of effect transfer is not between equal inputs.
-
Please report a metric that compares the edit with the reference for all methods. Examples are held-out VQA with a VLM from a family other than Gemma or Gemini, or the ICE-Bench judge. Please also add quantitative ablations of each feedback term, the diversity term and candidate weighting.
-
What does unadapted FLUX.2 Klein score on the same 16 requests as Table 2? If K = 1 scores below it, what does fine-tuning add beyond selection?
2 comments
We want to thank you for the detailed review. We wanted to ask whether any concerns remain after our response. If there are areas you would most want us to update to strengthen the paper, we will prioritize them before the discussion period ends.
We address whether adaptation improves on frozen sampling at matched compute, report runtime and GPU memory, and clarify reference-image inputs. We also report reference adherence under Gemini and Qwen3.5 and quantitative component ablations.
Adaptation versus sampling (Q1, Q4; W1)
The next table shows an equal-compute comparison on the same 16 requests used for ablations, testing whether the gains in main-paper Table 2 can be explained by sampling and selection alone. All methods receive the same compute budget on the same hardware: two 45-minute attempts per request, then best-of-two selection by overall quality. Frozen best-of-M generates and scores candidates without model updates.
PQ/SC: perceptual quality/source consistency. Judge scores: 0-10. Adherence: effect transfer, judged by Gemini. Higher is better except LPIPS; bold/underline: best/second best.
KINESYS achieves higher mean perceptual quality, source consistency, and reference adherence, with lower LPIPS, than frozen best-of-M sampling. Under matched compute, adaptation improves effect transfer and source preservation together, beyond sampling and selection alone.
For VFX, we prioritize overall quality and identity preservation: the edit must transfer the requested effect while preserving the source subject and scene composition.
Runtime costs (Q1; W1)
Runtime budgets are user-defined. We use the same 16 requests and best-of-two selection at each budget. Artists can select usable intermediate outputs without waiting for the full budget.
PQ changes little; preservation improves from 30 to 45 minutes. At 60, SC is nearly unchanged and LPIPS decreases further. We host the generator and feedback models on two RTX A6000 GPUs (48 GB each), with peak memory of 24.58 GiB for generation and 44.98 GiB for feedback.
Reference image inputs (Q2)
KINESYS, native FLUX.2 Klein 4B, Dual-Process, Nano Banana 2, Qwen-Image-Edit, OmniGen2, and ACE++ receive the source and effect reference as separate image inputs. Step1X receives a side-by-side composite containing both images. ICEdit receives only the source and is therefore labeled as a source-only baseline.
Reference adherence and Qwen3.5 evaluation (Q3; W2)
Selected baselines on the original 156 shared cases (81 requests), averaged over both seeds. KINESYS leads evaluated open-weight editors in overall quality under both judges and has the lowest LPIPS; adherence remains competitive.
For VFX, it is critical to balance adherence with visual quality and source preservation. Figure 1 in the main paper demonstrates this: Dual-Process follows the reference strongly but loses the source building and composition; KINESYS adds puddles while preserving both.
Strong adherence is undesirable when it imports reference content at the expense of the artist's source. Conversely, source similarity alone can favor under-editing. We therefore interpret adherence alongside overall quality and LPIPS, favoring edits that transfer the effect while preserving the source identity and scene composition. Automated scores complement visual and human evaluation.
Component ablations (Q3; W1)
We run ablations on the same 16 requests as above. Overall quality=(PQ+SC)/2.
Full KINESYS leads in overall quality and SC. Removing VQA causes a large adherence drop (8.125 to 6.594) despite better LPIPS; source similarity is insufficient without the requested effect being transferred well. Removing EditReward increases adherence but reduces SC, weakening source preservation. Full KINESYS uses EditReward to weight candidates, while uniform weights give all edits equal emphasis. Full KINESYS improves SC and LPIPS at the cost of adherence, balancing effect transfer with preservation of the artist's source.