- Rating
- 3(+2)
- Confidence
- 4
- Informativeness
- 2(+1)
- Presentation
- 3(+1)
- Soundness
- 3(+2)
Read review
Paper summary
RefGAP is a training-free method that uses regional reference-attention statistics to strengthen reference conditioning inside edited regions and suppress it elsewhere. Evaluation across seven image and video editors demonstrates consistent identity-fidelity improvements, with preservation trade-offs and mixed results on other tasks.
Critical strengths
-
The paper provides useful empirical insights into reference-attention behavior across different editors. It examines attention allocation across layers and between edited and preserved regions. These observations offer a useful diagnostic perspective on how editing models use reference information.
-
The method is simple and demonstrates practical value across multiple editors. It requires no additional training or test-time optimization. Consistent identity improvements across seven editors, including three excluded from coefficient calibration, support its transferability.
Critical weaknesses
-
The novelty over existing attention-control methods is insufficiently established. AnyScene [1], BooW-VTON [2], and KV-Edit [3] provide related spatial-control precedents, while SaaS [4] already uses regional attention statistics for adaptive reweighting. The paper should clarify the distinct contribution of its particular reference-attention correction.
-
The analysis does not adequately justify the proposed mass-dependent correction. The attention profiles and parameter ablations do not establish that reference-attention mass reliably predicts editing failures or the required correction strength. In particular, the keep-region ablations demonstrate the benefit of suppression without establishing the benefit of its adaptive form.
-
The computational cost analysis does not fully substantiate the claimed tuning savings. Although Table 6 reports inference overhead, the costs of offline statistical analysis, initial calibration, and profiling each new model remain unquantified. These development and setup costs should be reported separately and included in comparisons with simpler alternatives.
-
The writing and presentation need improvement, particularly the clarity of Figure 1. Its unbalanced layout and small annotations make the proposed attention intervention difficult to follow.
References
[1] AnyScene: Customized Image Synthesis with Composited Foreground. [2] BooW-VTON: Boosting In-the-Wild Virtual Try-On via Mask-Free Pseudo Data Training. [3] KV-Edit: Training-Free Image Editing for Precise Background Preservation. [4] Scale Your Instructions: Enhance the Instruction-Following Fidelity of Unified Image Generation Model by Self-Adaptive Attention Scaling.
Questions for authors
Could the authors further explain the novelty of the proposed method and provide more evidence of its effectiveness, addressing the concerns raised in the weaknesses? I also suggest improving the writing and presentation, as this would help readers better understand the paper.
9 comments
Mass-dependent correction
W2. Mass-dependent correction. Thank you; this comment helped us sharpen the claim. We use the measured mass to set the correction strength, not to predict how much an editor will gain or which samples will fail, and Sec. 3.2 now says so explicitly.
Does the mass track the strength an editor needs? We find that it does (App. H.5, Figure 6). RefGAP's online offset ranges from 1.48 on Klein and Qwen-Image-Edit to 3.21 on LoomVideo, which has the lowest mass and is also the editor that needs the largest tuned constant (2.5). How the offset varies across layers and steps matters as well: a constant equal to LoomVideo's mean offset breaks preservation (ΔLPIPS_ne +.018 vs +.003).
Why not a fixed strength? A simple baseline would remove the dynamic part of our framework and use the best fixed strength instead, chosen with the same validation procedure as RefGAP. We tested it, and it cannot adapt to every case:
- the best fixed strength depends on the editor (b from 1.5 to 2.5 when tuned per editor), so every new editor needs its own search;
- one strength shared across editors is too weak on LoomVideo, the lowest-mass editor (ID −.018);
- on the four image editors, the selected shared strength damages the background more (details below).
This is why RefGAP sets the strength from the measured mass. Table 4 compares it with the best fixed edit-side strength chosen per editor (highest ID on validation with ΔLPIPS_ne ≤ .005; only the edit side differs). Without that search, RefGAP comes within .006 in ID (head-swap test set, whole-frame LPIPS):
| Editor | RefGAP ID / Kpt / LPIPS | Tuned b | Constant ID / Kpt / LPIPS |
|---|---|---|---|
| JoyAI | .644 / .085 / .215 | 1.5 | .644 / .075 / .207 |
| LoomVideo | .665 / .098 / .275 | 2.5 | .671 / .098 / .271 |
| Klein | .790 / .153 / .217 | 2.0 | .794 / .171 / .229 |
| Klein-base | .799 / .236 / .261 | 2.0 | .801 / .236 / .260 |
For the shared case, we selected one pair of fixed offsets with RefGAP's rule on the development editors (+1.5 edit, −1 keep) and ran it on the head-swap test set. Here we also count near-copies: outputs whose whole-frame LPIPS to the reference is below 0.2, where the editor reproduces the reference frame instead of editing the input (Figure 11 shows examples from the constant). Because copying inflates ID, one column scores near-copies as 0. Results as RefGAP / constant:
| Editor | Near-copy % ↓ | ID, includes near-copy ↑ | ID, near-copy = 0 ↑ | LPIPS_ne ↓ |
|---|---|---|---|---|
| Klein | 7.0 / 9.4 | .790 / .788 | .730 / .708 | .071 / .083 |
| Klein-base | 6.4 / 7.0 | .799 / .795 | .744 / .736 | .087 / .093 |
| JoyAI | 1.5 / 1.7 | .644 / .644 | .631 / .631 | .079 / .072 |
| LoomVideo | 7.5 / 6.2 | .665 / .647 | .609 / .598 | .136 / .133 |
| VACE | 2.3 / 2.3 | .729 / .736 | .709 / .716 | .047 / .047 |
| Qwen-Image-Edit | 9.1 / 22.3 | .723 / .738 | .645 / .560 | .075 / .129 |
| OmniGen2 | 5.3 / 10.3 | .693 / .745 | .650 / .663 (n.s.) | .192 / .202 |
Bold: better arm (difference ≥ .005 or 1 point, 95% CI excluding 0). The near-copy difference on Klein-base is borderline, and the constant's higher raw ID on Qwen-Image-Edit and OmniGen2 partly comes from copying; on VACE it is .008 higher (App. H.1, Tables 24–26).
On the keep side, suppression itself is needed: the keep-side ablations (Table 5) show that lowering reference attention in the keep region helps, and the edit side alone damages the background (table in W1). Its mass-dependent form is a design choice (Sec. 3.3); we do not claim it outperforms a well-chosen fixed keep offset, and App. C.1 now says so. The evidence for mass-dependent strength is on the edit side, above.
W3. Setup cost.
W3. Setup cost. Thank you for asking for this; App. A now reports the setup cost. Every option below needs some parameter selection. The difference is that RefGAP does it once, while a per-editor constant has to repeat it for each new editor:
| RefGAP | Shared constant pair | Per-editor constant | |
|---|---|---|---|
| Parameter selection | Once, on 4 editors (18 configs, 32 GPU hours), then fixed | Once, on 4 editors (5 × 5 grid of (b, c), 44 GPU hours), then fixed | Repeated on every editor (6 values, Table 4) |
| Adding a new editor | 0.02–1.8 GPU hours (one profiling run) | 0.02–1.8 GPU hours (one profiling run) | 0.1–11 GPU hours (a new sweep) |
GPU hours are estimated from the per-clip times in Table 6.
Novelty over the cited works.
Thank you for the careful review, and we are sorry the paper was hard to follow. To restate the idea in one sentence: RefGAP measures how much attention the edited region pays to the reference image, and adds an offset sized by that measurement, raising reference attention inside the edit and lowering it outside.
W1. Novelty over the cited works. Thank you for pointing out these prior works; the revised Related Work section (Sec. 2) now cites and discusses all four. We believe, however, that RefGAP is a new design that differs significantly from them: none of them both raises attention to the conditioning tokens in the edit region and lowers it in the keep region, with a strength set from measured attention. AnyScene and BooW-VTON learn region control through training; at inference, AnyScene also zeroes foreground-word cross-attention outside a fixed mask, a hard cut rather than a graded reduction, with no edit-side gain. KV-Edit copies the source's keys and values into the background. SaaS is the closest, but it rescales attention only inside the edit region:
A natural question is whether SaaS would work just as well if it were applied to reference tokens instead of text. We tried this: inside our edit mask, we rescaled reference attention with SaaS's rule, choosing its scale ξ = 2 with the same procedure we use for RefGAP. We also ran KV-Edit. The table reports ID ↑ / background LPIPS (LPIPS_ne ↓) on the head-swap test set; ✗ marks a background damaged by more than .005 relative to the base:
Even with a tuned scale, SaaS's rule raises ID by at most .079 on its five editors (JoyAI in Table 30), while RefGAP raises it by +.06 to +.49. The table also shows why both sides are needed: RefGAP's edit side alone damages the background, and the keep side prevents this without extra forward passes. KV-Edit's hard lock can replace the keep side and preserves the background even better, but it doubles the forward passes, needs token-aligned output, forbids any change outside the mask, and raises Kpt on Qwen-Image-Edit (.213 vs .179; Table 30).
We hope this makes the distinction clearer, and we are happy to discuss any of these comparisons further.