- Rating
- 3
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper presents PiDiT, a drag-based image editing framework built on an in-context diffusion transformer. It represents source–target point pairs as sparse index maps, encodes them into spatially aligned features, and injects these features into the image-token representations. The point encoder and LoRA parameters are trained using flow matching, while inference requires only a source image and point pairs without user-provided masks or test-time optimization. Experiments on DragBench and ReD-Bench report improved point reachability, alongside qualitative examples of joint drag-and-text editing.
Critical strengths
-
Simple and useful interaction interface. PiDiT requires only the source image and sparse point pairs, avoiding the need for users to specify editable masks. Its formulation integrates spatial control into the generation process without iterative point tracking or test-time optimization. This makes the approach practically appealing for interactive editing.
-
Consistent reported improvements in point reachability. Tables 1 and 2 report the lowest mean distance among the evaluated methods on DragBench and across the operation categories of ReD-Bench. The qualitative results also cover varied objects, geometric transformations, and non-rigid edits. Subject to clarification of the sampling protocol, these results provide promising evidence of effective spatial control.
Critical weaknesses
-
Table 3 explicitly reports the best seed for each sample selected by mean distance, and its default LPIPS and mean-distance values match those in Tables 1 and 4. This does not establish that the main tables use the same selection rule. However, the supplied PDF leaves the candidate counts, output-selection criteria, and aggregation across seeds for the main comparisons unclear. The authors should specify these details for each method and distinguish averages over all outputs from prespecified seeds from best-of-N results. Any best-of-N comparison should use documented, comparable selection criteria and candidate budgets where applicable.
-
The paper describes the point encoder and several training settings, but the supplied PDF does not identify the exact pretrained checkpoint or the LoRA rank and target modules. Figure 2 places point-feature addition before the DiT blocks, but its relation to the intermediate hidden states described in Section 4.4 and to the input projection needs clarification. Section 5.1 identifies video frame pairs and generated image edits, but not the specific source collections or editing generator. The supplied PDF also leaves the curation procedure under-specified and does not state whether overlap with the evaluation benchmarks was checked. These omissions hinder faithful reproduction and make it difficult to assess the provenance of the training data and their independence from the evaluation benchmarks.
-
The G_PQ margins over the strongest reported alternatives in Tables 1 and 2 are small, while the supplied PDF gives neither uncertainty estimates nor the VIEScore evaluator configuration. This quality score does not by itself establish identity preservation. The authors correctly acknowledge that LPIPS can favor conservative edits, so its results alone cannot establish which method better preserves identity while completing the requested motion. The abstract and conclusion appropriately describe visual quality as competitive, whereas the stronger preservation wording in the introduction and Figure 3 needs more direct support. Documenting the evaluator and providing a targeted preservation analysis that accounts for achieved drag accuracy would strengthen this narrower claim.
Questions for authors
-
How many candidates are generated per sample for each method, and which tables use selection by mean distance? What criteria select the reported outputs, and how are metrics aggregated across samples and seeds? Can the authors distinguish averages over all outputs from prespecified seeds from any best-of-N results, documenting comparable selection criteria and candidate budgets where applicable?
-
Which backbone checkpoint, LoRA rank, and LoRA target modules are used? Where exactly are point features added relative to the input projection and transformer blocks? Please identify the specific source collections and editing generator, and explain whether and how overlap with the evaluation benchmarks was checked.
-
Which evaluator model, version, and prompting configuration are used for VIEScore? Can the authors provide uncertainty estimates for the reported quality scores? What evidence supports identity and background preservation after accounting for differences in achieved drag accuracy?
2 comments
Thanks for the authors' response. My main concerns have been addressed and thus I keep my positive score.
Thank you for your careful review. We address each weakness together with its corresponding question.
[B1] Evaluation protocol. For every method in Tables 1–4, we generate two candidates per test sample and select the one with the lowest MD. MD, LPIPS, and GPQ come from that same output and are averaged equally across test samples. These are best-of-2 by MD results, rather than averages over all candidates. Within each benchmark, all test samples use the same prespecified seeds.
[B2] Implementation and data provenance.
Backbone and LoRA. We use
black-forest-labs/FLUX.1-Kontext-dev. Base weights, VAE, and text encoders are frozen; we train the point-map encoder, point-feature projection, and LoRA parameters. LoRA rank/alpha/dropout are 32/32/0.05. Targets are:attn.to_q,attn.to_k,attn.to_v,attn.to_out.0,ff.net.0.proj, andff.net.2.attn.to_q,attn.to_k, andattn.to_v.Injection. Packed 64-dimensional point features are projected to 3072 dimensions and added to corresponding image tokens after
x_embedderand before the first DiT block. These projected tokens are the “hidden states” in §4.4, consistent with Figure 2.Data and curation. The 2,213 paired records contain 1,366 video-frame pairs (1,232 from videos collected from the Pexels website; 134 from OpenVid-1M) and 847 generated edits. Video processing includes motion filtering, CoTracker tracking/frame-pair selection, foreground-point/displacement filtering, and manual review.
Generated edits use Pexels source images. Qwen2.5-VL-7B-Instruct assesses suitability and generates geometric instructions; Qwen-Image-Edit-2511 generates targets. Point-pair annotation, filtering, and manual review follow.
Benchmark overlap. We compared DINOv2 embeddings of training source/target images against DragBench and ReD-Bench reference images using cosine similarity. Manual review of high-similarity candidates identified no duplicates or near-duplicates.
[B3] Quality evaluation and preservation.
Evaluator. We use VIEScore's PQ component [3], GPT-4o (
gpt-4o-2024-11-20), and its official zero-shot PQ prompt. Naturalness and artifacts are each scored 0–10; their minimum is the per-image PQ. The full evaluation prompt is provided in the appendix.The additional 95% confidence intervals for GPQ are:
Preservation. Following ConsistentID [1], we use CLIP-I as complementary evidence of source-content consistency. Following BrushEdit [2], we compute background masked LPIPS outside official DragBench editing masks.
PiDiT has the highest CLIP-I in this comparison. Its background LPIPS is numerically close to LightningDrag, DragDiffusion, and DragFlow, and lower than FastDrag and DragNeXt. We interpret these preservation metrics jointly with MD, global LPIPS, GPQ, and qualitative results to assess motion completion, content consistency, background preservation, and visual quality. We describe visual quality as competitive.
[1] Huang et al. ConsistentID: Portrait Generation With Multimodal Fine-Grained Identity Preserving. TPAMI, 2026. [2] Li et al. BrushEdit: All-in-One Image Inpainting and Editing. TPAMI, 2026. [3] Ku et al. VIEScore: Towards Explainable Metrics for Conditional Image Synthesis Evaluation. ACL, 2024.