acceptodds
Withdrawn from ICLR 2027

InsHuman: Towards Natural and Identity-Preserving Human Insertion

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Human insertion aims to naturally place specific individuals into a target background. Although existing image editing models may have such ability, they often produce many failure cases, including inappropriate human pose in new background, inconsistent number of people, and modified facial identity. Moreover, publicly available human datasets often lack full-body portraits and realistic physical interaction between humans and their background. To address these challenges, we propose InsHuman to achieve natural and identity-preserving human insertion. Specifically, we propose Human-Background Adaptive Fusion (HBAF), which detects foreground humans to obtain a binary mask and applies region-aware weighting to align the human regions between predicted and ground-truth latents, ensuring the person's pose, count, and overall appearance are coherently adapted to the target background. We further propose Face-to-Face ID-Preserving (FFIP), which detects and matches faces between the generated image and the source image in terms of face recognition features to enforce identity consistency for each face.In addition, we propose Bidirectional Data Pairing (BDP) strategy to construct BDP-InsHuman, a high-quality dataset with realistic human-background interactions. Extensive experiments demonstrate that InsHuman achieves significant improvements in generating plausible images while keeping human identity unchanged.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
Human Insertion、Human-Background Adaptation、ID-Preservation、Image Editing

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_Zos1OpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary
  • LoRA fine-tuning of Qwen-Image-Edit-2509 to insert the person(s) from a source image into a target background.
  • HBAF: timestep-adaptive weighting on the human bounding-box region (λ_max = 2.5, T from 900 to 808).
  • FFIP: ArcFace loss between Hungarian-matched faces.
  • BDP: 529 pairs built from real photographs and Nano Banana Pro composites via forward and reverse pairing.
  • Evaluated on 131 pairs against existing editing models, with ablations, a 10-annotator preference study, and transfer to FLUX.2-klein-base-4B.
Critical strengths
  • The fine-tuning is data-efficient: 529 training pairs and 2 A100 GPUs, and the effect holds when the base model is switched to FLUX.2-klein-base-4B (Table 4, CA 0.374→0.160).
  • Reverse pairing, which places real photographs on the ground-truth side, is a sensible idea, and the forward-only vs. reverse-only comparison in Fig. 9 is convincing.
Critical weaknesses
  • The CA metric is inconsistent across tables. Table 1 marks CA as lower-is-better, yet InsHuman has the highest value (96.18) and is marked best; in Tables 2–4 the same model reports 0.1298 (Qwen: 70.99 vs 1.0229). Appendix B gives a single definition (mean absolute error), so it is not possible to tell from the text what Table 1 measures. The Table 3 caption (“FR and IDS”) also does not match its columns.
  • The training data is constructed from Nano Banana Pro outputs, and the paper then reports outperforming Nano Banana Pro (IDS 0.55 vs 0.54, LPIPS 0.222 vs 0.264). There is no variance over the 131 pairs, and no description of how the test pairs are separated from the Unsplash images used for training.
Questions for authors

Q1) Does CA in Table 1 use a different definition from Tables 2–4? Could you align the definition, direction, and scale?

Q2) How were the 131 evaluation pairs separated from the 529 training pairs — are identities, source images, and backgrounds disjoint? Could you also report variance for the main comparisons?

Q3) How were the Nano Banana Pro outputs obtained, and did you confirm that using them as training data is permitted under the applicable service terms?

Reviewer_DFBEOpenReview ↗
Rating
2
Confidence
4
Informativeness
1
Presentation
3
Soundness
2
Read review
Paper summary

InsHuman studies human insertion: placing people from a reference image into a new background while preserving person count, body structure, and facial identity. It fine-tunes an existing image generation model, using three pieces: HBAF, a human-box mask whose loss weight is higher at large timesteps; FFIP, a face-matching identity loss applied at smaller timesteps; and BDP, a bidirectional set built from Nano Banana Pro composites and inpainted backgrounds. The authors also conducted extensive ablation studies to demonstrate their efficacy.

Critical strengths
  1. Given a dataset of only a few hundred pairs, introducing additional training losses—specifically for body regions and facial alignment—proves to be an effective way to stabilize the insertion process.
  2. Ablation studies confirm this: HBAF and FFIP are only effective when used in combination; setting the mask weight too high, however, compromises the body structure.
  3. The experimental results are also quite good.
Critical weaknesses
  1. This method cannot be compared to existing closed-source image editing tools such as GPT Image 2.5 or Nano Banana 2. Since Nano Banana Pro is used solely to synthesize "ground truth" data, the paper fails to demonstrate that InsHuman outperforms the very generator used to produce its annotated data. Furthermore, the "failures" attributed to the open-source editing tools mentioned in the paper are difficult to interpret; the claim that general-purpose editing tools lack prior knowledge regarding "human-scene interactions" remains unverified across various prompt selections.
  2. The "structure-first, detail-refinement" workflow is a standard practice in diffusion models. Moreover, the paper does not establish whether explicit loss functions remain effective when large-scale, real-world paired data is available.
  3. The baseline models selected for comparison lag behind current state-of-the-art open-source and closed-source image editing tools. While the paper discusses some earlier identity-preservation methods, it overlooks systems currently in practical use.
  4. Given that the training objective involves synthetic composite images or inpainted scenes, a lower failure rate may simply indicate that the model has better fitted the distribution of this synthetic data. The paper lacks experiments comparing the performance of InsHuman against open-source or closed-source models using diverse prompts. In the absence of such comparisons—and without supervision from real-world paired data—the proposed method does not demonstrate any clear advantage.
Questions for authors

N/A

Reviewer_8B9qOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper addresses the task of human insertion. The authors fine-tune Qwen-Image-Edit-2509 with LoRA and add three components: HBAF (Human-Background Adaptive Fusion) up-weights the diffusion loss inside detected person boxes at early timesteps; FFIP (Face-to-Face ID-Preserving) applies a cosine loss on ArcFace features of Hungarian-matched faces; BDP (Bidirectional Data Pairing) builds 529 training pairs from Unsplash photos, using Nano Banana Pro composites as forward ground truth and inpainted real photos in reverse. Experimental results demonstrate the effectiveness of the proposed method.

Critical strengths
  1. The studied problem is practically important, and the three failure modes identified (pose-scene mismatch, incorrect person count, identity drift) are well motivated and clearly illustrated.
  2. The proposed method is simple and operates only at training time, introduces no additional inference modules, and demonstrates transferability to a second base model.
Critical weaknesses
  1. Inconsistent definition of the CA metric. Appendix B defines CA as the mean absolute error of the person count (lower is better), however, InsHuman’s value of 96.18 is the largest in the column and is reported as the best. The Table 1 values correspond to k/131 (e.g., 96.18 = 126/131), suggesting exact-count accuracy, whereas Tables 2 through 4 report MAE for the same models (e.g., Qwen-Image-Edit-2509: 70.99 vs. 1.0229).
  2. Omission of a stronger baseline. FLUX.2-klein-base-4B appears in Figures 1, 6, and 14 and in Table 4, but is absent from Table 1. Its zero-shot IDS of 0.61 exceeds InsHuman’s 0.55, which contradicts the claim of best performance on all seven metrics.
  3. Insufficient evaluation of identity preservation. Tables 2 and 3 report only CA and VLM-T, so no IDS result is provided without FFIP; the caption of Table 3 refers to “FR and IDS” but neither is reported. In addition, if the face encoder used in FFIP and the InsightFace model used for IDS share the same checkpoint, the IDS improvement would be circular, and the VLM-T prompt does not evaluate identity.
  4. Limited contribution of the proposed losses relative to the dataset. Zero-shot Qwen-Image-Edit-2509 obtains CA 1.0229 / VLM-T 5.916; plain LoRA fine-tuning on BDP-InsHuman obtains 0.2137 / 7.2115; the full method obtains 0.1298 / 7.8626. The plain fine-tuned model should be included in Table 1. The fixed-λ variant performs nearly identically to the full method, the BDP and mask-design ablations are qualitative only, and all results are based on single runs without variance.
  5. Under-specified and partly circular evaluation protocol. The forward-path ground truth consists of selected Nano Banana Pro outputs, which is subsequently a compared baseline; the VLM judge is Qwen2.5-VL-72B; the construction of the test set and the train/test overlap are not described; and no human-specific insertion method (e.g., TelePortraits, Text2Place) is compared.
Questions for authors
  1. Does Table 1 report exact-count accuracy for CA while Tables 2 through 4 report MAE? Please clarify and report both consistently.
  2. Why is FLUX.2-klein-base-4B excluded from Table 1?
  3. Are the face encoders used in FFIP and in the IDS evaluation the same checkpoint? What is the IDS score without FFIP?
  4. Are the training and test sets disjoint with respect to identities, source images, and scenes?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.