acceptodds
Under review as a conference paper at ICLR 2027

IARouter: Identity–Attribute Routing for Compositional Text-to-Image Personalization

Abstract

Multi-reference personalization must preserve each identity and assign every requested attribute to the correct person. Shared attention often mixes these assignments. We introduce IARouter, a training-free method that routes reference identity information to the corresponding heads and textual attributes to the corresponding body regions. First, a reference-free pass locates the people and freezes their head and body ownership maps. Personalized generation first establishes a text-conditioned layout and then activates the two routes. The attribute route redistributes the existing attribute-attention budget across the person-specific body regions. Ordinary text conditioning, the frozen backbone, and self-attention remain unchanged. On our two-subject benchmark ( images per method), our variants achieve the best value in every reported metric. This includes the highest target-versus-decoy binding (0.663) and valid-face rate (0.997) among the evaluated baselines. Paired ablations show that attribute routing improves both binding and complete prompt response. These results support identity–attribute routing for improving two-subject compositional personalization.

Primary area:
generative models
Keywords:
Text-to-Image Generation, Diffusion Models, Image Personalization, Multi-Subject Generation, Controllable Generation

Reviews

Among:
Shape:
1234this paper, Bottom 2%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.33 over 3 reviews, above 2% of the others.
Reviewer_QGm9OpenReview ↗
Rating
1
Confidence
4
Informativeness
1
Presentation
2
Soundness
1
Read review
Paper summary

this paper proposes IARouter, a framework that improves multi-reference personalized text-to-image generation by routing each reference identity and its associated attributes to the correct person. It first performs reference-free localization to obtain head and body ownership maps, then delays identity injection to preserve layout and routes body-local attribute attention to matching regions. Without training or modifying the backbone, IARouter improves subject–attribute binding over identity-only baselines while maintaining strong identity preservation and face validity. Experiments and ablations show gains in binding, prompt response, and compositional control across two-subject settings.

Critical strengths

Clear problem framing. The paper identifies a concrete failure mode in multi-subject personalization: models may preserve identities but still swap or mix person-specific attributes.

Training-free design. IARouter works at inference time, keeping the backbone and reference encoder frozen while adding explicit identity and attribute routing.

Critical weaknesses

1. Architecture Scope. The proposed routing mechanism is tightly coupled with text-conditioned cross-attention in U-Net-based SDXL generation, while many recent personalization systems adopt stronger DiT or MMDiT backbones such as FLUX, SD3, and Qwen-Image. The paper does not discuss or compare against these newer DiT-based personalization pipelines, making the practical relevance and transferability of IARouter unclear. Moreover, the attribute-mixing failure mode demonstrated on SDXL/Nested Attention may be reduced or qualitatively different in stronger DiT models. Therefore, the paper should either validate this issue on DiT-based backbones or better justify why the proposed solution remains broadly applicable.

2. Limited Comparison. Even within the setting of U-Net-based backbones, the comparison experiments in this paper are limited. Several baselines [1–5] focus on multi-face personalization, but the paper does not discuss the advantages of IARouter over these methods. This weakens the novelty claim, since the distinction between IARouter’s identity–attribute routing and prior subject-routing or layout-guided personalization methods is not clearly established.

References

  • [1] High-fidelity Person-centric Subject-to-Image Synthesis. CVPR 2024
  • [2] OMG: Occlusion-friendly Personalized Multi-concept Generation in Diffusion Models. ECCV 2024
  • [3] LoRA-Composer: Leveraging Low-Rank Adaptation for Multi-Concept Customization in Training-Free Diffusion Models. TIP 2025
  • [4] Create Anything Anywhere: Layout-Controllable Personalized Diffusion Model for Multiple Subjects. ICME 2025
  • [5] MS-Diffusion: Multi-subject Zero-shot Image Personalization with Layout Guidance. ICLR 2025
Questions for authors
  • The paper motivates IARouter using attribute mixing in SDXL/Nested Attention. Could the authors provide evidence that this failure mode also persists in stronger recent personalization backbones, especially DiT- or MMDiT-based systems?

  • Most examples focus on clothing or body-local appearance attributes. Could the authors clarify whether the method generalizes to non-clothing attributes, object interactions, poses, or relational attributes between people?

  • Could the authors clarify the interpretation of Bind? Since Bind measures target-versus-decoy phrase ranking rather than actual attribute presence or human-perceived binding correctness, to what extent should improvements in Bind be interpreted as improvements in clothing recognition or attribute assignment?

1 comment
AuthorsOpenReview ↗

Response to Reviewer QGm9: FLUX.2 MMDiT, MS-Diffusion, VLM-Bind & Scope

We thank Reviewer QGm9 for highlighting our "Clear problem framing" and "Training-free design" and raising three incisive questions. Revisions are highlighted in the PDF (bold = 1st, underline‾\underline{\text{underline}} = 2nd).

1. MMDiT Evidence & Transfer to FLUX.2 [klein] 4B (W1 & Q1 -> Prop. 1 p.5, Sec. 4.3, Table 2 p.8, Fig. 5 p.9)

  • Architecture-agnostic formulation: Prop. 1(iv) (p.5, App. G p.25) shows IARouter is not tied to U-Net cross-attention: in MMDiT joint attention over prompt-text, generated-image, and reference-portrait tokens, routing image-to-attribute attention via body masks (Eq. 7) and image-to-reference-portrait attention via head masks preserves exact row-stochasticity (≤1.7×10−6\le 1.7\times 10^{-6}) and phrase budget (≤2.4×10−7\le 2.4\times 10^{-7}).
  • Failure evidence & fix on FLUX.2 [klein] 4B (216 images): Testing 144 canonical (<person1> left, <person2> right) and 72 non-canonical (<person1> right, <person2> left) conditions proves MMDiTs suffer from: (i) a rigid index-to-position shortcut, swapping 93.1% of assignments on non-canonical prompts (Joint = .069; Fig. 5a p.9); (ii) occupational gender erasure ("firefighter" turning female references into men, Fig. 5b p.9); and (iii) eyeglass leakage (Fig. 5c p.9). Zero-training IARouter-DiT fixes all three:
Suite (FLUX.2 [klein] 4B)MethodID Top-1 ↑\uparrowID Sim ↑\uparrowID-Centric Bind ↑\uparrowJoint ↑\uparrowJoint PR ↑\uparrow
Canonical (N=144N=144)Native / + IARouter-DiT.931 / 1.000.400 / .564.931 / 1.000.931 / 1.000.910 / 1.000
Non-Canonical (N=72N=72)Native / + IARouter-DiT.069 / .986.025 / .507.069 / .986.069 / .986.028 / .972
Combined (N=216N=216)Native / + IARouter-DiT.644 / .995.275 / .545.644 / .995.644 / .995.616 / .991

2. Multi-Face Baselines [1–5] & Interpretation of Bind (W2 & Q3 -> Sec. 2 p.2, Table 1 p.7, App. Table 9 p.25)

  • Distinction from [1–5]: Discussed in Sec. 2 (p.2) and Table 9 (p.25), with MS-Diffusion [5] (ICLR 2025) evaluated on all 144 SDXL (Table 1 p.7) and 36 R=3R=3 conditions (Fig. 6 p.9, Table 13 p.26). Unlike multi-pass fusion [1, 2], regional LoRA [3], or whole-person box gating [4, 5], IARouter needs no user boxes, decouples head identity from body text routing, and conserves phrase budget.
  • Clarifying Bind via blind 3-way VLM-Bind (Q3): Since Bind ranks target vs. decoy CLIP scores (.500 chance), unrendered outfits (FastComposer) score .486. Evaluating 1,728+ isolated person crops with 3-way Qwen2-VL-7B (Option A, B, or C = Neither matches) and Joint (ID Top-1 & VLM-Bind) separates outfit presence from swaps:
SDXL (N=144N=144 / 288 subj.)Box-FreeBind ↑\uparrowVLM-Bind ↑\uparrowSwap ↓\downarrowNeither ↓\downarrowID Top-1 ↑\uparrowJoint ↑\uparrow
FastComposer / MuDI (LoRA)Yes.486/.465.181/.323.129/.354.691/.323.462/.559.101/.174
Direct Nested AttentionYes.559‾\underline{.559}.493‾\underline{.493}.372.135.542.264
MS-Diffusion [5] (Auto / Box)Yes/No.542/.483.479/.208.292/.205‾\underline{.205}.229/.587.472/.934.250/.201
Ablation: Identity-onlyYes.538.458.372.170‾\underline{.170}.847‾\underline{.847}.396‾\underline{.396}
IARouter (Ours, step-5)Yes.628.531.281.188.837.458

(1) MS-Diffusion with oracle boxes preserves faces (.934) by flooding torsos with reference features (58.7% Neither, VLM-Bind = .208), whereas IARouter achieves >2.2x higher Joint (.458 vs. .201); (2) over Identity-only, IARouter raises VLM-Bind by +.073 (95% CI [.042, .104]) by cutting swaps from 37.2% to 28.1% (-24.4% relative).

3. Non-Clothing Attributes, Interactions, Overlap & Poses (Q2 -> Sec. 4.5 p.9, App. I Table 12 p.26 & Fig. 15 p.27)

Across 48 conditions (App. Fig. 15 p.27), IARouter generalizes cleanly to body/arm-attached attributes (distant detached props remain bounded by body-mask support):

Method (N=48N=48, R=2R=2)Overall Bind ↑\uparrowVLM-Bind ↑\uparrowJoint ↑\uparrowOverlap/InteractNeck AccessoriesHandheld PropsPoses
Direct Nested Attention.469.427.240.333.500.458‾\underline{.458}.583‾\underline{.583}
Identity-only (w/o Attr-Route).479‾\underline{.479}.458‾\underline{.458}.422‾\underline{.422}.375‾\underline{.375}.542‾\underline{.542}.583.417
IARouter (Ours).719.594.563.750.833.583.708
Reviewer_h2qjOpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
1
Soundness
1
Read review
Paper summary

The paper presents a method to improve ID and attibute handling in multi-reference conditional image generation, for the SDXL+Nested Attention setting. First a reference-free SDXL geneation is completed. Then body-ownerships and head-ownerships are estimated and then used to guide attentions in the final generation phase. The paper paper introduces two metrics to evaluate two-subject reference case, and Table 1 results suggest that the proposed method provides a valuable trade-off across different metrics (though not "the best" in most individual ones). For three-reference cases, only sample outputs are provided.

Critical strengths

The paper's contribution seems to be meaningful (though not all details are clear -- see below).

Proposed metrics & benchmark is valuable for future study on two-reference generation.

Promising results are reported in certain cases.

Critical weaknesses

Overall I find the paper hard to read (despite not necessarily being technically very challenging) for both writing-quality related reasons & lack of details. Some example (but not all) parts & details that made it hard for me to read (as a human being) is listed below.

Figure 1 caption reads: "(...) Columns 3–5 compare direct Nested Attention (M3), our identity-only baseline B1 (M4), and IARouter (M6);" The reader cannot possibly know the abbrevations, therefore, the caption doesn't really provide a good context summary. These abbreviations are first used later in Section 4, but even there, the presentation is confusing because one has to read them w/o knowing what they really are. The first corresponding sentence is "E1 evaluates M0–M4 and M6 over six pairs, 12 prompts, and four seeds, for 1,728 images." at line 281. M0 is explained later in Line 288. Similar puzzling non-linearities appear throughout the paper -- I wished for a moment that I was an LLM.

The paper clearly acknowledges its dependence on nested attention. However, in Section 3, what part already being done in Nested Attention is not clearly delineated.

In addition, I find that some of the crucial technical details are mentioned only briefly in the paper, such as "We associate candidates across checkpoints, retain the stable primary people, order them deterministically, and freeze their head and body masks. No reference image is used during localization, and the localization output is not reused as the personalized output.". Similarly, later in the paper: "We exclude ambiguous scene words and external objects." is said but isn't explained. What is explained here is not really trivial or obvious. A proper explanation is necessary for both clarity purposes and for understanding the robustness (or the fragility) of the proposed technique.

The paper's validation is purely based on SDXL. This is problematic in two ways: (i) Only validated over a single base model (SDXL combined with Nested Attention). (ii) SDXL is hardly the state-of-the-art open-weight generative model these days, and these problems may (have) simply dissapear(ed) with a better pre-trained model. Therefore, given the nature of the problem, i.e. trying to improve the capabilities of pre-trained large scale models, I think it is clearly important to report results over more recent base models, especially those which are ppre-trained to support text + image conditioning (eg. Flux Klein).

The paper text is clear about its focus to two-reference case. However, the title implies a more flexible coverage (e.g. 5-10 people), which is not studied in the paper.

"This separates identity and clothing control while conserving the attribute budget up to numerical toler- ance. Neither the backbone nor the reference encoder is trained, and the full U-Net feature map is not gated." -- Why is the last part emphasized in the Introduction? Is it for computational gains? If yes, it should have been explained (why/how not-gating is important).

Why no qualitative comparison for 3 and more references? Only two sample outputs seem to exist for the two-reference case, and no comparison in that case against strong baselines. No result at all for higher-reference cases.

Questions for authors

"Before denoising, token r is assigned to reference Ir and person slot r. " How is this done? Do you assume that persons are explained in separate sentences?

1 comment
AuthorsOpenReview ↗

Response to Reviewer h2qj: FLUX.2 [klein], Exposition Overhaul & Baselines

We sincerely thank Reviewer h2qj for the detailed reading. Your review identified genuine exposition and framing flaws in our draft, all fixed and highlighted in the revised PDF (bold = 1st, underline‾\underline{\text{underline}} = 2nd).

1. Tested Directly on FLUX.2 [klein] 4B ("Flux Klein") as Suggested (Sec. 4.3, Table 2 p.8, Fig. 5 p.9)

Following your exact suggestion ("pre-trained to support text + image conditioning (eg. Flux Klein)"), we evaluated FLUX.2 [klein] 4B across 216 conditions: 144 canonical (<person1> left, <person2> right) and 72 non-canonical (<person1> right, <person2> left).

  • Why the problem persists: Native FLUX.2 relies on a rigid index-to-position shortcut (Ref 1->left, Ref 2->right). On non-canonical prompts, it places clothing by text but hardcodes identities by index—causing a 93.1% swap (Joint = .069; Fig. 5a p.9). Even on canonical prompts, occupational priors ("firefighter") erase female faces into men (Fig. 5b p.9) and bleed glasses (Fig. 5c p.9).
  • Zero-training IARouter-DiT (Prop. 1iv p.5): In MMDiT joint attention, routing generated-image queries to reference-portrait tokens via head masks and to prompt-attribute spans via body masks (Eq. 7) resolves all three:
Suite on FLUX.2 [klein] 4BMethodID Top-1 ↑\uparrowID Sim ↑\uparrowID-Centric Bind ↑\uparrowJoint ↑\uparrowJoint PR ↑\uparrow
Canonical (N=144N=144)Native / + IARouter-DiT (Ours).931 / 1.000.400 / .564.931 / 1.000.931 / 1.000.910 / 1.000
Non-Canonical (N=72N=72)Native / + IARouter-DiT (Ours).069 / .986.025 / .507.069 / .986.069 / .986.028 / .972
Combined (N=216N=216)Native / + IARouter-DiT (Ours).644 / .995.275 / .545.644 / .995.644 / .995.616 / .991

2. Removed Cryptic Codes, Renamed B2, and Added MS-Diffusion (Fig. 1 p.1, Table 1 p.7)

  • Eliminated nonlinear codes: Forward-referenced M0–M6 and E1/E2 codes are removed from Fig. 1 (p.1) and Sec. 1–4.
  • Clarifying B2 & Table 1: B2 was actually our own step-0 ID-onset ablation of IARouter (with attribute routing enabled!), mislabeled with "B". Renaming it Ablation: IARouter (step-0 ID onset) and adding blind 3-way VLM-Bind, Joint, Layout IoU, and MS-Diffusion (ICLR 2025) (Table 1 p.7) clarifies the trade-off:
SDXL (N=144N=144)Box-FreeBind ↑\uparrowVLM-Bind ↑\uparrowID Top-1 ↑\uparrowID Sim ↑\uparrowJoint ↑\uparrowFaceValid ↑\uparrowLayout IoU ↑\uparrow
Direct Nested-AttnYes.559.493.542.013.264.941.797
MS-Diffusion (Auto/Box)Yes/No.542/.483.479/.208.472/.934.028/.241‾\underline{.241}.250/.201.892/.990‾\underline{.990}.671/.348
Ablation: ID-onlyYes.538.458.847‾\underline{.847}.210.396.990‾\underline{.990}.810
Ablation: step-0 IDYes.677.510‾\underline{.510}.816.245.441‾\underline{.441}.934.656
IARouter (step-5)Yes.628‾\underline{.628}.531.837.207.458.997.805‾\underline{.805}

Step-0 ID injection causes severe layout drift (Layout IoU .805 -> .656, LPIPS .232 -> .389), lowers FaceValid (.934 vs. .997), and loses to step-5 IARouter on both VLM-Bind and Joint.

3. Prompt Parsing (Q1), Nested-Attention Boundary, Localization & Non-Gating (Sec. 3.1–3.5, p.3–5)

  • Q1 (Token assignment & separate sentences): Separate sentences are not required; <person1> and <person2> can appear in one sentence ("<person1> wearing X and <person2> wearing Y") or inverted clauses. Sec. 3.1 (p.3) details our deterministic tokenizer audit: it locates token index srids_r^{\text{id}} of <person$r$> and extracts its local modifier span Ar(P)\mathcal{A}_r(P) up to the clause/conjunction boundary, keeping scene nouns in Sother\mathcal{S}_{\text{other}}.
  • Nested Attention vs. Ours & Localization: Sec. 3.2 (p.3–4) separates Nested Attention's single-subject reader (Eq. 2a) from our multi-reference norm-matched residual (Eq. 2b). Sec. 3.3 (p.4) specifies GroundingDINO+SAM+KeypointRCNN localization, Hungarian tracking, and our time-annealed halo (Eq. 3).
  • Why U-Net features are not gated (Sec. 1 p.2, App. H Table 11 p.26): Hard-gating U-Net features causes seams; routing attention weights with soft halos tolerates ±20%\pm 20\% mask erosion/dilation and SAM-free DINO boxes (.611 Bind).

4. Scope Alignment & R=3R=3 Comparisons (Fig. 6 p.9, App. Table 13 p.26)

We aligned title/abstract scope, added side-by-side R=3R=3 baseline visuals in Main Fig. 6 (p.9), and added a 36-image R=3R=3 stress test (App. Table 13 p.26: IARouter leads Joint at .250 all / .321 valid-3 vs. .056 Direct Nested-Attn and .148 MS-Diffusion).

Reviewer_uL14OpenReview ↗
Rating
2
Confidence
2
Informativeness
1
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes that IARouter is a multi person personalized text to image method that does not require additional training, aiming to solve the problem of identity and clothing attribute mismatch. It first generates and locates characters through pure text, then injects reference identity information into the corresponding head, and assigns clothing descriptions to corresponding body areas while keeping the total amount of attribute attention basically unchanged. Experimental results have shown that facial similarity and attribute binding have been improved compared to baseline.

Critical strengths

This paper disentangle subjects from their attributes and use budget-conserving body-local attribute routing to rebalance attention across attributes while preserving awareness of the remaining text. This is an intuitive and effective method that does not require training.

Critical weaknesses
  1. The main concerns are the additional overhead incurred during inference, as well as whether the final effect is sensitive to the positioning module.
  2. The diversity of prompts and multi reference (three or more) testing in the experimental section are limited.
  3. In addition, the scalability of the method is unknown. sdxl (2024) is already a relatively old model.
Questions for authors
  1. Analysis of the impact of additional inference overhead and positioning module.
  2. More types of prompt words, such as overlapping characters, complex interactions, and testing effects under multiple references.
  3. Is this method effective in the latest DIT based model which has stronger instruction following ability?
1 comment
AuthorsOpenReview ↗

Response to Reviewer uL14: Fast Routing, Mask Sensitivity, Complex & DiT

We thank Reviewer uL14 for recognizing IARouter as "an intuitive and effective method that does not require training" and rating Presentation 3. Revisions are highlighted in the PDF (bold = 1st, underline‾\underline{\text{underline}} = 2nd).

1. Inference Overhead & Positioning-Module Sensitivity (W1 & Q1 -> Sec. 4.4 p.8, App. H Tables 10–11 p.26)

  • Latency, VRAM & IARouter-Fast: Across all 24 D/Y trajectories, step K=10K=10 x^0(10)\hat{x}_0^{(10)} masks reach .860 head IoU / .845 body IoU (95.8% state agreement) with step 29. IARouter-Fast (K=10) stops localization at step 10, runs DINO+SAM once (0.61s), and resumes from cached z(5)z^{(5)} (11 + 25 = 36 UNet steps), cutting latency by -43.8% with 0.00 GB extra peak VRAM on one RTX 5090 (102421024^2, fp16; MuDI needs >720s LoRA training/pair).
  • Insensitivity to localization noise: On the matched 36-image / 72-subject grid (fixing z(5)z^{(5)}), perturbing head/body masks or dropping SAM entirely for rectangular GroundingDINO boxes preserves strong gains as soft halos (Eq. 3 p.4) and budget conservation avoid hard feature seams:
Variant (RTX 5090, 102421024^2, N=36N=36)StepsLatencyVRAMCLIP Bind ↑\uparrowVLM-Bind ↑\uparrowJoint ↑\uparrowLayout IoU ↑\uparrow
Direct Nested Attention (unrouted)304.81s15.93 GB.597‾\underline{.597}.556‾\underline{.556}.319.766
Ablation: Identity-only (w/o Attr-Route)5515.79s15.93 GB.458.472.431.800
IARouter (Full 8-ckpt research pass)5515.93s15.93 GB.569.500.444.793
IARouter-Fast (K=10K=10, cached z(5)z^{(5)})36‾\underline{36}8.95s (-43.8%)‾\underline{\text{8.95s (-43.8\%)}}15.93 GB.583.514.431.866
IARouter-Fast (K=15K=15, cached z(5)z^{(5)})419.37s (-41.2%)15.93 GB.583.528.458.870‾\underline{.870}
Mask Perturb: Dilate +20%+20\% / Erode −20%-20\%5510.45s15.93 GB.611 / .569.514 / .556‾\underline{.556}.472 / .486‾\underline{.486}.868 / .886
Mask Perturb: Shift ±16px\pm 16\text{px} / No-SAM Boxes5510.45s15.93 GB.597‾\underline{.597} / .611.500 / .611.389 / .556.868 / .846

2. Overlapping Characters, Interactions & R=3R=3 Testing (W2 & Q2 -> Sec. 4.5 p.9, Fig. 6 p.9, App. I Tables 12–13 & Fig. 15 p.26–27)

  • Diverse two-subject prompts (N=48N=48): Testing (i) overlapping-arm interactions (clinking glasses, handshakes), (ii) neck/collar accessories, (iii) handheld objects, and (iv) asymmetric poses (App. Fig. 15 p.27), IARouter substantially beats baselines.
  • Three-subject (R=3R=3) comparison (N=36N=36 / 108 subjects): Side-by-side visuals are in Main Fig. 6 (p.9) and metrics in App. Table 13 (p.26). While frozen SDXL generates 3 valid people in 75.0% of text-only runs due to 102421024^2 crowding, IARouter ranks #1 on every binding and joint metric:
Benchmark & SubsetMethodCLIP Bind ↑\uparrowPR ↑\uparrowVLM-Bind ↑\uparrowID Top-1 ↑\uparrowJoint ↑\uparrow
R=2R=2 Complex (N=48N=48; Fig. 15 p.27)Direct Nested Attention.469.167‾\underline{.167}.427.510.240
(Overlap Bind: .375 -> .750)Identity-only (w/o Attr-Route).479‾\underline{.479}.125.458‾\underline{.458}.948‾\underline{.948}.422‾\underline{.422}
IARouter (Ours).719.479.594.958.563
R=3R=3 Stress Test (N=36N=36 all / 2727 valid;Direct Nested Attention.417‾\underline{.417} / .420‾\underline{.420}.056‾\underline{.056} / .074‾\underline{.074}.333‾\underline{.333} / .333‾\underline{.333}.250 / .259.056 / .037
chance =.333=.333; Main Fig. 6 p.9)MS-Diffusion (3 Boxes).315 / .309.028 / .037.167 / .173.861 / .840.148‾\underline{.148} / .148
(Base SDXL 3-person rate =.750= .750)Identity-only (w/o Attr-Route).296 / .284.000 / .000.204 / .235.583‾\underline{.583} / .716‾\underline{.716}.148‾\underline{.148} / .185‾\underline{.185}
IARouter (Ours, R=3R=3).509 / .568.194 / .259.361 / .444.583‾\underline{.583} / .716‾\underline{.716}.250 / .321

3. Scalability to Latest DiT Models (W3 & Q3 -> Sec. 4.3, Table 2 p.8, Fig. 5 p.9)

We evaluated FLUX.2 [klein] 4B (2026 unified MMDiT) across 216 conditions (Fig. 5 p.9):

FLUX.2 [klein] 4B SuiteNative Multi-Ref (ID Sim / Joint)+ IARouter-DiT (ID Sim / Joint)Key Failure Fixed (Fig. 5 p.9)
Canonical (N=144N=144, P1 left).400 / .931.564 / 1.000Female->male uniform erasure (5b), glasses bleed (5c)
Non-Canonical (N=72N=72, P1 right).025 / .069.507 / .98693.1% identity–attribute swap & cloning (5a)
Combined (N=216N=216).275 / .644.545 / .995Architecture-agnostic zero-training transfer

Comments on the paper

1 comment
AuthorsOpenReview ↗

Global Author Response & Summary of Manuscript Revisions

We thank the AC and Reviewers (uL14, h2qj, QGm9) for their constructive feedback. All revisions are highlighted in the updated PDF (10-page main text + Appendix; bold = best, underline‾\underline{\text{underline}} = 2nd best). Without altering our core training-free operator, we added five major validations:

1. Modern MMDiT Validation on FLUX.2 [klein] 4B (Sec. 4.3, Table 2 p.8, Fig. 5 p.9)

We evaluated Black Forest Labs' unified multi-reference MMDiT FLUX.2 [klein] 4B over 216 conditions: 144 canonical (<person1> left, <person2> right) and 72 non-canonical (<person1> right, <person2> left).

  • Diagnostic finding: Native MMDiTs rely on a rigid index-to-position shortcut (Ref 1->left, Ref 2->right). Under non-canonical phrasing, native FLUX.2 places outfits by text but hardcodes identities by index—causing a 93.1% identity–attribute swap (Joint = .069; Fig. 5a p.9), plus female-to-male occupational erasure ("firefighter", Fig. 5b p.9) and eyeglass leakage (Fig. 5c p.9).
  • Zero-training IARouter-DiT (Prop. 1iv p.5, App. G p.25): In MMDiT joint attention, routing generated-image queries to each subject's reference-portrait tokens via head masks and to prompt-attribute spans via body masks (Eq. 7) resolves all three failures without training:
Suite (NN images)Method (4-step MMDiT)ID Top-1 ↑\uparrowID Sim ↑\uparrowID-Centric Bind ↑\uparrowJoint ↑\uparrowJoint PR ↑\uparrow
Canonical (144)FLUX.2 [klein] 4B (Native).931.400.931.931.910
Canonical (144)FLUX.2 + IARouter-DiT (Ours)1.000.5641.0001.0001.000
Non-Canonical (72)FLUX.2 [klein] 4B (Native).069.025.069.069.028
Non-Canonical (72)FLUX.2 + IARouter-DiT (Ours).986.507.986.986.972
Combined (216)FLUX.2 + IARouter-DiT (Ours).995.545.995.995.991

2. MS-Diffusion (ICLR 2025) & Blind 3-Way VLM-Bind (Sec. 4.2, Table 1 p.7, Fig. 3 p.7)

We added MS-Diffusion (both Box-Free and privileged Oracle Boxes) and blind 3-way VLM-Bind (Qwen2-VL-7B: Option A, Option B, or C = Neither matches) plus Joint (ID Top-1 & VLM-Bind):

Method (SDXL, N=144N=144 images / 288 subjects)Box-FreeCLIP Bind ↑\uparrowVLM-Bind ↑\uparrowSwap Err ↓\downarrowID Top-1 ↑\uparrowID Sim ↑\uparrowJoint ↑\uparrowLayout IoU ↑\uparrow
FastComposer / MuDI (LoRA)Yes.486 / .465.181 / .323.129 / .354.462 / .559.153 / .095.101 / .174.261 / .449
Direct Nested AttentionYes.559.493.372.542.013.264.797
MS-Diffusion (Auto-Layout)Yes.542.479.292.472.028.250.671
MS-Diffusion (Oracle Boxes)No.483.208.205‾\underline{.205}.934.241‾\underline{.241}.201.348
Ablation: Identity-only (w/o Attr-Route)Yes.538.458.372.847‾\underline{.847}.210.396.810
Ablation: IARouter (step-0 ID onset)Yes.677.510‾\underline{.510}.288.816.245.441‾\underline{.441}.656
IARouter (Ours, delayed step-5)Yes.628‾\underline{.628}.531.281.837.207.458.805‾\underline{.805}

Whole-box features in MS-Diffusion preserve faces (.934) by flooding torsos with reference appearance, collapsing VLM-Bind (.208; low Swap Err in FastComposer/MS-Diffusion stems from 69.1%/58.7% missing outfits, whereas IARouter cuts true swaps to .281 with >2.2x higher Joint, .458 vs. .201, box-free).

3. -43.8% Faster Inference & Mask Robustness (Sec. 4.4 p.8, App. H Tables 10–11 p.26)

Stopping localization at K=10K=10 (0.860 head IoU to K=29K=29) and reusing cached z(5)z^{(5)} (IARouter-Fast) cuts RTX 5090 latency from 15.93s to 8.95s (36 UNet steps, 0 GB extra VRAM) while keeping gains (Bind .583, VLM-Bind .514). ±20%\pm 20\% mask dilation/erosion, ±16px\pm 16\text{px} shifts, and SAM-free DINO boxes preserve .569–.611 Bind.

4. Interactions, Non-Clothing & R=3R=3 Comparisons (Sec. 4.5, Fig. 6 p.9, App. I Tables 12–13 & Fig. 15 p.26–27)

Across 48 overlapping-arm, accessory, object, and pose conditions (App. Fig. 15 p.27), IARouter lifts Bind .479 -> .719 and Joint .422 -> .563. On a 36-image R=3R=3 stress test (Main Paper Fig. 6 p.9), IARouter ranks #1 on all binding/joint metrics (Joint .250 overall / .321 on valid 3-person base layouts vs. .056 Direct Nested-Attn and .148 MS-Diffusion).

5. Presentation Overhaul (Sec. 1–4, Fig. 1 p.1)

Removed forward-referenced codes (M0–M6, E1/E2), renamed B2 to Ablation: IARouter (step-0 ID onset), and moved full localization, halo (Eq. 3 p.4), and parsing rules into Sec. 3.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.