acceptodds
Withdrawn from ICLR 2027

Rethinking Information Propagation in Diffusion-Based Pose-Guided Person Image Synthesis

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Pose-guided person image synthesis aims to generate realistic human images under arbitrary target poses while preserving appearance consistency. Existing diffusion-based methods have achieved remarkable progress, yet they still suffer from erroneous information propagation, where appearance features are indiscriminately propagated across semantically unrelated body regions under large pose variations. To address this issue, we propose a diffusion-based framework that explicitly rethinks information propagation during denoising. Specifically, selective self-attention suppresses redundant feature interactions through dual-sided channel gating, while the proposed query-gated cross-attention dynamically regulates appearance injection according to semantic relevance, effectively preventing erroneous feature propagation. Furthermore, a transformer refiner equipped with window-based attention and perceptual loss is introduced to further correct residual geometric inconsistencies and recover high-frequency texture details lost during latent-space compression. Extensive experiments demonstrate that our method achieves state-of-the-art generative quality, attaining competitive FID and LPIPS while requiring substantially fewer parameters and lower GPU memory.

Primary area:
generative models
Keywords:
Pose-guided person image synthesis, Information propagation, Selective self-attention, Query-gated cross-attention, Refiner

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_AkyROpenReview ↗
Rating
1
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes a two-stage diffusion framework for pose-guided person image synthesis. It introduces query-gated cross-attention and selective self-attention to suppress erroneous appearance information propagation, followed by a transformer refiner to improve visual details.

Critical strengths

The paper addresses an important problem in pose-guided person image synthesis, with a clear motivation regarding appearance inconsistency under large pose variations.

The proposed framework achieves competitive FID and LPIPS results with relatively low GPU memory consumption.

The paper provides ablation studies and attention visualizations to analyze the proposed components.

Critical weaknesses

Limited methodological novelty. The proposed modules mainly rely on existing gating, attention, and refinement techniques. The authors should better clarify the technical contributions beyond combining these established mechanisms.

Unclear mathematical formulations. In Eq. (4), both query and key gates appear to be computed from Q, while Eq. (6) does not explicitly incorporate the gated features. Furthermore, the query-dependent gating in Eq. (1) does not explicitly model semantic relevance between target queries and reference keys. These formulations need clarification to support the claimed selective information propagation mechanism.

Insufficient evidence from the ablation study. The improvements over the original baseline B1 are relatively small, and the model parameter counts vary substantially across variants. Therefore, the current experiments do not clearly isolate the effectiveness of the proposed attention mechanisms.

Limited evaluation under challenging pose variations. Since the main motivation is to address erroneous information propagation under large pose changes, an evaluation grouped by pose discrepancy would provide more direct evidence for the claimed benefits.

Inconsistent efficiency reporting. The model parameter counts are inconsistent across the main text, Table 1, and Table 2 (885M, 855M, and 772M). Moreover, the paper does not report end-to-end inference latency despite emphasizing computational efficiency.

Questions for authors

See weakness.

Reviewer_NEMsOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper attributes the main bottleneck in PGPIS to misdirected information flow rather than poor pose correspondence. Dense cross-attention injects reference textures into incorrect body regions, standard self-attention introduces redundant dependencies, and latent diffusion fails to preserve fine details. To address these issues, the authors combine query-gated cross-attention, selective self-attention with low-rank channel gates, and a window-attention Transformer. In addition, this mehod also supports local appearance editing.

The author also do a series of experiments to understand their proposed method. these ablations show that standard self-attention degrades performance, while selective self-attention and query gating progressively improve FID and LPIPS. The refiner further reduces LPIPS from 0.1756 to 0.1704 with about 5M additional parameters.

Critical strengths
  1. The paper provides a clear analysis of erroneous propagation in PGPIS. Rather than attributing the problem solely to insufficient pose correspondence, it identifies several issues. Cross-attention can inject irrelevant reference appearance into target regions, particularly under large pose discrepancies; Dense self-attention introduces redundant feature interactions; Latent-space compression removes high-frequency details. Based on these analysis, the author proposed their method, where their main goal is to control how appearance information is propagated and recover details lost during generation.

  2. The proposed modules are designed to address these issues at different stages. QGCA applies query-dependent gating to value features before cross-attention aggregation, allowing each spatial location to control the amount of reference appearance injected. SSA uses dual-sided channel gating to suppress redundant feature interactions in self-attention. A separate refiner with window-based attention and a scaled residual connection are used to recover fine details. Although gating, selective attention, and refinement are publised in previous work, this paper makes a reasonable combination to address their findings.

Critical weaknesses
  1. The paper identifies erroneous propagation as a key issue and proposes SSA and QGCA to address it. However, the ablation results mainly show improvements in generation metrics, without directly demonstrating how these methods reduce the propagation errors discussed in the paper.

  2. The ablations show that SSA and QGCA improve performance, but it is unclear whether the proposed designs are better than other ways of implementing gating or selective attention. Moreover, the improvement from B1 to B4 is small in FID (6.61 to 6.57), while the refiner brings more gains in LPIPS (0.1756 to 0.1704) with about extra 5M parameters.

Questions for authors
  1. The paper identifies irrelevant appearance injection and redundant self-attention as sources of erroneous propagation, while attributing over-smoothing to high-frequency loss during latent compression. How much does each contribute to the final appearance inconsistency? Can the authors distinguish cases dominated by high-frequency loss, which QGCA and SSA may not address? if not, the diagnosis of erroneous propagation may be incorrect.

  2. QGCA predicts gates from query features. Fig. 5 provides some results of region-specific gating, but how reliable is it under severe occlusion, unusual poses, or out-of-distribution inputs? Have the authors examined such failure cases or considered lightweight semantic priors to improve robustness?

Reviewer_gETbOpenReview ↗
Rating
2
Confidence
3
Informativeness
1
Presentation
2
Soundness
2
Read review
Paper summary

The paper addresses an important problem in pose-guided person image synthesis by focusing on erroneous appearance propagation under large pose variations. The proposed framework achieves strong quantitative and qualitative results, outperforming existing methods in FID and LPIPS while substantially reducing model parameters and GPU memory usage.

Critical strengths

The paper addresses a practical problem in pose-guided person image synthesis by focusing on erroneous appearance propagation under large pose variations. The proposed framework achieves strong experiment results, outperforming existing methods in FID and LPIPS while substantially reducing model parameters and GPU memory usage.

Critical weaknesses
  • W1. Limited Methodological Novelty. The methodological novelty of the proposed framework appears somewhat limited. The two main attention modules, SSA and QGCA, primarily extend conventional attention mechanisms through relatively simple projection and gating operations, while the transformer refiner employs established techniques such as window-based self-attention and perceptual loss. Since these components are largely based on existing techniques, their technical distinctions from prior approaches, particularly gating-based attention methods, are not sufficiently clarified or validated.
  • W2. Insufficient Validation of the Central Claim on Information Propagation. The paper argues that erroneous information propagation is a primary limitation of existing diffusion-based methods, but this central claim lacks direct quantitative validation. Although attention visualizations provide qualitative insights, the evaluation mainly relies on general image quality metrics such as FID, LPIPS, SSIM, and PSNR. More targeted evaluations of semantic feature leakage or region-level appearance consistency are needed to demonstrate that the proposed modules effectively suppress erroneous information propagation.
  • W3. Limited Experimental Generalization. All quantitative experiments are conducted exclusively on the DeepFashion dataset, limiting the assessment of generalization. Although the paper emphasizes appearance inconsistency under large pose variations, it does not systematically evaluate performance across different pose-change magnitudes or occlusion levels. The effectiveness of the proposed method on diverse subjects, challenging pose configurations, and in-the-wild images therefore remains unclear.
  • W4. Limited Improvements and Insufficient Ablation Analysis. The performance improvements attributable to individual components are relatively modest. Although SSA outperforms conventional self-attention, its FID is slightly worse than the original baseline (6.63 vs. 6.61), while QGCA provides only a marginal improvement (6.63 to 6.57). Furthermore, the refiner improves LPIPS but not FID, and comparisons with alternative gating strategies or refinement architectures are missing, making it difficult to establish the effectiveness of the proposed designs.
Questions for authors

I would appreciate it if the authors could address the concerns raised in the Weaknesses section, particularly regarding methodological novelty, direct validation of information propagation, limited generalization, and the effectiveness of individual components.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.