acceptodds
Under review as a conference paper at ICLR 2027

AnatoGuard: Anatomy-Guided Pose–Semantic Diffusion for Human Amodal Completion

Abstract

Human amodal instance segmentation aims to recover the complete extent of a person, including body regions hidden by occlusion. This task is challenging because invisible regions cannot be directly observed. We observe that the anatomical structure of the human body allows a detected full-body pose to serve as an explicit spatial prior for locating occluded body parts. However, pose alone provides limited information about appearance, scene context, and the continuation of hidden body regions, and its reliability may decrease under severe occlusion. We introduce AnatoGuard, an anatomy-guided pose-semantic diffusion framework for human amodal completion. The detected pose is rendered as a spatial condition and jointly provided with the occluded image and visible-person mask. To complement unreliable pose predictions, a Qwen-based module extracts typed semantic attributes related to appearance, contextual relations, and hidden-body continuation. These attributes are localized using pose-derived body-part supports and transformed into spatial semantic residuals, providing complementary evidence beyond pose. The proposed framework therefore combines anatomical structure with semantic cues for robust amodal completion. We evaluate pose-only and pose-semantic variants through controlled comparisons and component ablations, with particular emphasis on severe occlusion.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
Amodal Instance Segmentation, Pose-Guided Diffusion, Spatial Semantic Conditioning

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 4 reviews, above 27% of the others.
Reviewer_KZDkOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

AnatoGuard proposes a pose-semantic diffusion framework for human amodal instance segmentation. Given an occluded image and a visible-person mask, it fine-tunes LocLLM to predict full-body keypoints, renders them as a pose spatial condition, and uses a conditional latent diffusion model to reconstruct a complete person image. The amodal mask is then extracted from the generated image. In parallel, Qwen2.5-VL produces six types of attributes: visible geometry, pose state, scene relation, hidden-completion cue, occlusion relation, and appearance. CLIP features of these attributes are combined with pose-derived soft body-part supports to form an Attribute-Part Spatial Residual, which is added to the image latent. On Pix2-Human, AnatoGuard reports 87.24 mIoU_amodal and 70.02 mIoU_occ, compared with 85.78 and 67.12 for pix2gestalt. In zero-shot evaluation on 3DOH50K, it reports 78.24 and 20.17.

Critical strengths
  • The paper addresses a meaningful problem. Recovering a complete human mask from an occluded image requires inferring body structure that visible appearance alone cannot provide. Using full-body pose as an explicit spatial condition is a sensible approach.

  • The conditioning design has a clear structure. The pose raster, visible-person mask, image token, generic-human prompt, and Attribute-Part Spatial Residual enter the diffusion model through distinct paths, with separate roles for structural and semantic cues.

  • The experiments include useful controlled comparisons. Table 3 separates the effects of LocLLM fine-tuning, the generic prompt, and the Spatial Residual, while Table 4 compares several pose representations. Results on Pix2-Human and zero-shot evaluation on 3DOH50K test the method on two datasets.

Critical weaknesses
  1. The evidence for the semantic contribution is limited. In Table 3, adding the Attribute-Part Spatial Residual to the fine-tuned pose model with the generic prompt increases mIoU_amodal from 87.14 to 87.24 and mIoU_occ from 69.64 to 70.02. The paper does not ablate the six attribute groups or compare Qwen2.5-VL attributes with simpler text or non-text conditions. These results therefore do not show which attributes contribute, or whether the gains come from semantic content rather than the additional conditioning capacity.

  2. The paper does not quantify the accuracy of its full-body pose estimates. LocLLM is fine-tuned on 8,000 Pix2-Human images with full-body keypoint targets, and its predictions provide the pose spatial condition and the soft body-part supports used by the Attribute-Part Spatial Residual. The experiments report final mask metrics and include qualitative pose examples, but provide no keypoint error, oracle-pose comparison, or controlled pose perturbation. This leaves the effects of pose estimation, spatial binding, and diffusion generation difficult to separate. The claim that semantic attributes compensate for unreliable pose is also not directly tested.

  3. The paper describes safeguards for generating and filtering Qwen2.5-VL attributes, but does not measure their accuracy or spatial binding. Appendix B specifies evidence-first prompts, source restrictions for appearance attributes, rule-based filtering, and deterministic mapping from phrases to seven learned support maps. However, no attribute accuracy, hallucination rate, support-map quality, or sensitivity to incorrect attributes is reported. This gap is especially relevant to the hidden-completion cue, which concerns body regions not directly visible in the image. The reported results establish a gain from the full pose-semantic model, but do not show whether that gain depends on correctly grounded attributes.

  4. The aggregate evaluation does not establish how performance changes with occlusion severity. The paper emphasizes severe or heavy occlusion, but Tables 1 and 2 report only aggregate mIoU_amodal and mIoU_occ. They provide no breakdown by visible ratio, occlusion severity, body part, or pose-estimation difficulty. The selected qualitative examples illustrate heavily occluded cases, but do not show performance trends across occlusion levels.

  5. The mask extraction and baseline descriptions leave reproducibility gaps. Section 3.1 says that the amodal mask is extracted from the completed image using a fixed post-processing step, but does not specify the threshold, image channel or score used, or connected-component processing. The reported segmentation results cannot be reproduced from the described extraction procedure alone.

Questions for authors

See weaknesses

Reviewer_8UiTOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper introduces AnatoGuard, a framework for recovering complete human appearance and amodal segmentation masks from partially occluded images. It uses a fine-tuned LocLLM to predict full-body keypoints and Qwen2.5-VL to extract semantic attributes describing appearance, pose, and occlusion relationships. The proposed Attribute–Part Spatial Residual module binds these attributes to pose-derived body regions and injects them into the image latent. Together, the skeletal and semantic conditions guide a diffusion model to generate a complete person image, from which the amodal mask is extracted. Experiments on Pix2-Human and zero-shot evaluation on 3DOH50K show improvements over the evaluated baselines.

Critical strengths

The Attribute–Part Spatial Residual block is an interesting design. It binds semantic information to relevent human body regions. This allows localized semantic guidance to the skeleton control during the completion process.

Critical weaknesses
  1. Most of the components build on existing models. And the overall framework largely follows an established pose-conditioned diffusion model with inpainting block. The paper needs further clarification what novelty this integration can bring to the diffusion society, especially for human body generation and restoration.
  2. The experiments do not take general-purpose inpainting models or pose-conditioned completion baseline using the same predicted skeletons into consideration. Without such comparisons, It is very hard to determine whether the proposed framework offers benefits beyond other existing completion methods, as well as the significance of the Attribute–Part Spatial Residual part.
Questions for authors
  1. How does the proposed framework differ from a standard pose-conditioned inpainting pipeline?
  2. How does the Attribute–Part Spatial Residual compare with simpler alternatives, such as incorporating the same semantic attributes through cross-attention?
  3. How does the framework handle inaccurate skeleton predictions under severe occlusion, particularly when semantic attributes are spatially localized using those predictions?
Reviewer_fc83OpenReview ↗
Rating
2
Confidence
4
Informativeness
1
Presentation
2
Soundness
3
Read review
Paper summary
  • The paper focuses on the problem of human amodal instance segmentation; given an rgb image and binary mask for the visible human, predict the mask for the complete (occluded + visible) human.
  • The key idea is to leverage a conditional latent diffusion setup to predict the amodal masks. The model is conditioned on 2d keypoints, CLIP features, semantic attributes and the input mask.
  • Additionally, the setup uses LLM backbones and Qwen2.5-VL for pose and semantic attribute extraction.
  • Datasets: Pix2Human (30k human images) and 3DOH50K
  • Baseliens: pix2gestalt (base), MaskRCNN, oRCNN, BCNet, C2F etc
  • The results show that AnatoGuard outperforms pix2gestalt by +1.46 / +2.90 mIoU (amodal / occluded) on Pix2-Human and +4.01 / +3.13 on 3DOH50K.
Critical strengths
  • The proposed method uses modern components to address the problem. Amodal segmentation is a generative task, so the use of a latent diffusion model is a natural choice. Additionally, the method uses VLM to benefit from the large scale prior of the visual models for generalizable segmentation.

  • The key idea of the paper is simple, cleanly presented and intuitive.

Critical weaknesses
  • Limited technical novelty: The core setup of the work is very similar to PHAC (CVPR 2026) and SDHDO (ACM MM 2025). The paper does cite PHAC, but argues that it requires user prompts. However, PHAC also supports a prompt-free mode, and the full-body keypoints predicted by AnatoGuard could directly serve as PHAC's pose prompt, making it fully automatic. Moreover, the combination of pose + VLM attributes + diffusion has already been explored in SDHDO by the same authors as PHAC, which uses occluded-joint heatmaps and VQA-derived human attributes to guide Stable Diffusion for human de-occlusion.

  • Marginal gains from the semantic module: The conditioning on the semantic attributes is presented as one of the important components of the proposed method. However, the Qwen spatial residual gives +0.1 / 0.38. mIoU (Table 3) on top of pose + generic prompt. The gains are incremental in comparison to the gains from finetuned pose (+1.21/+2.28).

  • Missing evaluations on established human amodal benchmarks: The main evaluations are done on two datasets.

  1. Pix2-Human, a person subset of the pix2gestalt dataset. Pix2-Human consists of synthetic occluders pasted on top of SA-1B images
  2. 3DOH50K: Although 3DOH50K has real occlusions, its GT amodal masks are rendered from SMPL meshes (excludes clothing and hair). To strengthen the case of the paper, it would be great to evaluate AnatoGuard on the AHP dataset (Zhou et al. CVPR21) - another popular dataset with manually curated annotations. This allows for an opportunity to directly compare with the closest baselines too like PHAC and Zhou et al (CVPR 21), both of which report on it.
  • Presentation: The visual presentation of the paper needs work, especially the figures. In my opinion, they do not yet meet the bar for ICLR and need further polishing. For example, Fig. 2 and Fig. 3 are cluttered, with small, bold, rotated text. The qualitative results (Fig. 4, Fig. 5) consist of only a few small binary-mask examples, and the people in Fig. 5 are barely visible.
Questions for authors
  • Would love to know the authors thoughts on how they would position the technical contributions of this work compared to the other papers.
  • The marginal gains of the core components of the proposed method.
  • Evaluations with the closest baselines on an additional benchmark.
  • Improved qualitative results and presentation.
Reviewer_FKJgOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

This paper proposes AnatoGuard, an anatomy-guided pose-semantic diffusion framework for human amodal instance segmentation. The method aims to recover complete human silhouettes, including body regions hidden by occlusion, by incorporating explicit anatomical and semantic guidance into a latent diffusion model.

The proposed framework consists of two main components. First, a LocLLM-based pose estimator is fine-tuned to predict full-body keypoints from occluded human images. The predicted keypoints are rendered into a skeleton image and incorporated as a spatial condition into the diffusion model. Second, Qwen2.5-VL extracts semantic attributes describing human appearance, pose, scene context, and hidden-body continuation. These attributes are associated with pose-derived body-part support maps and converted into spatial semantic residuals, which are injected into the diffusion model to complement potentially unreliable pose estimates.

The authors evaluate the proposed method on Pix2-Human and 3DOH50K. AnatoGuard achieves 87.24% amodal mIoU and 70.02% occluded-region mIoU on Pix2-Human, outperforming the pix2gestalt baseline by 1.46 and 2.90 percentage points, respectively. The method also demonstrates improvements on 3DOH50K under a cross-dataset evaluation setting.

Critical strengths
  • The paper addresses an important challenge in human amodal segmentation: recovering anatomically plausible body structures when substantial portions of a person are occluded. Incorporating full-body pose as an explicit structural prior is a reasonable and well-motivated approach, particularly because appearance-based generative models may produce incomplete or anatomically inconsistent results under severe occlusion.
  • The proposed method achieves the best reported performance on both Pix2-Human and 3DOH50K in terms of amodal and occluded-region mIoU. The improvements on 3DOH50K, where the model is evaluated without target-domain fine-tuning, are particularly encouraging and suggest that pose-based anatomical guidance may improve cross-dataset generalization.
Critical weaknesses
  • The main idea of incorporating human pose into a diffusion model is closely related to existing pose-conditioned generation and segmentation frameworks. The proposed method primarily combines established components, including LocLLM, Qwen2.5-VL, CLIP, and latent diffusion, with a spatial residual mechanism. Although the attribute-part spatial residual is a potentially interesting contribution, its methodological novelty and advantages over simpler semantic-conditioning alternatives are not sufficiently established. In particular, the paper does not compare the proposed spatial residual with conventional global semantic conditioning, direct attribute concatenation, or cross-attention-based attribute injection.
  • According to Table 3, incorporating fine-tuned pose conditioning improves occluded-region mIoU from 67.12% to 69.40%, representing a gain of 2.28 percentage points. Adding the generic prompt further improves performance to 69.64%, while the proposed spatial residual increases it to 70.02%. Thus, the additional improvement attributable to the spatial residual is only 0.38 percentage points in occluded-region mIoU and 0.10 percentage points in amodal mIoU over the pose-conditioned model with the generic prompt. These results suggest that most of the overall improvement comes from fine-tuned pose conditioning rather than the proposed semantic module. Furthermore, the paper does not report repeated experiments or statistical variability, making it difficult to assess the significance of these relatively small gains.
  • One of the central motivations of the paper is that semantic attributes can compensate for unreliable pose predictions under severe occlusion. However, the experimental results do not directly validate this claim. The paper does not systematically investigate how segmentation performance changes with pose estimation accuracy, nor does it demonstrate that the semantic residual becomes particularly beneficial when pose predictions are inaccurate. Moreover, because the spatial semantic residual is localized using pose-derived body-part supports, errors in the predicted pose may also affect the localization of semantic information. This raises an important question about whether the proposed semantic mechanism can actually correct pose-related errors or instead propagates them into the diffusion model.
  • Although the paper includes component-level ablations, several important design choices remain insufficiently investigated. For example, the paper does not evaluate the individual contributions of the six semantic attribute groups, the importance of pose-derived spatial binding, or the effectiveness of the proposed residual formulation compared with simpler alternatives. Additionally, Table 3 does not include a configuration using the spatial residual without the generic prompt. Therefore, the independent contribution of each semantic-conditioning component and their potential interactions cannot be fully determined.
  • Although robustness under severe occlusion is a central motivation, the quantitative results are reported mainly as aggregate mIoU values. The paper does not provide a systematic performance breakdown according to occlusion severity. Furthermore, the framework involves multiple computationally expensive components, including LocLLM, Qwen2.5-VL, CLIP, and a diffusion model. However, the paper does not report inference latency, computational complexity, or memory requirements.
  • The Pix2-Human dataset is constructed from person images in the pix2gestalt dataset, and the paper states that Sapiens-0.3B is used to generate pseudo-labels. However, the precise roles of these pseudo-labels in training and evaluation are not sufficiently explained. Furthermore, the paper evaluates a reproduced SAMEO-style baseline rather than the original SAMEO implementation. While this is understandable given the reported unavailability of the official implementation, the extent to which the reproduced baseline faithfully represents the original method remains uncertain.
Questions for authors

Please refer to the weaknesses section.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.