- Rating
- 2
- Confidence
- 4
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
AnatoGuard proposes a pose-semantic diffusion framework for human amodal instance segmentation. Given an occluded image and a visible-person mask, it fine-tunes LocLLM to predict full-body keypoints, renders them as a pose spatial condition, and uses a conditional latent diffusion model to reconstruct a complete person image. The amodal mask is then extracted from the generated image. In parallel, Qwen2.5-VL produces six types of attributes: visible geometry, pose state, scene relation, hidden-completion cue, occlusion relation, and appearance. CLIP features of these attributes are combined with pose-derived soft body-part supports to form an Attribute-Part Spatial Residual, which is added to the image latent. On Pix2-Human, AnatoGuard reports 87.24 mIoU_amodal and 70.02 mIoU_occ, compared with 85.78 and 67.12 for pix2gestalt. In zero-shot evaluation on 3DOH50K, it reports 78.24 and 20.17.
Critical strengths
-
The paper addresses a meaningful problem. Recovering a complete human mask from an occluded image requires inferring body structure that visible appearance alone cannot provide. Using full-body pose as an explicit spatial condition is a sensible approach.
-
The conditioning design has a clear structure. The pose raster, visible-person mask, image token, generic-human prompt, and Attribute-Part Spatial Residual enter the diffusion model through distinct paths, with separate roles for structural and semantic cues.
-
The experiments include useful controlled comparisons. Table 3 separates the effects of LocLLM fine-tuning, the generic prompt, and the Spatial Residual, while Table 4 compares several pose representations. Results on Pix2-Human and zero-shot evaluation on 3DOH50K test the method on two datasets.
Critical weaknesses
-
The evidence for the semantic contribution is limited. In Table 3, adding the Attribute-Part Spatial Residual to the fine-tuned pose model with the generic prompt increases mIoU_amodal from 87.14 to 87.24 and mIoU_occ from 69.64 to 70.02. The paper does not ablate the six attribute groups or compare Qwen2.5-VL attributes with simpler text or non-text conditions. These results therefore do not show which attributes contribute, or whether the gains come from semantic content rather than the additional conditioning capacity.
-
The paper does not quantify the accuracy of its full-body pose estimates. LocLLM is fine-tuned on 8,000 Pix2-Human images with full-body keypoint targets, and its predictions provide the pose spatial condition and the soft body-part supports used by the Attribute-Part Spatial Residual. The experiments report final mask metrics and include qualitative pose examples, but provide no keypoint error, oracle-pose comparison, or controlled pose perturbation. This leaves the effects of pose estimation, spatial binding, and diffusion generation difficult to separate. The claim that semantic attributes compensate for unreliable pose is also not directly tested.
-
The paper describes safeguards for generating and filtering Qwen2.5-VL attributes, but does not measure their accuracy or spatial binding. Appendix B specifies evidence-first prompts, source restrictions for appearance attributes, rule-based filtering, and deterministic mapping from phrases to seven learned support maps. However, no attribute accuracy, hallucination rate, support-map quality, or sensitivity to incorrect attributes is reported. This gap is especially relevant to the hidden-completion cue, which concerns body regions not directly visible in the image. The reported results establish a gain from the full pose-semantic model, but do not show whether that gain depends on correctly grounded attributes.
-
The aggregate evaluation does not establish how performance changes with occlusion severity. The paper emphasizes severe or heavy occlusion, but Tables 1 and 2 report only aggregate mIoU_amodal and mIoU_occ. They provide no breakdown by visible ratio, occlusion severity, body part, or pose-estimation difficulty. The selected qualitative examples illustrate heavily occluded cases, but do not show performance trends across occlusion levels.
-
The mask extraction and baseline descriptions leave reproducibility gaps. Section 3.1 says that the amodal mask is extracted from the completed image using a fixed post-processing step, but does not specify the threshold, image channel or score used, or connected-component processing. The reported segmentation results cannot be reproduced from the described extraction procedure alone.
Questions for authors
See weaknesses