acceptodds
Under review as a conference paper at ICLR 2027

PhasePose: Wavelet Phase Decomposition for Pose Estimation on Low-Resolution Feature Maps

Abstract

Accurate human pose estimation is commonly associated with restoring a high-resolution feature map. We show that this requirement is not intrinsic to the task. In a controlled degradation study, heatmap regression collapses when its output lattice is reduced from \(64\times48\) to \(8\times6\), whereas coordinate-based decoders remain near their full-resolution accuracy. The remaining obstacle is therefore not simply a small lattice, but how downsampling mixes sampling phases and how a decoder represents localization uncertainty. We propose PhasePose, an upsampling-free top-down estimator that addresses both issues. First, a critically sampled Haar analysis decomposes the four sampling phases of a stride-16 feature into one low-frequency and three directional-detail bands. The low-frequency band is kept fixed, while the three detail bands admit a grouped learnable \(SO(3)\) rotation before being stored as compact correction memory; no dense feature map is reconstructed. Second, a weight-shared spectral corrector represents each SimCC prediction by expectation, entropy, and a DCT-anchored spectrum whose analysis basis can adapt through a zero-start low-rank residual. Entropy can further condition a band-wise prior over the cached phase memory, while skeletal message passing and learned step embeddings support three recurrent gated logit corrections. Feature diagnostics show that the Haar anchor reduces mean inter-band correlation from 0.31 to 0.03, while the subsequently adapted directional-detail bands retain 48.1% of the illustrated response energy. On COCO val2017, PhasePose obtains 73.8 AP with ResNet-50 and 76.7/77.5 AP with CSPNeXt-L at \(256\times192\)/\(384\times288\). It also reaches 56.2 AP from a \(64\times48\) input, 92.5 PCKh on MPII, 74.7 AP on CrowdPose, and 64.5 whole-body AP on COCO-WholeBody.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
Human Pose Estimation, Low-Resolution Feature Maps, Wavelet Phase Decomposition, SimCC, Spectral Correction

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_nmZyOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
1
Soundness
2
Read review
Paper summary

This paper proposes PhasePose, an upsampling-free top-down estimator for pose estimation in low-resolution feature maps. Authors combine Haar analysis, SO(3) rotation, DCT, Simcc and etc for pose estimation. Experiments are preformed on COCO, MPII, and others.

Critical strengths
  • Experimental results show that the method achieves good performance.
  • Authors perform ablation studies.
Critical weaknesses
  • The motivation for this paper appears somewhat weak. The authors claim to address the problem of low-resolution pose estimation but directly employ a large number of frequency-domain-related components, such as Haar analysis, DCT, and SO(3). Why are these techniques relevant to the problem at hand?
  • Although the frequency domain is presented as the primary approach to solving the low-resolution feature problem, the authors actually utilize frequency-domain information from an earlier stage, which does not demonstrate the effectiveness of the proposed frequency-domain analysis for low-resolution scenarios.
  • The decoupling of feature map resolution from output coordinate accuracy is already a key premise of coordinate-based classification methods such as SimCC. It is unfair to use this as a basis for analyzing the degradation of heatmaps, since heatmaps inherently benefit from high resolution.
  • For low-resolution results, the backbones used by different methods vary. Can the authors provide results based on HRNet or the VIT backbone?
  • The overall framework is complex and lacks intuitiveness.
Questions for authors
  • Please clarify the motivation more deeply.
  • Please provide experimental results using frequency information from low resolution feature maps.
  • Please provide the results under different backbone.
  • There seems to be a problem with the citation format, which makes the paper difficult to read.
2 comments
AuthorsOpenReview ↗

Response to Reviewer nmZy (Part 1/2): Motivation, Heatmap Fairness, and Presentation Clarity

We sincerely thank the reviewer for the constructive comments regarding our motivation, frequency-domain design, fairness of heatmap comparisons, backbone diversity, and presentation clarity. We have uploaded a revised manuscript and supplementary PDF addressing your feedback, and we are also running additional experiments on deepest-stage frequency analysis and low-resolution ViT/HRNet backbones, which we will report in a follow-up comment during the discussion period. Due to the character limit, Part 1 addresses Weaknesses 1, 3 & 5 and Questions 1 & 4; Part 2 addresses Weaknesses 2 & 4 and Questions 2 & 3.

1. Fair Motivation Without Heatmap-Degradation Premise (Weakness 3 & Question 1)

We completely agree with the reviewer that decoupling coordinate precision from feature-map resolution is already a core premise of SimCC, and that using heatmap degradation on tiny grids as the main motivation is unfair because heatmaps inherently benefit from high spatial resolution.

In the uploaded revision, we removed the heatmap degradation study from the main paper (moving it to Appendix A solely as supplementary context) and rewrote Section 1 starting directly from coordinate classification: while SimCC eliminates spatial grid quantization, simply increasing the number of 1D output bins neither restores local directional contrasts lost during downsampling nor distinguishes a flat unimodal spread from multimodal ambiguity under occlusion.

2. Why Haar + SO(3) and DCT Are Relevant and Intuitive (Weaknesses 1 & 5, Question 1)

Rather than combining unrelated frequency tools, our two modules form a direct, intuitive closed loop (Section 3, illustrated step-by-step in Figures 2 and 3):

  1. Spatial Evidence Preservation (Haar + SO(3)): Repeated stride-2 downsampling averages 2x2 neighborhoods and washes out local directional edges (e.g., forearm contours near an occluded wrist). Applying a fixed 2x2 Haar transform reorganizes the four spatial polyphase samples into 1 fixed low-frequency average (sent to the coarse semantic stream) and 3 orthogonal directional-detail bands (48.1% energy share; reducing inter-band correlation from 0.31 to 0.03 in Figure 1). A lightweight 24-scalar grouped SO(3) rotation then adapts the 3-band detail basis to task-relevant limb orientations without upsampling the grid (Table 5: +0.77 mean AP over no detail memory across 5 seeds).
  2. Uncertainty-Guided Retrieval (DCT + Entropy): Taking argmax on initial SimCC logits discards distribution shape. A truncated DCT-II basis (with a zero-start low-rank adapter) compresses the full 1D coordinate distribution into a compact state vector (Table 4(b): +1.06 AP over coordinates alone), while distribution entropy explicitly routes cross-attention toward the cached directional-detail bands when predictions are uncertain (Figure 5: 2.25x, 1.94x, and 2.34x near-target attention enrichment in detail bands D1-D3 across 1,499 joints, vs. 0.67x in Main).

3. Citation Formatting and Readability Improvements (Question 4 & Presentation)

Thank you for catching the citation formatting issue: bare \cite commands under natbib author-year style broke sentence flow in the initial submission. We have audited and corrected all citations to proper parenthetical and textual \citep / \citet formatting throughout the revised paper, streamlined the exposition in Sections 1 and 3, and added a dedicated step-by-step diagram of the shared recurrent corrector (Figure 3) to make the framework easy to follow.

AuthorsOpenReview ↗

Response to Reviewer nmZy (Part 2/2): Low-Resolution Frequency Analysis, New ViT-S Results, and Ongoing Experiments

We thank the reviewer again for the constructive suggestion on low-resolution Vision Transformer evaluation. Following up on our previous response, we have completed the controlled 64x48 comparison on the ViT-S backbone and updated Table 2 and Appendix B.1 in the uploaded revision.

New Controlled Low-Resolution ViT-S Experiment (64x48, COCO val2017)

  • Identical Backbone and Bit-Exact Initialization: Both PhasePose ViT-S and the ViTPose-S s8 DARK baseline use the identical mmpretrain.VisionTransformer ViT-Small backbone (12 layers, embed_dims=384, 12 heads, MLP width 1536) with patch_size=16 and patch_cfg=dict(stride=8, padding=4). At 64x48 input resolution, both output an identical 8x6 feature map with 384 channels, and all 149 backbone initialization tensors are verified bit-for-bit identical. Both models use the exact same training schedule and evaluation protocol (flip_test=True for AP; batch-1 FP32 eager timing for latency).
  • Neck and Head Comparison:
    • PhasePose ViT-S: Uses a parameter-free PhasePosePoolNeck that routes the 8x6 (stride-16) feature map directly as the detail/phase branch and applies 2x2 average pooling to form the 4x3 (stride-32) semantic branch, followed by PhasePoseV2Head (embed_dim=256, 8 heads, 4 visual blocks, PixelUnshuffle(2) detail injection, 3-step recurrent phase correction with spectral_dim=96 and OKS coupling) predicting 1D SimCC logits (96x128 bins, split ratio 2.0) trained with KLDiscretLoss + OKSLoss and decoded via 1D DARK.
    • ViTPose-S s8 DARK: Connects the 8x6 backbone feature directly (no neck) to a HeatmapHead with two ConvTranspose2d(kernel=4, stride=2, ch=256) layers and a 1x1 conv, upsampling 4x to 24x32 2D heatmaps (17 channels) trained with KeypointMSELoss + MSRAHeatmap(unbiased=True) and evaluated with 2D DARK second-order Taylor decoding.
MethodBackboneInputParamsFLOPsCPU (ms)GPU (ms)AP
ViTPose-S s8 DARKViT-S64x4824.2M (24.235M)1.3G (1.338G)17.75.153.4
PhasePose (ours)ViT-S64x4827.0M (26.988M)1.2G (1.169G)17.15.358.9 (+5.5)

Under bit-for-bit identical backbone initialization, PhasePose ViT-S reaches 58.9 AP (0.5894), outperforming ViTPose-S s8 DARK (53.4 AP) by +5.5 AP. Although the recurrent phase head adds 2.75M parameters, operating on the 4x3 grid and 1D coordinate sequences avoids 2D transposed convolutions over 16x12 -> 32x24 spatial grids, reducing FLOPs by 12.6% (1.169G vs. 1.338G, i.e., ViTPose-S uses 14.5% higher FLOPs) and improving CPU latency (17.1 ms vs. 17.7 ms). Thank you again for this helpful suggestion.

Reviewer_WiBhOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

In the paper the authors proposed a new method of pose estimation that uses a coarse visual workspace and then predicts body joint coordinates by combining several classic building blocks like Harr and DCT. The method uses a classic Haar transform to four phases from an earlier backbone feature, keeps the low-frequency path fixed, and learns grouped rotations of the three detail bands.

These captured bands then form information for a shared recurrent corrector to function, which reads a compact representation of the current coordinate distributions using DCT coefficients, expected coordinates, and entropy, then exchanges information over a fixed skeleton graph and updates the output logits.

The authors evaluated on standard and small input resolutions on COCO, with additional results on MPII, CrowdPose, and COCO-WholeBody, with good results claimed at 76.7 AP with CSPNeXt-L at 256 x 192 and 56.2 AP at 64 x 48.

Critical strengths
  1. I think the paper's idea of combining cached phase details with recurrent distribution correction seems reasonable and interesting. The proposed method is able to allow the corrector access to an earlier low-res features without running a full high-resolution visual inference, so that makes sense to me from an inference efficiency perspective. The Table 4 in the paper also tests raw phases, centered phases, fixed Haar, and learned mixing, which is helpful ablation.

  2. Several useful settings beyond the usual COCO input size were evaluated in the paper. The gains at low-res input 64 x 48 are worth paying attention: PhasePose-CSPNeXt-L reaches 56.2 AP versus 53.0 for RTMPose-L, and the same general design works with several joint layouts. The separate CPU/GPU timing and the detailed training and skeleton specifications in the supplement are also helpful in parsing the results.

Critical weaknesses
  1. I think the Table 1 in the paper does not establish the claimed matched improvement over CIGPose directly. The ground-truth-box block lists CIGPose-L at 77.6/78.5 AP, then compares PhasePose's 78.5/79.7 AP against these values to claim a 0.9/1.2 AP gain. However, CIGPose paper describes evaluation with detected person boxes, with ground-truth boxes used only for UBody; the Table 2 there reports exactly these COCO values. The paper does not clearly explain whether CIGPose was re-evaluated with ground-truth boxes. If these are the published detected-box results, then the claimed gains would mix two evaluation protocols and cannot establish a pose-model improvement. If they are new ground-truth-box runs, the authors would need to identify the checkpoints and evaluation configurations.

  2. Also, the ablations somehow support the overall design more clearly than the specific phase/spectral explanation. Table 4 does show a benefit from detail memory (+0.77 AP) and recurrent correction (+0.96 AP). My concern is that what the particular orthogonal and spectral choices add. Haar+SO(3) improves over learned phase mixing by 0.22 AP; the DCT adapter adds 0.26 AP over fixed DCT-128; entropy routing adds 0.21 AP over static routing. No variation across training runs were reported, so the reliability of these narrower conclusions is unclear. All of these controls also use 256 x 192 inputs. At 64 x 48, the model changes both the feature stages and the intermediate supervision weight, so the standard-resolution ablations do not isolate which components produce the larger low-input gains.

The broader premise also predates this method: SimCC already removes upsampling and separates coordinate precision from the feature lattice. The degradation curve is useful context, but it does not establish the benefit of PhasePose's new components.

Figure 1's lower correlation and 48.1% energy share are feature diagnostics, but they do not show that phase mixing caused the localization errors or that the Haar basis recovered task-relevant evidence that learned mixing lost. A different basis could help the network use the same information more effectively, and I think Table 4 provides some evidence for that benefit. What remains not answered is whether this specific mechanism explains the low-input improvement.

Questions for authors
  1. I'm curious are the CIGPose-L entries in Table 1 published detected-box results or new ground-truth-box evaluations? Please can the authors give the checkpoint/configuration and box source for these rows. Which comparison supports the claimed 0.9/1.2 AP gains once the box protocol is matched?

  2. Also, which exact pretrained CSPNeXt checkpoint did the authors use, and what data had it seen? Which baseline results were quoted and which were re-trained? These would hlep to clarify flip testing for the AP results and whether it is included in the latency measurements.

  3. How many independent training runs were used for the ablations, and were variation estimates available for the 0.21-0.26 AP differences? Do any existing 64 x 48 controls hold feature stages and loss weights fixed while varying the phase representation or correction?

4 comments

Thank you for your constructive review. The authors have posted their rebuttal. Could you please review their response to see if you have any remaining questions?

AuthorsOpenReview ↗

Response to Reviewer WiBh (Part 1/2): Manuscript Updates, Pretraining, and Evaluation Protocols

We sincerely thank the reviewer for the prompt follow-up and for reconsidering our work. Following your request, we have uploaded a revised manuscript and standalone supplementary PDF that include the dual-protocol CIGPose-L re-evaluation details, complete pretraining/testing specifications, 5-run variation studies at 256x192, fixed-stage 3-run controls at 64x48, and direct readout diagnostics. Due to the character limit, Part 1 addresses CIGPose integration and Question 2 (training/test protocols), and Part 2 addresses Weakness 2 and Question 3 (multi-run variation, 64x48 controls, and mechanism evidence).

1. Inclusion of CIGPose-L Dual-Protocol Results in the Paper (Question 1 & Follow-up)

As requested in your follow-up comment, we have incorporated both the detected-box and ground-truth-box re-evaluations of the official CIGPose-L checkpoints (cigpose-l_coco_256x192-best_355e.pth and cigpose-l_coco_384x288-best_389e.pth) into Main Table 1 and provided the full metric breakdown (AP, AP50, AP75, APM, APL, AR) in Appendix B.2 (Table 1):

  • Matched Detected Boxes (COCO_val2017_detections_AP_H_56_person.json, 56.4 human AP): PhasePose achieves 76.7 / 77.5 AP, improving over CIGPose-L (75.3 / 76.2 AP) by +1.4 / +1.3 AP at 256x192 and 384x288.
  • Matched Ground-Truth Boxes: PhasePose achieves 78.5 / 79.7 AP, improving over CIGPose-L (77.6 / 78.5 AP) by +0.9 / +1.2 AP at 256x192 and 384x288.

2. Pretraining Data, Quoted vs. Retrained Baselines, and Flip Testing (Question 2)

We have consolidated all initialization, baseline provenance, and flip-testing details in Section 4.1 of the revised manuscript and Appendix B of the supplementary PDF:

  • Exact CSPNeXt-L Checkpoint and Pretraining Data: PhasePose uses MMDetection's CSPNeXt P5 backbone and loads only the backbone. parameters from the official MMPose checkpoint cspnext-l_udp-aic-coco_210e-256x192-273b7631_20230130.pth, which was UDP-pretrained on AI Challenger (AIC) + COCO. Crucially, this is the exact same backbone checkpoint used to initialize the official COCO RTMPose-L baseline (75.8 AP) and CIGPose-L at both 256x192 and 384x288 (as well as on MPII, CrowdPose, and COCO-WholeBody). After loading backbone. weights, PhasePose's head is initialized from scratch and the model is trained strictly on COCO train2017 for 210 epochs (no extra pose datasets or distillation are used during PhasePose training). ResNet-50 models initialize from TorchVision ImageNet-1K classification weights, and our newly added ViT-S variant (Table 1: 75.1 AP detected / 77.1 AP GT, +1.2 AP over ViTPose-S with 14.3% fewer FLOPs) initializes from official ViTPose-S backbone weights.

  • Quoted vs. Re-evaluated vs. Retrained Baselines:

    1. Quoted: Standard-resolution baselines in Table 1 and Table 3 quote published papers or official MMPose documentation.
    2. Re-evaluated: CIGPose-L rows in Table 1 (CIGPose-L with dagger symbol) evaluate official released checkpoints under matched detected and GT boxes.
    3. Retrained: In Table 2 (low-resolution 64x48 and 128x96), SimCC, SAR, RTMPose-L, and CIGPose-L were all independently retrained from scratch at the exact rectangular 64x48 and 128x96 resolutions on COCO train2017 using their official configurations (marked (retrained)). Under matched rectangular geometry, PhasePose-R50 reaches 43.9 / 64.9 AP (vs. SimCC's 40.1 / 62.8 and SAR's 39.8 / 62.0), and PhasePose-CSPNeXt-L reaches 56.2 / 70.5 AP (vs. RTMPose-L's 53.0 / 69.6 and CIGPose-L's 52.4 / 68.6). (Note: we also corrected a transcription typo in initial Table 2 where PhasePose-CSPNeXt-L GPU latency at 64x48 was listed as 9.8 ms; actual measured latency under our unified protocol is 6.0 ms GPU / 20.3 ms CPU, matching RTMPose-L's 6.0 / 21.1 ms.)
  • Flip Testing in AP vs. Latency: In all tables, horizontal flip testing is enabled for every AP/AR evaluation and strictly disabled for every CPU and GPU latency measurement (batch-one FP32 forward-plus-decoding in PyTorch eager mode on 8 Xeon Platinum 8352V threads and 1 RTX 4090 GPU, reporting the median of 3x100 timed iterations after 100 warm-ups).

AuthorsOpenReview ↗

Response to Reviewer WiBh (Part 2/2): Multi-Run Variation, 64x48 Controls, and Readout Diagnostics

Continuing from Part 1, we address Weakness 2 and Question 3 using the 5-run stability experiments at 256x192, fixed-stage 3-run controls at 64x48, token-matched memory controls, and attention-readout diagnostics in Section 4.4 (Tables 4-6, Figure 5) and Appendices A & C.

3. Multi-Run Variation for Phase and Spectral Choices at 256x192 (Weakness 2 & Question 3)

In our initial submission, Table 4 reported single-run ablations. To test the reliability of the narrower orthogonal and spectral gains, we independently trained five runs (seeds 2027-2031) at 256x192 (CSPNeXt-L) under identical supervision, augmentation, and EMA selection, and added a Token-count-matched control (3 independent 256-to-256 linear projections of Main tokens matching the 192-token memory length). Results (Tables 5 and 6 in the uploaded revision):

Configuration (256x192, CSPNeXt-L)Single5-Run MeanMin-MaxDeltaParams / FLOPs
No detail memory (48 tokens)75.9375.9175.81-75.99-0.7732.02M / 4.38G
Token-count-matched (Main-proj., 192 tok.)--75.9775.88-76.06-0.7132.21M / 4.42G
Raw phase (P00, P01, P10)76.0476.0775.98-76.16-0.6132.02M / 4.41G
Centered phase76.1976.1876.09-76.26-0.5032.02M / 4.41G
Fixed Haar (B1, B2, B3)76.3576.3276.22-76.41-0.3632.02M / 4.41G
Unconstrained learned 3x3 mixing76.4876.4576.36-76.54-0.2332.02M / 4.41G
Fixed DCT-128 (no adapter)76.1676.1876.08-76.27-0.5032.02M / 4.41G
Static learned routing76.4976.4676.37-76.55-0.2232.02M / 4.41G
Query-only routing (no entropy)--76.5376.41-76.65-0.1532.02M / 4.41G
No explicit entropy in state/router--76.3176.18-76.39-0.3732.02M / 4.41G
Full (Haar+SO(3), Adapter, Entropy)76.7076.6876.59-76.76--32.02M / 4.41G

Across 5 runs, Full PhasePose (76.59-76.76) shows strictly non-overlapping min-max intervals over unconstrained phase mixing (76.36-76.54, +0.23 mean AP with 24 scalars vs. 72), Fixed Haar (76.22-76.41, +0.36), Fixed DCT-128 (76.08-76.27, +0.50; single-run fixed DCT-64 reaches 76.32 in Table 4(b)), Static routing (76.37-76.55, +0.22), and No explicit entropy (76.18-76.39, +0.37). Five-run controls in Table 6 also confirm consistent gains over Identity adjacency (76.48, -0.20) and Fixed DCT synthesis (76.37, -0.31).

4. Fixed-Stage and Fixed-Supervision Controls at 64x48 (Weakness 2 & Question 3)

To isolate low-resolution gains without confounding stage or loss changes, we ran three independent runs (seeds 2027-2029) at 64x48 (CSPNeXt-L) while holding stride-8/16/32 stages and loss weights (lambda_int = 0.25, lambda_oks = 0.10) strictly fixed (Table 6):

Configuration (64x48, fixed stages & loss)Single3-Run MeanMin-MaxDelta
Full PhasePose56.2056.2256.12-56.31--
w/o phase detail memory--55.0854.98-55.19-1.14
Raw phase memory--55.3255.21-55.44-0.90
Fixed Haar--55.7355.62-55.85-0.49
Unconstrained learned phase mixing--55.9555.86-56.07-0.27
w/o spectral distribution state--55.3955.28-55.49-0.83
w/o recurrent correction--54.8854.76-54.99-1.34

At 64x48, Raw (55.32) to Fixed Haar (55.73) to Learned mixing (55.95) to Haar+SO(3) (56.22, +0.27 over learned mixing and +0.49 over fixed Haar) holds with larger margins and non-overlapping 3-run ranges.

5. Premise Framing and Direct Readout Evidence (Weakness 2)

We agree that SimCC already decouples output bins from feature grids and Figure 1 alone is a feature diagnostic. We moved the heatmap degradation curve to Appendix A and reframed Section 1 around coarse-feature ambiguity. Two new results directly link our basis and routing to localization: (i) Token-count-matched (75.97 AP, Table 5) adds only +0.06 AP over 48 Main tokens (75.91), proving the +0.77 AP gain requires preserved intermediate directional detail; and (ii) Figure 5 (128 GT crops, 1,499 joints, 95% cluster-bootstrap CIs) shows final-step attention in detail bands D1-D3 near GT joints is enriched by 2.25x, 1.94x, and 2.34x over uniform (vs. 0.67x in Main), beating both other-joint and other-image controls. Thank you again for your guidance.

AuthorsOpenReview ↗

Thank You for Facilitating the Discussion

Dear Area Chair,

We sincerely thank you for your time and effort in coordinating the review process and actively facilitating the discussion. Your timely follow-up helped foster a very constructive exchange that substantially improved our revised manuscript.

Reviewer_om9AOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes PhasePose, an upsampling-free framework for top-down human pose estimation. It applies a fixed Haar transform to intermediate backbone features and adapts the three detail bands through grouped SO(3) rotations, retaining them as correction memory. An initial SimCC prediction is refined through three weight-shared correction steps using DCT-anchored distribution summaries, coordinate expectations, entropy, and skeletal message passing. Experiments cover COCO, MPII, CrowdPose, and COCO-WholeBody, including reduced input resolutions. The paper reports competitive accuracy and investigates the contributions of phase representations and recurrent refinement.

Critical strengths

Coherent architectural design. The framework combines preservation of intermediate spatial detail with distribution-aware coordinate refinement. Shared correction weights and cached visual keys and values are sensible choices for limiting refinement overhead.

Broad empirical evaluation. The experiments cover multiple benchmarks, keypoint topologies, input resolutions, and backbones. Separating detected-box and ground-truth-box evaluations on COCO helps distinguish pose estimation performance from detector effects.

Relevant ablations. Table 4 includes comparisons against raw phases, centered phases, fixed Haar decomposition, and unconstrained learned phase mixing, together with component-removal and recurrence-depth experiments. These results provide evidence that both detail memory and recurrent correction contribute to performance.

Critical weaknesses

The provenance and input dimensions of some low-resolution baselines require clarification. Table 2 reports SimCC results of 39.3/62.6 AP and SAR results of 39.2/62.5 AP at 64×48/128×96. These values coincide with results reported at 64×64/128×128 in the original SimCC paper (Table 4) and SAR paper (Table 2). The SimCC parameter counts and FLOPs also coincide with the corresponding original configurations. This does not establish that the comparisons are incorrect: the authors may have independently reproduced these values. However, clarification is necessary to assess the claim of shared geometry and the resulting low-resolution improvements.

The evidence is stronger for the overall framework than for the advantages of its specific parameterizations. Haar+SO(3) improves over fixed Haar by 0.35 AP and over unconstrained learned mixing by 0.22 AP. Without variability estimates, the stability of these relatively small differences is uncertain. Moreover, the reported difference between KL-only supervision and the complete objective is 0.51 AP, making consistent supervision important when attributing gains to architecture. Since SimCC and SAR already demonstrate accurate localization from coarse features, the paper’s incremental contribution depends particularly on establishing the benefits of its phase routing and distribution correction.

The efficiency advantage depends on the operating setting. At 64×48 with CSPNeXt-L, PhasePose improves over RTMPose-L by 3.2 AP, but GPU latency increases from 6.0 to 9.8 ms. At 256×192, GPU latency is slightly lower, while CPU latency is higher. These are useful accuracy–latency trade-offs, but the efficiency claims should be qualified accordingly. The paper’s acknowledgment of recurrent small-kernel overhead is appropriate.

Questions for authors

Were the SimCC and SAR rows in Table 2 independently trained and evaluated at the stated rectangular resolutions, or taken from published results? Please provide the configurations and explain their numerical correspondence with the original square-input results.

For the principal comparisons, which results were reproduced under a common protocol? Please clarify pretraining data, training schedules, augmentation, flip testing, decoding, and applicable loss terms.

How stable is the 0.22 AP improvement over unconstrained learned phase mixing across training runs? Were the phase-representation ablations independently retrained with identical supervision and checkpoint-selection procedures?

The 64×48 configuration changes the phase source and main-feature stride. Do component ablations at this resolution support the same conclusions about phase memory and spectral correction as the standard-resolution experiments?

2 comments
AuthorsOpenReview ↗

Response to Reviewer om9A (Part 1/2): Low-Resolution Baselines, Protocols, and Efficiency

We sincerely thank the reviewer for the constructive comments. We have uploaded a revised manuscript and supplementary PDF incorporating all new retraining experiments, multi-seed controls, and protocol clarifications. Part 1 addresses Weaknesses 1 & 3 and Questions 1 & 2; Part 2 addresses Weakness 2 and Questions 3 & 4.

1. Low-Resolution Baselines and Rectangular Retraining (Weakness 1 & Question 1)

We sincerely thank the reviewer for carefully spotting this oversight on our part. In our initial submission, due to our negligence in cross-checking input aspect ratios, we directly cited the published low-resolution results from the original SimCC (64x64 and 128x128, Table 4) and SAR (64x64 and 128x128, Table 2) papers without noticing their square-crop difference from our rectangular 64x48 and 128x96 settings. We apologize for this oversight.

To guarantee a strictly fair comparison under shared rectangular geometry, we independently retrained SimCC, SAR, RTMPose-L, and CIGPose-L from scratch at 64x48 and 128x96 on COCO train2017 using their official configs. SimCC-R50 and SAR-R50 initialize from ImageNet-1K ResNet-50 weights with their official heads and losses (3 deconv stages, split ratio 2.0, and KL loss for SimCC; official joint regression-classification head and loss for SAR) for 210 epochs. All models are evaluated on val2017 under identical Faster R-CNN boxes (COCO_val2017_detections_AP_H_56_person.json, 56.4 AP) and profiled under the same graph-tracing and batch-1 FP32 eager timing setup. Also, the 9.8 ms GPU latency originally listed for PhasePose-CSPNeXt-L at 64x48 was a transcription error; re-measuring under our unified protocol gives 6.0 ms GPU and 20.3 ms CPU.

The retrained results (Table 2 in the uploaded revision) are summarized below:

InputMethodBackboneParamsFLOPsCPU (ms)GPU (ms)AP
64x48SAR (retrained)ResNet-5034.0M0.4G16.56.039.8
64x48SimCC (retrained)ResNet-5034.1M0.4G17.05.640.1
64x48PhasePose (ours)ResNet-5029.6M0.4G13.96.243.9
64x48RTMPose-L (retrained)CSPNeXt-L27.5M0.3G21.16.053.0
64x48CIGPose-L (retrained)CSPNeXt-L27.8M0.3G20.56.152.4
64x48PhasePose (ours)CSPNeXt-L31.8M0.4G20.36.056.2
128x96SAR (retrained)ResNet-5034.0M1.4G25.35.862.0
128x96SimCC (retrained)ResNet-5034.3M1.4G27.46.162.8
128x96PhasePose (ours)ResNet-5029.5M1.1G19.66.064.9

Under matched rectangular inputs, PhasePose-R50 beats SimCC/SAR by +3.8/+4.1 AP at 64x48 and +2.1/+2.9 AP at 128x96, and PhasePose-CSPNeXt-L reaches 56.2/70.5 AP (+3.2/+0.9 over RTMPose-L; +3.8/+1.9 over CIGPose-L).

2. Evaluation Protocols and Training Configurations (Question 2)

Details are consolidated in Section 4.1 and Appendix B:

  • Provenance: Standard baselines in Table 1 quote cited papers or MMPose docs; CIGPose-L is re-evaluated from official checkpoints under matched boxes (Table 1; Appendix B.2); (retrained) rows in Table 2 are trained from scratch.
  • Pretraining & Schedule: ResNet-50 uses ImageNet-1K weights; CSPNeXt-L loads only backbone. weights from the official AIC+COCO UDP checkpoint (cspnext-l_udp-aic-coco_210e-256x192-273b7631_20230130.pth, matching RTMPose-L and CIGPose-L) and trains strictly on COCO train2017 for 210 epochs (4x RTX 4090, batch 1024, AdamW lr = 1e-3, backbone lr * 0.2, weight decay 0.05, cosine decay from epoch 150 to 5e-5, EMA 0.0002).
  • Augmentation, Loss & Flip Test: Epochs 1-180 use flip (0.5), half-body (0.3), shift (0.3), scale [0.6, 1.4], rotation +/-80 deg, HSV, blur (0.1), and dropout (1.0), tightening in epochs 181-210 (no shift, scale [0.75, 1.25], +/-60 deg, dropout 0.5). Objective: final SimCC KL + intermediate KL (lambda_int = 0.10 standard, 0.25 low-res) + OKS (lambda_oks = 0.10), split ratio 2.0, DARK decoding. Flip testing is ON for all AP evaluations and OFF for all latency measurements.

3. Qualification of Efficiency Claims (Weakness 3)

With the corrected 64x48 timing, PhasePose-CSPNeXt-L runs at 6.0 ms GPU / 20.3 ms CPU (vs. RTMPose-L's 6.0 / 21.1 ms, +3.2 AP). Still, trade-offs depend on the setting: at 256x192, GPU latency is slightly lower than RTMPose-L (10.2 vs. 10.7 ms) while CPU is higher (46.3 vs. 40.6 ms); with ResNet-50 at 64x48, CPU is lower than SimCC (13.9 vs. 17.0 ms) while GPU is slightly higher (6.2 vs. 5.6 ms) due to small-kernel overhead. We have qualified these trade-offs in Sections 4.2-4.4.

AuthorsOpenReview ↗

Response to Reviewer om9A (Part 2/2): Multi-Seed Stability, Mechanism Diagnostics, and 64x48 Ablations

Continuing from Part 1, we present the multi-run stability analyses, controlled supervision checks, readout diagnostics, and 64x48 component ablations added to Section 4.4 (Tables 4-6 and Figure 5) of our uploaded revised manuscript and Appendix C of the supplementary PDF.

4. Multi-Run Stability, Controlled Supervision, and Readout Diagnostics (Weakness 2 & Question 3)

We confirm that every phase-representation and corrector ablation (single-run controls in Table 4 / Appendix C Table 2 and multi-run studies in Tables 5-6) was independently trained from scratch under the exact same complete objective (final KL + intermediate KL + OKS), optimization schedule, two-stage augmentation, and EMA checkpoint-selection procedure; only the named component was modified.

To quantify run-to-run variability, we trained five independent runs (seeds 2027-2031) at 256x192 with CSPNeXt-L for all memory variants and key corrector controls, plus a Token-count-matched control (appending three independent 256-to-256 linear projections of Main tokens to match the 192-token memory length). Results (Tables 5 and 6 in the uploaded revision):

Configuration (256x192, CSPNeXt-L)Single5-Run MeanMin-MaxDeltaParams / FLOPs
No detail memory (48 tokens)75.9375.9175.81-75.99-0.7732.02M / 4.38G
Token-count-matched (Main-proj., 192 tok.)--75.9775.88-76.06-0.7132.21M / 4.42G
Raw phase (P00, P01, P10)76.0476.0775.98-76.16-0.6132.02M / 4.41G
Centered phase76.1976.1876.09-76.26-0.5032.02M / 4.41G
Fixed Haar (B1, B2, B3)76.3576.3276.22-76.41-0.3632.02M / 4.41G
Unconstrained 3x3 phase mixing76.4876.4576.36-76.54-0.2332.02M / 4.41G
Fixed DCT-128 (no spectral adapter)76.1676.1876.08-76.27-0.5032.02M / 4.41G
Static learned routing76.4976.4676.37-76.55-0.2232.02M / 4.41G
No explicit entropy in state or router--76.3176.18-76.39-0.3732.02M / 4.41G
Full PhasePose (Haar + SO(3) & Adapter)76.7076.6876.59-76.76--32.02M / 4.41G

Across five runs, Full PhasePose spans 76.59-76.76 AP (mean 76.68), achieving strictly non-overlapping min-max intervals over unconstrained mixing (76.36-76.54, +0.23 mean AP with 24 rotation scalars vs. 72), Fixed Haar (76.22-76.41, +0.36), Fixed DCT-128 (76.08-76.27, +0.50), and Static routing (76.37-76.55, +0.22). Token-count-matched reaches only 75.97 mean AP despite +0.20M more parameters, confirming that gains come from orthogonal detail preservation rather than extra slots.

Furthermore, Figure 5 in the uploaded revision analyzes final-step cross-attention across 128 validation GT crops (1,499 joints, 95% cluster-bootstrap CIs): near-target attention in detail bands D1-D3 is enriched by 2.25x, 1.94x, and 2.34x over a uniform grid (vs. 0.67x in Main), significantly exceeding both same-person other-joint and other-image same-joint controls.

5. Controlled Component Ablations at 64x48 Resolution (Question 4)

Holding the stride-8 phase source, stride-16 workspace, pooled stride-32 context, and loss weights (lambda_int = 0.25, lambda_oks = 0.10) strictly fixed, we ran a three-run ablation study (seeds 2027-2029) with CSPNeXt-L at 64x48 (Table 6 in the uploaded revision):

Configuration (64x48, CSPNeXt-L)Single3-Run MeanMin-MaxDelta
Full PhasePose56.2056.2256.12-56.31--
w/o phase detail memory--55.0854.98-55.19-1.14
Raw phase memory--55.3255.21-55.44-0.90
Fixed Haar--55.7355.62-55.85-0.49
Unconstrained learned phase mixing--55.9555.86-56.07-0.27
w/o spectral distribution state--55.3955.28-55.49-0.83
w/o recurrent correction--54.8854.76-54.99-1.34

Removing detail memory (-1.14 AP), spectral state (-0.83 AP), or recurrent correction (-1.34 AP) yields larger drops than at 256x192, and Raw (55.32) to Fixed Haar (55.73) to Learned mixing (55.95) to Haar + SO(3) (56.22) remains consistent with non-overlapping ranges. Thank you again for your valuable feedback.

Comments on the paper

1 comment
AuthorsOpenReview ↗

Thank You and Summary of Revisions for Area Chair

Dear Area Chair,

We sincerely thank you for your proactive facilitation during the discussion period. Your reminder on November 17 directly prompted constructive follow-up engagement and positive feedback from Reviewer WiBh, and we are truly grateful for your support in ensuring a thorough review process.

As the author-reviewer discussion window closes, we provide a concise summary of all revisions, newly completed experiments, and uploaded PDFs for your reference during the AC-reviewer discussion:

1. Strict Protocol Parity and Matched Baseline Retraining (Reviewers om9A & WiBh)

  • CIGPose-L Dual-Box Verification (Table 1; Appendix B.2): Re-evaluating official CIGPose-L checkpoints (256x192 and 384x288) yields 77.64 / 78.48 AP on GT boxes and 75.27 / 76.17 AP on standard MMPose Faster R-CNN detections (56.4 AP). Under strictly matched protocols, PhasePose outperforms CIGPose-L by +1.4 / +1.3 AP (detected boxes) and +0.9 / +1.2 AP (GT boxes).
  • Matched Rectangular Low-Resolution Retraining (Table 2): To eliminate the initial 64x64 / 128x128 aspect-ratio discrepancy in SimCC and SAR, we independently retrained SAR, SimCC, RTMPose-L, and CIGPose-L from scratch at 64x48 and 128x96 on COCO train2017 under identical Faster R-CNN boxes and unified latency profiling (also correcting a 9.8 ms GPU typo to 6.0 ms GPU / 20.3 ms CPU). Under matched rectangular geometry, PhasePose-R50 beats SimCC/SAR by +3.8/+4.1 AP (64x48) and +2.1/+2.9 AP (128x96), and PhasePose-CSPNeXt-L beats RTMPose-L/CIGPose-L by +3.2/+3.8 AP (64x48) and +0.9/+1.9 AP (128x96).
  • Full Provenance and Hyperparameters (Section 4.1; Appendix B.1): Consolidated exact pretraining checkpoints (including AIC+COCO UDP backbone initialization), optimization schedules, loss weights, and flip-test conventions.

2. Multi-Seed Stability, Token-Matched, and Readout Diagnostics (Reviewers om9A & WiBh)

  • 5-Run Memory Controls at 256x192 (Table 5, seeds 2027-2031): Full PhasePose (76.68 mean AP, range 76.59-76.76) outperforms no detail memory (75.91, +0.77), a token-count-matched 192-token stride-32 baseline (75.97, +0.71 with 0.20M fewer parameters), Raw (76.07), Centered (76.18), Fixed Haar (76.32), and Learned 3x3 mixing (76.45).
  • 5-Run Corrector Controls at 256x192 (Table 6): Full outperforms Fixed DCT-128 (76.18), Static routing (76.46), Query-only routing (76.53), No explicit entropy (76.31), Identity adjacency (76.48), Same-rank head (76.42), and Fixed DCT synthesis (76.37).
  • 3-Run Low-Resolution Controls at 64x48 (Table 6, seeds 2027-2029): Under fixed stages and lambda_int=0.25, Full (56.22 mean AP) consistently beats w/o detail memory (55.08), Raw (55.32), Fixed Haar (55.73), Learned mixing (55.95), w/o spectral state (55.39), and w/o recurrent correction (54.88).
  • Cluster-Bootstrap Attention Diagnostics (Figure 5): Across 128 GT crops (1,499 joints, 2,000 image-cluster resamples), near-target (r=48 px) enrichment reaches 2.25x, 1.94x, 2.34x in Haar detail bands D1-D3 vs. 0.67x in Main.

3. Vision Transformer (ViT-S) Generalization at 256x192 and 64x48 (Reviewer nmZy)

  • Standard 256x192 ViT-S (Table 1): PhasePose (ViT-S) achieves 75.1 AP / 80.5 AR on detected boxes (+1.2 AP over ViTPose-S 73.9 AP with 14.3% fewer FLOPs, 4.8G vs. 5.6G; +0.9 AP over SHaRPose 74.2 AP) and 77.1 AP / 79.8 AR on GT boxes (+0.5 AP over ProbPose-S 76.6 AP with 32% fewer parameters, 27.3M vs. 40.3M).
  • Newly Added Controlled 64x48 ViT-S Comparison (Table 2; Appendix B.1): Using a bit-for-bit identical mmpretrain.VisionTransformer ViT-Small backbone (patch_size=16, stride=8, padding=4, 8x6 output, 149 identical initial tensors), PhasePose ViT-S reaches 58.9 AP (0.5894) vs. 53.4 AP for ViTPose-S s8 DARK (+5.5 AP), while lowering FLOPs by 12.6% (1.169G vs. 1.338G) and improving CPU latency (17.1 ms vs. 17.7 ms).

4. Cross-Topology Transfer and Organization (Reviewer nmZy)

  • MPII, CrowdPose, and COCO-WholeBody (Table 3): Retained in the main text, achieving 92.5 [email protected] on MPII, 74.7 AP on CrowdPose (+1.0 over CIGPose-L), and 64.5 AP on COCO-WholeBody (+1.9 over CIGPose-L; +1.4 over DWPose-L).

We hope this summary is helpful for the upcoming AC-reviewer discussion. Thank you again for your time, fairness, and guidance.

Best regards, The Authors

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.