acceptodds
Under review as a conference paper at ICLR 2027

STANCE: Scene-Consistent Real-Time Motion Estimation from Sparse Wearables via Posterior-Corrected Residual Diffusion

Abstract

Wearable tracking began as a way to drive game avatars and has since become an instrument for embodied intelligence, where human demonstrations captured from a few body-worn sensors train agents inside simulated environments. Configurations vary widely: a headset and two controllers give three 6-DoF signals, and optional hip and foot trackers extend this to six. Methods that accept arbitrary sensor sets fix each tracker's influence in advance, zero-padding or masking absent channels with hand-crafted weights that encode measurement presence rather than reliability. A second gap opens only inside the scene: signals captured in a physical room are replayed in a virtual one, and the mismatch surfaces as limbs passing through the very geometry the motion is meant to interact with. We present STANCE, which predicts and then corrects. A configuration-agnostic predictor forecasts a temporally coherent prior from a fixed set of core sites, so continuity never depends on which trackers are present, while a single-frame residual denoiser rewrites only what the trackers contradict, keeping accuracy and latency from trading against each other. How far a partial inverse-kinematics solve travels from the prior, and how closely it meets the trackers, give each joint a continuous trust gating how much the denoiser may rewrite; a configuration change then alters feature values and nothing else. Inside the denoiser, a training-free posterior-sampling procedure corrects the predicted residual against scene geometry at every diffusion step, removing penetration without retraining or a learned collision model. STANCE runs in real time, reaches state-of-the-art accuracy across most tracker counts, and holds continuity through dynamic reconfiguration.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
sparse sensor, motion capture, conditional diffusion model, human-scene interaction, diffusion posterior sampling

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_dJdgOpenReview ↗
Rating
2(+1)
Confidence
4
Informativeness
2
Presentation
1
Soundness
2(+1)
Read review
Paper summary

This paper studies human motion reconstruction (HMR) from body-worn IMU sensor trackers under a configuration-agnostic setting: 3 compulsory trackers (always available) and 3 optional trackers (whose signals may occasionally be lost). To mitigate occasional signal loss, it proposes STANCE, a diffusion-based framework. It starts from a coarse pose prediction, calculates the per-joint confidence using available tracker measurements, and refines the poses through diffusion refinement. It also designs training-free guidance based on scene geometry to reduce scene penetration, but it does not show how to obtain this scene. Overall, the methodology design is not clearly structured and is questionable.

Critical strengths

The research tries to improve the robustness of IMU-based HMR and improve temporal consistency and accuracy during sensor signal loss. It is a meaningful research direction for real-world deployment.

The proposed method is claimed to support real-time Unity rendering.

Critical weaknesses
  1. Mathematical symbols are not well organized: lower-case s_n^k, upper-case S^c, and u^k_m all stand for sensor measurements. H stands for both the future window size and the feature representation. Both n and m are used in timestamps. In Equation 5, the historical poses x are input to the model rather than \bar{x}. Please clarify whether you use GT history or predicted history. These misuses of symbols make the method difficult to comprehend.

  2. Missing important details: How do you obtain y_floor for coordinate transformation? Moreover, FABRIK is undefined, and there is no reference. Do you use human limb lengths during IK? How do you obtain or estimate GT human limb lengths?

  3. The geometric term on the reconstructed pose in L_geo is undefined.

  4. The paper does not describe how the prior scene is constructed in Sec. 3.5. If it is not estimated from head observations, it raises fairness concerns since other methods do not have access to this prior scene.

Questions for authors

No

8 comments
AuthorsOpenReview ↗

Thank you for your valuable feedbacks.(Notice1&Q1)

We thank reviewer dJdg for the careful reading and for pointing out several aspects that require clarification.

Notice 1: Fundamental Clarification of the Input Modality

We would first like to clarify a fundamental misunderstanding in the review: STANCE does not use IMU signals at all. It operates exclusively on body-worn position trackers that provide positional observations, not inertial measurements. The three compulsory and three optional sensors described in the paper are all position trackers, with the optional trackers potentially becoming unavailable during motion. This distinction is fundamental, not merely terminological: IMU-based motion reconstruction relies on inertial signals, whereas STANCE addresses motion reconstruction from sparse and intermittently missing positional observations. Consequently, characterizing our work as IMU-based HMR misrepresents the input modality, the underlying problem setting, and the technical motivation of the proposed method. We respectfully ask that this distinction be taken into account when evaluating the paper.

Q1: Organization of Mathematical Symbols

Clarifications of Existing Notation. The three sensor symbols represent successive levels of the same data pipeline, rather than three names for the same quantity. Specifically, snks^k_n denotes the raw measurement from tracker kk at frame nn (Eq. 2). umku^k_m denotes the single-frame encoding derived from this measurement for the Predictor, incorporating frame-to-frame rotation and displacement (Eq. 4). SmcS^c_m concatenates these encodings across the three core trackers at frame mm, and the Predictor takes the window Sn−L:ncS^c_{n-L:n} as input. Thus, these symbols represent three successive levels of the same data pipeline, from an individual tracker's raw measurement to its per-frame encoding and finally to the Predictor's windowed input.

The two time indices also have distinct meanings: nn denotes the current frame being estimated, whereas mm indexes individual frames within the input window n−L,…,nn-L,\ldots,n. This separate index is necessary because the encoding involves frame-to-frame differences at each position in the window, rather than only at the current frame.

Corrections in the Revised Manuscript. We acknowledge that reusing HH for the prediction horizon and token matrices could cause ambiguity, as they were distinguished only by boldface. We have therefore renamed the token matrices as Zs\mathbf{Z}^{s}, Zm\mathbf{Z}^{m}, and Z\mathbf{Z}, reserving HH exclusively for the prediction horizon.

We have also corrected Eq. (5) and the corresponding denoiser conditioning in Sec. 3.4. Throughout the paper, x\mathbf{x} denotes the ground-truth pose, xˉ\bar{\mathbf{x}} denotes the Predictor's prior prediction, and x^=xˉ+r^\hat{\mathbf{x}}=\bar{\mathbf{x}}+\hat{\mathbf{r}} denotes the reconstructed pose after residual refinement. Therefore, the historical input should consist of reconstructed poses, and Eq. (5) should use x^n−L:n−1\hat{\mathbf{x}}_{n-L:n-1} instead of xn−L:n−1\mathbf{x}_{n-L:n-1}. Accordingly, the denoiser conditioning in Sec. 3.4 is corrected to (x^n−L:n−1,xˉn:n+H)(\hat{\mathbf{x}}_{n-L:n-1},\bar{\mathbf{x}}_{n:n+H}).

AuthorsOpenReview ↗

Thank you for your valuable feedbacks.(Q2&Q3&Q4)

Q2: Missing details

yfloory_{\mathrm{floor}} is the height of the floor plane in the world frame and is a session constant rather than a per-frame quantity. In the live system it comes from the headset's room calibration, which places the floor at zero; on AMASS it is read from the capture. It enters only the origin of the local frame FnF_{n} in Eq. (3), so what matters is that the same convention holds at training and at deployment, which calibration guarantees.

FABRIK is Forward And Backward Reaching Inverse Kinematics (Aristidou & Lasenby, 2011), a classical and widely used IK solver whose name we had taken to be common knowledge and therefore left unexpanded, which we agree is not appropriate for a general audience. The reference will be given in the revised paper.

The solve does use bone lengths, and they are an input rather than a quantity we estimate: on AMASS the per-sequence rest skeleton supplied with the dataset, and in the live system the calibrated avatar skeleton of the session. No limb length is inferred from the tracker readings, so the solve has nothing subject-specific to estimate at run time.

Q3: Definition of the Geometric Loss

We have now explicitly defined both terms. Lres\mathcal{L}_{\mathrm{res}} uses an MSE loss to measure the error between the confidence-gated target residual and the residual predicted by the denoiser, aggregated over all joints. Lgeo\mathcal{L}_{\mathrm{geo}} is a weighted combination of position MSE, squared rotational-angle error, Huber loss, and cosine loss, evaluated on the reconstructed pose xˉn+r^n\bar{\mathbf{x}}_n+\hat{\mathbf{r}}_n against the ground-truth pose xn\mathbf{x}_n. The two terms provide complementary supervision: Lres\mathcal{L}_{\mathrm{res}} supervises pose residuals, while Lgeo\mathcal{L}_{\mathrm{geo}} explicitly constrains the geometric and rotational consistency of the reconstructed motion. The full set of components and their weights is given in the revised appendix.

Q4: Construction of the Prior Scene and Fairness

The scene is authored rather than estimated. We reconstruct the real furniture geometry from photographs and dimensional references, place it in Unity, and attach collision proxies whose signed distance fields (SDFs) provide the distance and gradient queried by the correction module. This design matches our target application: when motion is replayed in a virtual room, the room geometry is already available as part of the environment, making it an input rather than something that must be recovered from head observations. Estimating unknown scene geometry is a separate problem and is outside the scope of this work.

Importantly, our main quantitative comparisons do not give STANCE access to scene information unavailable to other methods. The evaluations in Tables 1–3 use AMASS under the A-P2 protocol, where no scene geometry is involved and the scene correction module is inactive. Thus, all reported accuracy, velocity, and jitter results are obtained using the same input information available to the baselines. Scene geometry is introduced only in the additional scene-consistency evaluation, where we compare STANCE with and without posterior correction under the same scene conditions. We therefore present scene consistency as an additional capability of STANCE, rather than as an advantage in the main benchmark comparisons.

Dear Reviewer dJdg,

Thank you again for the great efforts and valuable comments. We hope you find the response satisfactory. As the discussion phase is about to close, we are eagerly looking forward to hearing from you regarding any further feedback. We will be more than happy to address any additional concerns you may have.

Best,

STANCE Authors

Reviewer_Cn7BOpenReview ↗
Rating
3
Confidence
4
Informativeness
2
Presentation
2
Soundness
3
Read review
Paper summary

This paper tackles the problem of estimating full-body pose from sparsely placed 6DoF trackers in the presence of 3D scene geometry that constrains the human pose. The authors use a Diffusion framework to solve this problem. Unlike traditional diffusion-based pose estimation algorithms, the authors first predict a rough pose sequence using a neural network prior. The diffusion model is then used to refine the original prediction, guided by the scene geometry and other posterior-refinement losses.

The prior neural network model assumes that three trackers are always present: the head and the wrists. The sparse 6Dof signals from these three trackers are fed directly into the prior model to produce the initial pose sequence. The predicted poses are then individually corrected using a posterior-guided diffusion framework. One of the guidance terms is the presence of other trackers that are used opportunistically when available. This correction is performed using an Inverse-Kinematic guidance signal that refines the prior pose predictions within the sequence, one by one. To allow for opportunistic tracker signal dropouts without causing discontinuous transitions, the authors propose a slerp-based pose interpolation algorithm.

To incorporate scene geometry constraints, a signed distance field of the scene is first constructed in advance. During inference, if the predicted pose mesh intersects the scene geometry, an additional posterior guidance signal is used to modify the pose so that the mesh is pushed away from the geometry it intersects. This removes the need for any training-based method for incorporating the scene.

The proposed method compares favorably with other baselines in real-world datasets.

Critical strengths

The authors propose a new SoTA technique for full-body pose estimation using sparse 6DOF sensors. Their technique manages to beat existing baselines with a neural network that runs with comparable latency to existing works.

Secondly, the authors propose a training-free strategy to account for 3D scene geometry. Unlike prior works that require encoding the scene using a trained model, this training-free strategy can be applied at inference time to any previously unseen 3D scenes to prevent generated pose meshes from penetrating the scene geometry.

Critical weaknesses

The main weakness, in my opinion, is the lack of comparison between the opportunistic tracker incorporation mechanism proposed in the paper, and existing techniques in the literature. The authors propose using a PostEdit[1] style guidance mechanism. But there have been previously proposed guidance mechanisms such as EgoAllo[2], which uses a Levenberg-Marquardt solver to opportunistically incorporate tracking signals, and InPose[3], which uses a Π\Pi-GDM-Based diffusion inverse guidance mechanism.

A minor weakness is the pipeline explanation in Figure 2, which is hard to follow and doesn't show the numbering used in the labels.

[1] Feng Tian, Yixuan Li, Yichao Yan, Shanyan Guan, Yanhao Ge, and Xiaokang Yang. Postedit: Posterior sampling for efficient zero-shot image editing. In the International Conference on Learning Representations, 2025.

[2] Brent Yi, Vickie Ye, Maya Zheng, Yunqi Li, Lea Müller, Georgios Pavlakos, Yi Ma, Jitendra Malik and Angjoo Kanazawa. Estimating Body and Hand Motion in an Ego-sensed World. In CVPR 2025.

[3] Sahil Bhandary Karnoor and Romit Roy Choudhury. Zero-shot Human Pose Estimation using Diffusion-based Inverse Solvers. In the International Conference on Learning Representations, 2026.

Questions for authors

See weaknesses.

One question is whether the authors can pull the pose mesh towards the scene geometry to prevent the mesh from floating in space. This has previously been achieved using physics-based pose generation methods such as MaskedMimic [1].

[1] Chen Tessler, Yunrong Guo, Ofir Nabati, Gal Chechik and Xue Bin Peng. MaskedMimic: Unified Physics-Based Character Control Through Masked Motion Inpainting. In SIGGRAPH Asia 2024.

5 comments
AuthorsOpenReview ↗

Thank you for your valuable feedbacks.(Q1&Q2)

Q1: Comparison with existing opportunistic guidance mechanisms

We thank Reviewer Cn7B for pointing out the lack of comparisons with existing guidance mechanisms. To address this concern, we conduct controlled comparisons with two alternative scene-correction strategies: an LM-based optimizer, motivated by EgoAllo [2], and a Π\PiGDM-based guidance mechanism, motivated by InPose [3].

We compare three mechanisms while keeping the Predictor, DiT weights, input conditions, DDIM sampling schedule, and final pose projection unchanged:

  • Ours: Gradient-based optimization of the predicted clean motion residual at each reverse diffusion step.
  • LM: Replaces the inner gradient-descent optimizer with a Levenberg–Marquardt solver, retaining the same objective and correction location.
  • Π\PiGDM: Replaces clean-residual editing with diffusion posterior guidance, propagating the scene constraint through the reverse process and applying the guidance increment to the noisy-state update.

All three methods share the same implementation framework and automatic-differentiation backend. Their respective inner iterations, step sizes, damping factors, guidance strengths, and covariance scales are independently tuned.

MethodSampling time ↓ (ms/frame)Relative timeMean penetration depth ↓ (mm)
LM correction67.762.50×0.010
Π\PiGDM scene guidance37.391.38×4.704
Ours27.091.00×0.335

All results are measured on the same GPU under the same sampling schedule and scene configuration. Sampling time includes condition preparation, denoising, scene correction, and final projection, but excludes the Predictor. Penetration depth is averaged over valid capsule–box pairs.

LM achieves the lowest penetration depth but at substantially higher computational cost. It further reduces the mean penetration depth by 0.325 mm compared with ours, while requiring 2.50× the sampling time. Its repeated Jacobian construction and damped linear solves at every denoising step introduce considerable overhead, making the sampler alone exceed the per-frame budget; the actual end-to-end latency is higher because Predictor time is excluded.

Π\PiGDM is both slower and less effective in this few-step setting. Unlike our approach, which edits the predicted clean residual before the DDIM update, Π\PiGDM applies its guidance to the noisy state after the update. Each correction is therefore partly re-noised by the following step, with limited opportunities for recovery under a short sampling schedule. Moreover, computing the pseudo-inverse requires differentiation through the denoiser, whereas our correction only differentiates through forward kinematics and the SDF lookup.

Overall, these results show that directly correcting the predicted clean residual offers a more favorable trade-off between scene consistency and computational efficiency for real-time, few-step motion tracking.

Q2: Clarification of the pipeline illustration in Fig. 2

We thank the reviewer for this helpful suggestion. Fig. 2 is organized into three parts. The left part shows (a) the Predictor followed by (b) the Kinematic-Consistency Confidence Estimator, which operate sequentially. The middle part illustrates two consecutive DiT denoising steps, with red and blue indicating steps TT and T−1T-1, respectively. Each denoising step includes (d) the M-step Posterior Correction, which is shown separately on the right for clarity. The output of each posterior correction is fed into the subsequent denoising step. We have added number in the Fig. 2 to make this organization and iterative data flow clearer. The revised Fig. 2 is added to the supplementary materials.

AuthorsOpenReview ↗

Thank you for your valuable feedbacks.(Q3)

Q3: Preventing the mesh from floating in space

We thank the reviewer for raising this important question. Foot floating can be addressed by contact detection and IK post-processing alone, without involving the posterior correction. To verify this, we conducted a controlled ablation on standing sequences with valid foot-tracker observations, comparing results with and without contact-aware IK post-processing (foot anchoring, leg IK, and a small pelvis-height adjustment). Both settings retain the same anti-penetration processing and share identical support labels derived independently from raw tracker measurements.

Metricw/o contact IKw/ contact IK
Floating ratio, clearance > 1 cm8.39%0.00%
Floating ratio, clearance > 5 mm49.60%1.21%
Mean clearance5.37 mm2.12 mm
P95 clearance10.18 mm2.11 mm
Max. ground penetration0.00 mm0.00 mm

Clearance is measured from the lowest skinned foot vertex to the ground and time-weighted over both feet. This causal post-processing stage adds only 0.76 ms per displayed frame.

The two mechanisms serve complementary roles. Our posterior energy is one-sided by construction: it penalizes penetration and provides no attraction toward scene geometry. We therefore handle attraction separately, where the target contact is unambiguous. Foot-ground contact during standing is a well-defined planar constraint that local IK can resolve when reliable foot-tracker observations are available. In contrast, interactions with scene objects involve multiple body regions, differently oriented surfaces, and dynamically changing contacts. Such constraints are difficult to resolve through isolated IK corrections without disrupting other body configurations. STANCE's posterior correction instead incorporates scene geometry into the reverse diffusion process, jointly balancing these constraints against motion fidelity. Thus, the two mechanisms are complementary rather than interchangeable.

MaskedMimic pursues a different objective: it synthesizes physically plausible motion, with sparse targets serving as guidance that the policy is encouraged to follow. In this setting, deviations from the targets can be acceptable if they improve physical plausibility. STANCE, in contrast, aims to faithfully reconstruct the user's actual motion, making such deviations a direct reconstruction error.

This ablation is limited to standing sequences with both foot trackers valid, where support states can be determined directly from sensor measurements. Scenarios with non-planar support surfaces would require a dedicated contact predictor to provide support labels.

I thank the authors for their detailed response to my questions. I still have one question regarding their partial inverse-kinematics solver, FABRIK, that reviewer dJdg raised. I assumed it was the same technique as the inverse guidance mechanism that was used for geometry penetration removal.

Since this doesn't seem to be the case, can the authors elaborate on what FABRIK is? Is it the "FABRIK: A fast, iterative solver for the Inverse Kinematics problem" algorithm by Andreas Aristidou and Joan Lasenby? Why not use this algorithm to solve for geometry penetration, since both fundamentally solve the same partial inverse kinematics problem?

Furthermore, are body joint constraints respected by the inverse guidance mechanisms and by FABRIK? Do they affect joint angles in such a way that a joint could exceed the joint limits defined by the prior?

Reviewer_hADeOpenReview ↗
Rating
2
Confidence
3
Informativeness
1
Presentation
3
Soundness
3
Read review
Paper summary

This paper proposes a framework to estimate full-body motion from a varying set of sparse trackers. To this end, a temporal predictor first generate a full-body motion prior using only fixed core trackers. Then per-joint confidence weights are determined from the gap between the motion prior and IK solution. A residual diffusion transformer refines the prior conditioned on the confidence-gated features. During diffusion inference, a posterior correction optimizes the denoised pose residual by using SDF of the scene to remove penetration.

Critical strengths

The strategy to support varying number of trackers without training networks for each case is practical for VR applications where tracker failure occurs often. Addressing abrupt transitions and snapping artifacts due to tracker dropout and reconnection also raises the practicality of the framework. The adoption of single-frame residual denoising rather than multi-frame trajectory generation keeps the diffusion sampling relatively compact. Overall, the framework is well-motivated for real-time applications.

Critical weaknesses
  • The overall pipeline is largely an engineering integration of known components, such as single-frame residual DiT and test-time Diffusion Posterior Sampling. I could not find conceptually novel aspect from the framework.

  • Confidence is a distinguishing factor of this work, but the way it determine confidence is rather hueristic. I wonder how the method will improve if it used a learned probabilistic uncertainty as confidence rather than kinematic distance.

  • Scene consistency is highlighted in the text, but the way it solves for scene consistency does not seem novel. In addition, it has been validated only for a very limited case.

Questions for authors
  • How does the proposed confidence compare empirically agains simple binary availability mask? As the authors argues that simple binary mask approach is limiting, such comparion would be necessary.

  • Please clarify why the Area Under Jerk during tracker loss is substantially higher than RPM baselines. Wouldn't this lead to visible sluggishness or floating artifacts during sudden dropouts?

6 comments
AuthorsOpenReview ↗

Thank you for your valuable feedbacks. (Q1&Q2)

We sincerely thank Reviewer hADe for the careful reading, and in particular for recognising that "confidence is a distinguishing factor of this work". We take the novelty concern seriously and use this response to state our claim more precisely than the submission did.

Q1: Clarification on the conceptual novelty

We agree that both the single-frame residual DiT and test-time posterior sampling are established tools. Our contribution is a predict-confidence-correct mechanism for real-time motion estimation from sparse sensors, where both tracker configurations and observation reliability can change dynamically. Crucially, these components are tightly coupled rather than independently assembled: the predictor provides the reference pose for residual construction, while the confidence mn\mathbf{m}_n defines the gated residual target during training, conditions the denoiser, and gates the posterior correction. Moreover, performing diffusion and posterior correction in this shared residual space anchors scene guidance to the denoiser's motion estimate. Thus, each component is designed to work in concert with the others. Specifically, our innovations are threefold:

  1. We decouple a configuration-independent motion prior from an observation-dependent correction, allowing a single set of weights to support tracker configurations that change at run time. The same checkpoint achieves state-of-the-art performance on most metrics under both 3- and 6-tracker configurations.
  2. We propose kinematic consistency-driven confidence, replacing binary availability with a per-joint trust score derived from partial inverse kinematics (IK). The key insight is that a reliable tracker measurement should be consistent with the body's kinematic structure: its constraint should be satisfiable when propagated through the corresponding kinematic chain. We therefore use the gap between the partial-IK solution and the tracker target to quantify confidence, with a larger gap indicating lower confidence. This confidence is propagated to the joints governed by each tracker, enabling targeted corrections based on observation reliability.
  3. We impose scene constraints through training-free posterior correction at every reverse diffusion step, rather than applying a post-hoc collision fix to the final SMPL pose. Each step balances motion fidelity against scene consistency while the pose is being refined. Since no scene information is required during training and no weights are updated, the same checkpoint can accommodate different scene geometries at inference without retraining.

Q2: Comparison against a binary availability mask

We thank the reviewer for this suggestion and agree that this comparison is necessary. We therefore added an ablation under the same A-P2 protocol as Tab. 1, replacing continuous confidence mn\mathbf{m}_n with a binary availability mask while keeping all else fixed.

\hline \textbf{Config} & \textbf{Variant} & \textbf{MPJRE}\downarrow\\, & \textbf{MPJPE}\downarrow\\, & \textbf{MPJVE}\downarrow\\, & \textbf{Jitter}\downarrow \\\\ \hline \text{3-point} & \text{binary mask} & 4.99 & 5.58 & 21.92 & 0.962 \\\\ \text{3-point} & \text{confidence} & \mathbf{4.66} & \mathbf{5.31} & \mathbf{21.23} & \mathbf{0.958} \\\\ \hline \text{6-point} & \text{binary mask} & 3.73 & 1.16 & 8.41 & 1.17 \\\\ \text{6-point} & \text{confidence} & \mathbf{3.37} & \mathbf{1.07} & \mathbf{8.29} & \mathbf{1.16} \\\\ \hline \end{array}$$ Continuous confidence consistently improves accuracy in both configurations, with larger gains at 6 trackers: MPJRE improves by 9.7% vs. 6.6%, and MPJPE by 7.8% vs. 4.8%, under 6- and 3-point settings, respectively. **This is counterintuitive**, as more trackers might be expected to reduce the need to distinguish among readings. However, **tracker drift and noise can make additional readings mutually inconsistent. Binary gating admits every available reading at full strength, so more trackers can introduce more conflicting constraints.** In contrast, continuous confidence can downweight unreliable measurements, explaining its increasing advantage as the number of trackers grows. The largest gains are in rotation accuracy, consistent with confidence helping resolve rotational ambiguity along kinematic chains, which is important for motion retargeting and replay. Jitter changes little, as expected: $m_n$ controls the joint-wise residual correction strength, whereas the ramped temporal admission $\alpha_n^k$, which smooths tracker reconfiguration, remains unchanged. Overall, these results demonstrate the advantage of reliability-aware correction over binary availability.
AuthorsOpenReview ↗

Thank you for your valuable feedbacks. (Q3&Q4)

Q3: On the limited validation of scene consistency

We thank the reviewer for this suggestion. We have added two complementary scenarios: (1) sitting on a chair, involving coupled contacts with the backrest and seat across different planes; and (2) leaning against a wall, testing horizontal non-penetration constraints. Together with the original armrest-tapping scenario, which captures dynamic, localized contact, these cover diverse scene interactions.

We further quantify scene consistency using three mutually exclusive frame-level metrics. Over the manually verified frames in which contact is expected, each frame is classified by the minimum signed distance of the region as Penetration (<−ε<-\varepsilon), Contact Preservation (−ε-\varepsilon to δ\delta), or Separation (>δ>\delta), with δ=5\delta=5 mm and ε=\varepsilon= 1 mm. The three sum to 100%, and the frame set is identical for both rows of each scenario. We also report the mean penetration depth over violating frames, so that severity is reported separately from frequency.

Scene / RegionMethodPenetration ↓Contact ↑Separation ↓MeanDepth ↓ (mm)
Armrest / Left armWithout correction45.79%26.64%27.57%5.51
STANCE0.00%55.14%44.86%0.00
Chair / BackWithout correction99.43%0.57%0.00%12.91
STANCE19.54%39.66%40.80%2.69
Chair / ThighsWithout correction100.00%0.00%0.00%29.21
STANCE42.59%57.41%0.00%1.90
Wall / BackWithout correction100.00%0.00%0.00%35.31
STANCE75.44%24.56%0.00%4.42

Penetration falls and contact rises in every region. The penetration that remains is shallow, since the correction takes a fixed number of gradient steps within each denoising step and therefore reduces deep violations rather than removing them. The separation it leaves in the armrest and backrest scenarios is a posture rather than floating, as the arm parts from the armrest between taps and the backrest supplies only a horizontal reaction, while the weight stays on the thighs.

All evaluations use the same checkpoint, with no scene-specific training and no weight updates. Qualitative demonstrations of the three scenarios are provided in the supplementary material.

Q4: On the higher Area Under Jerk during tracker loss

AUJ measures the cumulative frame-wise absolute difference between predicted and ground-truth jerk over the transition window, reflecting deviations in high-order motion dynamics rather than motion roughness. Jitter, our smoothness metric, is lowest for STANCE among all methods in Tab. 1 (0.96 and 1.16 under 3- and 6-point configurations, respectively). We further verified that all AUJ contributions arise from frames where the predicted mean jerk is below the ground truth, indicating attenuated rather than excessive jerk.

To assess whether this attenuation causes perceptible motion artifacts, we evaluated the 0.5s following each dropout across all 138 A-P2 test sequences (702 dropout events), measuring the speed and position of the wrist whose tracker was lost, as well as foot sliding during contact.

MethodAUJ↓Speed ratio to GT (→1)Wrist err. (cm)↓Foot sliding (cm/s)↓
RPM-Reactive11.730.79516.1421.84
RPM-Smooth6.430.83116.5422.85
STANCE22.790.85312.2517.02

Despite its higher AUJ, STANCE retains the largest fraction of ground-truth wrist speed (0.853 vs. 0.795 and 0.831) and achieves the lowest wrist position error (12.25cm vs. 16.14cm and 16.54cm), indicating less motion sluggishness. Foot sliding during ground-truth contact frames is also lowest with STANCE (17.02cm/s vs. 21.84cm/s and 22.85cm/s), indicating fewer foot-ground skating artifacts.

Notably, although STANCE has 3.54× the AUJ of RPM-Smooth, its wrist position error is 26% lower and its speed ratio is closer to ground truth. A plausible explanation is that STANCE preserves the dominant wrist trajectory while attenuating small-amplitude, high-frequency secondary motions. Such motions have limited impact on position and speed errors but can contribute substantially to AUJ due to the high-order nature of jerk. Thus, a higher AUJ does not necessarily imply poorer trajectory fidelity.

AuthorsOpenReview ↗

Thank you for your valuable feedbacks. (Q5)

Q5: Kinematics-based confidence vs. learned probabilistic uncertainty

We thank the reviewer for this insightful suggestion. STANCE targets motion tracking and reconstruction, where faithfully following user-provided observations is essential. This differs from motion generation (e.g., MaskedMimic), where tracker inputs can serve as flexible guidance toward plausible motions. Learned uncertainty based primarily on a motion prior may favor prior-plausible motions over valid but less common user inputs. In contrast, STANCE first uses the motion prior to predict a reference pose, then assesses observation reliability through kinematic consistency. This allows us to selectively correct unreliable measurements while preserving reliable user inputs. Our ablation results in Q2 further demonstrate the benefit of continuous kinematics-based confidence over binary availability masks.

Comments on the paper

1 comment
AuthorsOpenReview ↗

Rebuttal summary for the Reviewers and Area Chair.

We thank all reviewers for their careful reading and the Area Chair for overseeing the discussion. Our responses address every point raised; here we summarise what we take to be the substance of the paper and the evidence added during the discussion. All additions appear in the revised manuscript.

What is distinctive. STANCE is a predict-confidence-correct framework for real-time reconstruction from sparse trackers whose configuration and reliability change at runtime. Its components are coupled, not independently assembled: the predictor gives the residual's reference pose; confidence defines the gated training target, conditions the denoiser, and gates the posterior correction; and a shared residual space anchors scene guidance to the denoiser's estimate. This integration enables two capabilities that, to our knowledge, no prior sparse-tracker method supports jointly. First, one set of weights handles tracker configurations that change within a session, at SOTA accuracy with 3 and 6 trackers and fixed per-frame cost. Second, scene constraints are enforced with no scene seen in training, so the room may change without retraining or a learned collision model. Together they suit collecting embodied-interaction data, where a demo whose hand passes through the object it grasps teaches an agent nothing. Both are illustrated in the supplementary demos.

Confidence from kinematic consistency. A binary mask indicates tracker presence, not measurement reliability. We derive continuous per-joint confidence from the discrepancy between tracker targets and kinematically constrained IK solutions, enabling reliability-aware correction; to our knowledge, prior sparse-tracker methods primarily rely on binary availability. We addressed this in Q6 for Reviewer hADe, who maintained the concern. A matched-condition ablation compares continuous confidence against binary availability, all else fixed, and shows improvements in every metric under both configurations: MPJPE from 5.58 to 5.31 cm and MPJVE from 21.92 to 21.23 cm/s under 3 trackers, and from 1.16 to 1.07 cm and 8.41 to 8.29 cm/s under 6.

Scene consistency without training. Beyond the armrest sequence of Fig. 5 we added two scenarios (Fig. 8, appendix): sitting on a chair, with coupled contact across a backrest and seat in different planes, and leaning against a wall, a horizontal constraint with no support. Over manually verified contact frames in the 4 regions we report 3 mutually exclusive frame-level metrics plus mean penetration depth, separating severity from frequency. Penetration frames fall and contact frames rise in every region, with mean depth dropping from 5.51-35.31 mm to 0.00-4.42 mm, all on one checkpoint with no scene-specific training or weight updates. Measured in our laboratory, these scenarios show that a training-free correction transfers to geometry the model never saw, though they are not a large-scale benchmark, which does not exist for sparse-tracker capture. Where the correction acts also matters: against a Levenberg-Marquardt optimizer (EgoAllo lineage) and a ΠGDM-based mechanism (InPose lineage), everything else held fixed, we sample at 27.09 ms/frame against 67.76 ms (2.50×) and 37.39 ms (1.38×), at 0.335 mm mean depth against 0.010 and 4.704 mm. LM's additional 0.325 mm reduction comes at 2.50× the sampling cost, while ΠGDM corrects the noisy state after the DDIM update and is partly re-noised at each step. Both comparisons are in the revised appendix.

Interpreting the dropout AUJ. AUJ accumulates the frame-wise absolute difference from ground-truth jerk, registering deviation in high-order dynamics, not roughness; jitter, our smoothness metric, is the lowest in Tab. 1. All AUJ contributions arise from frames whose predicted jerk is below ground truth, and we evaluated the half second after each of 702 dropout events in the 138 A-P2 sequences. Despite a higher AUJ, STANCE keeps the largest fraction of ground-truth wrist speed (0.853 vs 0.795 and 0.831), the lowest wrist error (12.25 cm vs 16.14 and 16.54), and the least foot sliding (17.02 cm/s vs 21.84 and 22.85). A plausible explanation is that it preserves the dominant wrist trajectory while attenuating small-amplitude, high-frequency secondary motions, so a higher AUJ need not imply poorer trajectory fidelity.

Presentation. The notation problems raised are now resolved throughout: no symbol serves as both gate and frame index, the inner iterations of the correction carry their own index, token matrices are labelled by type, slerp, clip, yfloory_\mathrm{floor} and both loss terms are defined, FABRIK is cited, and Fig. 2 is redrawn with numbered stages. We are grateful these were pointed out; they concern how the method is described rather than what it does.

We welcome any remaining questions in the final hours. If the revised manuscript and the added evidence address your concerns, we would be grateful if you would reconsider your assessments.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.