- Rating
- 3
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 3
Read review
Paper summary
This paper presents StableCrowd, a feed-forward method for reconstructing 3D human motion from monocular large-scene videos. The method predicts the height of multiple body joints above the ground and uses camera geometry to recover their 3D positions. A temporal fusion network then combines these estimates to produce stable global trajectories. The authors also introduce camera normalization to improve the generalization across different person crops. Experiments on VirtualCrowd and WorldPose show strong results in both spatial accuracy and temporal consistency without per-sequence optimization.
Critical strengths
-
The paper addresses an important and challenging problem. Using multi-joint heights for global localization is a simple but effective idea with a clear geometric motivation.
-
The overall framework is well designed. The proposed components work together to improve both spatial and temporal consistency.
-
The experimental results are strong. The authors compare with several recent methods on both synthetic and real-world datasets. The ablation studies also support the effectiveness of the main components.
Critical weaknesses
-
Incomplete ablation of temporal fusion. The idea of MJH is good, but I wonder how much of the improvement actually comes from GTF. In Table 5, HVIP is tested without temporal fusion, while MJH uses GTF. Could the authors also test HVIP with GTF? This would help show the benefit of MJH itself.
-
Limited analysis of Phase-shifted Coverage. The current ablation only compares whole-sequence prediction with the full phase-shifted method. It would be useful to compare with fixed-length windows with 0% or 50% overlap and simple blending. The effect of different numbers of refinement rounds is also unclear.
-
Sensitivity to camera and ground errors. The method relies on accurate camera and ground geometry, but the experiments do not fully evaluate its robustness to errors in these inputs. This is important for real-world videos where camera and ground parameters need to be estimated.
Questions for authors
-
Could the authors add an experiment using HVIP with the same temporal GTF to better separate the contributions of the representation and temporal fusion?
-
Could the authors compare Phase-shifted Coverage with fixed-length windows using 0% or 50% overlap and report results with one, two, and three refinement rounds?
-
How sensitive is the method to errors in camera and ground estimation? It would be helpful to test different error levels (e.g., 1%–10%) in focal length and camera height, angular errors in the ground normal. Camera height errors are especially worth studying.
-
CCN appears related to existing perspective crop methods, such as PersPose[1]. Could the authors briefly clarify the differences?
-
Could the authors provide a detailed runtime breakdown of the full pipeline, including detection, tracking, camera and ground estimation, CCN, MJH, MHL, and GTF? This would help readers better understand the actual inference cost.
[1] Hao, Xiaoyang, and Han Li. "Perspose: 3d human pose estimation with perspective encoding and perspective rotation." 2025 IEEE/CVF International Conference on Computer Vision (ICCV). IEEE, 2025.
5 comments
Response to Reviewer t2M5 (2/3)
(2/3)
Sensitivity to the scene (Weakness 3; Question 3).
- Input. Like Crowd3D, Crowd4D and DyCrowd, StableCrowd takes the camera and scene surface as its input, calibrated or estimated with existing tools; this input is not among our contributions, and every method we can re-run is given the same one (Sec. 4.2).
- Real-world videos. Large-scene videos from deployed cameras, such as surveillance, are the main target of StableCrowd; even with unknown intrinsics and lens distortion, these cameras can be calibrated accurately and automatically against a 3D reconstruction of the site from an automatically flown drone survey, as shown in Appendix L, Fig. 13, with median reprojection residuals from 0.60 to 2.37 pixels. The revision states this dependence in the limitations, together with the automatic calibration of deployed cameras (Appendix M.4).
- Error levels. The revision perturbs the focal length, camera pitch, camera height and ground orientation one at a time (Table Rc-3): an error of 1 to 10% in the camera height moves each person's ground point by the same share of its distance.
- Estimated scene. Table 2 and Table Rc-4 evaluate StableCrowd with an estimated ground, on VirtualCrowd and, under the protocol of the FIFA Skeletal Light challenge, on WorldPose, where the per-frame intrinsics and lens distortion are given and the camera rotation of every frame after the first, and with it the ground in each camera's frame, is estimated from the pitch lines (Appendix I, Table 23). Against the given camera, the estimated one changes the MPJPE of StableCrowd little and leaves its spatial consistency (PPDS, PA-PPDS) essentially unchanged, StableCrowd remaining far more accurate than TRAM; apart from the one sequence in which the court lines are lost, its rotation error lies far below the angles of Table Rc-3, its pitch component the smallest of the three (Appendix I, Fig. 9). In the wild, on three in-the-wild clips, two of whose scenes are shown in Figs. 12 and 15, estimating the scene surface, or the camera intrinsics as well, moves each person by 4.5 to 6.2% of the distance to the camera on average, while the layout of the people is largely kept (Appendix L, Table 24; Fig. 12 shows the camera intrinsics and scene surface estimated from the video).
Table Rc-3. Error propagation of the focal length, camera pitch, camera height and ground orientation (new; same as Table Ra-3). Ground error as a share (%) of the distance, one factor at a time, in the worse direction. VirtualCrowd: means over the person-frames of the six evaluated videos, whose rays meet the ground at a median of 9°, that the lifting still takes (under the 1, 2 and 5° tilts toward or away, 9.8, 15.5 and 29.4% fall below the 2° threshold and are left out); other columns: a single ray meeting the ground at that angle. Full table: Appendix F, Table 10.
| Error | VirtualCrowd | 45° | 30° | 10° |
|---|---|---|---|---|
| Camera height, 1 / 5 / 10% | 1 / 5 / 10 | 1 / 5 / 10 | 1 / 5 / 10 | 1 / 5 / 10 |
| Focal length, 1 / 5 / 10% | 0.56 / 2.8 / 5.6 | 0.71 / 3.5 / 7.1 | 1.0 / 5.1 / 10 | 3.0 / 16 / 38 |
| Camera pitch or ground tilt toward or away, 1 / 2 / 5° | 13 / 27 / 81 | 1.8 / 3.7 / 10 | 3.1 / 6.5 / 18 | 11 / 25 / 99 |
| Ground tilt sideways, 1 / 2 / 5° | 2.0 / 3.9 / 9.5 | – | – | – |
Table Rc-4. Absolute localization on WorldPose with an estimated ground (new; same as Table Ra-4). The per-frame intrinsics and lens distortion, which change with the zoom, are given in both. Given: the standard protocol (Table 3). Estimated: the protocol of the FIFA Skeletal Light challenge; ours is re-run with it, and TRAM is the row of Table 3, placed in the world by the same camera. Best in bold.
| Ground | Method | MPJPE (m) ↓ | SEQ-RT-MPJPE (mm) ↓ | ACC error (mm/f²) ↓ | PPDS ↑ | PA-PPDS ↑ |
|---|---|---|---|---|---|---|
| Given | TRAM | 2.71 | 1928 | 628.77 | 0.841 | 0.845 |
| Given | Ours | 0.33 | 199 | 2.26 | 0.984 | 0.985 |
| Estimated | TRAM | 2.72 | 1936 | 634.91 | 0.841 | 0.845 |
| Estimated | Ours | 0.36 | 244 | 2.35 | 0.983 | 0.984 |
Response to Reviewer t2M5 (3/3)
(3/3)
CCN and PersPose (Question 4).
Differences. CCN differs from PersPose in two respects.
- Source of the warp rotation. PersPose turns the camera along the shortest arc onto the box center, with no ground plane; CCN fixes the rotation about the optical axis against the ground normal.
- Network input. PersPose encodes the cropped camera as a per-pixel map (its Sec. 4.1 and 4.2); CCN passes the focal length and the residual pitch to the network, which regresses the pose with the pitch set to zero and adds it back analytically.
CCN as a design choice. Both apply the perspective warp of a camera rotating about its optical center and zooming (Hartley & Zisserman), each where its task needs it. On reflection, we regard the StableCrowd pipeline and its core component, MJH, as the substantive technical contributions; CCN is a design choice that serves them. Sec. 4.4 cites PersPose alongside PCL (Yu et al., CVPR 2021), and Appendix M.3 discusses CCN together with PersPose, PCL and Crowd3D++.
Runtime (Question 5). The revision reports the runtime of each stage (Appendix M.2):
- Detection, tracking and keypoints precede any method in a large scene, where cropping is necessary; on 1920$×$1080 frames with about 207 people, they run at 14.9 frames per second (29.0 batched), 70.6 frames per second and 166 person-frames per second (255 batched).
- The camera and scene surface are estimated offline, once per video or once per site, within minutes.
- The stages of StableCrowd run at 4262 (CCN), 174 (image features), 1227 (MJH, 4741 batched), 7825 (MHL) and 1623 (GTF) person-frames per second.
- After the detection, tracking and keypoints, StableCrowd reconstructs 103 person-frames per second, so that a scene of 100 people over 200 frames takes about 195 seconds, against about four hours reported by Crowd4D and DyCrowd (Appendix M.1).
Response to Reviewer t2M5 (1/3)
(1/3)
We thank the reviewer for finding MJH "a simple but effective idea with a clear geometric motivation", the framework "well designed" and that "the experimental results are strong". We would be grateful to hear whether the revision addresses the concerns raised, and will answer any further question promptly within the discussion period.
HVIP with temporal fusion (Weakness 1; Question 1). The revision adds GTF retrained on a single point, HVIP or the anchor height of the body root, in place of the per-joint estimates of MJH, with nothing else changed (Table Rc-1, also added to Table 5). Under the same GTF, MJH is the most accurate, on WorldPose the root height with GTF being slightly smoother at a 30% higher MPJPE, and even with per-frame softmax fusion and no GTF MJH has a lower MPJPE than either single point with GTF (Table 5), so the accuracy does not come from temporal fusion alone.
Table Rc-1. HVIP, root height and MJH with matched temporal decoders (same as Table Ra-1). Cells read VirtualCrowd / WorldPose; GTF† is GTF trained with a single point as its input.
Phase-shifted coverage (Weakness 2; Question 2). The whole-sequence row of Table 5 showed that bounded windows give the accuracy on long sequences. Within windows of one length, a single round with 50% overlap and linear blending already bridges the window boundaries, and the three shifted rounds of the submitted model, whose windows within a round lie end to end, smooth them further at unchanged accuracy (Table Rc-2). The revised appendix reports this table (Appendix I, Table 19).
Table Rc-2. Phase-shifted coverage against windows of one length, by overlap and rounds (new). Cells read VirtualCrowd / WorldPose; best in bold.