- Rating
- 2
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
The paper intended to improve the generalization capability, across different scenes or configurations, for multi-view pedestrian detection (MVPD). The paper proposed a query-based framework PGMVG, which iteratively refine the 3D quadrics on the ground and their 2D conics in individual camera views by explicit projective geometry. Thus, there is no scene-specific BEV representation needed. As the major technical contribution, the Supporting Constraints simplify and resolve the ill-posed problem by imposing a physical shape prior on the quadric parameterization (9 to 4 parameters) in Eq.5, e.g., the quadric of a standing person has roughly equal horizonal semi-axes and similar vertical axe on the ground. Further, the structural decoupling reduces to sequential center translation and radius estimation. PGMVG is implemented as a decoder-only DETR using pre-trained ResNet50 as the backbone. The performance of the proposed MVPD is competitive on 3 public benchmark datasets.
Critical strengths
The proposed PGMVG employed a simplified upright ellipsoid representation of a pedestrian to allow iteratively refinement of the 3D quadrics and 2D conics using explicit projective geometry. This approach relaxes the scene-specific limitation of previous MVPD methods.
The experiments show very competitive performance of MVPD in the cross-scene and cross-configuration settings.
The query based PGMVG is a reasonable way to leverage the simplified assumption of upright standing persons. This work is a perfect match to the scope of CVPR/ICCV, yet loosely related to learning representations.
Critical weaknesses
The purpose of enhancing the generalization capability for MVPD is to improve the robustness of MVPD in practical video surveillance systems. The major concern is whether the physical shape prior of pedestrians can hold in real-world video surveillance systems, any measurement? How sensitive of the proposed method if persons are not perfectly upright and ground is not perfectly flat? Or what if some pedestrians are kids?
If the specific multiple cameras have to be well calibrated and perfectly synchronized in one scene as pre-processing, then it means each MVPD system has to be tuned and deployed scene-specific, right?
The query compression (QC) including two operations, i.e., center selector and query aggregator, appears some heuristics to make the query-based method work.
Questions for authors
What are the “typical standing human dimensions” (Line 250) and their distributions?
Line 169, what does “the orientation is immaterial to a ground position” means? The orientation is irrelevant to its position? Line 170: => parameterization
4 comments
W2. If the specific multiple cameras have to be well calibrated and perfectly synchronized in one scene as pre-processing, then it means each MVPD system has to be tuned and deployed scene-specific, right?
Response
Calibration and synchronization do not make deployment scene-specific. They are one-time, label-free steps of the camera installation for scene setup, not of the model. It is a standard assumption of MVPD shared by every method we compare against. Bottom-up methods warp image features onto the ground plane through the same camera matrices for any scenes, and multi-view 3D detection in autonomous driving (nuScenes, Waymo) relies on the same assumption (calibrated and synchronized cameras).
Small errors in both are tolerated in practice. WildTrack is a real installation with ~50 ms synchronization accuracy and calibration from manually annotated correspondences (Chavdarova et al., 2018), so our real-world results already include real system errors.
What makes deployment scene-specific is retraining or adapting the detection model for each scene (like MVUDA), and PGMVG skips this. Camera parameters enter its parameter-free geometric path (projection and triangulation), so a trained model is deployed to a new scene or camera layout by ONLY supplying its camera matrices, with NO retraining on target-scene data.
W3. The query compression (QC) including two operations, i.e., center selector and query aggregator, appears some heuristics to make the query-based method work.
Response:
We respectfully clarify that QC is not a heuristic needed to make the query-based method work, but an answer to a known failure mode of one-to-one matching. Different scenes have different dimensions. To stay scene-agnostic, PGMVG initializes its queries on a uniform grid over the ground plane rather than at learned positions (Sec. 3). This causes several queries converging on the same pedestrian. Under one-to-one matching, these near-duplicates of a matched query are forced to be negatives, which weakens or even reverses the positive’s gradient, as analyzed in DDQ-DETR (Zhang et al., CVPR 2023). Crowding and occlusion in MVPD amplify this negative behavior of query-based framework, since neighboring queries share most of their multi-view evidence.
QC removes these duplicates in two steps. (1) The Center Selector picks distinct queries by their learned classification scores, using soft-NMS instead of DDQ’s hard NMS so that genuine neighbors in dense crowds are down-weighted rather than suppressed. Its main hyperparameter, the compression ratio , gives stable performance between 0.2 and 0.3 (Fig. 4) and is fixed across all datasets and generalization settings. (2) The Query Aggregator is fully learned (Eqs. 12–13) with no hyperparameter. Each selected query attends to all queries to recover the scene context it lacks on its own (+1.6 MODA, Tab. 11).
Finally, QC is not what makes PGMVG work. Without it ( = 1). PGMVG still trains and detects pedestrians, and QC improves its MODA by about 6 points (Fig. 4).
Grammar. Line 170: => parameterization
Response
"Parametrization" and "parameterization" are both accepted spellings of the same word, and both are in common use in the literature. However, to address the reviewer's concern, we will update it as parameterization.
Dear Reviewer,
Thank you again for your thoughtful review. We have posted a response addressing each of your points:
-
Shape prior. We measured the quadric dimensions triangulated from ground-truth boxes on WildTrack, MultiViewX, and GMVD, showing that the quadric adapts to multiple sizes of different pedestrians. We also addressed children as pedestrians, non-upright pedestrians, and non-flat ground.
-
Calibration and deployment. Calibration is a one-time, label-free installation step required by all MVPD methods, and PGMVG transfers to new scenes without retraining.
-
Query compression. QC adapts our query-based framework to dense crowds, removing near-duplicate queries with a learned aggregator and hyperparameters fixed across all datasets.
We also clarified the typical human dimensions, and the orientation statement in Line 169.
We would be grateful to know whether these address your concerns, and we would welcome further discussion if anything remains unclear. As only two days remain in the discussion period, we would appreciate any additional comments at your earliest convenience.
Thank you for your time and consideration.
Follow up on Response
Dear Reviewer T6Py,
Thank you again for your review. We have addressed your concerns and questions.
We are looking forward to having your feedback, especially if there are any remaining questions.
As your evaluation weighs significantly in the final decision, we would greatly appreciate your thoughts before the discussion period closes on November 18.
Thank you.
We sincerely thank the reviewer for the careful reading and constructive feedback. We appreciate the recognition that our upright ellipsoid representation, combined with explicit projective geometry, relaxes the scene-specific limitation of previous MVPD methods, and that PGMVG achieves competitive performance in cross-scene and cross-configuration settings. We address each concern below.
W1a. The purpose of enhancing the generalization capability for MVPD is to improve the robustness of MVPD in practical video surveillance systems. The major concern is whether the physical shape prior of pedestrians can hold in real-world video surveillance systems, any measurement?
Q1. What are the “typical standing human dimensions” (Line 250) and their distributions?
Response:
The physical shape prior (Sec. 2.2) fixes only the form of the quadric: upright, horizontally isotropic (s1=s2=r), and touching the ground. Since MVPD annotates only ground positions, without 3D extent or orientation, these constraints make the triangulation well-posed. Removing the shape prior costs 5.2 MODA (Tab. 7). The quadric’s size and position are not fixed. They are adaptively updated per pedestrian by triangulation from the refined 2D detections (Eqs. 7 and 9). The “typical standing human dimensions” (Line 250) (tz = 0.8 m (height 1.6 m) and r = 0.25 m (App. E)) refer to the query initialization, and these dimensions are subsequently updated per pedestrian.
As MVPD benchmarks provide no human dimension labels, we provide approximate human dimensions by triangulating a quadric from the ground-truth boxes of every annotated pedestrian:
On WildTrack, the boxes are projections of an annotated 3D cylinder (Chavdarova et al., 2018), so the recovered dimensions are nearly constant and reflect the cylinder rather than the pedestrians (the 0.8 cm error confirms the accuracy of the solver). On the synthetic MultiViewX and GMVD, the quadric adapts to heights from 1.17 to 2.17 m and radii from 0.17 to 0.44 m.
W1b. How sensitive of the proposed method if persons are not perfectly upright and ground is not perfectly flat? Or what if some pedestrians are kids?
Response:
The quadric acts as an approximate bounding volume that follows each pedestrian’s observed extent. Hence, a child yields a smaller quadric. Moreover, the 2D boxes of a crouching, sitting, or leaning pedestrian enclose its actual extent in every view, so the triangulated quadric from those detections will also adapt to its pose. Although only approximate, its center stays close to the annotated foot point. The deviation in the above table, which reflects foot stride, arm swing, and similar differences in body shape, is 2.7–10.6 cm on average, well below the 0.5 m matching threshold (App. D).
Regarding the non-flat ground, both the center and the extent of the quadric are triangulated in full 3D, so ground elevation does not affect the estimated ground position (t^x,t^y). The ground plane constraint only affects the quadric’s vertical placement, not its position. Bottom-up methods, by contrast, localize from foot-region features projected onto the plane z=0, so they rely on both visible feet and a flat ground.
Q2. Line 169, what does “the orientation is immaterial to a ground position” means? The orientation is irrelevant to its position?
Response:
Yes, the orientation of the quadric does not affect its ground position. (tx,ty) is the quadric’s center, and rotating a quadric about its center does not move that center. Since MVPD annotates only ground positions, the orientation is neither supervised nor needed for the output, so we fix R(θ)=I3.