acceptodds
Under review as a conference paper at ICLR 2027

Projective Geometry for Generalizable Multi-View Pedestrian Detection

Abstract

Multi-view pedestrian detection (MVPD) seeks to localize pedestrians on the ground plane by gathering observations from multiple cameras. Recent MVPD methods achieve strong in-domain performance by projecting per-view features onto a shared Bird's-Eye-View (BEV) plane, where localization is performed. Since BEV construction couples learned representation with camera geometry, their spatial reasoning becomes strongly biased toward scene-specific occupancy patterns, limiting generalization to unseen environments. Rather than expanding the training distribution, we address generalization by integrating projective geometry into a scene-agnostic framework. Specifically, each pedestrian is represented as a ground-anchored 3D quadric and its 2D representation in image view is modeled as a projected 2D conic, connected by an exact, invertible () mapping under the pinhole model. As 3D localization is fully lifted from multi-view 2D predictions through camera parameters, the framework learns no scene-specific priors and naturally inherits the cross-scene transferability of pretrained image detectors. Based on this hypothesis, we propose **PGMVG** , a query-based framework that iteratively refines per-view detections and yields pedestrian locations through constrained projective triangulation. Extensive experiments across multiple benchmarks show that PGMVG improves MODA by on cross-scene and on sim-to-real settings, while maintaining strong in-domain performance.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
pedestrian detection, multi-view detection, multi-view geometry, 3D localization, BEV localization

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_T6PyOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper intended to improve the generalization capability, across different scenes or configurations, for multi-view pedestrian detection (MVPD). The paper proposed a query-based framework PGMVG, which iteratively refine the 3D quadrics on the ground and their 2D conics in individual camera views by explicit projective geometry. Thus, there is no scene-specific BEV representation needed. As the major technical contribution, the Supporting Constraints simplify and resolve the ill-posed problem by imposing a physical shape prior on the quadric parameterization (9 to 4 parameters) in Eq.5, e.g., the quadric of a standing person has roughly equal horizonal semi-axes and similar vertical axe on the ground. Further, the structural decoupling reduces to sequential center translation and radius estimation. PGMVG is implemented as a decoder-only DETR using pre-trained ResNet50 as the backbone. The performance of the proposed MVPD is competitive on 3 public benchmark datasets.

Critical strengths

The proposed PGMVG employed a simplified upright ellipsoid representation of a pedestrian to allow iteratively refinement of the 3D quadrics and 2D conics using explicit projective geometry. This approach relaxes the scene-specific limitation of previous MVPD methods.

The experiments show very competitive performance of MVPD in the cross-scene and cross-configuration settings.

The query based PGMVG is a reasonable way to leverage the simplified assumption of upright standing persons. This work is a perfect match to the scope of CVPR/ICCV, yet loosely related to learning representations.

Critical weaknesses

The purpose of enhancing the generalization capability for MVPD is to improve the robustness of MVPD in practical video surveillance systems. The major concern is whether the physical shape prior of pedestrians can hold in real-world video surveillance systems, any measurement? How sensitive of the proposed method if persons are not perfectly upright and ground is not perfectly flat? Or what if some pedestrians are kids?

If the specific multiple cameras have to be well calibrated and perfectly synchronized in one scene as pre-processing, then it means each MVPD system has to be tuned and deployed scene-specific, right?

The query compression (QC) including two operations, i.e., center selector and query aggregator, appears some heuristics to make the query-based method work.

Questions for authors

What are the “typical standing human dimensions” (Line 250) and their distributions?

Line 169, what does “the orientation is immaterial to a ground position” means? The orientation is irrelevant to its position? Line 170: => parameterization

4 comments

We sincerely thank the reviewer for the careful reading and constructive feedback. We appreciate the recognition that our upright ellipsoid representation, combined with explicit projective geometry, relaxes the scene-specific limitation of previous MVPD methods, and that PGMVG achieves competitive performance in cross-scene and cross-configuration settings. We address each concern below.

W1a. The purpose of enhancing the generalization capability for MVPD is to improve the robustness of MVPD in practical video surveillance systems. The major concern is whether the physical shape prior of pedestrians can hold in real-world video surveillance systems, any measurement?

Q1. What are the “typical standing human dimensions” (Line 250) and their distributions?

Response:

The physical shape prior (Sec. 2.2) fixes only the form of the quadric: upright, horizontally isotropic (s1=s2=rs_1 = s_2 = r), and touching the ground. Since MVPD annotates only ground positions, without 3D extent or orientation, these constraints make the triangulation well-posed. Removing the shape prior costs 5.2 MODA (Tab. 7). The quadric’s size and position are not fixed. They are adaptively updated per pedestrian by triangulation from the refined 2D detections (Eqs. 7 and 9). The “typical standing human dimensions” (Line 250) (tzt_z = 0.8 m (height 1.6 m) and rr = 0.25 m (App. E)) refer to the query initialization, and these dimensions are subsequently updated per pedestrian.

As MVPD benchmarks provide no human dimension labels, we provide approximate human dimensions by triangulating a quadric from the ground-truth boxes of every annotated pedestrian:

DatasetNumber of pedestriansHeight (m) min / mean / maxRadius (m) min / mean / maxGround error from GT boxes (cm) mean / p95
WildTrack3131.73/1.79/1.810.20/0.25/0.340.8/2.0
MultiViewX3501.17/1.82/2.170.17/0.29/0.4110.6/23.6
GMVD2,8001.53 /1.74/1.890.26/0.31/0.442.7/8.0

On WildTrack, the boxes are projections of an annotated 3D cylinder (Chavdarova et al., 2018), so the recovered dimensions are nearly constant and reflect the cylinder rather than the pedestrians (the 0.8 cm error confirms the accuracy of the solver). On the synthetic MultiViewX and GMVD, the quadric adapts to heights from 1.17 to 2.17 m and radii from 0.17 to 0.44 m.

W1b. How sensitive of the proposed method if persons are not perfectly upright and ground is not perfectly flat? Or what if some pedestrians are kids?

Response:

The quadric acts as an approximate bounding volume that follows each pedestrian’s observed extent. Hence, a child yields a smaller quadric. Moreover, the 2D boxes of a crouching, sitting, or leaning pedestrian enclose its actual extent in every view, so the triangulated quadric from those detections will also adapt to its pose. Although only approximate, its center stays close to the annotated foot point. The deviation in the above table, which reflects foot stride, arm swing, and similar differences in body shape, is 2.7–10.6 cm on average, well below the 0.5 m matching threshold (App. D).

Regarding the non-flat ground, both the center and the extent of the quadric are triangulated in full 3D, so ground elevation does not affect the estimated ground position (t^x,t^y)(\hat t_x, \hat t_y). The ground plane constraint only affects the quadric’s vertical placement, not its position. Bottom-up methods, by contrast, localize from foot-region features projected onto the plane z=0z = 0, so they rely on both visible feet and a flat ground.

Q2. Line 169, what does “the orientation is immaterial to a ground position” means? The orientation is irrelevant to its position?

Response:

Yes, the orientation of the quadric does not affect its ground position. (tx,ty)(t_x, t_y) is the quadric’s center, and rotating a quadric about its center does not move that center. Since MVPD annotates only ground positions, the orientation is neither supervised nor needed for the output, so we fix R(θ)=I3R(\theta) = I_3.

W2. If the specific multiple cameras have to be well calibrated and perfectly synchronized in one scene as pre-processing, then it means each MVPD system has to be tuned and deployed scene-specific, right?

Response

Calibration and synchronization do not make deployment scene-specific. They are one-time, label-free steps of the camera installation for scene setup, not of the model. It is a standard assumption of MVPD shared by every method we compare against. Bottom-up methods warp image features onto the ground plane through the same camera matrices for any scenes, and multi-view 3D detection in autonomous driving (nuScenes, Waymo) relies on the same assumption (calibrated and synchronized cameras).

Small errors in both are tolerated in practice. WildTrack is a real installation with ~50 ms synchronization accuracy and calibration from manually annotated correspondences (Chavdarova et al., 2018), so our real-world results already include real system errors.

What makes deployment scene-specific is retraining or adapting the detection model for each scene (like MVUDA), and PGMVG skips this. Camera parameters enter its parameter-free geometric path (projection and triangulation), so a trained model is deployed to a new scene or camera layout by ONLY supplying its camera matrices, with NO retraining on target-scene data.

W3. The query compression (QC) including two operations, i.e., center selector and query aggregator, appears some heuristics to make the query-based method work.

Response:

We respectfully clarify that QC is not a heuristic needed to make the query-based method work, but an answer to a known failure mode of one-to-one matching. Different scenes have different dimensions. To stay scene-agnostic, PGMVG initializes its queries on a uniform grid over the ground plane rather than at learned positions (Sec. 3). This causes several queries converging on the same pedestrian. Under one-to-one matching, these near-duplicates of a matched query are forced to be negatives, which weakens or even reverses the positive’s gradient, as analyzed in DDQ-DETR (Zhang et al., CVPR 2023). Crowding and occlusion in MVPD amplify this negative behavior of query-based framework, since neighboring queries share most of their multi-view evidence.

QC removes these duplicates in two steps. (1) The Center Selector picks distinct queries by their learned classification scores, using soft-NMS instead of DDQ’s hard NMS so that genuine neighbors in dense crowds are down-weighted rather than suppressed. Its main hyperparameter, the compression ratio α\alpha, gives stable performance between 0.2 and 0.3 (Fig. 4) and is fixed across all datasets and generalization settings. (2) The Query Aggregator is fully learned (Eqs. 12–13) with no hyperparameter. Each selected query attends to all queries to recover the scene context it lacks on its own (+1.6 MODA, Tab. 11).

Finally, QC is not what makes PGMVG work. Without it (α\alpha = 1). PGMVG still trains and detects pedestrians, and QC improves its MODA by about 6 points (Fig. 4).

Grammar. Line 170: => parameterization

Response

"Parametrization" and "parameterization" are both accepted spellings of the same word, and both are in common use in the literature. However, to address the reviewer's concern, we will update it as parameterization.

Dear Reviewer,

Thank you again for your thoughtful review. We have posted a response addressing each of your points:

  • Shape prior. We measured the quadric dimensions triangulated from ground-truth boxes on WildTrack, MultiViewX, and GMVD, showing that the quadric adapts to multiple sizes of different pedestrians. We also addressed children as pedestrians, non-upright pedestrians, and non-flat ground.

  • Calibration and deployment. Calibration is a one-time, label-free installation step required by all MVPD methods, and PGMVG transfers to new scenes without retraining.

  • Query compression. QC adapts our query-based framework to dense crowds, removing near-duplicate queries with a learned aggregator and hyperparameters fixed across all datasets.

We also clarified the typical human dimensions, and the orientation statement in Line 169.

We would be grateful to know whether these address your concerns, and we would welcome further discussion if anything remains unclear. As only two days remain in the discussion period, we would appreciate any additional comments at your earliest convenience.

Thank you for your time and consideration.

AuthorsOpenReview ↗

Follow up on Response

Dear Reviewer T6Py,

Thank you again for your review. We have addressed your concerns and questions.

We are looking forward to having your feedback, especially if there are any remaining questions.

As your evaluation weighs significantly in the final decision, we would greatly appreciate your thoughts before the discussion period closes on November 18.

Thank you.

Reviewer_FHuwOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

This paper studies 3D pedestrian detection from multi-view images, focusing on the cross-scene generalization task. While learning-based BEV detection is a popular paradigm for 3D object detection, the authors argue that such a paradigm can overfit to specific scene and camera settings, and they propose the PGMVG framework to address this limitation. Specifically, the key components of this proposed framework are:

  1. Pedestrian location triangulation with strong geometric assumptions: Instead of regressing the 3D locations of pedestrians in a learning-based manner, the authors propose to infer 3D locations from 2D box observations via triangulation. Each pedestrian is represented as a quadric on the ground plane, and the observed conics in each image are used to triangulate 3D pedestrian parameters using the LfD approach (Rubino et al., 2017). To solve the triangulation problem with weak 2D observations, the authors simplify the pedestrian quadric parameterization using assumptions about a standing human and a physical shape prior, where pedestrian centers are first obtained through DLT from conic centers.

  2. Multi-task training: The triangulation process is embedded within a DETR-style detector trained with a multi-task objective. For each pedestrian query, its quadric is projected onto each image for feature extraction and cross-view feature fusion. The resulting pedestrian features compute a residual to update the projected conic, and the updated conics (2D observations) are then used to estimate 3D pedestrian parameters via triangulation. Training objectives include 2D box supervision and ground location supervision.

  3. Query compression: To facilitate model convergence, the authors propose selecting a subset of pedestrian queries via soft-NMS during training. This prevents many near-ground-truth queries from being misclassified as negative predictions, which would otherwise produce conflicting gradients that disturb model convergence.

The authors validate the proposed approach on cross-scene generalization and provide a helpful ablation study to justify the key components of the framework. However, two out of the three benchmarks used in this study are synthetic. On the only realistic benchmark, the proposed framework does not outperform previous studies. Along with other weaknesses and questions detailed below, my current recommendation is a weak rejection prior to the discussion period.

Critical strengths
  1. The ablation study is well-designed, effectively justifying the model design choices and the query compression strategy.

  2. The cross-scene performance of the proposed approach is strong and well-supported by extensive quantitative results.

Critical weaknesses
  1. Weak In-domain Performance and Over-reliance on Geometry: The in-domain quantitative results are not strong enough for ICLR. For 3D pedestrian detection (and 3D object detection more broadly), both in-domain performance and cross-scene generalization are critical. Only one realistic benchmark is used in this study, on which the in-domain results are inferior compared to prior work like MCIVA. Furthermore, the proposed geometry-based triangulation framework actually degrades in-domain performance as shown in Table 6. This is likely due to overly strict geometric assumptions; a better balance should be struck rather than optimizing solely for the cross-domain problem using such a rigid geometric solution.

  2. Overly Restrictive Geometric Assumptions: The framework relies on a rigid geometric assumption—while most pedestrians can be approximated by an upright quadric, they deform significantly in many real-world situations. This quadric assumption limits the modeling of diverse pedestrian postures. The paper lacks an analysis of how posture variations or structural asymmetry introduce errors into the triangulation process. The framework uses hard triangulation with no learnable parameters to correct pedestrian locations when geometric priors fail. A more robust design might blend learning-based approaches with soft geometric constraints.

Questions for authors
  1. What happens when a pedestrian is heavily occluded across most views and visible in only one or two cameras? Is the geometric path still robust if the available conic centers are inaccurate?

  2. A large portion of the paper is dedicated to quadric geometry, but is this quadric formulation strictly necessary? A conventional 3D bounding box can also represent pedestrians. What are the specific advantages of the quadric representation over standard bounding boxes?

  3. The geometric formulation assumes an ideal setup, but real-world applications frequently suffer from camera calibration and synchronization errors. How robust is this framework to those common adverse conditions?

8 comments

We sincerely thank the reviewer for the thorough review and constructive feedback. We appreciate the recognition that our ablation study effectively justifies the design choices, including query compression, and that PGMVG’s cross-scene performance is strong and well supported by extensive quantitative results. We address each concern below.

General comment: two out of the three benchmarks used in this study are synthetic.

W1a. Weak In-domain Performance and Over-reliance on Geometry: The in-domain quantitative results are not strong enough for ICLR. For 3D pedestrian detection (and 3D object detection more broadly), both in-domain performance and cross-scene generalization are critical. Only one realistic benchmark is used in this study, on which the in-domain results are inferior compared to prior work like MCIVA.

Response:

Our real-world evaluation is not limited to the in-domain setting: 5 of 6 generalization settings test on real WildTrack data, through cross-configuration (Tab. 3) and sim-to-real transfer (Tab. 4). These settings follow the generalization protocol of GMVD (Vora et al., 2023), also adopted by MVFP, TMVD, and MVUDA. We strictly follow the protocol for fair comparison:

Setting (test: WildTrack)PGMVGBest prior, no target dataMVUDA (unlabeled target data)
Cross-config, Exp. A81.878.7 (TMVD)79.2
Cross-config, Exp. B81.170.4 (TMVD)81.2
Sim-to-real, 6 views85.076.7 (MVFP)–
Sim-to-real, 7 views85.182.6 (MVFP)85.4
Sim-to-real (GMVD)87.285.6 (MVFP)–

Among methods that use no target-domain data, PGMVG is best in all five settings, and it has the highest MODP in all five (by 3.1–5.6 points). MVUDA leads by 0.1 and 0.3 MODA in two settings, but it needs unlabeled target-scene data, while PGMVG does not. We agree that more real-world benchmarks would be valuable. However, annotating calibrated, synchronized multi-camera data is costly. WildTrack remains the only real-world benchmark on which recent MVPD methods are evaluated, and every MVPD benchmark introduced since then is synthetic (MultiViewX, GMVD, MVP-Occ, SynMVCrowd), as are recent multi-camera tracking challenges (AI City 2024–2025). Sim-to-real transfer is therefore a central focus of our work, and it is where PGMVG gains the most (+8.3 MODA).

We agree that in-domain accuracy matters, and PGMVG is competitive there. PGMVG is NOT INFERIOR to MCIVA overall. It achieves higher MODA (94.0 vs. 93.4) and precision (97.5 vs. 95.3)]. On MultiViewX, it achieves the best MODP (94.1). Moreover, WildTrack and MultiViewX take their training and test frames from one continuous recording (App. C; also noted by GMVD), so in-domain gains can come from memorizing the scene. Specifically, among those evaluated on both benchmarks, the 4 methods lead on WildTrack (93.4–94.2) and MultiViewX (95.5–96.4) (in-domain setting), yet span nearly 10 MODA on GMVD (71.2–81.0) (Tabs. 2 & 5) (cross-scene generalization).

MethodWildTrack MODAMultiViewX MODAGMVD MODA
MVDGC94.295.571.2
MVFP94.195.773.3
PGMVG94.095.981.0
MCIVA93.496.475.9

To provide the evidences for the visual overfitting, we replace only the geometric branch with direct regression in our model (Tab. 6). The gain is 0.3–1.3 MODA in-domain but loses 23.5–30.4 MODA under transfer, a clear sign of scene overfitting.

W1b. Furthermore, the proposed geometry-based triangulation framework actually degrades in-domain performance as shown in Table 6. This is likely due to overly strict geometric assumptions; a better balance should be struck rather than optimizing solely for the cross-domain problem using such a rigid geometric solution.

Response: PGMVG does not apply its geometry rigidly. Each view’s conic is corrected by a learned residual Δcv\Delta c_v over 4 decoder layers. View attention (Eq. 10) exchanges evidence across views (+1.3 MODA, Tab. 10), and learned weights wvw_v softly down-weight unreliable views in the triangulation (Eq. 24). Removing this soft corrections costs −4.7 MODA (Tab. 12).

W2. Overly Restrictive Geometric Assumptions: The framework relies on a rigid geometric assumption—while most pedestrians can be approximated by an upright quadric, they deform significantly in many real-world situations. This quadric assumption limits the modeling of diverse pedestrian postures. The paper lacks an analysis of how posture variations or structural asymmetry introduce errors into the triangulation process. The framework uses hard triangulation with no learnable parameters to correct pedestrian locations when geometric priors fail. A more robust design might blend learning-based approaches with soft geometric constraints.

Response:

Since the quadric is triangulated from 2D detections that enclose the pedestrian in any pose, it acts as an approximate bounding volume that adapts its size and position to that pose. A child, or a crouching, sitting, or leaning pedestrian, yields a smaller or wider quadric. The shape prior only makes this triangulation well-posed, as MVPD annotates no 3D extent or orientation (Sec. 2.2). As MVPD benchmarks provide no posture labels, we provide approximate human dimensions by triangulating a quadric from the ground-truth boxes of every annotated pedestrian:

DatasetNumber of pedestriansHeight (m) min / mean / maxRadius (m) min / mean / maxGround error from GT boxes (cm) mean / p95
WildTrack3131.73/1.79/1.810.20/0.25/0.340.8/2.0
MultiViewX3501.17/1.82/2.170.17/0.29/0.4110.6/23.6
GMVD2,8001.53 /1.74/1.890.26/0.31/0.442.7/8.0

On WildTrack, the boxes are projections of an annotated 3D cylinder (Chavdarova et al., 2018), so the dimensions are nearly constant, and the 0.8 cm error confirms the accuracy of the solver. On the synthetic MultiViewX and GMVD, the quadric adapts to heights from 1.17 to 2.17 m and radii from 0.17 to 0.44 m. As an approximate bounding volume, the quadric still localizes each body well. Its center deviates from the annotated foot point by 2.7–10.6 cm, well below the 0.5m matching threshold (App. D).

Q1. What happens when a pedestrian is heavily occluded across most views and visible in only one or two cameras? Is the geometric path still robust if the available conic centers are inaccurate?

Response:

In PGMVG, View attention exchanges features across all views (Eq. 10), so the features used for classification and per-view prediction also draw on the views where the pedestrian is visible. The quadric center is then recovered by confidence-weighted triangulation (Eq. 24): each view’s constraints are scaled by a learned confidence wvw_v, so its influence on the solution scales with wv2w_v^2 (App. B.4). Therefore, the one or two clear views will dominate the quadric center triangulation, while the inaccurate conic centers of occluded views contribute little. Removing this weighted triangulation costs 4.7 MODA (Tab. 12). Fig. 14 (ID 1, App. G) shows a pedestrian visible in only one camera, detected by PGMVG but missed by EarlyBird. PGMVG can still degrade when nearly all views are occluded (Sec. 4), which we leave to future work on part-aware representations.

Q2. A large portion of the paper is dedicated to quadric geometry, but is this quadric formulation strictly necessary? A conventional 3D bounding box can also represent pedestrians. What are the specific advantages of the quadric representation over standard bounding boxes?

Response:

MVPD provides NO 3D extent or orientation labels, so the 3D params must be recovered from per-view 2D boxes in closed form, which is also what lets localization inherit the 2D detector’s generalization. The quadric allows this and a 3D box does not:

(1) Unique, closed-form relation to 2D boxes. A dual quadric projects to a dual conic linearly (Eq. 3), so triangulation is a closed-form least-squares solve. Moreover, our ellipsoid of revolution is invariant to rotation about the vertical axis, so its projection does not depend on orientation. The 2D projection of 3D box encloses its 8 projected corners, which depend on the box’s yaw even when its width and length are equal. With an arbitrary, unannotated orientation, a 3D box thus has non-unique 2D (box) projections. Any assumed orientation yields 2D boxes inconsistent with those observed across views, so the 3D box has no unique, closed-form inverse.

(2) End-to-end differentiable geometry. Projection and triangulation are closed-form matrix operations, so gradients propagate from the ground loss back to the per-view features (Sec. 2.2). The tight box of a conic is likewise a smooth function of its parameters (App. B.1), giving well-defined gradients to the 2D box loss. For a 3D bounding box, the 2D box is a non-smooth min/max over 8 projected corners, and recovering the box would require differentiating through an unstable iterative fit.

Q3. The geometric formulation assumes an ideal setup, but real-world applications frequently suffer from camera calibration and synchronization errors. How robust is this framework to those common adverse conditions?

Response:

Calibrated, synchronized camera setup is a standard assumption of MVPD, shared by every method we compare against, as they are in multi-view 3D detection for autonomous driving (nuScenes, Waymo). Within this setting, WildTrack is itself a real installation, with ~50 ms synchronization accuracy and calibration from manually annotated correspondences. All our WildTrack results, including the five real-target transfer settings, therefore already include real calibration and synchronization error.

To quantify robustness to calibration error, we perturb each camera’s extrinsics at test time with zero-mean Gaussian noise, independently per axis, following the noise model of BEVFormer (Li et al., 2022). We compare against EarlyBird, a representative bottom-up method with public code and weights, which receives identical perturbations. Results are extracted from MultiViewX, averaged over 5 seeds:

Noise (radians, cm)PGMVG (MODA / MODP)EarlyBird (MODA / MODP)
None95.9 / 94.194.2 / 90.1
Rotation 0.002594.7 / 91.793.3 / 88.7
Rotation 0.00591.5 / 86.789.6 / 83.7
Rotation 0.007585.6 / 80.981.2 / 78.0
Rotation 0.0177.0 / 75.070.3 / 72.9
Translation 2.5 cm95.3 / 91.893.7 / 89.4
Translation 5 cm93.1 / 87.592.1 / 85.2
Translation 7.5 cm88.6 / 82.386.4 / 79.4
Translation 10 cm82.2 / 76.877.0 / 73.7
(R+T) 0.0025 + 2.5 cm93.7 / 88.992.9 / 86.5
(R+T) 0.005 + 5 cm88.0 / 79.986.4 / 77.5
(R+T) 0.0075 + 7.5 cm76.9 / 70.171.9 / 67.4
(R+T) 0.01 + 10 cm62.4 / 60.352.1 / 57.7

PGMVG outperforms EarlyBird in both MODA and MODP at every noise level. Its MODA advantage grows with the noise, from 1.7 points without noise to 6.8, 5.2, and 10.3 points at the largest rotation, translation, and combined levels, while its MODP consistently leads of 2–3 points throughout. Together, these results show that PGMVG, under calibration errors, degrades more gracefully than a bottom-up method that relies on the same camera matrices.

Reviewer_xnmqOpenReview ↗
Rating
3(+1)
Confidence
2
Informativeness
2
Presentation
3(+1)
Soundness
2
Read review
Paper summary

This paper proposes PGMVG, a multi-view pedestrian detector designed to generalize across scenes and camera configurations. It represents pedestrians as upright ellipsoids anchored to the ground, projects them into each camera view, and iteratively refines the resulting 2D ellipses using image features. The 3D estimates are then updated through confidence-weighted triangulation with separate center and radius estimation. The framework also uses attention over sampled points and camera views to combine visual evidence, along with query compression to consolidate redundant predictions. Experiments on WildTrack, MultiViewX, and GMVD cover cross-scene, cross-configuration, and sim-to-real transfer, as well as in-domain detection. The reported results show strong transfer performance and competitive in-domain accuracy, with ablations examining the contributions of the geometric formulation, attention, and query compression.

Critical strengths
  1. The method show strong results under scene and camera changes. These gains, obtained without target-scene adaptation, address a practical limitation of multi-view pedestrian detection. The transfer results are the strongest evidence for the value of the proposed method.

  2. The ablations support the geometric localization design.

Critical weaknesses

The claimed exactness of the geometric formulation is overstated. Eqs. (7) and (21) equate the center of the projected ellipse with the projection of the 3D ellipsoid center, which does not generally hold under perspective projection. This may be a reasonable approximation in the evaluated settings, but the paper does not state its assumptions or quantify the resulting localization bias. Since an exact 2D–3D geometric mapping is central to the paper’s motivation, this leaves a gap between the theoretical claims and the proposed solver.

Questions for authors

The first stage already recovers (t_z), but Eqs. (9) and (25) include (tz2)(t_z^2) among the quantities solved by SVD. Is it fixed to the first-stage estimate or re-estimated in the second stage? If re-estimated, is that value used in the final quadric, or discarded after extracting the radius? Please make the treatment of this variable and the associated constraint explicit.

5 comments

We sincerely thank the reviewer for the careful and constructive review. We appreciate the recognition that PGMVG achieves strong results under scene and camera changes without target-scene adaptation, addressing a practical limitation of MVPD, and that our ablations support the geometric localization design. We address the reviewer’s concern and question below.

Weakness: The claimed exactness of the geometric formulation is overstated. Eqs. (7) and (21) equate the center of the projected ellipse with the projection of the 3D ellipsoid center, which does not generally hold under perspective projection. This may be a reasonable approximation in the evaluated settings, but the paper does not state its assumptions or quantify the resulting localization bias. Since an exact 2D–3D geometric mapping is central to the paper’s motivation, this leaves a gap between the theoretical claims and the proposed solver.

Response:

The quadric–conic formulation itself is exact. The projection (Eq. 3) and the triangulation of the dual quadric (Eq. 4) involve no approximation. The approximation is induced by our structural decoupling (Sec. 2.2). The center estimation (Eqs. 7 & 21) treats the conic center as the projected quadric center, while under perspective projection, the conic center is slightly offset from it. The exact quadric triangulation is ill-conditioned, and decoupling fix this problem (+ 18.7 MODA, Tab. 7) at the cost of the small error quantified below. We will explicitly state this approximation in the paper.

We placed 800 human-size quadrics (rr = 0.25 m, tzt_z = 0.8 m) at positions sampled uniformly over the ground plane of WildTrack and MultiViewX. For each quadric, we computed its exact projected conic (Eq. 3) in every view, then triangulated the conic centers with Eq. 7. For each quadric with center X=(tx,ty,tz,1)⊤X = (t_x, t_y, t_z, 1)^\top and each view vv that observes it, the image offset is the pixel distance between the center cvc_v of its exact projected conic (Eq. 3) and the projection of its center:

δv=∥cv−π(PvX)∥2,\delta_v = \lVert c_v - \pi(P_v X) \rVert_2,

where π(⋅)\pi(\cdot) denotes dehomogenization, measured at the original 1920×1080 resolution. The ground error is the ground-plane distance between the center triangulated from {cv}\{c_v\} with Eq. 7 and the true center:

e=∥(t^x,t^y)−(tx,ty)∥2.e = \lVert (\hat t_x, \hat t_y) - (t_x, t_y) \rVert_2.

DatasetDepth(m)Image offset (pixel) median / p95 / maxGround error (cm) median / p95 / max
WildTrack<516.3/38.9/66.80.63/4.72/9.10
5–105.0/11.0/15.70.22/0.85/3.98
10–201.5/3.2/4.60.21/0.55/0.78
>200.45/0.78/1.240.46/0.61/0.62
Avg1.23/9.81/66.80.34/1.44/9.10
MultiViewX<513.7/44.6/60.00.62/2.11/3.82
5–103.1/6.6/7.840.27/0.70/1.03
10–200.89/1.76/2.140.34/0.56/0.73
>200.38/0.48/0.510.27/0.46/0.48
Avg0.90/8.00/60.00.34/1.03/3.82

The image (center projection) offset decreases with distance. Its median is 14–16 pixels within 5 m of a camera (max ≈ 67 px), 0.9–1.5 px at 10–20 m, and below 0.5 px beyond 20 m. Because the triangulation draws on every view observing the pedestrian, a large offset in a nearby view is outweighed by the small offsets of farther views. Therefore, the ground error (of the reconstructed center) stays small: 0.34 cm median, under 5 cm for over 99% of positions, and never above 10 cm. The largest errors (up to 9.1 cm) occur only within 5 m of the nearest camera, and even these are more than 5× below the 0.5 m matching threshold (App. D).

We will update our revisions as follows: We state the approximation explicitly at Eqs. 7 & 21 and add the analysis above to the appendix.

Question: The first stage already recovers (t_z), but Eqs. (9) and (25) include among the quantities solved by SVD. Is it fixed to the first-stage estimate or re-estimated in the second stage? If re-estimated, is that value used in the final quadric, or discarded after extracting the radius?

Response:

tzt_z is re-estimated in the second stage, and the re-estimated value is used in the final quadric. The first-stage tzt_z is used to center the quadric, which removes the scale disparity between translation and shape, so stage 2 is well-conditioned (structural decoupling, Sec. 2.2). The condition s3=tzs_3 = t_z (physical shape prior, Sec. 2.2) is not imposed in either solve, but when the final quadric is assembled with Eq. 5. We use the second-stage value because stage 1 estimates tzt_z from conic centers alone, which are subject to the center approximation discussed above, whereas stage 2 uses the conics for shape triangulation.

We will update our revisions as follows: we will state explicitly in Sec. 2.2 and Appendices B.5–B.6 that the final tzt_z is taken from the second stage, with the first-stage value serving only as the reference center for the shape estimation.

Dear Reviewer,

Thank you again for your thoughtful review. We have posted a response addressing your concern and question. We agree that the center triangulation is an approximation, and we quantified the resulting error over the full ground plane of WildTrack and MultiViewX. We also revised the “exact” claim in the center triangulation accordingly. In addition, we clarified the treatment of tzt_z. It is re-estimated in the second stage and used in the final quadric. Both of them has been revised in the paper, with all changes highlighted in red.

We would be grateful to know whether these address your concerns, and we would welcome further discussion if anything remains unclear. As only two days remain in the discussion period, we would appreciate any additional comments at your earliest convenience.

Thank you for your time and consideration.

AuthorsOpenReview ↗

Follow up on Response

Dear Reviewer xnmq,

Thank you again for your review. We have addressed your concerns and questions.

We would love to hear your feedback, especially if there are any remaining questions. As your evaluation weighs significantly in the final decision, we would greatly appreciate your thoughts before the discussion period closes on November 18.

Thank you.

Comments on the paper

1 comment
AuthorsOpenReview ↗

General Response: Summary of Strengths and How We Addressed the Main Concerns

Dear Area Chair and Reviewers,

We thank the reviewers for their careful and constructive reviews. We have answered each review point by point and uploaded a revised PDF, with all changes highlighted in red. Below, we summarize the strengths the reviewers recognized and how we addressed their main concerns.

Contributions and strengths recognized by the reviewers

  • Contribution
    • A new representation for MVPD: each pedestrian is a 3D quadric, lifted from 2D detections through explicit projective geometry, which relaxes the scene-specific limitation of previous MVPD methods (T6Py). This representation and the following framework enables generalization.
    • All three reviewers recognized the strong generalization under scene and camera placement changes without target-scene adaptation, addressing a practical limitation of MVPD (xnmq); strong cross-scene performance, well supported by extensive results (FHuw); and very competitive cross-scene and cross-configuration performance (T6Py).
  • Soundness
    • The ablation study is well designed and justifies the model design choices, including query compression (FHuw), and supports the geometric localization design (xnmq).
  • Presentation
    • Reviewers FHuw and T6Py found the paper clear and easy to follow.

How we addressed the main concerns

  1. Center triangulation and treatment of tzt_z (xnmq). We agree that the center triangulation is an approximation and quantified it over the ground plane of 2 datasets. The ground-plane error is 0.34 cm median, which is negligible compared to pedestrian size. We also specified that tzt_z is re-estimated in the second stage. Both points are now stated explicitly in the paper, and the error analysis has been added to the ablation study.
  2. Real-world evaluation (FHuw). 5 of 6 generalization settings test on real WildTrack data, where PGMVG is best among all methods without target-domain data. In-domain, PGMVG is comparable to MCIVA (higher MODA and precision) and competitive to existing methods. Table 6 shows that in-domain gains on continuous-video splits can reflect scene overfitting.
  3. Shape prior and posture (FHuw, T6Py). The shape prior makes quadric triangulation well-posed under MVPD’s weak supervision. Quadric size and position adapt to varied sizes of pedestrians.
  4. Rigidity and occlusion (FHuw). The quadric geometry is not rigid. The triangulation is corrected by learned conic residuals, view attention, and confidence-weighted, so that occluded views are less likely to impede the quadric recontruction (Tab. 12).
  5. Calibration and synchronization (FHuw, T6Py). Calibrated, synchronized cameras are part of the MVPD problem setting. We further evaluate robustness to calibration error by perturbing camera extrinsics at test time, and PGMVG outperforms EarlyBird in both MODA and MODP at every noise level.
  6. Quadric vs. 3D bounding box (FHuw). The quadric admits a unique, closed-form, orientation-free relation to 2D boxes, which a 3D bounding box does not.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.