- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
The authors study 3D object detection using videos from multiple cameras. Some existing methods continuously pass object queries from the previous frame to the next, so processing the current frame depends on previously accumulated state. PQR3D builds an independent temporal window for each target frame, progressively updates object queries within the window, and passes the queries to the final target frame using ego-motion and object velocity. In addition, the authors adjust the attention rules for denoising queries and the stages at which position, size, orientation, and velocity information is introduced into the network. Experiments are mainly conducted on nuScenes.
Critical strengths
1、The method has practical value. PQR3D builds an independent temporal window for each target frame, using information from multiple frames without relying on query states accumulated over previous frames. This makes it easier to randomly sample target frames during training and process individual frames independently during inference.
2、The experiments provide some support for the effectiveness of the method. The authors show performance improvements with several backbones and conduct ablations on query propagation strategies, the number of propagation steps, the number of retained queries, and positional encoding designs. These experiments help explain how different design choices affect the final performance.
Critical weaknesses
The source of the gains from the core method is not sufficiently established. PQR calls the decoder multiple times, and each refinement step can access the entire temporal window, including the target frame. The experiments do not include a comparison that uses the same inputs and the same number of decoding passes but always updates queries in the target frame’s coordinate system. It is therefore unclear whether the improvement comes from the proposed progressive propagation mechanism or the additional decoding computation. This comparison is needed to establish whether the core design provides a benefit of its own.
The comparison with strong baselines is incomplete, which weakens the claimed performance advantage. The authors compare against a ResNet101 version of Sparse4D v3 trained for 24 epochs. However, the original paper reports 62.3 NDS after the full 100-epoch training schedule with the same backbone and image resolution, which is higher than the 61.2 NDS reported in this paper. Although the proposed method uses fewer training epochs, the number of epochs does not directly reflect the actual computational cost. The current results therefore do not establish an advantage over this strong baseline in either final accuracy or computational efficiency. Results from the full training schedule and comparisons under the same computational budget are needed. Sparse4D v3, Section 4 and Table 1.
Questions for authors
1、With the same input frames and number of decoding passes, does progressive query propagation still outperform repeatedly updating queries in the target frame’s coordinate system?
2、How does PQR3D compare with a fully trained Sparse4D v3 in accuracy and training cost? Can you report the GPU hours for both methods under the same hardware setup?
5 comments
Question 1. With the same input frames and number of decoding passes, does progressive query propagation still outperform repeatedly updating queries in the target frame’s coordinate system?
Thank you for the question. The sequential baseline follows the approach used in StreamPETR and Sparse4D v3: it caches queries from the previous frame and propagates them into the current frame. With one refinement step, PQR also makes a single transfer into the target frame, but it computes the historical queries within the current window. This makes the one-step setting a more comparable test of how each method brings historical queries into the target frame.
In Table 4, one-step PQR achieves 56.2 NDS and 46.1 mAP, compared with 55.7 NDS and 45.7 mAP for the sequential baseline. These gains of 0.5 NDS and 0.4 mAP points show that PQR’s advantage over sequential propagation does not depend on the second refinement step.
Question 2. How does PQR3D compare with a fully trained Sparse4D v3 in accuracy and training cost? Can you report the GPU hours for both methods under the same hardware setup?
Thank you for the question. All runs below use eight NVIDIA RTX 4090 GPUs. We calculate GPU-hours by multiplying wall-clock training time by eight.
| Method | Backbone | Epochs | NDS ↑ | mAP ↑ | Training time (h) | GPU-hours |
|---|---|---|---|---|---|---|
| Sparse4D v3 | ResNet-50 | 24 | 51.3 | 41.6 | 9 | 72 |
| Sparse4D v3 | ResNet-50 | 100 | 56.1 | 46.9 | 36 | 288 |
| PQR3D | ResNet-50 | 24 | 58.2 | 48.6 | 13 | 104 |
| Sparse4D v3 | ResNet-101 | 24 | 56.1 | 48.0 | 15 | 120 |
| PQR3D | ResNet-101 | 24 | 61.2 | 52.0 | 22 | 176 |
At 24 epochs, PQR3D achieves higher NDS and mAP with both backbones, but uses more GPU-hours than Sparse4D v3.
Comparison with the full 100-epoch schedule. With ResNet-50, Sparse4D v3 achieves 56.1 NDS and 46.9 mAP in 36 hours (288 GPU-hours). On the same hardware, PQR3D trained for 24 epochs improves on those scores by 2.1 NDS points and 1.7 mAP points while using 104 GPU-hours. We plan to extend the comparison to ResNet-101 by training both PQR3D and Sparse4D v3 for 100 epochs.
Thank you for the clarification and additional GPU-hour comparisons. The R50 results provide useful evidence of improved accuracy with lower training cost relative to the fully trained Sparse4D v3 baseline, partially addressing my second concern.
However, my main concern remains unresolved. The comparison with sequential propagation does not isolate the benefit of progressive reference changes from additional decoding computation. The requested baseline—using the same input frames and number of decoder passes while repeatedly updating queries in the target-frame coordinate system—is still missing. The proposed single-pass baseline with 2,700 queries changes the refinement depth and query interactions, so it would not directly answer this question; moreover, its results have not yet been provided.
For R101, PQR3D’s reported 61.2 NDS / 52.0 mAP remains below Sparse4D v3’s published full-schedule result of 62.3 / 53.7. The additional 24-epoch comparison provides useful context but does not establish an advantage under a matched computational budget. The planned longer-training experiments cannot yet be assessed.
I therefore maintain my rating of 2 (Weak rejection). The additional cost measurements are appreciated, but the controlled comparison needed to establish the core mechanism’s contribution remains absent.
Weaknesses 1. The source of the gains from the core method is not sufficiently established. PQR calls the decoder multiple times, and each refinement step can access the entire temporal window, including the target frame. The experiments do not include a comparison that uses the same inputs and the same number of decoding passes but always updates queries in the target frame’s coordinate system. It is therefore unclear whether the improvement comes from the proposed progressive propagation mechanism or the additional decoding computation. This comparison is needed to establish whether the core design provides a benefit of its own.
Thank you for pointing this out. In our implementation, the input window shifts with each refinement step. With nine input frames, the pass at t-1 uses frames (t-9,...,t-1), and the pass at t uses frames (t-8,...,t). The earlier pass therefore does not see frame t. At the target frame, the decoder combines propagated and regular queries to produce the final detections.
Table 3(a) shows that adding PQR improves NDS from 55.5 to 56.6 and mAP from 45.3 to 46.5. Table 4 shows diminishing returns: a third refinement step improves NDS by only 0.2 points and reduces mAP by 0.1 point. In separate experiments, expanding the window to ten frames or increasing the query count to 1,600 does not yield a clear improvement.
We will compare PQR3D with a single-pass baseline that uses nine input frames and 2,700 regular queries without propagation. This matches the total number of regular query updates across PQR3D’s three passes (900 per pass). We would compare NDS and mAP to see whether using more queries in one pass can recover the accuracy gains from progressive refinement.
Weaknesses 2. The comparison with strong baselines is incomplete, which weakens the claimed performance advantage. The authors compare against a ResNet101 version of Sparse4D v3 trained for 24 epochs. However, the original paper reports 62.3 NDS after the full 100-epoch training schedule with the same backbone and image resolution, which is higher than the 61.2 NDS reported in this paper. Although the proposed method uses fewer training epochs, the number of epochs does not directly reflect the actual computational cost. The current results therefore do not establish an advantage over this strong baseline in either final accuracy or computational efficiency. Results from the full training schedule and comparisons under the same computational budget are needed. Sparse4D v3, Section 4 and Table 1.
Thank you for pointing this out. Sparse4D v3 reports 62.3 NDS and 53.7 mAP after 100 epochs of training with ResNet-101 at 1408 * 512 on the nuScenes validation split. However, the official release does not provide the files needed to reproduce this result:
we adapted the released ResNet-50 configuration to ResNet-101 and trained Sparse4D v3 for 24 epochs at 1408 * 512, obtaining 56.1 NDS and 48.0 mAP. This gives us a more comparable evaluation with PQR3D because both runs use the same backbone, input resolution, and number of epochs.
We plan to train PQR3D with ResNet-101 for 100 epochs at the same resolution and evaluate it on the same validation split. This run is specifically for comparison with Sparse4D v3's reported 100-epoch result. For baselines that use substantially shorter training schedules, such as SparseBEV and RayDN, we will retain the existing comparisons.