acceptodds
Under review as a conference paper at ICLR 2027

PQR3D: Progressive Query Refinement over Reference-Conditioned Temporal Windows for Multi-View 3D Object Detection

Abstract

Temporal context is essential for camera-only multi-view 3D object detection. Existing streaming detectors maintain and propagate query states from one frame to the next, requiring sequence-aware training and chronological inference. We propose PQR3D, which performs progressive query refinement within referenceconditioned temporal windows. This design enables random frame sampling and independent inference without persistent query memory. Within each window, PQR3D progressively transfers motion-aligned high-confidence queries from earlier timestamps toward the target frame. We further introduce masked selfattention to regulate interactions among regular, propagated, and denoising queries while keeping denoising supervision isolated from detection queries. In addition, a stage-decoupled anchor embedding injects position before self-attention and size, orientation, and velocity afterward, reducing interference from temporally inconsistent attributes. Extensive experiments on nuScenes demonstrate consistent improvements across multiple backbones. With a ViT-L backbone, PQR3D sets a new state of the art on the nuScenes test set, achieving 71.6 NDS and 64.9 mAP. Source code is available at https://anonymous.4open.science/r/pqr3d-review-B5E8

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
Computer Vision, Camera-Only 3D Object Detection, Multi-View 3D Perception, Progressive Query Refinement

Reviews

Among:
Shape:
1234this paper, Bottom 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.25 over 4 reviews, above 49% of the others.
Reviewer_PR33OpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The authors study 3D object detection using videos from multiple cameras. Some existing methods continuously pass object queries from the previous frame to the next, so processing the current frame depends on previously accumulated state. PQR3D builds an independent temporal window for each target frame, progressively updates object queries within the window, and passes the queries to the final target frame using ego-motion and object velocity. In addition, the authors adjust the attention rules for denoising queries and the stages at which position, size, orientation, and velocity information is introduced into the network. Experiments are mainly conducted on nuScenes.

Critical strengths

1、The method has practical value. PQR3D builds an independent temporal window for each target frame, using information from multiple frames without relying on query states accumulated over previous frames. This makes it easier to randomly sample target frames during training and process individual frames independently during inference.

2、The experiments provide some support for the effectiveness of the method. The authors show performance improvements with several backbones and conduct ablations on query propagation strategies, the number of propagation steps, the number of retained queries, and positional encoding designs. These experiments help explain how different design choices affect the final performance.

Critical weaknesses

The source of the gains from the core method is not sufficiently established. PQR calls the decoder multiple times, and each refinement step can access the entire temporal window, including the target frame. The experiments do not include a comparison that uses the same inputs and the same number of decoding passes but always updates queries in the target frame’s coordinate system. It is therefore unclear whether the improvement comes from the proposed progressive propagation mechanism or the additional decoding computation. This comparison is needed to establish whether the core design provides a benefit of its own.

The comparison with strong baselines is incomplete, which weakens the claimed performance advantage. The authors compare against a ResNet101 version of Sparse4D v3 trained for 24 epochs. However, the original paper reports 62.3 NDS after the full 100-epoch training schedule with the same backbone and image resolution, which is higher than the 61.2 NDS reported in this paper. Although the proposed method uses fewer training epochs, the number of epochs does not directly reflect the actual computational cost. The current results therefore do not establish an advantage over this strong baseline in either final accuracy or computational efficiency. Results from the full training schedule and comparisons under the same computational budget are needed. Sparse4D v3, Section 4 and Table 1.

Questions for authors

1、With the same input frames and number of decoding passes, does progressive query propagation still outperform repeatedly updating queries in the target frame’s coordinate system?

2、How does PQR3D compare with a fully trained Sparse4D v3 in accuracy and training cost? Can you report the GPU hours for both methods under the same hardware setup?

5 comments

Weaknesses 1. The source of the gains from the core method is not sufficiently established. PQR calls the decoder multiple times, and each refinement step can access the entire temporal window, including the target frame. The experiments do not include a comparison that uses the same inputs and the same number of decoding passes but always updates queries in the target frame’s coordinate system. It is therefore unclear whether the improvement comes from the proposed progressive propagation mechanism or the additional decoding computation. This comparison is needed to establish whether the core design provides a benefit of its own.

Thank you for pointing this out. In our implementation, the input window shifts with each refinement step. With nine input frames, the pass at t-1 uses frames (t-9,...,t-1), and the pass at t uses frames (t-8,...,t). The earlier pass therefore does not see frame t. At the target frame, the decoder combines propagated and regular queries to produce the final detections.

Table 3(a) shows that adding PQR improves NDS from 55.5 to 56.6 and mAP from 45.3 to 46.5. Table 4 shows diminishing returns: a third refinement step improves NDS by only 0.2 points and reduces mAP by 0.1 point. In separate experiments, expanding the window to ten frames or increasing the query count to 1,600 does not yield a clear improvement.

We will compare PQR3D with a single-pass baseline that uses nine input frames and 2,700 regular queries without propagation. This matches the total number of regular query updates across PQR3D’s three passes (900 per pass). We would compare NDS and mAP to see whether using more queries in one pass can recover the accuracy gains from progressive refinement.

Weaknesses 2. The comparison with strong baselines is incomplete, which weakens the claimed performance advantage. The authors compare against a ResNet101 version of Sparse4D v3 trained for 24 epochs. However, the original paper reports 62.3 NDS after the full 100-epoch training schedule with the same backbone and image resolution, which is higher than the 61.2 NDS reported in this paper. Although the proposed method uses fewer training epochs, the number of epochs does not directly reflect the actual computational cost. The current results therefore do not establish an advantage over this strong baseline in either final accuracy or computational efficiency. Results from the full training schedule and comparisons under the same computational budget are needed. Sparse4D v3, Section 4 and Table 1.

Thank you for pointing this out. Sparse4D v3 reports 62.3 NDS and 53.7 mAP after 100 epochs of training with ResNet-101 at 1408 * 512 on the nuScenes validation split. However, the official release does not provide the files needed to reproduce this result:

  1. Training configurations. The official configuration directory provides a Sparse4D v3 training configuration for ResNet-50, but not for ResNet-101, VoV-99, or EVA-02-L (ViT-L).
  2. Checkpoints and training logs. The official repository provides a trained detector checkpoint and training log for the ResNet-50 variant, but does not list either for the ResNet-101, VoV-99, or EVA-02-L variants.

we adapted the released ResNet-50 configuration to ResNet-101 and trained Sparse4D v3 for 24 epochs at 1408 * 512, obtaining 56.1 NDS and 48.0 mAP. This gives us a more comparable evaluation with PQR3D because both runs use the same backbone, input resolution, and number of epochs.

We plan to train PQR3D with ResNet-101 for 100 epochs at the same resolution and evaluate it on the same validation split. This run is specifically for comparison with Sparse4D v3's reported 100-epoch result. For baselines that use substantially shorter training schedules, such as SparseBEV and RayDN, we will retain the existing comparisons.

Question 1. With the same input frames and number of decoding passes, does progressive query propagation still outperform repeatedly updating queries in the target frame’s coordinate system?

Thank you for the question. The sequential baseline follows the approach used in StreamPETR and Sparse4D v3: it caches queries from the previous frame and propagates them into the current frame. With one refinement step, PQR also makes a single transfer into the target frame, but it computes the historical queries within the current window. This makes the one-step setting a more comparable test of how each method brings historical queries into the target frame.

In Table 4, one-step PQR achieves 56.2 NDS and 46.1 mAP, compared with 55.7 NDS and 45.7 mAP for the sequential baseline. These gains of 0.5 NDS and 0.4 mAP points show that PQR’s advantage over sequential propagation does not depend on the second refinement step.

Question 2. How does PQR3D compare with a fully trained Sparse4D v3 in accuracy and training cost? Can you report the GPU hours for both methods under the same hardware setup?

Thank you for the question. All runs below use eight NVIDIA RTX 4090 GPUs. We calculate GPU-hours by multiplying wall-clock training time by eight.

MethodBackboneEpochsNDS ↑mAP ↑Training time (h)GPU-hours
Sparse4D v3ResNet-502451.341.6972
Sparse4D v3ResNet-5010056.146.936288
PQR3DResNet-502458.248.613104
Sparse4D v3ResNet-1012456.148.015120
PQR3DResNet-1012461.252.022176

At 24 epochs, PQR3D achieves higher NDS and mAP with both backbones, but uses more GPU-hours than Sparse4D v3.

Comparison with the full 100-epoch schedule. With ResNet-50, Sparse4D v3 achieves 56.1 NDS and 46.9 mAP in 36 hours (288 GPU-hours). On the same hardware, PQR3D trained for 24 epochs improves on those scores by 2.1 NDS points and 1.7 mAP points while using 104 GPU-hours. We plan to extend the comparison to ResNet-101 by training both PQR3D and Sparse4D v3 for 100 epochs.

Thank you for the clarification and additional GPU-hour comparisons. The R50 results provide useful evidence of improved accuracy with lower training cost relative to the fully trained Sparse4D v3 baseline, partially addressing my second concern.

However, my main concern remains unresolved. The comparison with sequential propagation does not isolate the benefit of progressive reference changes from additional decoding computation. The requested baseline—using the same input frames and number of decoder passes while repeatedly updating queries in the target-frame coordinate system—is still missing. The proposed single-pass baseline with 2,700 queries changes the refinement depth and query interactions, so it would not directly answer this question; moreover, its results have not yet been provided.

For R101, PQR3D’s reported 61.2 NDS / 52.0 mAP remains below Sparse4D v3’s published full-schedule result of 62.3 / 53.7. The additional 24-epoch comparison provides useful context but does not establish an advantage under a matched computational budget. The planned longer-training experiments cannot yet be assessed.

I therefore maintain my rating of 2 (Weak rejection). The additional cost measurements are appreciated, but the controlled comparison needed to establish the core mechanism’s contribution remains absent.

Reviewer_VDtxOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

This paper proposes PQR3D, a method for 3D object detection within multi-view video. The main idea of his paper is to progressively propagete query in discrete temporal windows. Take high-confidence candidates from earlier keyframes and propagate to subsequent ones with motion compensation. This paper uses masked self-attention to control the interaction among various types of query, and stage-decoupled anchor embedding to avoid conflicts during attention calculation. The experimental results show state-of-the-art perfoamance across multiple configureations.

Critical strengths
  1. The proposed method is straightforward and reasonable. Restricting the query propagation within independent windows enbales the model to leverage information from previous frames while avoiding state dependencies and error accumulation.
  2. The experiments across various backbone show the proposed method outperforms exisiting baselines.
Critical weaknesses
  1. The authors claim the main motivation as the sequential propagation, error propagation, and memory management, but the experiments mainly report the detection metrics, without analyzing the actual impact of the above issues. For example, the current experimental results could not demonstrate the capacity of the proposed method for error prapogation.

  2. While this paper provides inference FPS and VRAM usage across various propagation settings, it does not report the training time, number of parameters, FLOPs. Additionally, in Figure 5, it is insufficient to demonstrate the proposed method holds a clear advantage under an equivalent inference budget comparing with SparseBEV, RayDN, and StraemPETR.

Questions for authors

Please refer to the weaknesses.

1 comment
  1. The authors claim the main motivation as the sequential propagation, error propagation, and memory management, but the experiments mainly report the detection metrics, without analyzing the actual impact of the above issues. For example, the current experimental results could not demonstrate the capacity of the proposed method for error prapogation.

Thank you for pointing this out. PQR3D refines high-confidence queries within independent windows, so each target frame can be processed without a persistent query bank.

In Table 1, with a ResNet-50 backbone, PQR3D achieves higher NDS and mAP than StreamPETR, Sparse4Dv2, and Sparse4Dv3. Compared with StreamPETR, it achieves lower translation (mATE), scale (mASE), orientation (mAOE), velocity (mAVE), and attribute (mAAE) errors. It improves four of these five metrics over each Sparse4D variant, with lower orientation and velocity errors than all three baselines. In the controlled comparison in Table 3(b), replacing sequential propagation with within-window refinement improves NDS from 55.7 to 56.6 and mAP from 45.7 to 46.5 under the same detector and training settings.

These results establish better final detection quality. They do not directly measure how individual query errors propagate. Our claim is that an inaccurate query can affect refinement within its own window, but its state is not carried into the next window through persistent memory.

  1. While this paper provides inference FPS and VRAM usage across various propagation settings, it does not report the training time, number of parameters, FLOPs. Additionally, in Figure 5, it is insufficient to demonstrate the proposed method holds a clear advantage under an equivalent inference budget comparing with SparseBEV, RayDN, and StraemPETR.

Thank you for pointing this out. Following the reporting practices of baselines such as SparseBEV, RayDN, and StreamPETR, we did not include training time, parameter count, or FLOPs in the original submission.

The main 900-query PQR3D configuration with a ResNet-50 backbone has 46 million parameters and takes approximately 13 hours to train for 24 epochs on eight RTX 4090 GPUs (104 GPU-hours). The training time is also reported in the anonymous repository provided for review. We have not yet measured full-model FLOPs.

PQR3D refines queries sequentially across historical keyframes within each window. Because each step uses queries from the previous step, refinement adds inference latency compared with a single pass. Figure 5 shows this accuracy–speed trade-off within PQR3D; it does not establish an advantage over the baselines at the same inference budget. Future work could explore parallel query propagation to reduce this overhead.

Reviewer_7DoeOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

PQR3D is a camera-only multi-view 3D detector that progressively refines object queries within a self-contained temporal window for each target frame. At each refinement step, a shared decoder uses a selected timestamp as the reference while aggregating image features from the entire window. The refined queries are aligned to the next reference frame using ego-motion and predicted object velocities. High-confidence queries are then selected and combined with the queries initialized for that frame before further decoding. This process proceeds toward the target timestamp, where the final queries predict object classes and 3D bounding boxes. Each window is processed independently, without retaining query states across target-frame predictions. The method also incorporates attention masking and stage-decoupled anchor embeddings. Experiments on nuScenes evaluate the framework across multiple backbones and examine its components through ablation studies.

Critical strengths
  1. PQR3D consistently outperforms SparseBEV across the evaluated backbone configurations. The component ablations further support the effectiveness of the proposed design.

  2. The framework supports temporal query refinement within self-contained windows without maintaining query states across target samples. This provides flexibility in training and inference by allowing random target-frame sampling and independent processing of temporal windows.

  3. The paper is well organized, and the main exposition is generally easy to follow. The method is presented in a logical sequence, with clear descriptions of its principal components.

Critical weaknesses
  1. The methodological distinction from prior work is not sufficiently established. Query propagation, denoising masks, and anchor encoding are established components of StreamPETR (Wang et al., 2023) and Sparse4Dv3 (Lin et al., 2023), while the sampling and fusion operations are adopted from SparseBEV (Liu et al., 2023). A more thorough analysis of the proposed design choices would help clarify the paper’s methodological insights and substantiate its claims.

  2. The efficiency evaluation is somewhat incomplete. Appendix A.2 reports inference speed, and Tables 4–5 report memory consumption; however, the speed measurements use a 625-query model trained for 48 epochs, whereas the main results use 900 queries and 24 epochs. The computational cost of the main configuration remains insufficiently characterized. Training GPU-hours, inference latency, and peak memory should be reported under consistent settings, with ablations separating the cost of temporal context from that of repeated decoder refinement.

  3. Table 2 omits a strong and directly relevant baseline. Although Sparse4Dv3 appears in the ResNet comparisons, its offline configuration—71.9 NDS and 66.8 mAP with EVA02-Large and future frames—is absent from the test-set comparison. These results exceed the reported 71.6 NDS and 64.9 mAP of PQR3D, albeit under different configurations. The authors should include this baseline, document differences in backbone, pretraining, external data, and temporal inputs, and discuss the resulting trade-offs.

  4. The choice of temporal window is not adequately justified. The default configuration uses the current frame and eight historical frames sampled at approximately 0.5-second intervals, but neither window length nor sampling interval is ablated. Table 4 examines refinement depth, which is a separate design choice. Controlled experiments varying the temporal window at a fixed refinement depth are needed to assess sensitivity to these settings.

  5. The SDAE ablation does not fully explain the proposed design. Table 5(b) compares position-only, coupled, and stage-decoupled embeddings, but does not isolate the contributions of size, orientation, and velocity or systematically examine where each attribute should be introduced. Attribute-wise ablations and alternative injection placements would provide a stronger justification. The claim that SDAE reduces interference between regular and propagated queries also requires evidence beyond improvements in aggregate detection metrics.

  6. The figures are less clear than the accompanying text. Figure 2 does not explicitly explain the query colors or clearly distinguish initial from propagated queries. Reference changes and motion compensation should be indicated more clearly. Notation is also inconsistent: (P_t) denotes detection outputs in Figure 1 but projection matrices in the method. Clearer annotations, consistent notation, and larger labels would improve the presentation.

  7. The qualitative evaluation is too limited to characterize the method’s behavior. Figures 4 and 6 show a small number of successful examples, with limited coverage of scene conditions and object categories. Additional examples, including failure cases, would make the evaluation more informative. Enlarged views of the highlighted regions would also facilitate comparison.

Questions for authors
  1. What are the training and inference costs of the main 900-query configuration, and how do these costs scale with temporal-window size and refinement depth?

  2. How does PQR3D compare with Sparse4Dv3-offline when differences in backbone, pretraining, external data, and temporal inputs are explicitly accounted for?

  3. How sensitive is the method to window length and temporal sampling interval?

  4. Which anchor attributes account for the SDAE improvement, and what evidence supports the proposed injection order and the claim of reduced attribute interference?

  5. Could you revise Figures 1 and 2 during the rebuttal period to make the processing flow easier to follow? The current illustrations are difficult to interpret. Clearer labels and annotations distinguishing temporal windows, reference changes, and query propagation would improve readability.

13 comments

Weakness 1. The methodological distinction from prior work is not sufficiently established. Query propagation, denoising masks, and anchor encoding are established components of StreamPETR (Wang et al., 2023) and Sparse4Dv3 (Lin et al., 2023), while the sampling and fusion operations are adopted from SparseBEV (Liu et al., 2023). A more thorough analysis of the proposed design choices would help clarify the paper’s methodological insights and substantiate its claims.

Thank you for highlighting the connections to prior work. PQR3D builds on established ideas in query propagation, denoising, and anchor encoding, and adopts adaptive spatio-temporal sampling and mixing from SparseBEV. The design differences lie in how PQR3D propagates queries within each temporal window, controls interactions among query types, and injects anchor attributes at different decoder stages.

Query propagation. In their standard streaming setups, StreamPETR and Sparse4D carry query states across successive predictions. PQR3D instead uses each refinement keyframe as the reference for sampling multi-frame features from the same window and jointly refines newly initialized and motion-aligned propagated queries. We discard the query state after predicting the target frame, so target frames can be processed independently. Figure 1 illustrates this distinction by contrasting PQR3D with StreamPETR and Sparse4D v2. Sparse4D v3 retains v2’s recurrent query propagation strategy. Using the same detector, our approach improves NDS by 0.9 points and mAP by 0.8 points over the StreamPETR-style sequential variant (Table 3(b)).

Denoising mask. StreamPETR and SparseBEV already use query denoising, and Sparse4D v3 isolates denoising groups and propagates selected noisy groups during training. PQR3D applies the established isolation principle to within-window refinement: regular and propagated queries can attend to each other, while each denoising group is kept separate from both types of detection queries and from other denoising groups. This lets detection queries exchange temporal information without accessing denoising inputs. Adding masked self-attention improves NDS by 0.4 points and mAP by 0.7 points (Table 3(a)).

Anchor encoding. Unlike Sparse4D v3, which uses the full anchor embedding to construct attention queries and keys, PQR3D adds the position embedding before self-attention and the size, orientation, and velocity embeddings afterward, before feature sampling (Figure 3). This lets position guide query interactions without directly injecting potentially inconsistent box and motion estimates into self-attention. Within PQR3D, this stage-decoupled design achieves 57.7 NDS and 48.2 mAP, compared with 57.2 NDS and 46.9 mAP for coupled encoding and 57.0 NDS and 47.2 mAP for position-only encoding (Table 5(b)).

Weakness 2. The efficiency evaluation is somewhat incomplete. Appendix A.2 reports inference speed, and Tables 4–5 report memory consumption; however, the speed measurements use a 625-query model trained for 48 epochs, whereas the main results use 900 queries and 24 epochs. The computational cost of the main configuration remains insufficiently characterized. Training GPU-hours, inference latency, and peak memory should be reported under consistent settings, with ablations separating the cost of temporal context from that of repeated decoder refinement.

Thank you for raising this concern. Our main ResNet-50 PQR3D configuration uses 900 queries and is trained for 24 epochs, following the SparseBEV training schedule. PQR3D-Light uses 625 queries and is trained for 48 epochs to improve accuracy at the smaller query budget.

We will update Figure 5 to show the main PQR3D configuration alongside PQR3D-Light. We will report accuracy and inference speed for each configuration, timing both on the same GPU with the same procedure. For the main configuration, we will report training GPU-hours, per-sample inference latency, and peak GPU memory.

Table 4 already compares zero to three refinement steps in terms of accuracy and GPU memory. Compared with no propagation, two steps improve NDS by 1.1 points and mAP by 1.2 points, while GPU memory increases from 8.1 to 8.7 GB. We will extend Table 4 with inference latency measurements at a fixed temporal window to quantify the additional cost of repeated refinement.

Weakness 3. Table 2 omits a strong and directly relevant baseline. Although Sparse4Dv3 appears in the ResNet comparisons, its offline configuration—71.9 NDS and 66.8 mAP with EVA02-Large and future frames—is absent from the test-set comparison. These results exceed the reported 71.6 NDS and 64.9 mAP of PQR3D, albeit under different configurations. The authors should include this baseline, document differences in backbone, pretraining, external data, and temporal inputs, and discuss the resulting trade-offs.

Thank you for suggesting this comparison. Sparse4Dv3-offline reports 71.9 NDS and 66.8 mAP on the nuScenes test split using EVA-02-L at an input resolution of 1600 × 640. However, the official release does not provide the files needed to reproduce this result:

  1. Training configurations. The official configuration directory provides a Sparse4D v3 training configuration for ResNet-50, but not for ResNet-101, VoV-99, or EVA-02-L (ViT-L).
  2. Checkpoints and training logs. The official repository provides a trained detector checkpoint and training log for the ResNet-50 variant, but does not list either for the ResNet-101, VoV-99, or EVA-02-L variants.
SettingPQR3DSparse4Dv3-offline
BackboneEVA-02-L (ViT-L)EVA-02-L (ViT-L)
Pretrained checkpoint referenceeva02_L_coco_seg_sys_o365.pth (COCO instance segmentation)eva02_L_coco_det_sys_o365.pth (COCO object detection)
Future frames88
Training epochs24100
LiDAR depth supervisionNoYes
External 3D detection dataNoneNone

We trained PQR3D for 24 epochs to match SparseBEV’s training schedule and limit computational cost. The ViT-L run took about two weeks on eight NVIDIA A100 GPUs with 80 GB of memory each. At the same training rate, 100 epochs would take about two months. Our ViT-L model had not yet converged at epoch 24. We will report the 48-epoch result to quantify the effect of longer training.

Without the offline configuration, we cannot reproduce Sparse4Dv3-offline under a matched 24-epoch schedule. We provide PQR3D’s source code and training configurations to support reproducibility.

Weakness 4. The choice of temporal window is not adequately justified. The default configuration uses the current frame and eight historical frames sampled at approximately 0.5-second intervals, but neither window length nor sampling interval is ablated. Table 4 examines refinement depth, which is a separate design choice. Controlled experiments varying the temporal window at a fixed refinement depth are needed to assess sensitivity to these settings.

Thank you for pointing this out. Table 4 varies the number of refinement steps, not the temporal window length. We retain an approximately 0.5-second sampling interval, which matches the 2 Hz nuScenes keyframe rate and is common among temporal baselines. SparseBEV, for example, uses the current frame and seven historical frames at this interval.

With the sampling interval and refinement depth held fixed, we compared seven, eight, and nine historical frames. Using eight instead of seven improved NDS by about 0.1 points. Adding a ninth historical frame increased computation without further improving NDS. We therefore use eight historical frames and will include this window-length ablation in the revised paper.

Weakness 5. The SDAE ablation does not fully explain the proposed design. Table 5(b) compares position-only, coupled, and stage-decoupled embeddings, but does not isolate the contributions of size, orientation, and velocity or systematically examine where each attribute should be introduced. Attribute-wise ablations and alternative injection placements would provide a stronger justification. The claim that SDAE reduces interference between regular and propagated queries also requires evidence beyond improvements in aggregate detection metrics.

Thank you for pointing this out. The embedding widths are an empirical design choice that matches the allocation in the released Sparse4D v3 anchor encoder. We allocate 128 dimensions to position because it provides a common spatial reference for regular and propagated queries, 64 to velocity to capture motion across frames, and 32 each to size and yaw to represent object extent and heading.

Table 5(b) shows that adding size, yaw, and velocity together after self-attention improves NDS by 0.7 points and mAP by 1.0 point over the position-only baseline. To measure each additional attribute’s contribution, we will add size, yaw, or velocity one at a time after self-attention, keeping position before self-attention in all runs.

Regular queries start from learned anchors, whereas propagated queries carry box estimates from earlier frames. Even when two queries are spatially aligned, their size, yaw, or velocity estimates may differ. In the coupled design, these differences can directly affect how strongly the queries attend to each other. SDAE injects position before self-attention and adds size, yaw, and velocity afterward. This removes a direct way for those attribute mismatches to influence attention in that layer while preserving geometry and motion cues for box refinement. Compared with the coupled design, SDAE improves NDS by 0.5 points and mAP by 1.3 points (Table 5(b)), supporting the proposed injection order.

Weakness 6. The figures are less clear than the accompanying text. Figure 2 does not explicitly explain the query colors or clearly distinguish initial from propagated queries. Reference changes and motion compensation should be indicated more clearly. Notation is also inconsistent: PtP_t denotes detection outputs in Figure 1 but projection matrices in the method. Clearer annotations, consistent notation, and larger labels would improve the presentation.

Thank you for the helpful suggestions. Please see the revised Figures 1 and 2 in the updated submission.

In Figure 1, we replaced PtP_t with Y^t\widehat{\mathcal{Y}}_t to denote the predicted 3D detections. We reserve Pt\mathbf{P}_t for the projection matrix in Section 3.1.

In Figure 2, we added two examples of mixed query inputs. In each example, the bars on the left represent propagated queries, and those on the right represent newly initialized queries. The colors indicate when the queries were initialized or last refined: gray for t−2t-2, green for t−1t-1, and purple for tt. We also standardized the legend’s font size and spacing.

Section 3.1 (Eqs. 2–5) describes the reference change and motion compensation. When the reference advances from t−1t-1 to tt, we use ego-motion and velocity compensation to align the queries refined at t−1t-1 to the coordinate frame at tt. We then select the top-kk aligned queries by classification confidence and concatenate them with the newly initialized queries at tt.

Reviewer_6T4pOpenReview ↗
Rating
2(−1)
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper presents PQR3D, a temporal query refinement framework for multi-view 3D object detection. It replaces persistent frame-to-frame query memory with reference-conditioned temporal windows, progressively aligns and refines high-confidence queries, and introduces masked self-attention and stage-decoupled anchor embeddings. On nuScenes validation, its ResNet-50 configuration achieves 58.2 NDS / 48.6 mAP.

Critical strengths

1.This paper presents PQR3D, a progressive query refinement framework that processes each reference-conditioned temporal window independently and refines object queries in temporal order toward the target frame, without carrying memory across target samples.

  1. The proposed PQR3D achieves promising performance on the nuScenes.
Critical weaknesses

1.PQR3D repeatedly selects high-quality queries through Top-K selection and further refines them. However, distant objects may provide limited information and have low confidence, making them prone to being missed. How does PQR3D recover objects that are missed during the initial Top-K selection?

2.In Figure 4, the ground truth is shown in blue, while predictions are shown in other colors. However, it is difficult to distinguish them. I recommend enlarging the relevant regions to better illustrate PQR3D’s advantage in reducing false negatives. Figure 6 has a similar issue.

3.It is recommended that the authors compare PQR3D with other methods on the KITTI and Waymo datasets to further demonstrate the effectiveness and generalizability of the proposed method.

Questions for authors

1.PQR3D repeatedly selects high-quality queries through Top-K selection and further refines them. However, distant objects may provide limited information and have low confidence, making them prone to being missed. How does PQR3D recover objects that are missed during the initial Top-K selection?

2.In Figure 4, the ground truth is shown in blue, while predictions are shown in other colors. However, it is difficult to distinguish them. I recommend enlarging the relevant regions to better illustrate PQR3D’s advantage in reducing false negatives. Figure 6 has a similar issue.

3.It is recommended that the authors compare PQR3D with other methods on the KITTI and Waymo datasets to further demonstrate the effectiveness and generalizability of the proposed method.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.