acceptodds
Withdrawn from ICLR 2027

TempCalib: Diagnosing and Recovering Extrinsic Drift with a Frozen 3D Detector

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Multi-camera 3D detectors rely on accurate extrinsics to align cross-view observations, yet post-deployment drift can degrade downstream perception. Existing solutions either use dedicated online calibration or improve robustness without explicitly restoring corrupted extrinsics. The former adds a separate estimation procedure, while the latter leaves geometric errors unresolved. We investigate whether the deployed detector itself contains evidence for diagnosis and recovery. We propose TempCalib, a framework for unknown-camera fault diagnosis, localization, and rotational recovery using a frozen multi-camera 3D detector. It extracts reusable detector-derived evidence, detects persistent cross-view inconsistency on a camera graph, and generates target-excluded rotational proposals after localization. Cross-view feature consistency and fixed-correspondence epipolar geometry refine these proposals, and final states are compared on common visible support. Fixed clean references calibrate these signals; neither additional training nor per-candidate backbone or decoder passes are required. On a nuScenes-based benchmark, TempCalib detects and localizes every injected fault across three splits sharing a 96-condition single-camera rotational perturbation grid. On blind Test-A, it achieves a mean rotational residual of under the matched 40-sequence setting, reduced to with the extended recovery budget. On the two validation splits, the L4 configuration restores approximately 91–97% of the condition-level mAP/NDS loss. These results show that a frozen detector can provide evidence for explicit geometric recovery and restoration of its own perception.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
Multi-Camera 3D Object Detection, Camera Extrinsic Calibration, Calibration Drift, Fault Diagnosis, Test-Time Geometric Recovery

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_Hks1OpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper formulates the problem of post-deployment rotational extrinsic drift in multi-camera 3D detectors and proposes TempCalib, a three-stage framework that uses the frozen detector's own internal representations to diagnose, localize, and correct the fault. Stage 0 extracts cross-view descriptors from decoder anchors; Stage 1 detects and localizes the faulty camera via persistent contrast on a camera-pair graph; Stage 2 recovers the rotation through target-excluded sparse proposals, cross-view feature refinement (pitch/roll), and epipolar refinement (yaw), with a dual-branch common-support selector. Evaluated on a nuScenes-based 96-condition benchmark across three splits (including a blind test partition), TempCalib achieves perfect fault localization and substantially outperforms known-target matching baselines, recovering roughly 87–97% of corruption-induced mAP/NDS loss.

Critical strengths
  1. Novel and well-motivated problem formulation. The insight that a detector is not merely a calibration consumer but also a calibration evidence source is original and relevant to real-world autonomous driving deployment. The end-to-end evaluation (from unknown-camera diagnosis through geometric recovery to downstream perception restoration) goes beyond the typical calibration paper and provides a compelling demonstration. The protocol itself — with dev/val/blind-test partitions, condition-wise metrics, retained failures, and oracle diagnostics — sets a good standard.

  2. Efficient inference-time design. The method avoids per-candidate backbone or decoder passes by reprojecting and re-sampling only the target camera's features. This makes the dense axis scan tractable without training any additional networks, which is an appealing property for deployment.

Critical weaknesses
  1. Baseline fairness and the source of the performance gap. TempCalib has access to clean commissioning references (camera-specific yaw offsets βtclean\beta^{\text{clean}}_t, clean rank curves from Reference150) that compensate for systematic biases in the detector's evidence. The external baselines receive no equivalent correction. Given nuScenes' asynchronous camera triggering, standard relative-pose estimators are likely systematically biased, and TempCalib's commissioning implicitly corrects for this. Until the baselines are given comparable debiasing, the magnitude of the claimed advantage remains uncertain. This matters because the headline contribution is that detector evidence outperforms dedicated geometric methods.

  2. Generality claims exceed the evidence. The paper frames its contribution as applicable to frozen multi-camera 3D detectors in general, but all experiments use StreamPETR (three checkpoints). Several Stage 2 components — target-excluded decoder passes, per-head query/key projections, layer-specific anchor readout — are tightly coupled to the PETR architecture. No experiment tests a BEV-lifting or spatiotemporal-attention detector (e.g., BEVFormer, BEVDepth). The scope of the contribution should match the evidence.

Questions for authors
  1. (Clarification, decision-relevant) Can you apply the same Reference150-derived camera-specific offset correction to the external baselines (DeDoDe-B, XFeat) and report updated results? This would help isolate the contribution of detector evidence from asymmetric access to clean references.

  2. (Clarification, decision-relevant) What are the per-camera values of βtclean\beta^{\text{clean}}_t, and do they correlate with inter-camera timestamp offsets or typical ego-motion speed? This would clarify whether the offset primarily compensates for capture asynchrony.

  3. (Clarification, decision-relevant) Was the L4 configuration specified before any Test-A evaluation? The abstract highlights the best numbers across L3, L4, and HighAcc75 on Test-A; clarifying the pre-registration would strengthen the blind-test claim.

  4. (Suggestion for future work) Have you considered a per-camera visual odometry baseline that estimates extrinsics via ego-pose consistency (hand–eye style)? This would avoid the inter-camera overlap and asynchrony issues that disadvantage the current baselines and provide a stronger geometric reference point.

  5. (Clarification) What is the wall-clock time for the full pipeline (Stage 0 through Stage 2) under the Fast40 and HighAcc75 budgets? The paper claims low overhead but provides no runtime measurements.

Reviewer_dzSmOpenReview ↗
Rating
2
Confidence
3
Informativeness
1
Presentation
2
Soundness
1
Read review
Paper summary

This paper proposes TempCalib, a framework for detecting and correcting rotational extrinsic drift using representations extracted from a frozen multi-camera 3D detector. The method combines cross-view feature consistency, detector-derived 3D anchors and epipolar geometry to identify a faulty camera and estimate its rotational correction. On a nuScenes-based benchmark with a frozen StreamPETR, the authors report high fault-localization accuracy and substantial restoration of downstream 3D detection performance.

While the problem is practically relevant, I have several concerns regarding the restrictive problem formulation, the strength of the proposed geometric constraints, and the distinction from existing perception-driven calibration methods.

Critical strengths

The problem is practically relevant, and reusing the deployed detector as a source of calibration evidence is an appealing idea, closing the loop by checking whether its own detection performance is restored. The evaluation keeps failed recoveries in all aggregates and reports results on a blind test split.

Critical weaknesses

My main concern is the restrictive setting. The method assumes that only one camera drifts, that all others remain correctly calibrated and serve as references, and that only the rotation is wrong. No simultaneous drift of several cameras is evaluated. More importantly, only three rotational parameters are estimated. The justification by detector sensitivity is weak. That the detector-derived evidence can only constrain rotation suggests that the constraint itself is weak. I would encourage the authors to evaluate multi-camera and 6-DoF faults and the observability of the objective in all six parameters, or at least to narrow the claims to single-camera rotational recovery.

Missing geometric baseline. The matching baselines estimate a full essential matrix per pair, while TempCalib uses rotation-only geometry, clean references and the current calibration as initialization. Their residuals are identical for medium and large faults, which suggests their output ignores the perturbation. A fair baseline would optimize only the target rotation from LightGlue or RoMa matches under the same priors. Similarly, it is not shown that detector features are needed over photometric or generic features such as DINOv2.

Evaluation protocol. The protocol is new, and FloSR and TESO are adapted to it rather than evaluated under their own protocols, so it is hard to separate method from setting. The faults also change the extrinsics but not the images; warping the target image with K R K^{-1} would simulate a physical rotation.

Overlap and visibility. The method relies on anchors seen by two cameras, but the number of jointly valid anchors per camera pair is not reported. Visibility is checked with the possibly corrupted extrinsics and ignores occlusion.

Generalization. All results use the six-camera nuScenes rig and StreamPETR. The checkpoint study covers only two further StreamPETR variants (R50 and V2-99) on 12 conditions per split, so it does not show transfer to other detector families such as BEV-based models. Many vehicles also do not have a six-camera surround rig, and with fewer cameras or less overlap the camera graph has fewer healthy references. Results on a different rig, such as Argoverse 2, and with a non-PETR detector would show which parts transfer.

Questions for authors

Please see the weaknesses above.

Reviewer_LeVHOpenReview ↗
Rating
2
Confidence
3
Informativeness
3
Presentation
3
Soundness
3
Read review
Paper summary

This paper investigate whether a frozen multi-camera 3D detector can provide evidence for diagnosing and recovering camera extrinsic drift. TempCalib combines persistent camera-graph inconsistency for fault detection and localization, target-excluded detector queries for rotational proposals, cross-view feature consistency and fixed-correspondence epipolar geometry for refinement, and common support comparison for final state selection. It requires clean commissioning references but no additional training or per-candidate backbone/decoder passes.

Critical strengths
  1. The Paper propose a novel approach and direction that use a pre-trained 3D detector as an evidence provider for camera extrinsic drift detection and correction.

  2. The paper connects unknown-camera fault diagnosis, rotational recovery, and restoration of the affected detector. This makes the contribution more consequential than demonstrating lower calibration error alone.

  3. Target exclusion addresses contamination of proposal evidence by the faulty camera, while fixed correspondences make geometric candidate comparisons more consistent. Reusing frozen detector features avoids complete detector inference for every candidate. The ablations support the importance of target exclusion and complementary feature/geometric refinement. Together with post-development and blind evaluation, these results provide a credible demonstration of the central idea.

Critical weaknesses
  1. All experiments use nuScenes.Appendix B.9 provide useful portability results across StreamPETR checkpoints and backbones, but these remain within the same detector family. Since the method relies on decoder queries, query/key projections, and camera-specific clean references, transfer to a different architecture or camera configuration may require adaptation and recommissioning. The paper does not clearly establish which hyperparameters transfer unchanged and which components requires retuning. This limits the evidence for the broader applicability of the proposed framework.

  2. Inaccurate 3D anchors, unreliable features, and camera-specific perception degradation can also reduce cross-view consistency, potentially resembling extrinsic drift. Clean-reference normalization, graph contrast, and persistent aggregation provide reasonable mitigation, but the reported experiments do not isolate these alternative causes. Target exclusion is introduced only after localization and therefore does not resolve this ambiguity in the initial diagnosis. Without controlled evaluations under correct calibration but degraded perception, it remains unclear whether the system could misdiagnose perception errors and propose unnecessary geometric corrections.

  3. Clean-reference construction uses Graph40 and Reference150, while deployment diagnosis aggregates 40 scenes before issuing a fault decision. The reference-subset and recovery-budget experiments provide partial evidence, but do not establish the minimum data needed for reliable diagnosis or stable reference statistics. In particular, the 20-scene recovery experiment retains the 40-scene diagnosis budget. The absence of diagnostic performance versus observation budget and acquisition-time measurements makes it difficult to assess commissioning cost and responsiveness, especially when informative cross-view observations are scarce or drift changes during accumulation.

Questions for authors
  1. Which parts of TempCalib can be transferred unchanged to another dataset, camera rig, or structurally different 3D detector? Please distinguish fixed algorithmic hyperparameters, clean statistics that require recommissioning, and architecture-specific implementation changes. Can the authors provide transfer evidence beyond nuScenes and the StreamPETR family, with any additional tuning explicitly identified?

  2. How does the method distinguish calibration drift from inaccurate detector anchors or persistent camera-specific feature degradation? Can the authors provide a controlled evaluation with correct extrinsics but degraded perception, reporting false-alarm rates and any induced corrections, alongside the corresponding calibration-fault cases?

  3. How do fault recall, false-alarm rate, localization accuracy, and recovery error vary as the commissioning and deployment observation budgets decrease? Please distinguish the data required to construct clean references from the data required after a fault occurs, and quantify the acquisition time before a reliable decision. Does the method provide an observable criterion for determining when sufficient evidence has been collected?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.