acceptodds
Withdrawn from ICLR 2027

EasyCalib: Online Spatio-Temporal Radar-Camera Calibration from Object Observations

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Radar and cameras are widely paired in vehicles, roadside infrastructure, and robotics, yet online calibration from routine object observations remains difficult. Camera detection centers and radar returns may describe different physical points; measurement errors differ, and object motion couples pose with timing. We present EasyCalib, which estimates six spatial perturbations and an added radar timestamp offset from observations accumulated over time. Its XYT representation encodes planar position and reported time while retaining visual height as an attribute. A coarse pose estimate is refined through rigid-transform composition, then soft candidate association estimates the offset after spatial compensation. In one forward pass, EasyCalib outputs 3D translation, 3D rotation, and the offset without supplied point correspondences or a pre-estimated delay. We evaluate pose perturbations, offset ranges, training continuations, and perception frontends on an internal nuScenes split with disjoint recording logs. At the widest R4–D4 setting ( m per translation axis, per rotation axis, and s added offset), mean per-axis translation and rotation MAEs are 51.89 cm and , and added-offset MAE is 238.45 ms. Spatial refinement reduces translation and rotation errors by 10.5% and 23.1%; offset error is 52.3% below a zero-offset control.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
Radar–camera calibration, Spatiotemporal calibration, Multimodal perception, End-to-end learning, Extrinsic calibration, Temporal alignment

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_fEpMOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
1
Soundness
2
Read review
Paper summary

EasyCalib is a spatial-temporal method for calibrating the extrinsic between a camera and radar sensor and the time offset between their respective data sequences. Specifically, EasyCalib uses a BEV-like representation, denoted as XYT, to accomplish this joint task. For evaluation, they evaluate on NuScenes and perturb the ground-truth extrinsic with previous works' perturbation values and demonstrate that their method outperforms previous baselines. Other experiments and ablation validate their 3 stage coarse to final processing and other architecture decisions.

Critical strengths

The model is able to output the extrinsic between the camera and radar sensors and also output the time offset. The model indeed outperforms its baselines in both reproduced results and results reported by earlier works.

Critical weaknesses

Although the EasyCalib does outperform its baselines, its translation error is at least 30 cm. In the context of calibration, translation is indeed a much harder task compared to rotation, and the high error could partly be explained by the larger perturbation R2-D2. Still, it remains unexplained why EasyCalib struggles on the translation correction. Additionally, assuming the y-axis corresponds to the height dimension (if extrinsic is w.r.t the camera frame), the y-axis error being lower than the x and z axes is a surprising considering that the domination representation is BEV-like.

More broadly, table results are provided but are not elaborated on. For example, in Table 1, it is not surprising that the more difficult R4-D4 perturbation level yields worse calibration. Aside from that, what are the takeaways?

Questions for authors

Could the authors provide reasoning for why EasyCalib struggles with high error on the evaluation benchmark?

Are baselines such as LCCNet and CalibDepth, which are trained for lidar-camera calibration, retrained on NuScenes for the camera-radar calibration task?

Could the authors clarify the difference between previously-developed BEV representation and XYT representation? Is the only difference the time dimension?

Could qualitative visualizations be provided to visually observe the calibration quality?

1 comment
AuthorsOpenReview ↗

Response to Reviewer fEpM and withdrawal

Thank you for your careful review and constructive suggestions. Your questions helped us identify where the results and the calibration mechanism needed clearer explanation.

  1. Translation error and coordinate frames. Monocular depth uncertainty and the difference between visual detection centers and radar scattering points are relevant sources of translation error. The output errors use the optical camera frame (x right, y down, z forward), whose axes differ from the planar X and Y used in XYT. Visual height is retained as an input attribute, and the model also receives nominal mounting information. A planar positional encoding therefore does not mean that vertical information is absent; attributing the smaller y-axis error to a particular input requires a controlled ablation.

  2. Baseline provenance. We did not retrain LCCNet or CalibDepth for the submitted tables. Their values were taken from the published comparison in RC-AutoCalib. The source of these numbers and the differences in evaluation protocol should have been clearer in our presentation.

  3. XYT and BEV. XYT represents irregular object observations using planar position and reported time, while retaining visual height as an attribute. It is not a rasterized BEV occupancy map. Its role is to organize asynchronous observations over a window for spatial correction, followed by temporal association after spatial compensation. We agree that this role needs to be explained through the complete calibration mechanism, rather than through the addition of a time coordinate alone.

  4. Interpretation and visualization. The coarse-to-refined comparison demonstrates the benefit of spatial refinement; temporal estimation needs its own controlled evaluation. We appreciate your suggestion to show real observations before correction, after spatial correction, and after temporal correction. Such examples are important for connecting the numerical results to observable calibration quality.

After considering the reviews, we have decided to withdraw this submission from ICLR 2027. Thank you again for the time and guidance you have given this work.

Reviewer_TaKwOpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
2
Soundness
1
Read review
Paper summary

The authors propose EasyCalib, which estimates the radar-camera extrinsics (6 DoF) and a radar timestamp offset from objects observed during normal driving. Camera 3D detections and raw radar points from a window of about 20 seconds are encoded as (X, Y, time) points. A coarse pose network and a residual pose network predict the spatial correction, and a temporal branch votes over candidate radar–camera time differences to estimate the offset. Training and testing use nuScenes with synthetic perturbations: recorded camera detections are perturbed by up to ±10 m / ±30°, and up to ±1 s is added to the radar timestamps. At R2–D2 (±1.5 m, ±20°, ±250 ms), the reported errors are 38.7 cm, 0.78°, and 93.5 ms.

Critical strengths

This work addresses a practical problem. Radar-camera calibration drifts over time, and checking it from ordinary driving data, without targets, would be useful for vehicles and roadside units.

Critical weaknesses
  1. The authors claim to estimate the time offset jointly with pose, but their own controls show the temporal branch is weaker than a simple heuristic. On nuScenes, a window endpoint statistic reaches 68.5 ms vs. 93.5 ms for EasyCalib (Table 9). On V2X-Radar, EasyCalib (124.1 ms) is about the same as predicting zero (125.9 ms), while the frame-phase + endpoint statistic reaches 60.0 ms. The endpoint statistic works because both streams are cropped to their common time range, so the injected offset shows up directly at the window boundaries. This is a leak in the evaluation task, and it would likely disappear on continuous data where windows are cut on one sensor’s clock. With the boundaries excluded, the learned branch is only 25% better than predicting zero.
  2. The spatial results do not support practical use, and there is no controlled comparison. With uniform perturbations, predicting zero gives a MAE of half the range. At ±0.25 m (Table 4), the translation error is about 16.6 cm averaged over axes, which is worse than predicting zero (12.5 cm). This is the range where real calibration drift usually lies. At ±1.5 m, it is 38.7 cm vs. 75 cm for zero. The only baseline actually run is planar CPD, which fails to converge on 8,504 of 8,544 views. The RC-AutoCalib numbers are copied from its paper under a different protocol. In addition, the test set was used during development, and radar windows are admitted using annotated vehicle boxes, so the reported numbers may be optimistic.
Questions for authors
  1. Can you evaluate RC-AutoCalib [1] under your protocol, or EasyCalib under its R1/R2 protocol, so the two can be compared directly? [1]: https://arxiv.org/abs/2505.22427
  2. At ±0.25 m, camera-side perturbation gives about 16.6 cm translation error (Table 4), but radar-side perturbation gives 1.04 cm (Table 12). Why is there such a large gap?
  3. How does the temporal branch perform when windows are cut on one sensor’s clock from continuous data, so that the boundaries carry no offset information?
1 comment
AuthorsOpenReview ↗

Response to Reviewer TaKw following withdrawal

Thank you for the detailed review and the concrete suggestions on temporal evaluation, small calibration errors, and baseline comparability. We appreciate the care you took in checking the experimental settings.

  1. Temporal evaluation. The reported-time overlap used in the submitted evaluation can preserve offset-related boundary information. We agree that windows defined on one sensor's clock, independently of the injected offset, are needed to assess temporal association beyond that cue. The submitted endpoint comparison does not establish an advantage of the learned branch over the endpoint statistic.

  2. Small perturbations and the two injection settings. Your comparison with no correction is relevant for evaluating behavior near the nominal calibration. The gap between Tables 4 and 12 also involves different estimators and constraints: Table 12 uses a specialized radar-side estimator with planar and nominal-mounting constraints, whereas Table 4 reports the camera-side model under different timing conditions. These differences should have been explained alongside the results; the gap cannot be attributed to the perturbation side alone.

  3. Baselines and evaluation population. Your request for a common R1/R2 comparison with RC-AutoCalib is well taken. In the submitted manuscript, its numbers were quoted from the published paper under a different protocol. A controlled comparison requires matched evaluation settings. We also recognize your points about development-time test exposure and annotation-based window selection; both affect how the reported results should be interpreted.

We have now withdrawn this submission from ICLR 2027. Your comments have helped us sharpen the evaluation questions and make the experimental presentation clearer. Thank you again for your time and guidance.

Reviewer_s9CGOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper proposes EasyCalib for estimating radar-camera pose and timing corrections from object observations collected over a window. It first predicts a coarse spatial correction, refines it, and then estimates the time offset through learned associations. The observations are represented using planar position and time. The experiments apply synthetic pose and timestamp perturbations to nuScenes and V2X-Radar, with additional detector and component comparisons.

Critical strengths
  • The problem is useful, and the staged design makes sense. Spatial misalignment makes matching radar and camera observations difficult, so correcting it before estimating timing is a reasonable approach. The method also handles asynchronous observations without requiring supplied correspondences at inference.

  • I appreciate that the paper is explicit about its limitations and includes simple controls that do not always favor the proposed method. The spatial refinement improves over the coarse network and the results across different perturbation ranges help show how the system behaves beyond one specific setting.

Critical weaknesses
  • My main concern is why the timing problem needs this learned pipeline when a much simpler statistic works better. In Table 9, the window-endpoint control achieves 68.48 ms error on nuScenes, compared with 93.47 ms for the model. On V2X-Radar, the model is just a bit slightly better than predicting zero offset, while the endpoint control is again substantially better. Could the authors give more explanation on: what useful temporal alignment has the network learned beyond information already available in the timestamps?
Questions for authors

See Weaknesses.

1 comment
AuthorsOpenReview ↗

Response to Reviewer s9CG following withdrawal

Thank you for your careful review and for identifying the key question about the temporal branch. We also appreciate your recognition of the staged spatial design and the use of asynchronous observations.

You are right that the submitted Table 9 does not demonstrate a temporal advantage over the endpoint control. Forming windows from the reported-time overlap can retain information about the injected offset at the boundaries. The endpoint statistic can exploit this information without establishing object-level temporal correspondence. The V2X-Radar comparison likewise provides little improvement over predicting zero offset.

Your question makes the needed evaluation clear: use windows defined independently of the injected offset and test whether the model depends on the correct association between observation content and time. These controls are essential for measuring what learned temporal association adds beyond timestamp cues. The spatial-refinement results address a separate part of the calibration pipeline.

We have now withdrawn this submission from ICLR 2027. Your feedback has been valuable in clarifying the temporal evaluation and its interpretation. Thank you again for your time and constructive guidance.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.