acceptodds
Withdrawn from ICLR 2027

Glean: Recovering Underused Signal via Targeted Augmentation in Classical 3D Multi-Object Tracking

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Classical tracking-by-detection pipelines for 3D multi-object tracking (MOT) provide a modular ap- proach to trajectory estimation, yet potentially useful information available around individual track- ing stages is often reduced to coarse representations or left unused. We present Glean, which sys- tematically exploits such underused information at selected stages of an established LiDAR-camera tracking pipeline while preserving its core association mechanism. Built on Fusion-Poly, Glean enriches detection scoring with contextual quality cues, replaces rectangular geometric alignment targets with pixel-level instance masks, adapts Kalman uncertainty based on ego speed, and re- fines completed trajectories through Gaussian smoothing. These components target complementary sources of information while leaving the underlying FACM association cascade unchanged. Only the detection-quality component requires additional training. On the nuScenes validation set, Glean improves AMOTA from 77.294 to 77.775 over the reproduced Fusion-Poly baseline, corresponding to a gain of 0.481%p. Across five alternating timing runs, Fusion-Poly and the Glean tracking loop with cached SAM2 masks both average 409.8 ms per frame, with run-level standard deviations of 3.5 ms and 4.1 ms, respectively; SAM2 inference is excluded from this comparison. Experimen- tal results demonstrate the effectiveness of the proposed Glean and its individual components, as evidenced by improvements in AMOTA.

Primary area:
applications to robotics, autonomy, planning
Keywords:
3D multi-object tracking, tracking-by-detection, autonomous driving, LiDAR-camera fusion

Reviews

Among:
Shape:
1234this paper, Bottom 1%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.00 over 3 reviews, above 0% of the others.
Reviewer_PEVoOpenReview ↗
Rating
1
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes Glean, a 3D multi-object tracking framework built on Fusion-Poly to make better use of underutilized information in tracking-by-detection pipelines. It combines SAM2-based geometric alignment, MLP-based detection-confidence refinement, ego-speed-aware Kalman filtering, and GSI-based trajectory smoothing. Experiments on nuScenes show improvements over Fusion-Poly, and include ablation studies, sensitivity analyses, and runtime evaluations.

Critical strengths
  1. The paper is well organized and clearly written. Its methodological descriptions make the proposed components and their integration easy to follow.
  2. The evaluation uses a controlled setup with consistent detection inputs and includes ablations, hyperparameter sensitivity analyses, and runtime measurements. This provides reasonable empirical support for the reported improvements.
Critical weaknesses
  1. Glean primarily makes incremental engineering changes to the existing Fusion-Poly pipeline, rather than introducing a fundamentally new technique or methodological insight. This raises concerns regarding the novelty of the proposed approach.
  2. Mask-GAAM replaces rectangular alignment targets with instance masks generated by SAM2 and leaves the underlying geometric optimization procedure unchanged. SG-LKF applies ego-speed-dependent scaling to the Kalman-filter noise matrices, while GSI adopts the trajectory-smoothing method introduced in StrongSORT [1]. These above components are essentially minor adaptations or direct usages of prior algorithms, rather than original methodological designs.
  3. Table 3 reports AMOTA increasing from 77.294 for Fusion-Poly to 77.775 for Glean, with a total gain of 0.481 percentage points. Adding the three proposed online components raises AMOTA to 77.494, a gain of 0.200 percentage points. Adding GSI then contributes a further 0.281 percentage points, or approximately 58.4% of the total gain. Thus, most of the improvement comes from an established method, while the new components provide a smaller gain. This raises questions about the significance and practical value of the methodological contributions.

[1] “StrongSORT: Make DeepSORT Great Again.” IEEE Transactions on Multimedia, vol. 25, 2023, pp. 8725–8737.

Questions for authors
  1. Beyond incremental engineering improvements, what fundamental methodological or theoretical problem does Glean address, and what novel insight does it provide?
  2. The performance gain brought by the newly proposed online components is marginal. Could the authors justify the necessity of introducing these relatively complex modules?
Reviewer_BhFBOpenReview ↗
Rating
1
Confidence
2
Informativeness
1
Presentation
3
Soundness
2
Read review
Paper summary

Glean adds four modules to Fusion-Poly for LiDAR-camera 3D MOT: SAM2 masks for geometric alignment, a learned detection-score rescaling, ego-speed scaling of Kalman noise, and post-hoc GSI smoothing. AMOTA on nuScenes val goes from 77.294 to 77.775.

Critical strengths

Clear and honest reporting. each module reduces exactly to the baseline when disabled.

Critical weaknesses
  • The gain is very small (+0.48 AMOTA), and more than half comes from GSI, an existing post-hoc method from StrongSORT. The three online modules add +0.20 together, less than the gap between the published and reproduced Fusion-Poly (77.1 vs 77.294). SG-LKF alone gives +0.019, and alpha = 0.3 is already below the baseline.
  • All hyperparameters (w, alpha, interval, tau) are tuned on the validation set that is also used for the final results, and there is no test-set evaluation, not even as a final check. Without confidence intervals, the gains cannot be separated from validation overfitting.
  • The latency claim excludes SAM2 inference, which is the most expensive part of an online deployment. The contribution is also mostly engineering, which is a weak fit for ICLR.
Questions for authors
  • What does Fusion-Poly + GSI alone achieve? This is needed to know what the online modules add.
  • Can you report nuScenes test results with all hyperparameters frozen from validation?
  • What is the end-to-end per-frame latency including SAM2 hiera-large?
Reviewer_SwjZOpenReview ↗
Rating
1
Confidence
4
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

Building upon the FACM association from Fusion-Poly, Glean introduces four additional modules: Detection-quality Filter, contextual reweighting based on pure 3D detection scores; Mask-GAAM, which replaces rectangular GAAM alignment targets with SAM2 instance masks; SG-LKF, which scales the process and measurement noise of the Kalman filter according to the ego-vehicle's speed; and Gaussian Smoothing (GSI) applied during the post-processing stage. Among these, only the Detection-quality Filter module requires training. On the nuScenes validation set, this method achieves an AMOTA score of 77.775, compared to 77.294 for the authors' reproduction of the Fusion-Poly baseline. In cumulative ablation experiments, the three online modules bring the metric to 77.494, while the inclusion of GSI further improves the result to 77.775. The authors note that the complete system is not strictly an online system with GSI and SAM2 cached operation.

Critical strengths
  • This paper is clearly structured. Each module targets a specific stage of the processing pipeline, resulting in improved accuracy compared to the Fusion-Poly baseline.

  • The experimental results are reported by incrementally adding each module, with a distinction made between the online processing loop and the subsequent Global Trajectory Optimization (GSI) step. The AMOTA metric stands at 77.494 prior to the application of GSI, and reaches 77.775 following this post-processing step. Consequently, the ablation study clearly demonstrates the performance gains achieved by leveraging future frames.

Critical weaknesses
  • The online modules provide a limited gain, and much of the reported improvement comes from post-processing that uses future information and the paper does not analyze why these modules work. SG-LKF assumes that higher ego speed should increase both process and measurement noise, but it does not report how prediction or update error varies with speed relative to ground truth. GSI is adopted directly as post-processing, without comparison to an RTS smoother or to other trajectory-smoothing methods.

  • Evaluation is conducted only on the nuScenes validation set. There are no test-set results. The relevant hyperparameters are also selected on this same validation split, so the reported numbers do not show whether the gains generalize.

Questions for authors
  • Could the authors provide results on the nuScenes test set? Including comparative results on other datasets would also be highly valuable.

  • Furthermore, could the authors provide a more detailed analysis of how each module contributes to performance improvements? Specifically, it would be helpful to compare GSI with other smoothing algorithms (such as RTS) and to analyze the state estimation error variance of the SG-LKF across different ego-vehicle speed ranges.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.