- Rating
- 3(+1)
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper presents FFBL-Coop, a query-based framework for cooperative 3D multi-object tracking. Its main idea is to admit and refine cooperative observations before assigning ego-track identities. CSA admits cooperative queries into available ego slots, URA refines the augmented query bank using ego observations, and CPIA resolves identities after decoding through learned association and persistent cross-agent ID mappings. A shared codebook compresses transmitted query features. Experiments on V2X-Seq and Griffin-25M evaluate detection, tracking, communication cost, and sensitivity to calibration errors and latency. Module ablations and additional identity-management interventions examine the contributions of the proposed components.
Critical strengths
The paper has some originality and reports promising results. The authors have also put considerable effort into the system design and experimental evaluation.
Critical weaknesses
My main concern is that the experiments do not clearly isolate the benefit of the proposed processing order from the other identity-management mechanisms. The interpretation of the scaled association scores and the role of unmatched supervision also need clearer explanation. CSA uses different update rules during training and inference, but the reasons for this difference and its effect on performance have not been sufficiently examined. Some implementation and experimental reporting details are incomplete, limiting reproducibility and making the stability of the reported gains difficult to assess.
Questions for authors
-
Tables 2 and 11 support the effectiveness of CSA, URA, and CPIA relative to their respective replacements. In particular, replacing slot-ID inheritance with CPIA while retaining Hungarian allocation and the URA architecture provides useful evidence for the proposed identity-management scheme. However, this comparison also introduces learned association, persistent ID mapping, and different rules for identity inheritance and transfer. It therefore remains unclear how much of the improvement comes specifically from deciding identities after fusion and refinement. In Table 14, Map-only and Association-only examine how correspondences are established and reused, while Early prefix admission changes when cooperative evidence enters relative to the temporal update; it does not compare identity assignment before and after fusion. A method that assigns identities earlier, or in parallel with fusion, could still use the same admission, refinement, and persistent mapping mechanisms. I suggest comparing independent identity assignment before and after refinement, with the admission policy, fusion network, association architecture, mapping rules, and training budget held consistent. Both variants should be trained under their respective processing orders and evaluated using AMOTA and IDS. This would clarify whether delaying identity assignment provides an additional benefit beyond the other mechanisms in CPIA, and give more direct support for the paper’s central claim.
-
Eqs. (14)–(16) include unmatched supervision, but page 7 states that unmatched scores are not used to reject identity associations at inference, without explaining this choice. Please clarify how this supervision contributes to the final identity decisions and whether this setting is intended to keep the comparison conditions consistent. For (m>1) and a non-square association matrix, (P) in Eq. (16), constructed using the scaling factors from Eq. (15), is not row-normalized; please clarify its target marginals, scale, and interpretation in the negative-log loss, and explain why multiple unmatched queries share a single unmatched column with unit total mass.
-
During training, CSA adds the cooperative features to the original slot features and averages the original and cooperative centers, while leaving the other anchor components unchanged. At inference, it overwrites the features and the entire anchor. These differences go beyond the choice between soft and hard assignment. Please explain why different update rules are used during training and inference, and assess their impact through a controlled comparison.
-
Some implementation and experimental reporting details are missing. These include the exact rule for computing the mapping aging factor (\alpha), the number of epochs in the initial cooperative training stage on V2X-Seq, which objects are included in the coverage statistics, and the variation across the three training runs. Please provide these details to help readers reproduce the method and interpret the results.
4 comments
Follow-up response, summary of manuscript revisions
To complement our response, we summarize the main manuscript revisions below. All additions and clarifications are marked in blue.
-
Added matched CPIA binding-stage and component ablations (Sec. 4.2, Tables 3–4, p. 9): compared three binding stages, each trained once under matched settings, and separately evaluated persistent mapping and association components using fixed-checkpoint interventions, clarifying their respective contributions to tracking performance and identity continuity.
-
Clarified Sinkhorn constraints and unmatched-query handling (Appendix E.3, Tables 17–18, p. 19): retained legacy normalization for the main method and results, evaluated an augmented Sinkhorn variant with compatible marginals, an additional unmatched row, and one-to-one targets, and assessed unmatched-aware rejection while clarifying new-ID allocation for rejected queries.
-
Expanded CSA training–inference strategy analysis (Appendix E.2, Table 16, p. 18): defined the complete soft and hard initialization rules and compared S/S, S/H, and H/H configurations to clarify the rationale for soft training and hard inference.
-
Improved implementation and reproducibility details (Appendix G; Table 23, p. 24): clarified the implementation settings, training schedules, filtering criteria, and three-run training variation, and specified the planned release of source code, configurations, and model weights.
Dear Reviewer,
Thank you for your review. The authors have submitted their rebuttal. Could you please take a moment to review their responses?
Thank you for your time and contribution!
Best regards, Area Chair
Sincere Appreciation for Your Valuable Feedback
Dear Reviewer,
Thank you very much for raising your score. Your recognition is truly encouraging to us.
We sincerely appreciate the time and effort you have devoted to reviewing our work, particularly your insightful comments on association constraints, reproducibility, and the underlying mechanisms driving the performance improvements. Your questions have encouraged us to revisit our research motivation and strengthen our work.
If time permits, we would greatly appreciate hearing from you before the discussion period closes, particularly whether our revisions have adequately addressed your concerns or whether there are any remaining aspects we may have overlooked.
Thank you again for your valuable time, thoughtful feedback, and guidance. We wish you continued success in your research.
Fusion–Identity Decoupling, CPIA Components, and Association Constraints
We thank the reviewer for the constructive suggestions. Our motivation is to decouple cooperative feature integration from identity inheritance, preventing fusion-slot correspondence from determining ego-track identities. Original experiments did not sufficiently isolate decoupling, association design, binding stage, and persistent mapping. We therefore conducted additional experiments to clarify these contributions.
W1/Q1: Fusion–Identity Decoupling and CPIA Ablation
(1) Matched Comparison of CPIA Binding Stages
We compare three binding stages: Pre-decoding, before decoding; Post-temporal (ours), after temporal inheritance and initial refinement but before remaining decoder layers; and Post-refinement, after the final decoder layer. All share identical admission, fusion/refinement, association, candidate construction, spatial gating, mapping, and new-ID policies, trained once under matched recipes. Results are single-run without variance estimates. Metrics are AP/AMOTA/IDS/FRAG.
Post-temporal improves AMOTA over Pre-decoding by 6.87/11.54 points and over Post-refinement by 4.02/4.23 points on V2X/Griffin, with AP gains of 1.88/0.62 points over Post-refinement. On V2X, Post-refinement yields fewer IDS (55.7 vs. 70.5) but more FRAG (82.3 vs. 61.0); on Griffin, Post-temporal leads on all four metrics. These results support Post-temporal while revealing the V2X identity-switch trade-off (AP/AMOTA: Sec. 4.2, Table 3, p. 9; IDS/FRAG are additional diagnostics).
(2) Effect of Persistent Mapping and Association Components
Fixed-checkpoint interventions examine persistent mapping and learned association (Sec. 4.2, Table 4, p. 9).
On V2X, removing mapping causes severe degradation (60.6→24.2) regardless of association method, highlighting its importance for continuity at 2 Hz with many infrastructure-only objects. On Griffin, degradation is smaller (67.9→66.1), and learned association outperforms geometric matching (66.1 vs. 59.3 AMOTA; 30 vs. 98 IDS). Thus, mapping maintains continuity; learned association provides complementary recovery.
W2/Q2: Sinkhorn Constraints, Unmatched Queries, and Inference Rules
Legacy Sinkhorn imposes incompatible unit marginals on an m×(n+1) matrix with a shared unit-mass unmatched column, so its identity loss supervises scaled rather than generally valid probabilities. We implemented augmented Sinkhorn with compatible marginals, an extra unmatched row, and one-to-one targets (App. E.3, Eqs. (21)–(23) and Table 17, p. 19). Main-table checkpoints retain legacy normalization. Under matched 20-epoch training, AMOTA changes from 60.4 to 60.5 on V2X and 68.1 to 67.8 on Griffin, with slightly higher AP@2m; these single-run results do not establish equivalence. Originally, unmatched supervision was training-only. The augmented formulation enables dustbin-aware rejection: ON/OFF changes AMOTA by 0.2–0.3 points, with mixed IDS changes, not a consistent gain (App. E.3, Table 18, p. 19; Eq. (24), p. 20). Rejected queries remain eligible for insertion and new-ID allocation.
W3/Q3: CSA Training–Inference Update Rules
S denotes soft assignment with additive cooperative features, averaged slot/cooperative anchor centers, and unchanged remaining anchor components. H denotes exclusive hard assignment with feature and full-anchor overwrite. S/S uses soft updates in both phases, S/H uses soft training and hard inference, and H/H uses hard updates throughout (App. E.2, Table 16, p. 18). These comparisons evaluate complete initialization rules, not assignment softness alone.
S/H achieves the highest AP and competitive AMOTA: AP improves over S/S by 0.14/1.08 points on V2X/Griffin, while H/H reduces AMOTA relative to S/H by 1.42/0.13 points. This supports soft training with hard inference. Implementation details (α, training schedules, filtering criteria, and three-run variance) are in App. G. Source code, configurations, and weights will be released.
We thank the reviewer for helping strengthen our analysis and presentation. We hope these additional experiments and clarifications address the concerns and merit reconsideration of our work.