acceptodds
Under review as a conference paper at ICLR 2027

FFBL-Coop: Association-Decoupled Cooperative 3D Multi-Object Tracking

Abstract

Cooperative 3D tracking must integrate complementary observations across agents and time while maintaining consistent identities. When evidence integration and identity inheritance share a matching decision, errors arising from cross-view appearance differences and spatial misalignment can compromise both feature fusion and track continuity. We propose FFBL-Coop, a fuse first, bind later framework that separates instance admission from identity management. Confidence-ranked Slot Admission (CSA) allocates cooperative queries to available ego slots using confidence and spatial proximity. Unified Representation Aggregation (URA) uses cooperative semantic features and aligned anchors to guide ego-feature retrieval, refining the augmented query bank within a shared transformer decoder. After refinement, Cooperative-Priority Identity Anchoring (CPIA) combines learned association with persistent mappings to establish accepted identity assignments across frames. A shared codebook reduces transmitted payload while retaining AP and AMOTA close to the uncompressed variant. FFBL-Coop achieves AMOTA/AP of 0.611/0.548 on V2X-Seq and 0.688/0.653 on Griffin-25M. Code will be released.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
cooperative perception, 3D multi-object tracking, sparse queries, association decoupling

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_qcxoOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper presents FFBL-Coop, a query-based framework for cooperative 3D multi-object tracking. Its main idea is to admit and refine cooperative observations before assigning ego-track identities. CSA admits cooperative queries into available ego slots, URA refines the augmented query bank using ego observations, and CPIA resolves identities after decoding through learned association and persistent cross-agent ID mappings. A shared codebook compresses transmitted query features. Experiments on V2X-Seq and Griffin-25M evaluate detection, tracking, communication cost, and sensitivity to calibration errors and latency. Module ablations and additional identity-management interventions examine the contributions of the proposed components.

Critical strengths

The paper has some originality and reports promising results. The authors have also put considerable effort into the system design and experimental evaluation.

Critical weaknesses

My main concern is that the experiments do not clearly isolate the benefit of the proposed processing order from the other identity-management mechanisms. The interpretation of the scaled association scores and the role of unmatched supervision also need clearer explanation. CSA uses different update rules during training and inference, but the reasons for this difference and its effect on performance have not been sufficiently examined. Some implementation and experimental reporting details are incomplete, limiting reproducibility and making the stability of the reported gains difficult to assess.

Questions for authors
  1. Tables 2 and 11 support the effectiveness of CSA, URA, and CPIA relative to their respective replacements. In particular, replacing slot-ID inheritance with CPIA while retaining Hungarian allocation and the URA architecture provides useful evidence for the proposed identity-management scheme. However, this comparison also introduces learned association, persistent ID mapping, and different rules for identity inheritance and transfer. It therefore remains unclear how much of the improvement comes specifically from deciding identities after fusion and refinement. In Table 14, Map-only and Association-only examine how correspondences are established and reused, while Early prefix admission changes when cooperative evidence enters relative to the temporal update; it does not compare identity assignment before and after fusion. A method that assigns identities earlier, or in parallel with fusion, could still use the same admission, refinement, and persistent mapping mechanisms. I suggest comparing independent identity assignment before and after refinement, with the admission policy, fusion network, association architecture, mapping rules, and training budget held consistent. Both variants should be trained under their respective processing orders and evaluated using AMOTA and IDS. This would clarify whether delaying identity assignment provides an additional benefit beyond the other mechanisms in CPIA, and give more direct support for the paper’s central claim.

  2. Eqs. (14)–(16) include unmatched supervision, but page 7 states that unmatched scores are not used to reject identity associations at inference, without explaining this choice. Please clarify how this supervision contributes to the final identity decisions and whether this setting is intended to keep the comparison conditions consistent. For (m>1) and a non-square association matrix, (P) in Eq. (16), constructed using the scaling factors from Eq. (15), is not row-normalized; please clarify its target marginals, scale, and interpretation in the negative-log loss, and explain why multiple unmatched queries share a single unmatched column with unit total mass.

  3. During training, CSA adds the cooperative features to the original slot features and averages the original and cooperative centers, while leaving the other anchor components unchanged. At inference, it overwrites the features and the entire anchor. These differences go beyond the choice between soft and hard assignment. Please explain why different update rules are used during training and inference, and assess their impact through a controlled comparison.

  4. Some implementation and experimental reporting details are missing. These include the exact rule for computing the mapping aging factor (\alpha), the number of epochs in the initial cooperative training stage on V2X-Seq, which objects are included in the coverage statistics, and the variation across the three training runs. Please provide these details to help readers reproduce the method and interpret the results.

4 comments
AuthorsOpenReview ↗

Fusion–Identity Decoupling, CPIA Components, and Association Constraints

We thank the reviewer for the constructive suggestions. Our motivation is to decouple cooperative feature integration from identity inheritance, preventing fusion-slot correspondence from determining ego-track identities. Original experiments did not sufficiently isolate decoupling, association design, binding stage, and persistent mapping. We therefore conducted additional experiments to clarify these contributions.

W1/Q1: Fusion–Identity Decoupling and CPIA Ablation

(1) Matched Comparison of CPIA Binding Stages

We compare three binding stages: Pre-decoding, before decoding; Post-temporal (ours), after temporal inheritance and initial refinement but before remaining decoder layers; and Post-refinement, after the final decoder layer. All share identical admission, fusion/refinement, association, candidate construction, spatial gating, mapping, and new-ID policies, trained once under matched recipes. Results are single-run without variance estimates. Metrics are AP/AMOTA/IDS/FRAG.

StrategyV2XGriffin-25M
Pre-decoding54.08/54.11/98.0/46.062.35/57.36/272.0/37.3
Post-temporal (Ours)54.66/60.98/70.5/61.065.38/68.90/50.7/35.3
Post-refinement52.78/56.96/55.7/82.364.76/64.67/72.0/45.3

Post-temporal improves AMOTA over Pre-decoding by 6.87/11.54 points and over Post-refinement by 4.02/4.23 points on V2X/Griffin, with AP gains of 1.88/0.62 points over Post-refinement. On V2X, Post-refinement yields fewer IDS (55.7 vs. 70.5) but more FRAG (82.3 vs. 61.0); on Griffin, Post-temporal leads on all four metrics. These results support Post-temporal while revealing the V2X identity-switch trade-off (AP/AMOTA: Sec. 4.2, Table 3, p. 9; IDS/FRAG are additional diagnostics).

(2) Effect of Persistent Mapping and Association Components

Fixed-checkpoint interventions examine persistent mapping and learned association (Sec. 4.2, Table 4, p. 9).

ConfigV2X AMOTA (%)V2X IDSGF AMOTA (%)GF IDS
Map + learned60.66367.935
Learned, w/o map32.5126066.130
Geometric, w/o map23.6126359.398

On V2X, removing mapping causes severe degradation (60.6→24.2) regardless of association method, highlighting its importance for continuity at 2 Hz with many infrastructure-only objects. On Griffin, degradation is smaller (67.9→66.1), and learned association outperforms geometric matching (66.1 vs. 59.3 AMOTA; 30 vs. 98 IDS). Thus, mapping maintains continuity; learned association provides complementary recovery.

W2/Q2: Sinkhorn Constraints, Unmatched Queries, and Inference Rules

Legacy Sinkhorn imposes incompatible unit marginals on an m×(n+1)m\times(n+1) matrix with a shared unit-mass unmatched column, so its identity loss supervises scaled rather than generally valid probabilities. We implemented augmented Sinkhorn with compatible marginals, an extra unmatched row, and one-to-one targets (App. E.3, Eqs. (21)–(23) and Table 17, p. 19). Main-table checkpoints retain legacy normalization. Under matched 20-epoch training, AMOTA changes from 60.4 to 60.5 on V2X and 68.1 to 67.8 on Griffin, with slightly higher AP@2m; these single-run results do not establish equivalence. Originally, unmatched supervision was training-only. The augmented formulation enables dustbin-aware rejection: ON/OFF changes AMOTA by 0.2–0.3 points, with mixed IDS changes, not a consistent gain (App. E.3, Table 18, p. 19; Eq. (24), p. 20). Rejected queries remain eligible for insertion and new-ID allocation.

W3/Q3: CSA Training–Inference Update Rules

S denotes soft assignment with additive cooperative features, averaged slot/cooperative anchor centers, and unchanged remaining anchor components. H denotes exclusive hard assignment with feature and full-anchor overwrite. S/S uses soft updates in both phases, S/H uses soft training and hard inference, and H/H uses hard updates throughout (App. E.2, Table 16, p. 18). These comparisons evaluate complete initialization rules, not assignment softness alone.

DataTraining/InferenceAP (%)AMOTA (%)IDS
V2XS/S54.6761.0270
V2XS/H (Ours)54.8161.3269
V2XH/H54.6359.9061
GriffinS/S64.2468.9548
GriffinS/H (Ours)65.3268.8950
GriffinH/H64.7768.7659

S/H achieves the highest AP and competitive AMOTA: AP improves over S/S by 0.14/1.08 points on V2X/Griffin, while H/H reduces AMOTA relative to S/H by 1.42/0.13 points. This supports soft training with hard inference. Implementation details (α\alpha, training schedules, filtering criteria, and three-run variance) are in App. G. Source code, configurations, and weights will be released.

We thank the reviewer for helping strengthen our analysis and presentation. We hope these additional experiments and clarifications address the concerns and merit reconsideration of our work.

AuthorsOpenReview ↗

Follow-up response, summary of manuscript revisions

To complement our response, we summarize the main manuscript revisions below. All additions and clarifications are marked in blue.

  • Added matched CPIA binding-stage and component ablations (Sec. 4.2, Tables 3–4, p. 9): compared three binding stages, each trained once under matched settings, and separately evaluated persistent mapping and association components using fixed-checkpoint interventions, clarifying their respective contributions to tracking performance and identity continuity.

  • Clarified Sinkhorn constraints and unmatched-query handling (Appendix E.3, Tables 17–18, p. 19): retained legacy normalization for the main method and results, evaluated an augmented Sinkhorn variant with compatible marginals, an additional unmatched row, and one-to-one targets, and assessed unmatched-aware rejection while clarifying new-ID allocation for rejected queries.

  • Expanded CSA training–inference strategy analysis (Appendix E.2, Table 16, p. 18): defined the complete soft and hard initialization rules and compared S/S, S/H, and H/H configurations to clarify the rationale for soft training and hard inference.

  • Improved implementation and reproducibility details (Appendix G; Table 23, p. 24): clarified the implementation settings, training schedules, filtering criteria, and three-run training variation, and specified the planned release of source code, configurations, and model weights.

Dear Reviewer,

Thank you for your review. The authors have submitted their rebuttal. Could you please take a moment to review their responses?

Thank you for your time and contribution!

Best regards, Area Chair

AuthorsOpenReview ↗

Sincere Appreciation for Your Valuable Feedback

Dear Reviewer,

Thank you very much for raising your score. Your recognition is truly encouraging to us.

We sincerely appreciate the time and effort you have devoted to reviewing our work, particularly your insightful comments on association constraints, reproducibility, and the underlying mechanisms driving the performance improvements. Your questions have encouraged us to revisit our research motivation and strengthen our work.

If time permits, we would greatly appreciate hearing from you before the discussion period closes, particularly whether our revisions have adequately addressed your concerns or whether there are any remaining aspects we may have overlooked.

Thank you again for your valuable time, thoughtful feedback, and guidance. We wish you continued success in your research.

Reviewer_uZyJOpenReview ↗
Rating
2
Confidence
1
Informativeness
1
Presentation
2
Soundness
2
Read review
Paper summary

Summary: FFBL-Coop addresses cooperative 3D multi-object tracking. Existing sparse-query methods use one cross-agent match both to fuse a cooperative agent’s evidence and to decide which ego track it continues, so a matching error damages both. The paper proposes “fuse first, bind later”: integrate cooperative evidence first, and assign identities only after refinement.

Contributions: CSA: admits cooperative queries into free ego slots by confidence and distance, without assigning identities. URA: refines the admitted queries with ego image features in the shared decoder. CPIA: assigns identities after decoding, using a persistent cross-agent ID map, then learned association, then new IDs. Codebook compression: shrinks transmitted features to 1–2-byte indices.

Validation: V2X-Seq and Griffin-25M: AP/AMOTA of 54.8/61.1 and 65.3/68.8, above published baselines, at lower bandwidth. Ablations: module combinations (3 runs each), identity-mechanism variants, hyperparameter sweeps and codebook size. Robustness: calibration error and latency, plus breakdowns by camera coverage, occlusion and range.

Critical strengths

A controlled test of the central claim. With Hungarian allocation and URA held fixed, replacing the coupled single-match identity inheritance with post-decoding binding (CPIA) raises AMOTA from 50.4 to 60.1 on V2X-Seq and from 55.3 to 66.1 on Griffin-25M, and cuts ID switches from 116 to 69 and from 86 to 57 (Table 2). Module variants are trained three times, with the median run reported. Why it matters: this directly isolates the paper’s core idea of decoupling admission from identity, rather than relying only on system-level comparisons. It’s the main reason I find the contribution credible. Strong tracking at low, transparently accounted bandwidth. The method reaches high AMOTA on two benchmarks while transmitting 53–54 bytes per query. Codebook compression costs only about 0.1–0.2 AP/AMOTA versus uncompressed features, and the payload accounting is explicit and internally consistent (Tables 6 and 15, Appendix G). Why it matters: in cooperative perception, communication cost is a deployment constraint, so a design that keeps accuracy at a fraction of competitors’ bandwidth has practical significance beyond the leaderboard numbers.

Critical weaknesses

The headline gains over prior work aren’t attributable to the method. The Table 1 baselines are published numbers from different backbones, resolutions and training recipes, and “No Fusion” is a published baseline rather than the authors’ own single-agent model. The authors’ own Base variant, which has none of CSA, URA or CPIA, already reaches 47.7 AMOTA on V2X-Seq, above SparseCoop’s 42.1. So at least 5.6 of the claimed +19.0 AMOTA comes before any proposed module, likely from a stronger detector and training recipe (e.g., 800-epoch single-agent pretraining). Why it matters: the main comparison overstates the contribution. Same-recipe baselines (the authors’ own no-fusion and late-fusion models, and ideally a retrained SparseCoop) are needed to show what the framework adds. The “bind later” benefit is mostly the persistent ID map, and the remaining effects are within apparent noise. Map-only reaches 60.5 AMOTA versus 61.1 for the full model, while association-only collapses to 32.5 (Table 14), so learned association adds about 0.6. Meanwhile, on one fixed checkpoint, changing only the codebook size swings AMOTA from 59.2 to 61.1 (Table 6). That variation is larger than the CSA (+1.0) and learned-association (+0.6) effects, and no run-to-run variance is reported despite three training runs per variant. Why it matters: the mechanism that actually works (reusing the cooperating agent’s track IDs) is narrower than the framing suggests, and several claimed module contributions can’t be distinguished from noise without variance estimates.

Questions for authors

Same-recipe baselines. What do your own no-fusion and late-fusion models score with the same backbone, resolution and schedule? Can SparseCoop or CoopTrack be retrained under your recipe? If not, how much of the gap to them does Base already explain? Run variance. What is the spread across the three training runs for each Table 2/11 variant? Are the CSA (+1.0 AMOTA) and learned-association (+0.6) effects larger than that spread? Learned association. Why does association-only collapse to 32.5 AMOTA, below simple slot-ID inheritance (59.3)? Does the persistent map rely on the cooperating agent having stable track IDs, and how does performance degrade when those IDs switch? Robustness metric. Under 1° yaw error or 200 ms latency, AP drops by about 5–6 points but AMOTA by under 1. Does this hold at a tighter tracking match threshold than 2 m, or does it reflect threshold tolerance?

8 comments
AuthorsOpenReview ↗

Baseline Attribution, CPIA Components, and the Core Innovation of Decoupled Association

We thank the reviewer for the constructive suggestions. Our central motivation is to separate cooperative evidence integration from identity inheritance, shielding valid representations from distortion by erroneous fusion. We agree the original manuscript insufficiently distinguished system-level improvements from method-specific gains and provide supplementary experiments in Appendices E–G. We therefore distinguish overall system performance from controlled method comparisons and interpret dataset-dependent improvements cautiously.

W1/Q1: Baseline Comparability and Attribution of the Headline Gains

(1) Baseline Comparability and Cooperative Processing

The +19.0-point AMOTA gain over published SparseCoop (42.1% → 61.1%) is a system-level comparison, not a gain attributable solely to FFBL, because perception architectures and training protocols differ. Our Base is likewise not an exact SparseCoop implementation: it retains the FFBL-Coop decoder but changes cooperative-query insertion stages and depths (App. E.4, Table 20(a)). The following matched comparison better isolates the FFBL design.

ConfigV2X AP↑V2X AMOTA↑Griffin AP↑Griffin AMOTA↑
No Fusion23.4621.829.5131.43
Late Fusion52.0355.3851.7640.56
Full FFBL54.6660.9565.3068.82

Full FFBL outperforms Late Fusion on both datasets, especially Griffin-25M. This is consistent with visibility: cooperative-only objects form the largest V2X-Seq group (2699/5599, 48.2%), whereas 73.9% of Griffin-25M objects are visible to both agents. Late concatenation can already recover cooperative-only detections, while the larger shared-visibility group may benefit more from feature integration (Table 5; App. G).

(2) Open-Source Reproduction and Matched-Detector Comparison

We reproduced SparseCoop using its public code and closely matched its published results. With identical ego/cooperative detectors, initialization, resolution, preprocessing, detector configurations, and training budget, FFBL-Coop improves AMOTA by 16.45/17.5 points and AP by 0.83/10.4 points on V2X-Seq/Griffin-25M, respectively. Thus, the gains are not due to stronger detectors (App. E.4, Table 20(b)).

ConfigV2X AP/AMOTA↑Griffin AP/AMOTA↑
CoopTrack36.1 / 35.344.2 / 43.5
SparseCoop52.2 / 43.154.9 / 51.3
Ours53.03 / 59.5565.3 / 68.8

W2/Q2: Training Variation, CPIA Components, and Tracking Thresholds

(1) Training Variation and CPIA Components

Three runs show small AP/AMOTA standard deviations (App. G, Table 23). Source-ID switching at 25–100% also leaves performance nearly unchanged, as changed IDs trigger a new mapping lookup and correspondence recovery (App. E.4, Table 19).

Fixed-checkpoint tests identify persistent mapping as CPIA's primary contributor. Without mapping, V2X-Seq degrades severely under either association method, whereas Griffin-25M degrades less. This suggests V2X-Seq depends more on persistent cross-frame correspondences, while Griffin-25M can more effectively associate with existing ego tracks. On Griffin-25M, learned association outperforms geometric association; on V2X-Seq, they are similar without mapping. Thus, persistent mapping primarily maintains trajectory continuity, while learned association provides complementary recovery (Sec. 4.2, Table 4).

ConfigV2X AMOTA (%)Griffin AMOTA (%)
Map + learned60.667.9
Learned, no map32.566.1
Geometric, no map23.659.3

(2) Robustness under Stricter Tracking Matching Thresholds

We re-evaluated saved predictions at 1.5 m and 1.0 m, without retraining or rerunning inference; only the evaluator's matching distance changed. Representative results are shown below (full results: App. F, Table 22).

ConfigurationAP (%)AMOTA @ 2.0 / 1.5 / 1.0 m (%)
FFBL, None54.861.1 / 55.4 / 41.7
FFBL, Yaw 1°49.460.4 / 54.4 / 35.0
SparseCoop, None52.243.1 / 36.3 / 22.3
SparseCoop, Yaw 1°45.743.0 / 34.1 / 18.5

SparseCoop's clean and perturbed results use the calibration reproduction checkpoint, separate from Table 20(b). FFBL achieves higher AMOTA in every corresponding comparison. At 1.0 m under 1° yaw, FFBL retains 35.0% AMOTA versus 18.5% for SparseCoop. The advantage therefore persists under stricter matching, although these thresholds expose sensitivity to spatial misalignment.

We hope these clarifications highlight FFBL-Coop's central contribution: separating cooperative evidence admission from identity inheritance. The additional controlled comparisons and robustness analyses further clarify both the sources and limitations of the observed improvements. Thank you for helping us strengthen this analysis.

AuthorsOpenReview ↗

Follow-up response, summary of manuscript revisions

To complement our response, we summarize the main manuscript revisions below. All additions and clarifications are marked in blue.

  • Clarified baseline comparability and performance attribution (Appendix E.4, Table 20, p. 20; Table 5, p. 10; Appendix G): distinguished system-level comparisons from matched-detector evaluations, added an open-source SparseCoop reproduction under controlled detector and training settings, and discussed how cross-agent visibility relates to the benefits of cooperative feature integration.

  • Expanded CPIA component analysis and training stability evaluation (Sec. 4.2, Table 4, p. 9; Appendix E.4, Table 19, p. 20; Appendix G, Table 23, p. 24): added fixed-checkpoint comparisons of persistent mapping and association strategies, evaluated robustness to cooperative source-ID switching, and reported ablation performance with standard deviations from three independent training runs.

  • Added robustness evaluation under stricter tracking thresholds (Appendix F, Table 22, p. 21): re-evaluated saved predictions at 1.5 m and 1.0 m matching thresholds under clean and perturbed conditions, without retraining or rerunning inference.

Reviewer_uZyJOpenReview ↗

Thank you for the response and suggestions for the revision

Thank you for the thorough response and the substantial new experiments. The matched no-fusion and late-fusion models and the SparseCoop reproduction with matched detectors resolve my main concern about attribution, and I appreciate that the +19.0 AMOTA is now framed as a system-level comparison. The source-ID switching test and the stricter-threshold evaluation are also helpful additions.

A few suggestions that I think would make the revision stronger:

  1. Please state the three-run standard deviations from Table 23 next to the CSA and learned-association gains, so readers can see how they compare with the spread.

  2. Table 4 shows the persistent map is the main contributor on V2X-Seq. Saying so in the abstract and contributions would align the claims with the evidence, and it is an interesting finding in its own right.

  3. Reporting the 1.0 m results alongside the 2 m ones in the robustness discussion would give a fuller picture. Your advantage over SparseCoop holds there, which is a point in your favour.

  4. In Table 3, your mid-decoder binding outperforms binding after final refinement. A sentence on why, and how this fits the "fuse first, bind later" framing, would help.

I will take these additions into account in my final assessment.

Reviewer_estXOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper studies camera-only cooperative 3D multi-object tracking and proposes FFBL-Coop, a "fuse first, bind later" framework that separates cooperative evidence admission from track identity assignment. Confidence-ranked Slot Admission (CSA) places cooperative queries in available ego slots, Unified Representation Aggregation (URA) refines the augmented query bank using cooperative semantics and ego observations, and Cooperative-Priority Identity Anchoring (CPIA) assigns identities after decoding through learned association and persistent cross-agent mappings. A shared codebook compresses transmitted instance features. The method achieves car-class AP/AMOTA of 54.8/61.1 on V2X-Seq and 65.3/68.8 on Griffin-25M.

Critical strengths
  1. Separating cooperative evidence admission from track identity assignment has a clear motivation in recurrent cooperative tracking. With Hungarian allocation and URA retained, adding CPIA in Table 2 raises AMOTA from 50.4 to 60.1 on V2X-Seq and from 55.3 to 66.1 on Griffin-25M, supporting the value of the identity management component.
  2. It covers vehicle-infrastructure and vehicle-drone cooperation, with module ablations, feature compression, calibration perturbations, and latency tests. On V2X-Seq, the default codebook reduces the estimated query payload from approximately 172 to 8.48 kB/s, with a decrease of only 0.1 percentage points in each of AP and AMOTA.
Critical weaknesses
  1. Equation (15) alternates scaling toward unit row and column sums, which cannot both hold when the numbers of rows and columns differ. Multiple unmatched queries also share a unit-mass unmatched column, yet its score is not used to reject matches at inference. These choices need clarification to assess the identity loss and its relationship to the inference rules.
  2. The ablations support CPIA as a whole, but do not directly isolate the benefit of assigning identities after refinement from the benefits of learned association and persistent mapping. A controlled comparison of binding before versus after refinement, with comparable association and mapping mechanisms, would clarify the specific contribution of the central “fuse first, bind later” design.
Questions for authors
  1. Could you clarify the intended row and column constraints in Eq. (15), including how multiple unmatched queries should be handled, and confirm whether the implementation follows the stated equations? Why is the unmatched supervision used during training not used to reject matches at inference?

  2. Could you compare identity binding before and after refinement while keeping the perception frontend, training protocol, association scorer, and persistent mapping mechanism the same? Appendix G already states that each module variant is trained three times, so reporting the AMOTA variation across these runs would also help assess whether the smaller gains are consistent.

4 comments
AuthorsOpenReview ↗

CPIA's Association Constraints and the Sources of Tracking Improvements

We thank the reviewer for the careful assessment and constructive suggestions. Our central motivation is to separate cooperative evidence integration from identity inheritance, shielding valid representations from distortion by erroneous fusion. We agree the original manuscript insufficiently distinguished system-level improvements from method-specific gains and provide supplementary experiments in Appendices E–G. We therefore distinguish overall system performance from controlled method comparisons and interpret dataset-dependent improvements cautiously.

W1/Q1: Baseline Comparability and Attribution of the Headline Gains

(1) Baseline Comparability and Cooperative Processing

The +19.0-point AMOTA gain over published SparseCoop (42.1% → 61.1%) is a system-level comparison, not a gain attributable solely to FFBL, because perception architectures and training protocols differ. Our Base is not an exact SparseCoop implementation (Sec. 4.2, Table 2; App. G). Matched comparison (App. E.4, Table 20(a)):

ConfigV2X AP↑V2X AMOTA↑Griffin AP↑Griffin AMOTA↑
No Fusion23.4621.829.5131.43
Late Fusion52.0355.3851.7640.56
Full FFBL54.6660.9565.3068.82

Full FFBL outperforms Late Fusion on both datasets, especially Griffin-25M. This is consistent with visibility: cooperative-only objects form the largest V2X-Seq group (2699/5599, 48.2%), whereas 73.9% of Griffin-25M objects are visible to both agents. Late concatenation can already recover cooperative-only detections, while the larger shared-visibility group may benefit more from feature integration (Table 5, p. 10; App. G).

(2) Open-Source Reproduction and Matched-Detector Comparison

We reproduced SparseCoop using its public code and closely matched its published results. With identical ego/cooperative detectors, initialization, resolution, preprocessing, detector configurations, and training budget, FFBL-Coop improves AMOTA by 16.45/17.5 points and AP by 0.83/10.4 points on V2X-Seq/Griffin-25M, respectively. Thus, the gains are not due to stronger detectors (SparseCoop/FFBL: App. E.4, Table 20(b), p. 20; CoopTrack results are reported here).

ConfigV2X AP/AMOTA↑Griffin AP/AMOTA↑
CoopTrack36.1 / 35.344.2 / 43.5
SparseCoop52.2 / 43.154.9 / 51.3
Ours53.03 / 59.5565.3 / 68.8

W2/Q2: Training Variation, CPIA Components, and Tracking Thresholds

(1) Training Variation and CPIA Components

Three runs show small AP/AMOTA standard deviations (App. G, Table 23, p. 24). Source-ID switching at 25–100% also leaves performance nearly unchanged, as changed IDs trigger a new mapping lookup and correspondence recovery (App. E.4, Table 19, p. 20).

Fixed-checkpoint tests identify persistent mapping as CPIA's primary contributor. Without mapping, V2X-Seq degrades severely under either association method, whereas Griffin-25M degrades less. This suggests V2X-Seq depends more on persistent cross-frame correspondences, while Griffin-25M can more effectively associate with existing ego tracks. On Griffin-25M, learned association outperforms geometric association; on V2X-Seq, they are similar without mapping. Thus, persistent mapping primarily maintains trajectory continuity, while learned association provides complementary recovery (Sec. 4.2, Table 4, p. 9).

ConfigV2X AMOTA (%)Griffin AMOTA (%)
Map + learned60.667.9
Learned, no map32.566.1
Geometric, no map23.659.3

(2) Robustness under Stricter Tracking Matching Thresholds

We re-evaluated saved predictions at 1.5 m and 1.0 m, without retraining or rerunning inference; only the evaluator's matching distance changed. Representative results are shown below (full results: App. F, Table 22, p. 21).

ConfigurationAP (%)AMOTA @ 2.0 / 1.5 / 1.0 m (%)
FFBL, None54.861.1 / 55.4 / 41.7
FFBL, Yaw 1°49.460.4 / 54.4 / 35.0
SparseCoop, None52.243.1 / 36.3 / 22.3
SparseCoop, Yaw 1°45.743.0 / 34.1 / 18.5

SparseCoop's clean and perturbed results use the calibration reproduction checkpoint, separate from Table 20(b). FFBL achieves higher AMOTA in every corresponding comparison. At 1.0 m under 1° yaw, FFBL retains 35.0% AMOTA versus 18.5% for SparseCoop. The advantage therefore persists under stricter matching, although these thresholds expose sensitivity to spatial misalignment.

We hope these clarifications highlight FFBL-Coop's central contribution: separating cooperative evidence admission from identity inheritance. The additional controlled comparisons and robustness analyses further clarify both the sources and limitations of the observed improvements. Thank you for helping us strengthen this analysis.

AuthorsOpenReview ↗

Follow-up response, summary of manuscript revisions

To complement our response, we summarize the main revisions below. All additions and clarifications are marked in blue.

  • Clarified CPIA association and unmatched handling (Sec. 3.4; Appendix E.3, Eqs. (21)–(23) and Tables 17–18, p. 19; Eq. (24), p. 20): retained legacy Sinkhorn for the main method and results, compared it with an augmented dual-dustbin variant using compatible marginals and one-to-one targets, and evaluated unmatched-aware rejection, clarifying that rejected queries remain eligible for new-ID allocation.

  • Added binding-stage comparison (Sec. 4.2, Table 3, p. 9): compared Pre-decoding, Post-T (ours), and Post-refinement under matched experimental settings.

  • Added training variation analysis (Appendix G, Table 23, p. 24): reported ablation performance with standard deviations from three independent training runs for the full model and module ablations.

Dear Reviewer,

Thank you for your review. The authors have submitted their rebuttal. Could you please take a moment to review their responses?

Thank you for your time and contribution!

Best regards, Area Chair

AuthorsOpenReview ↗

We Have Thought Deeply About Your Concerns — Please Give Us Another Chance

Dear Reviewer,

As the discussion period draws to a close, we sincerely hope to have one more opportunity to hear your thoughts.

We have spent considerable time reflecting on your questions, particularly those concerning association constraints, fair comparisons, and the underlying sources of our performance gains. Your feedback has encouraged us to revisit the fundamental motivation behind our work.

Our research stems from a critical failure mode in cooperative tracking: erroneous cross-agent associations can assign incorrect identities to cooperative observations, causing identity errors to propagate across frames. To address this issue, we decouple cooperative evidence integration from persistent identity assignment, allowing valid cooperative features to contribute to perception without being directly compromised by incorrect identity associations.

Would you kindly take a moment to reconsider our responses before the discussion period closes? We would greatly appreciate knowing whether your concerns have been adequately addressed or whether there are aspects we may still have overlooked.

Thank you sincerely for your time, thoughtful feedback, and guidance. We wish you all the best in your research.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.