acceptodds
Under review as a conference paper at ICLR 2027

IsoComm: Heterogeneous Cooperative Perception Across Training-Isolated Systems

Abstract

Heterogeneous cooperative perception enables agents with different sensing modalities and perception models to collaborate by sharing complementary information. Recent methods have moved heterogeneous cooperation closer to real-world deployment by relaxing assumptions on modalities, architectures, and system extensibility. However, cross-agent compatibility is still commonly established before deployment through shared source training domains, coupled model components, or pre-established representations and communication protocols. We study a more open setting where complete perception systems are developed independently on private domains and future collaborators are unknown. We formulate this setting as . Under this setting, conventional direct fusion cannot reliably turn an unseen sender into a cooperative gain, because the sender representation and receiver fusion interface have been learned through separate development processes. To address this challenge, we propose with two mechanisms: Protocol-Basis Diversification (PBD) and Ownership-Preserving Attention (OPA). PBD diversifies the communication bases encountered during local training and reduces dependence on a specific local basis. OPA preserves sender-native value encoding while keeping relevance estimation and aggregation under receiver control. This design enables zero-shot cooperation between independently trained systems without pair-specific optimization. For persistent partnerships, we further introduce Communication-Aware Commissioning to calibrate lightweight communication interfaces while keeping both perception systems frozen. Across three unseen senders, IsoComm consistently improves over the ego-only receiver, achieving 88.32–88.54 versus 82.49 for ego-only, while reducing the cross-sender range to 0.22 points and supporting annotation-efficient commissioning.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
cooperative perception, heterogeneous cooperative perception, real-world cooperative perception

Reviews

Among:
Shape:
1234this paper, Top 15%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.00 over 3 reviews, above 85% of the others.
Reviewer_4fKzOpenReview ↗
Rating
3
Confidence
4(+1)
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper studies training-isolated heterogeneous cooperative perception: how systems independently trained on private domains can collaborate at their first encounter without pair-specific optimization. IsoComm enables zero-shot cooperation through Protocol-Basis Diversification (PBD) and Ownership-Preserving Attention (OPA), with lightweight communication interface calibration (CAC) for persistent partnerships.

Critical strengths
  1. The problem is practically relevant. It focuses on first-encounter cooperation between independently developed systems, reflecting the needs of cross-vendor deployment.
  2. The deployment regimes are clearly distinguished. The paper separately considers ad-hoc zero-shot cooperation and commissioning for persistent partnerships.
  3. The experiments demonstrate consistent cooperative gains. All three unseen senders improve receiver performance, with additional validation across different receivers and a camera-based system.
Critical weaknesses

My main concerns are the assumptions needed to connect the systems, the strength of the zero-shot comparisons, and the scope of the generalization evidence.

  1. The claimed openness rests on substantial interface assumptions. Although the formulation permits different fusion designs, every evaluated system uses the same V2X-ViT structure, communication channel count, and attention projection specifications. Collaborators must therefore agree on a structural interface before deployment. It is unclear how an existing system built around a different architecture could participate. The core augmentation also operates on BEV features and, as formulated, does not accommodate systems without such representations. These requirements materially limit the motivation for a fully open setting. These issues substantially affect whether the motivation is justified and should be addressed.

  2. The zero-shot evaluation needs stronger alternatives. Ego-only and Direct fusion are useful reference points, but directly connecting incompatible feature interfaces is a weak test of the proposed solution. Its failure establishes the problem rather than demonstrating that IsoComm is the best way to address it. Comparisons with simple feature normalization, Direct fusion using the same message safeguard, and late fusion of independently predicted detections would make the evidence more convincing.

  3. The deployment tests cover only one direction of domain shift. The systems are trained on OPV2V and V2XREAL, yet all main results and additional receiver experiments are evaluated on OPV2V. The receiver consequently operates in its own training domain throughout. Reverse pairings evaluated on V2XREAL, together with a deployment domain unseen by both systems, would provide a stronger test of the claimed first-encounter cooperation capability.

Questions for authors

My questions are the same as the concerns raised above.

4 comments

We sincerely thank the reviewer for recognizing the practical relevance of first-encounter cooperation in cross-vendor deployment, the clear distinction between ad-hoc zero-shot cooperation and persistent-partnership commissioning, and the consistent cooperative gains across unseen senders and diverse receiver architectures, including a camera-based system. We appreciate the constructive feedback and address the concerns below with additional experimental evidence and clarifications.

W1. Structural Interface Assumptions and Applicability

We agree that the current implementation requires a compatible structural interface, which limits its applicability to systems with incompatible communication architectures. As specified in Appendix A.1.2 and A.2.3, the evaluated systems share BEV feature dimensions and V2X-ViT-based attention specifications. Consequently, while our experiments include different perception encoders and sensing modalities (Table 1 and Appendix Table 6), they do not establish compatibility with arbitrary fusion architectures or systems without BEV representations. We acknowledge these limitations of the current implementation.

The need for a structural interface limits the range of compatible systems, but does not negate the motivation for studying first-encounter cooperation between independently developed systems. Our reference to a "more open setting" concerns systems developed independently without shared training data, jointly optimized components, or pre-established learned representations, rather than unrestricted interoperability without structural prerequisites. Even with a compatible interface, independently developed systems do not automatically achieve effective first-encounter cooperation, as illustrated by the pair-dependent Direct fusion results in Table 1. We believe removing the structural-interface requirement is an important and valuable direction for future research toward more general heterogeneous cooperation. We will explicitly discuss this limitation and future direction in a new Limitations section.

W2. Stronger Zero-Shot Comparisons

We thank the reviewer for suggesting stronger zero-shot alternatives and have conducted additional comparisons with feature normalization and safeguard-matched Direct fusion. Specifically, we evaluate Direct fusion with mean–variance normalization (MVNorm) and Direct fusion with the same norm-clipping safeguard used in zero-shot IsoComm. Neither baseline requires pair-specific training or adaptation, allowing us to directly examine whether these simpler mechanisms can address the incompatibility between independently trained feature representations.

MethodB1 AP50B1 AP70B2 AP50B2 AP70B3 AP50B3 AP70
Direct fusion57.6653.4185.0872.4981.0869.56
Direct + MVNorm58.3453.8287.3474.0681.5669.77
Direct + Clipping85.4468.2385.6772.6386.5968.33
IsoComm (Clipping OFF)88.5473.9988.5574.3888.3074.33

Direct fusion, Direct + MVNorm, and the reported IsoComm variant use clipping OFF; Direct + Clipping uses the original safeguard with ρ = 0.05.

These controlled comparisons demonstrate that IsoComm's effectiveness cannot be explained by simple normalization or norm clipping alone. MVNorm provides only limited improvements over Direct fusion, particularly on the challenging B1 pairing (58.34 vs. 57.66 AP50). Norm clipping substantially improves Direct fusion on B1 (85.44 AP50), confirming that the safeguard can mitigate harmful sender contributions. However, IsoComm without clipping still outperforms Direct fusion with clipping by 3.10 AP50 and 5.76 AP70 on B1, and achieves the highest AP50 and AP70 across all three unseen senders. Importantly, IsoComm maintains a cross-sender AP50 range of only 0.25 points without clipping, demonstrating that its consistent cooperative gains are not dependent on the safeguard. These results provide stronger evidence for the effectiveness of IsoComm beyond directly connecting incompatible feature interfaces.

Regarding Late Fusion, the independently trained OPV2V and V2XREAL detectors have different category definitions and prediction-score distributions, requiring additional cross-domain output harmonization for decision-level fusion. Moreover, Late Fusion combines final detection outputs rather than addressing the interoperability of intermediate representations, which is the central challenge studied in this work. We therefore prioritize comparisons that directly test intermediate-feature compatibility under the same zero-shot setting. We recognize Late Fusion as a complementary decision-level reference, but it addresses a different communication paradigm.

We will include the additional normalization and safeguard comparisons, together with their implementation details, in the revised Appendix.

W3. Reverse-direction domain shift and deployment generalization

We thank the reviewer for highlighting the importance of evaluating domain shifts in both directions, and agree that reverse-direction evaluation would provide valuable additional evidence of generalization. Our current experiments fix an OPV2V-trained receiver and evaluate three independently trained V2XREAL senders, demonstrating consistent zero-shot cooperative gains across unseen heterogeneous sender architectures (Table 1). Appendix A.4 (Table 6) further evaluates different receiver architectures, including a camera-based system. These results support the effectiveness of first-encounter cooperation across the evaluated independently trained systems, although they do not establish symmetric generalization across domain-shift directions.

We recognize reverse-direction evaluation as an important extension of the current experimental evidence. In particular, evaluating V2XREAL-trained receivers with independently trained OPV2V senders would directly test whether the observed cooperative gains extend to the opposite domain-shift direction. A rigorous evaluation requires additional receiver training and consistent ego-only and Direct fusion baselines, which could not be completed within the rebuttal period. We will include the reverse-direction evaluation in the revised manuscript's Appendix, together with the corresponding baselines and evaluation protocol. We will also clarify the scope of the current experiments and the remaining deployment-domain generalization boundary in the Limitations section.

While this method suffers from some overclaims—for instance, it is incompatible with non-BEV-based systems yet is claimed to be fully generalizable—this seriously undermines the soundness of the motivation. I will not lower my score on this condition if the authors soften such statements throughout the paper.

We sincerely thank the reviewer for the constructive clarification and for the opportunity to strengthen the precision of our claims.

We have revised the manuscript throughout to explicitly acknowledge the structural-interface assumptions and align our generalization claims with the scope of the experimental evidence. Specifically, we have clarified the compatible structural communication interface in the Abstract, situated our research problem within BEV-based cooperative perception in the Introduction, distinguished structural compatibility from learned representation alignment in the Contributions, §2.2, and §3.1, and refined the Conclusion accordingly. We have also added §6, Limitations and Future Work, explicitly discussing non-BEV representations, alternative fusion architectures, and broader generalization as future research directions. All revisions are highlighted in blue in the revised manuscript for ease of reference.

We would also like to clarify that our evaluation of BEV-based systems does not establish fundamental incompatibility with non-BEV systems; rather, such interoperability remains unverified. BEV-based communication is a well-established setting in heterogeneous collaborative perception, as adopted by representative frameworks including HEAL [1], STAMP [2], GT-Space [3], and GenComm [4]. Within this setting, heterogeneous systems may differ in sensing modalities and perception encoders while sharing a compatible spatial communication interface. Our experiments accordingly evaluate independently trained PointPillar, SECOND, PIXOR, and camera-based LSS–ResNet101 systems.

We fully agree that extending IsoComm to non-BEV representations and more diverse fusion architectures would further strengthen its generality. Our revised manuscript now consistently distinguishes the demonstrated zero-shot cooperation across independently trained systems under compatible structural interfaces from unrestricted architectural interoperability. This clarification preserves our central contribution—effective first-encounter cooperation without shared training or pair-specific optimization—while avoiding claims beyond the experimental evidence.

We appreciate the reviewer's guidance and hope the completed revisions satisfactorily address this concern.

References

[1] Lu et al., An Extensible Framework for Open Heterogeneous Collaborative Perception, ICLR 2024.

[2] Gao et al., STAMP: Scalable Task- and Model-agnostic Collaborative Perception, ICLR 2025.

[3] Wang et al., GT-Space: Enhancing Heterogeneous Collaborative Perception with Ground Truth Feature Space, ICLR 2026.

[4] Zhou et al., Pragmatic Heterogeneous Collaborative Perception via Generative Communication Mechanism, NeurIPS 2025.

Reviewer_p5fvOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper formalizes Training-Isolated Heterogeneous Cooperative Perception, where complete perception systems are developed independently on private domains—with no shared data, jointly optimized components, or pre-established communication protocols—and encounter unseen collaborators for the first time at deployment. It proposes IsoComm, which prepares the communication interface during isolated local training: Protocol-Basis Diversification (PBD) perturbs non-ego features with random orthogonal channel rotations and per-channel affine transforms so that the fusion module reduces its dependence on a specific representation basis, and Ownership-Preserving Attention (OPA) keeps relevance estimation under receiver control while preserving sender-native value encoding. An optional commissioning stage (CAC) further calibrates lightweight residual key/value adapters on selected labeled deployment frames. Experiments on OPV2V and V2XREAL with one receiver and three unseen senders show that IsoComm consistently improves over ego-only (88.32–88.54 vs. 82.49 AP50) with a cross-sender AP50 range of 0.22, outperforming baselines that are allowed unlabeled pair observations, and that CAC reaches 90.76 AP50 with 40 labeled frames.

Critical strengths
  1. The problem addresses a genuine deployment challenge in cooperative perception and has clear practical value.
  2. Through Protocol-Basis Diversification and Ownership-Preserving Attention, the paper achieves zero-shot cooperation at first encounter. The method is simple and effective: zero-shot deployment requires no new modules and no pair-specific optimization.
  3. Across the three unseen senders, the AP50 range of MPDA is 11.67 while that of IsoComm is only 0.22—demonstrating the paper’s core claim of robustness to unknown pairings.
Critical weaknesses
  1. The paper attributes zero-shot robustness to PBD’s protocol-basis diversification but lacks mechanism-level analysis. Why not use arbitrary linear perturbations, additive noise, or per-channel standardization? These perturbation-family comparison experiments should be added. Representation-level analysis is also missing (what does the fusion module actually learn? How is channel semantics bypassed?), e.g., CKA or probing analyses.
  2. CAC improves performance for persistent partnerships by training interface adapters, but this adaptation-based improvement contradicts the paper’s motivating scenario of zero-adaptation cooperation at first encounter. CAC does not help demonstrate IsoComm’s superiority in the zero-adaptation first-encounter scenario; instead, the full-commissioning result (93.01) underscores the importance of adaptation after the first encounter, weakening the value of the problem scenario itself. Moreover, CAC is a conventional combination of standard components (bottleneck adapters with zero-initialized residuals) and does not constitute a methodological contribution.
  3. The zero-shot safeguard (norm clipping) is enabled in all zero-shot results, yet disclosed in only one sentence in the main text (end of §4.2), with implementation details in Appendix A.2.4, and has never been separately ablated. Table 3 does not state the safeguard status of each row, and all ablations in Table 7 are run with clipping off while the reference (zero-shot) runs with it on. All three headline numbers in the abstract (88.46, the 0.22 cross-sender range, and the 93.01 commissioning result) are entangled with this component: the attribution of PBD’s contribution, the source of the cross-sender stability (basis-robust fusion vs. mechanical clipping of sender influence), and the decomposition of the commissioning gain cannot be determined from the existing experiments.
  4. The paper reports no wall-clock time or GPU resources for readers’ reference. There is no Limitations section and no Reproducibility Statement.
Questions for authors
  1. What is the level of generality? Can the method accommodate different sensing modalities and more diverse network architectures? The robustness induced by PBD holds only within the sampled distribution—the rotation family drawn from Gaussian sampling—and the bases of unseen systems may fall in the tails of this distribution.
  2. The generality claim holds only at the level of “independently trained weights”; the interface shape, attention mechanism, pose conventions, and feature dimensions are all pre-agreed at development time, and geometric mismatches are resolved by deployment-time reconfiguration. “Training isolation” isolates parameters, not protocols.
  3. In collaboration with B1, none of the compared methods exceeds ego-only even with pair adaptation. Why is this? Is the experimental setting unsuitable for these methods, or do B1’s feature differences exceed their adaptation capacity? What are the concrete settings of the baseline methods? If the setting is inherently unsuitable for these methods, is choosing them as references justified?
  4. PHCP adapts lightweight interfaces from unlabeled collaborator observations, which is nearly identical to the relaxed protocol, yet it is absent from the comparison table. Why?
8 comments

We sincerely thank the reviewer for recognizing the practical significance of training-isolated cooperative perception, the simplicity and effectiveness of zero-shot cooperation through PBD and OPA without pair-specific optimization, and the strong cross-sender robustness demonstrated by IsoComm's 0.22 AP50 range compared with MPDA's 11.67. We appreciate the constructive feedback and address the concerns below with additional experiments and clarifications.

W1. PBD Perturbation Design and Representation-Level Mechanism

We thank the reviewer for the constructive questions regarding PBD's perturbation design and the representation-level mechanism underlying its robustness. Our additional experiments demonstrate that the structured perturbations in PBD outperform simpler alternatives and substantially reduce the fusion module's sensitivity to unfamiliar channel bases while preserving useful cooperative information.

The combination of orthogonal rotations and channel-wise affine perturbations provides more effective representation diversification than the tested alternatives. PBD is designed to reduce dependence on a fixed channel basis during isolated training (§4.2, Eqs. 7–9). Orthogonal rotations diversify channel representations without amplifying feature norms, while channel-wise affine transformations introduce complementary scaling and offset variations. To examine whether simpler alternatives can provide the same benefit, we evaluated channel standardization, additive Gaussian noise, and unconstrained dense Gaussian linear perturbations, keeping the OPA architecture unchanged.

Table R1. Comparison of PBD with alternative feature perturbation strategies across three unseen heterogeneous senders (AP in %).

MethodB1B2B3
AP50AP70AP50AP70AP50AP70
OPA (no perturbation)80.0460.5978.3868.5275.5965.69
OPA + Channel standardization80.7064.5783.8770.9779.2969.43
OPA + Gaussian noise86.6773.0685.0071.6983.6869.67
OPA + Dense Gaussian linear86.7972.0286.0871.9885.9972.07
OPA + PBD (ours)88.4674.2288.5474.3688.3274.31

PBD achieves the highest AP50 and AP70 for all three unseen senders, exceeding dense Gaussian linear perturbations, Gaussian noise, and channel standardization by 2.15, 3.33, and 7.15 mean AP50 points, respectively. It also achieves the smallest cross-sender AP50 range (0.22 points). These results show that the proposed perturbation design offers a consistent empirical advantage over the tested alternatives, rather than obtaining its gains merely from adding arbitrary noise to the communication features.

IsoComm handles channel-semantic mismatch by reducing dependence on a particular channel basis, rather than explicitly recovering channel-to-channel semantic correspondence. PBD exposes the fusion module to varying representations during training, while OPA preserves sender-native Value projections instead of processing unfamiliar sender features entirely through receiver-owned Value weights (§4.2). To investigate whether this design produces the intended behavior, we conducted a frozen-model channel-basis sensitivity analysis on 128 multi-agent frames (A1 ← B1), with inference-time clipping disabled. We applied controlled orthogonal rotations to unseen sender features and measured the normalized change in sender-induced fused representations: S=∥ΔF(α)−ΔF(0)∥F∥ΔF(0)∥F+ϵ,ΔF=Fcoop−FegoS = \frac{\lVert \Delta F(\alpha)-\Delta F(0) \rVert_{F}}{\lVert \Delta F(0) \rVert_{F}+\epsilon}, \qquad \Delta F = F_{\mathrm{coop}}-F_{\mathrm{ego}}

Here, FcoopF_{\mathrm{coop}} and FegoF_{\mathrm{ego}} denote the fused representations with and without the unseen sender, respectively, and ΔF\Delta F isolates the sender-induced contribution to the fused representation. The parameter α\alpha controls the strength of the orthogonal rotation applied to the sender's features, with α=0\alpha=0 denoting the unrotated reference. The subscript FF denotes the Frobenius norm, and ϵ\epsilon is a small constant for numerical stability. Thus, SS measures the relative change in the sender-induced representation under channel-basis rotation; a lower SS indicates that the fusion output is less sensitive to the perturbation. We additionally measure the sender contribution and detection AP to distinguish robustness from simply suppressing the sender's information.

Table R2. Channel-basis sensitivity and cooperative detection performance under controlled orthogonal rotations on 128 A1 ← B1 frames (clipping OFF).

ReceiverRotation αSensitivity SSender contributionAP50AP70ΔAP50 vs. EgoΔAP70 vs. Ego
OPA-only00.0000.33578.8959.00−3.08−10.50
OPA-only0.250.7420.21274.2165.27−7.76−4.23
OPA-only0.500.7130.21872.7362.60−9.25−6.89
PBD + OPA00.0000.22488.0473.92+3.00+3.78
PBD + OPA0.250.4320.17487.8274.33+2.77+4.19
PBD + OPA0.500.3830.18087.8474.64+2.79+4.50

PBD + OPA reduces representation sensitivity by approximately 42–46% at the tested rotations while maintaining effective cooperation. Its AP50 remains nearly unchanged (88.04 → 87.82/87.84), with positive cooperative gains over Ego-only at both AP50 and AP70. In contrast, OPA-only degrades from 78.89 to 74.21/72.73 AP50 and remains below Ego-only. Importantly, PBD + OPA retains measurable sender-induced contributions after rotation (0.174/0.180, compared with 0.224 without rotation), indicating that the observed robustness is not simply achieved by discarding unfamiliar sender information.

These complementary results provide functional evidence that PBD reduces sensitivity to mismatched channel bases while preserving useful collaborative features. We do not claim exact basis invariance or explicit one-to-one channel-semantic alignment. We will add both analyses to the revised Appendix, including perturbation definitions and strengths, training and checkpoint-selection protocols, representation metrics, evaluation details, and limitations, and clarify the mechanism in §4.2.

W2. CAC and the Zero-Shot Motivation

We thank the reviewer for raising this concern. CAC does not contradict our zero-shot motivation, as first-encounter cooperation and persistent-partnership commissioning are explicitly distinguished in §3.2. Zero-shot cooperation requires no pair-specific adaptation, while CAC is an optional subsequent stage for persistent partnerships (§4.3). CAC is therefore not presented as evidence of zero-shot effectiveness; this is independently established in Table 1, where IsoComm improves over Ego-only without CAC (e.g., 88.46 vs. 82.49 AP50 for A1 ← B1). The higher full-commissioning result (93.01 AP50, Table 2) reflects a different deployment regime with additional labeled calibration data. The possibility of further improvement through adaptation does not diminish the practical value of effective cooperation at first encounter, when such calibration is unavailable.

The contribution of CAC lies in its communication-aware commissioning procedure, rather than architectural novelty in its residual adapters. We agree that the residual bottleneck adapters are conventional and do not claim otherwise. CAC instead combines communication-aware frame acquisition based on frozen QK mismatch (§4.3, Eq. 15) with lightweight interface calibration while keeping both base perception systems frozen (Eqs. 16–17). Its practical value is demonstrated in Table 2: CAC with 40 labeled frames achieves 90.76 AP50, outperforming Random with 60 frames (90.44 AP50), despite using 33.3% fewer labeled frames and approximately 32.1% fewer annotated boxes. These results support its role in improving commissioning efficiency, rather than serving as a prerequisite for zero-shot cooperation.

We will clarify the distinction between these two deployment regimes and the scope of CAC's contribution in the revised manuscript. In particular, we will emphasize that CAC is an optional extension to zero-shot cooperation and that its contribution lies in communication-aware data acquisition and interface calibration, not in inventing a new adapter architecture.

W3. Norm Clipping and Attribution of Zero-Shot Robustness

We thank the reviewer for highlighting the need to isolate the effect of norm clipping. Our new ablations show that IsoComm's zero-shot cooperative gains and cross-sender consistency do not depend on norm clipping, and that clipping alone cannot explain its performance. Norm clipping is an auxiliary inference-time safeguard, rather than a core component of PBD or OPA. To directly address the attribution concern, we evaluated Direct fusion, PBD, OPA, and their combination with clipping enabled and disabled (ρ=0.05\rho=0.05) across all three unseen senders, under otherwise identical settings.

Table R3. Ablation of PBD, OPA, and norm clipping (AP in %).

MethodClipB1 AP50B1 AP70B2 AP50B2 AP70B3 AP50B3 AP70Mean AP50Mean AP70AP50 Range
Ego-only—82.4970.0682.4970.0582.4970.0582.4970.05—
DirectOFF57.6653.4185.0872.4981.0869.5674.6165.1527.42
DirectON85.4468.2385.6772.6386.5968.3385.9069.731.15
PBD onlyOFF86.2573.6587.6672.8388.3872.4887.4372.992.13
PBD onlyON87.8473.3188.5474.3688.3074.3288.2374.000.70
OPA onlyOFF80.0460.5978.3868.5275.5965.6978.0064.934.45
OPA onlyON80.2560.3278.6268.7775.1765.3278.0164.805.08
PBD + OPAOFF88.5473.9988.5574.3888.3074.3388.4674.230.25
PBD + OPAON88.4674.2288.5474.3688.3274.3188.4474.300.22

IsoComm maintains nearly identical cooperative performance and cross-sender consistency when clipping is completely disabled. Without clipping, the full PBD + OPA model achieves 88.46 mean AP50 and a cross-sender range of only 0.25 points, closely matching the original clipping-enabled results (88.44 mean AP50; 0.22 range). All three unseen senders consistently improve over Ego-only at both AP50 and AP70. Thus, the principal zero-shot gains and their consistency across heterogeneous collaborators are preserved independently of the inference-time safeguard.

Clipping improves Direct fusion's robustness, but remains insufficient to reproduce IsoComm's cooperative performance. Enabling clipping reduces Direct fusion's cross-sender AP50 range from 27.42 to 1.15 points, confirming its ability to mitigate harmful neighbor messages. However, clipping-enabled Direct fusion still achieves only 69.73 mean AP70, below Ego-only (70.05). Under identical clipping, IsoComm achieves 2.54 higher mean AP50 and 4.57 higher mean AP70, while improving over Ego-only for every unseen sender at both detection thresholds. Moreover, even with clipping disabled, IsoComm outperforms clipping-enabled Direct fusion across all three senders at AP50 and AP70. These comparisons show that the observed benefits extend beyond simply bounding the magnitude of incompatible messages.

The clipping-controlled ablations also establish the complementary contributions of PBD and OPA. Without clipping, PBD alone achieves 87.43 mean AP50 with a cross-sender range of 2.13, whereas combining PBD with OPA improves mean AP50 to 88.46 and reduces the range to 0.25. OPA alone achieves substantially lower mean AP50 (78.00), indicating that sender-native Value preservation is most effective when the fusion module has been trained to tolerate heterogeneous representation bases. The combined framework also delivers the highest mean AP50 and smallest cross-sender range when clipping is enabled.

The different clipping configurations in zero-shot cooperation and commissioning reflect their distinct inference conditions. At first encounter, clipping conservatively bounds potentially incompatible neighbor messages. During commissioning, lightweight key/value adapters calibrate the communication interface; clipping is disabled to avoid unnecessarily restricting calibrated neighbor contributions. Accordingly, the reported 93.01 AP50 full-commissioning result uses the clipping-disabled protocol.

These controlled experiments reinforce our original conclusion that PBD and OPA enable effective and consistent zero-shot cooperation, with both properties preserved without norm clipping. We will clarify the auxiliary inference-time role of clipping in §4.2, explicitly document its zero-shot and commissioning configurations, and incorporate the complete controlled ablations into §5.4 and the Appendix, with safeguard status specified for each configuration.

Reviewer_VXB9OpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

his paper proposes a new and practical heterogeneous collaboration setting in which collaborators are unseen and not jointly trained during the training phase, yet can collaborate in a zero-shot manner at inference time. The authors introduce two key components that enable both zero-shot collaboration and progressive adaptation once communication is established. The method is evaluated on both simulated and real-world benchmarks.

Critical strengths
  1. The paper addresses the practical challenge of enabling collaboration without joint training. In real-world deployments, privacy concerns make it nearly impossible for different companies to share their perception models, so this setting is well motivated.

  2. The framework supports two modes: a zero-shot mode and a communication-aware mode that progressively improves collaboration as more information is received. This design is highly practical, and performance improves accordingly in the latter setting.

  3. The visualizations are clear, and the experiments include comparisons with state-of-the-art methods as well as ablations of the key modules.

Critical weaknesses
  1. While the zero-shot performance is impressive, the paper lacks ablations of the PBD module. How large a domain gap can it handle? How does the training procedure work? My understanding is that the diversification step may introduce noise during training. How is the noise level balanced, and how does it affect both single-agent and collaborative performance? A more in-depth ablation would strengthen the justification for this design.

  2. The improvement of CAC over the random baseline is minor (less than 0.5%). Further analysis demonstrating the effectiveness of CAC would strengthen the paper.

  3. Additional evaluation across different model architectures and parameter sizes would further support the claimed effectiveness of zero-shot collaboration.

Questions for authors
  1. What is the training setup for PBD? How does it perform as the number of heterogeneous collaborators increases, for example with three to five distinct heterogeneous agents?

  2. What are the failure cases? Further analysis of these would help readers better understand the method's limitations.

5 comments

We sincerely thank the reviewer for recognizing the practical significance of training-isolated cooperation under real-world privacy constraints, the practical value of combining zero-shot collaboration with progressive communication-aware commissioning, and the clear visualizations and comprehensive experiments, including state-of-the-art comparisons and key module ablations. We appreciate the constructive feedback and address the concerns below with additional experimental evidence and clarifications.

W1. PBD Perturbation Strength and Single-Agent Performance

We thank the reviewer for raising the question of PBD's sensitivity to perturbation strength and its potential effect on single-agent perception. Our additional experiments demonstrate that effective zero-shot cooperation is maintained across different PBD strengths without substantial degradation in ego-only detection performance. We evaluated three settings—Weak (α = 0.25, β = 0.15), Original (α = 0.50, β = 0.30), and Strong (α = 0.75, β = 0.45)—using the same OPA architecture and frozen unseen senders, with inference-time norm clipping disabled.

Table R1. Sensitivity of ego-only and heterogeneous cooperative detection to PBD perturbation strength (AP in %).

PBD StrengthEgo AP50Ego AP70B1 AP50B1 AP70B2 AP50B2 AP70B3 AP50B3 AP70
Weak (0.25, 0.15)82.4169.3288.2074.3888.1674.3788.2174.37
Original (0.50, 0.30)82.7870.9288.5473.9988.5574.3888.3074.33
Strong (0.75, 0.45)82.6669.8588.1974.6588.1774.5388.0974.68

Collaborative performance remains stable across the tested perturbation strengths, indicating that PBD does not require precise tuning of α and β within this range. Mean AP50 across B1–B3 varies by only 0.31 points (88.15–88.46), while mean AP70 remains between 74.23 and 74.62. The original setting achieves the highest mean AP50, whereas the stronger perturbation achieves slightly higher AP70. This pattern indicates that the cooperative gains persist under moderate changes in perturbation strength rather than depending on a single narrowly tuned configuration.

Importantly, PBD does not substantially compromise single-agent performance because its training-time perturbations are applied only to non-ego features. The ego features are never transformed, and the PBD continuation updates only the fusion module while keeping the encoder and detection head frozen. Consistent with this design, ego-only AP50 remains between 82.41 and 82.78, close to the original paper's ego-only reference of 82.49. Meanwhile, all three perturbation settings achieve over 88 AP50 in heterogeneous cooperation. Thus, the additional robustness to unfamiliar collaborators is achieved without substantial single-agent AP50 degradation in these experiments.

These results support the stability of the proposed PBD design across the evaluated perturbation strengths. We will include the full sensitivity results, training configurations, checkpoint-selection details, and single-agent comparisons in the revised Appendix. The additional Weak and Strong configurations use five-epoch, single-seed continuation; accordingly, these experiments establish local robustness rather than global hyperparameter optimality.

W2. Effectiveness of Communication-Aware Commissioning

We thank the reviewer for the comment on CAC's effectiveness. We would like to clarify that CAC's improvements over Random are not uniformly below 0.5 AP points, and its primary benefit is improved annotation efficiency. As shown in Table 2, at 40 labeled frames, CAC improves AP30/AP50/AP70 over Random by 0.95/0.57/0.20 points, respectively. At 60 frames, the corresponding improvements are 0.96/0.57/0.48 points.

More importantly, CAC achieves better detection performance with substantially fewer annotations. With only 40 labeled frames, CAC outperforms Random with 60 frames across all three metrics (94.25/90.76/74.09 vs. 93.57/90.44/73.98 AP30/AP50/AP70), despite using 33.3% fewer labeled frames and approximately 32.1% fewer annotated boxes. This comparison directly demonstrates the practical benefit of communication-aware frame selection: CAC can achieve a stronger calibrated interface while reducing the annotation burden.

The existing acquisition ablations further support the contribution of communication-aware selection beyond the calibration adapters themselves. Appendix A.5 (Table 8) compares the proposed NormalizedQK + diversity strategy against Random-diverse sampling and RawQK ranking under the same annotation budgets and adapter-training protocol. At 40 and 60 frames, the proposed strategy achieves higher AP30, AP50, and AP70 than both alternatives. While the gains are smaller at very low budgets (10–20 frames), the results consistently support the usefulness of communication-aware selection at moderate annotation budgets.

CAC's contribution is therefore best understood through the improved performance–annotation trade-off, rather than a single AP difference at a fixed budget. We will make these existing numerical comparisons and acquisition ablations more explicit in §5.3 of the revised manuscript.

W3. Generalization Across Model Architectures and Sizes

We thank the reviewer for suggesting broader architectural evaluation. Our existing experiments already cover multiple independently trained encoder architectures, backbone capacities, and sensing modalities on both the sender and receiver sides, demonstrating that IsoComm's zero-shot gains are not confined to one perception model family.

The sender-side results establish consistent zero-shot cooperation across three different feature extraction architectures. In Table 1, with the A1 PointPillar receiver fixed, we evaluate three unseen V2XREAL-trained senders: B1 (PointPillar with a shallow BEV backbone), B2 (SECOND), and B3 (PIXOR). Despite their distinct encoder architectures, IsoComm achieves 88.46, 88.54, and 88.32 AP50, respectively, with a cross-sender range of only 0.22 points. These results demonstrate consistent cooperation across the evaluated heterogeneous sender architectures without pair-specific adaptation.

Receiver-side evaluations further show that the gains extend beyond the original PointPillar receiver, including a camera-based perception system. Appendix A.4 (Table 6) fixes B1 as the unseen sender and evaluates receivers based on PointPillar, SECOND, and LSS–ResNet101.

Table R2. Zero-shot cooperative performance across different receiver architectures and sensing modalities (AP50 in %).

Receiver ArchitectureEgo-onlyDirectIsoCommGain over Ego
A1: PointPillar (deep BEV backbone)82.4957.6688.46+5.97
A3: SECOND78.5379.5683.58+5.05
A4: LSS–ResNet101 (camera)22.3723.7030.28+7.91

IsoComm improves over Ego-only for every evaluated receiver architecture, including the camera-based system, without pair-specific training or adaptation. The AP50 gains of 5.05 points on SECOND and 7.91 points on LSS–ResNet101 provide additional evidence that the observed zero-shot benefits are not specific to the original receiver architecture or LiDAR sensing modality.

These results already demonstrate practical architectural diversity, although they do not constitute a systematic parameter-count scaling study. The evaluated models span distinct encoder families and backbone capacities, but we have not independently varied model parameter counts while controlling other architectural factors. We will make the existing sender- and receiver-side evidence more prominent in the revised experimental discussion and clarify the scope of architectural generalization supported by these experiments.

Q1. PBD Training Setup and Scalability to Multiple Heterogeneous Collaborators

We thank the reviewer for asking about PBD's training procedure and scalability to multiple heterogeneous collaborators. Our training procedure requires only isolated local data, and our additional experiments demonstrate stable cooperative performance as independently trained heterogeneous agents are progressively introduced.

PBD is trained through a two-stage isolated local procedure, as detailed in Appendix A.2.1–A.2.2 and Table 5. Each system first undergoes OPA-compatible base training on its own source domain (30 epochs for A-domain systems and 40 epochs for B-domain systems). The checkpoint with the lowest validation loss then initializes a 20-epoch PBD continuation with batch size 1, using Adam and a MultiStepLR scheduler. During this continuation, PBD independently applies randomly sampled orthogonal rotations and channel-wise affine transformations to non-ego features, while keeping ego features unchanged. PBD is disabled at inference time, requiring no pair-specific adaptation when an unseen collaborator is encountered. Our additional perturbation-strength analysis in W1 further demonstrates that the resulting cooperative gains remain stable across the tested training perturbation strengths.

Our additional multi-agent experiments demonstrate that IsoComm maintains stable cooperative performance as independently trained agents with different encoder architectures are progressively introduced. We extend the evaluation from a single heterogeneous sender to three distinct senders—B1 (PointPillar), B2 (SECOND), and B3 (PIXOR)—while keeping the A1 receiver fixed.

Our evaluation follows the incremental heterogeneous collaboration setting adopted in prior frameworks, including STAMP (Gao et al., ICLR 2025), HEAL (Lu et al., ICLR 2024), and GenComm (Zhou et al., NeurIPS 2025). Specifically, we progressively introduce different sender types in the order A1+B1, A1+B1+B2, and A1+B1+B2+B3, without retraining or pair-specific adaptation. All configurations are evaluated on the complete 2,170-frame test set, using the same receiver and strictly nested physical collaborators without duplicate senders.

Table R3. Incremental heterogeneous collaboration with up to three independently trained sender architectures on the complete test set (AP in %).

ConfigurationAP50AP70
Ego-only (original Table 1)82.4970.05
A1+B188.8573.91
A1+B1+B288.8774.01
A1+B1+B2+B388.8774.03

The results demonstrate that introducing additional heterogeneous collaborators does not compromise cooperative performance in the evaluated setting. AP50 remains remarkably consistent across the three configurations (88.85, 88.87, and 88.87), varying by only 0.02 points, while AP70 slightly improves from 73.91 to 74.03. Importantly, this stability is achieved without modifying the original receiver or performing pair-specific optimization for the newly introduced sender architectures.

These findings extend our original single-sender results to incremental multi-agent cooperation, supporting IsoComm's ability to accommodate independently trained heterogeneous collaborators. Since the number of physically available senders varies across frames, the experiment evaluates dynamic collaborator availability rather than four-agent cooperation in every scene. We will include the results and detailed evaluation protocol in the revised Appendix.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.