acceptodds
Under review as a conference paper at ICLR 2027

Decoupling to Fuse: Learning Shared and Modality-Specific Representations for Multi-Sensor 3D Object Detection

Abstract

Fusing information from cameras, LiDAR, and 4D radar is an important approach to improving 3D object detection in autonomous driving. However, multimodal features contain a mixture of shared information, modality-specific cues, and background responses. A key challenge is to exploit cross-modal consistency while preserving useful differences and guiding their fusion according to object relevance. We introduce ObjDec, a multi-sensor fusion architecture that learns shared and modality-specific representations and uses object-related information to guide their fusion. Specifically, ObjDec applies constraints to aligned local representations within foreground regions, encouraging cross-modal consistency in shared features while preserving differences among modality-specific features. Based on these representations, foreground gating and modality control regulate each modality’s feature contribution, while object context further modulates cross-modal attention queries and fused outputs. Experiments on K-Radar dataset demonstrate that ObjDec achieves state-of-the-art performance, improves 8.04% @IoU=0.3 (88.36%)and 7.77% @IoU=0.5 (88.10%)at a confidence threshold of 0.3, with only approximately 1.7% more parameters and fast inference. Component ablations and analyses of learned representations and spatial gating further support the effectiveness of the proposed design. Our code will be made publicly available.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
Autonomous Driving, 3D Object Detection, Multi-Sensor Fusion, Multimodal Representation Learning, Representation Decoupling

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_TmAtOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes ObjDec, a camera–LiDAR–4D-radar fusion framework that explicitly decomposes aligned BEV patch tokens into shared and modality-specific representations. Foreground-grounded constraints are combined with foreground gating, modality contribution control, and object-context modulation. The method is evaluated mainly on K-Radar v1/v2, with additional V2X-Radar-V experiments and extensive representation/gating analyses.

Critical strengths
  1. The representation-decoupling perspective is meaningful for multi-sensor fusion because it explicitly separates cross-modal agreement from modality-specific complementary cues and ties both to detection-relevant foreground regions.
  2. The diagnostic analysis is relatively transparent: the paper reports confidence-threshold sensitivity, representation statistics, spatial-gate interventions, component ablations, and transfer to another dataset.
Critical weaknesses
  1. A highly relevant prior work is missing: 'Multi-Modal Decouple and Recouple Network for Robust 3D Object Detection' (Ding et al., 2026) already decomposes camera/LiDAR BEV features into modality-invariant and modality-specific parts before adaptive recoupling. Since this overlaps directly with ObjDec's core shared/specific decomposition, the paper must cite it and clearly establish what is methodologically new beyond object-guided supervision and ASF-based fusion control.
  2. The headline K-Radar v1 gain is strongly operating-point dependent. The large improvement at confidence 0.3 and IoU 0.3 shrinks substantially at stricter IoU; at IoU 0.5 the overall gain over ASF is very small, and sleet performance degrades markedly. The current 'SOTA' framing overemphasizes a favorable threshold/IoU configuration.
  3. The main comparison is not fully controlled. The manuscript explicitly states that published baselines use non-uniform labels, preprocessing, and evaluators, so ranking ObjDec against those numbers is not a clean SOTA comparison. The strongest claims should rely on reproduced baselines under an identical protocol.
  4. The ablation does not isolate the actual decoupling architecture. 'w/o decoupling supervision' keeps both shared/specific branches and their forward paths; therefore it tests the auxiliary losses, not whether explicit shared/specific decomposition itself is necessary. A true no-decoupling baseline such as simple concatenation or a single shared projection is required.
  5. The role of object-context modulation is internally unclear. Removing the query/output context increments at inference changes AP by almost zero, whereas retraining without object context causes a large drop. This suggests that the branch may act mainly through auxiliary supervision or optimization rather than through the claimed inference-time context modulation, and these effects should be separated.
Questions for authors
  1. How does ObjDec differ from Ding et al. (2026), 'Multi-Modal Decouple and Recouple Network for Robust 3D Object Detection,' and why is this closely related decomposition framework not cited or compared?
  2. Can the authors provide a fully matched ASF comparison with identical preprocessing, evaluator, confidence filtering, training schedule, and checkpoint selection?
  3. Can the authors add a true no-decoupling/Concat baseline to isolate the architectural contribution from the auxiliary supervision and extra branches?
  4. What is the exact origin of the 1,000-frame K-Radar v1 checkpoint-selection subset, and is it strictly disjoint from the final test set at the sequence/frame level? Please also explain the near-zero inference effect of the object-context increments.
  5. Also, i think more R+C 3D object detection methods should be included for discussion, such as SGDet3D, RaGS, SIFormer.
  6. I will improve my score if authors address the issues above.
2 comments
AuthorsOpenReview ↗

Thanks and Response to Reviewer TmAt

Thank you for pointing us to Ding et al. and for distinguishing the contribution of the shared/specific architecture from that of its supervision. We address both questions below and report two additional controlled experiments.

Q1. Relation to Ding et al.

Both methods learn shared and modality-specific BEV features. Ding et al. use an invariant encoder shared across camera and LiDAR, an auxiliary detection head and adaptive recoupling. ObjDec uses separate mappings for each sensor and applies its representation losses at aligned foreground patches. At each patch, the learned features determine how strongly to update each sensor token and how to weight the available sensors before local attention. We will cite Ding et al. and make this connection between foreground supervision and local fusion control explicit in the introduction and related work.

Q2. Matched ASF comparison

On V2X-Radar-V, ASF and ObjDec use the same three sensors, encoder and detection-head architectures, 0.16 m grid and 80-epoch schedule. Both use the same evaluator, head score threshold (0.001) and rotated NMS threshold (IoU 0.1). Both select checkpoints by mean Moderate 3D AP on the same validation split and use the same deduplicated test set.

V2X-Radar-V test AP (Table 4b).

MethodVehicle 3DPed. 3DCyclist 3DMean 3DMean BEV
ASF83.5474.1782.1879.9685.72
ObjDec84.0675.9484.1381.3886.27
Difference+0.52+1.77+1.95+1.42+0.55

K-Radar v1 3D AP at IoU 0.5 and export threshold 0.3 (Table 1).

MethodOverallRainSleet
ASF67.1967.3367.28
ObjDec67.5074.9257.60

On v1 the overall gain is 0.31 points, with an improvement in rain and a drop in sleet. Tables 1 and 9 use the official ASF checkpoint model_10.pt and evaluation archive exp_250303_200024_A2F_v1_0. Table 1 reports export threshold 0.3. Table 9 repeats 0.3 and adds 0.0, with the head prefilter fixed at 0.1. We will correct the Table 1 caption .

Q3. Does the shared/specific architecture improve on direct fusion?

All four variants retain task supervision. The first two omit both representation branches. Raw-token control predicts the fusion controls directly from the original tokens. The last two rows have identical architectures.

R2. Fusion ablation. Scores are 3D AP.

VariantParams (M)IoU 0.3IoU 0.5IoU 0.7
Simple fusion78.4681.3566.5717.48
Raw-token control79.4780.7965.8416.63
Two branches, representation losses off79.4780.2366.1616.49
Full ObjDec79.4788.3667.5022.04

At 79.47M parameters, full ObjDec exceeds raw-token control by 5.41 AP points at IoU 0.7 (22.04 vs. 16.63). With the representation losses off, the two branches score 16.49, close to raw-token control. Enabling the three losses within the same architecture raises AP by 5.55 points.

Q4. Validation split and object context

We reserved a fixed set of 1,000 frames to select the v1 ObjDec and ablation checkpoints. None appears among the 10,065 final test frames.

We retrained all four settings with the same budget, other losses and modules. Turning injection off disables both the query and output increments.

R1. Context ablation. Scores are 3D AP.

SettingContext lossInjectionIoU 0.3IoU 0.5IoU 0.7
AOnOn88.3667.5022.04
BOnOff87.0166.3518.53
COffOn86.2566.0217.24
DOffOff80.1466.2317.10

At IoU 0.7, enabling injection during training adds 3.51 points with context supervision (A−B) and 0.14 without it (C−D). Adding supervision gives 4.80 points with injection (A−C) and 1.43 without it (B−D). At the same IoU, disabling both increments only at inference changes AP from 22.0603 to 22.1096 (+0.0493, Table 14). In these runs, training with injection improves the supervised model’s final AP, while the trained checkpoint is only weakly sensitive to the residuals at inference.

Q5. Radar–camera detectors

We will discuss SGDet3D’s fusion of radar geometry and image semantics, RaGS’s use of radar and monocular cues to construct 3D Gaussians, and SIFormer’s cross-view instance correspondence alongside our approach to controlling fusion at aligned BEV patches.

AuthorsOpenReview ↗

To Reviewer TmAt

Dear Reviewer TmAt,

We hope this message finds you well. With the deadline of discussion period is approaching, we wanted to make sure we have fully addressed your concerns. If you have any further points or feedback you would like us to consider, please let us know. Your insights are invaluable to us, and we are open to any further suggestions you might have to improve the paper.

Thank you for the time and effort you have dedicated to reviewing our paper.

Best regards,

The Authors

Reviewer_iSNzOpenReview ↗
Rating
1
Confidence
3
Informativeness
2
Presentation
1
Soundness
2
Read review
Paper summary

Overall, ObjDec offers a promising approximate solution to the entanglement of shared, modality-specific, and background information in multi-sensor fusion. The architecture directly addresses the problem identified by the authors, and the experiments show that its components can improve detection under some reported settings. However, the evidence does not yet establish that the model semantically and causally separates object-level shared and modality-specific information, nor that the broader problem is resolved under realistic industry deployment conditions. The headline gain is highly sensitive to confidence filtering, the primary checkpoint-selection subset is undocumented, and repeated-seed uncertainty is not reported. In addition, The writing is occasionally awkward and overly dense! I would reconsider the recommendation if the authors use a clean held-out selection protocol, demonstrate gains under matched evaluation without threshold-dependent inflation, and provide reproducible controlled ablations.

Critical strengths
  1. The paper presents a coherent object-guided formulation of shared and modality-specific representation learning for local multi-sensor fusion. The shared/specific branches are not introduced only as auxiliary features: they are connected to foreground gating, modality contribution control, and local attention. This is a sensible design for camera, LiDAR, and radar, whose aligned observations contain both redundant evidence and complementary cues.

  2. The empirical study is unusually broad. It includes K-Radar v1 and v2, target-dataset training on V2X-Radar-V, an additional LiDAR-radar adaptation on VoD, component ablations, missing/corrupted-sensor interventions, representation and gate analyses, and latency measurements. The controlled V2X comparisons show modest improvements over the corresponding ASF and L4DR references, suggesting that the design may transfer beyond the primary K-Radar setting.

Critical weaknesses
  1. The central performance claim is not yet robust enough to support acceptance. On K-Radar v1, the headline gain over ASF at confidence 0.3 is 8.04 points for 3D AP at IoU 0.3, but at confidence 0.0 the gain is only 0.72 points (88.06 versus 87.34), and 3D AP at IoU 0.5 is slightly lower (72.83 versus 72.95). This large dependence on an additional confidence prefilter raises the possibility that much of the reported gain reflects score calibration or the selected operating point rather than improved ranked detections. The paper also states that published baselines do not share uniform protocols, uses a 1,000-frame subset of undocumented origin for v1 checkpoint selection, and reports no variation across training seeds. These factors weaken both fairness and statistical reliability.

  2. The evidence does not clearly establish that the claimed semantic decoupling and context mechanism cause the gains. For shared representations, same-frame and weather-matched cross-sequence cosine similarities are almost identical before centering, and the centered camera-related correspondence is near zero. The specific camera-radar similarity remains around 0.65 despite the low similarity margin. In addition, retraining without object context reduces AP substantially, whereas removing both context increments from the trained model changes all overall AP values by less than 0.08 points. This discrepancy suggests that the benefit may arise from optimization or auxiliary regularization rather than the stated inference-time context pathway. More controlled evidence is needed to distinguish these explanations.

  3. The writing is occasionally awkward and overly dense. Several sentences compress multiple technical points into long constructions, while transitions between the motivation, mechanism, and supporting evidence are sometimes abrupt. The experimental protocols and result provenance are also fragmented across the main text and appendices, and several comparisons use non-uniform evaluators or operating points. In addition, unresolved placeholders and incomplete references, including "Figure ??", "ResNet-50 (?)", "LSS (?)", and "VoD (?)", reduce the paper's polish and make some technical descriptions harder to verify. As a result, readers must repeatedly move between sections to reconstruct which comparisons are controlled and how the evidence supports the stated conclusions.

Questions for authors
  1. What is the exact origin of the 1,000-frame subset used for K-Radar v1 checkpoint selection? Is it disjoint from the 10,065-frame test set? Please provide the sequence/frame indices, the selection metric, and results using a clearly held-out validation protocol or a predeclared final checkpoint. If this subset contains test frames, it would materially change my assessment.

  2. Why does the large v1 gain over ASF at confidence 0.3 nearly disappear at confidence 0.0, with no improvement at 3D IoU 0.5? Please report paired precision-recall or score-calibration analyses and results for ObjDec and ASF under exactly the same evaluator, postprocessing, checkpoint-selection rule, and a range of export thresholds. This would clarify whether ObjDec improves detection quality or mainly shifts confidence scores.

  3. Can the authors provide results over multiple training seeds for the full model and the main ablations, and separate the effects of context supervision from the forward query/output context injections? Please also include a patch- or object-level correspondence test with hard mismatches. These experiments would determine whether the large ablation changes are reproducible and whether the learned branches encode instance-relevant shared/specific information rather than dataset- or weather-level structure.

5 comments
AuthorsOpenReview ↗

Thanks and Response to Reviewer iSNz

Thank you for recognizing how the shared and modality-specific features guide local fusion, and for specifying which comparisons would affect your assessment.

Q1. Which frames were used for checkpoint selection?

We reserved a fixed validation set of 1,000 K-Radar v1 frames and used it to select checkpoints for both full ObjDec and its ablations. None of these frames appears in the final 10,065-frame test set.

Q2. How much of the v1 gain survives a lower export threshold?

ObjDec minus ASF, in AP points, calculated from the displayed values in Table 9.

Export threshold[email protected][email protected][email protected][email protected]
0.3+8.05+0.31+3.19+7.77
0.0+0.72−0.12+3.17+0.21

At IoU 0.3, lowering the export threshold raises ASF’s 3D AP from 80.31 to 87.34, while ObjDec changes from 88.36 to 88.06. This accounts for the gap shrinking from 8.05 to 0.72 points. At export 0.0, ASF leads by 0.12 points at IoU 0.5, and ObjDec retains a 3.17-point lead at IoU 0.7. We will qualify the headline gain with its export threshold and IoU. Both export settings retain the 0.1 head prefilter.

The v1 ASF results in Tables 1 and 9 use the official checkpoint model_10.pt and evaluation archive exp_250303_200024_A2F_v1_0. Table 1 reports export 0.3. Table 9 repeats that setting and adds 0.0. We will correct the Table 1 caption. On V2X-Radar-V, where both methods use the same evaluator, postprocessing and validation selection rule, mean test 3D AP is 79.96 for ASF and 81.38 for ObjDec (+1.42, Table 4b).

Q3. Context and local correspondence

Context supervision and injection. We retrained all four settings with the same budget, keeping the remaining modules and losses fixed. Disabling injection removes both the query and output context increments.

R1. K-Radar v1 3D AP after retraining.

SettingContext lossInjectionIoU 0.3IoU 0.5IoU 0.7
AOnOn88.3667.5022.04
BOnOff87.0166.3518.53
COffOn86.2566.0217.24
DOffOff80.1466.2317.10

At IoU 0.7, enabling injection adds 3.51 points with context supervision (A−B) and 0.14 without it (C−D). Enabling supervision adds 4.80 points with injection (A−C) and 1.43 without it (B−D). At the same IoU, disabling both increments only at inference changes AP from 22.0603 to 22.1096 (+0.0493, Table 14). The clearest benefit from including injection during training appears when context supervision is active. Once trained, this checkpoint shows little dependence on the two increments at inference.

Local correspondence. Figure 7 and Appendix E.2 compare shared LiDAR–radar features averaged over foreground patches in each frame. Each mismatched frame comes from a different sequence in the same weather group.

Frame-level cosine similaritySame frameMismatched frames
Before centering0.9876970.987136
After subtracting each modality’s weather-group mean0.148740−0.014689

After centering, the LiDAR–radar features retain a frame-level correspondence signal. Matched-pair scores involving the camera are near zero. At matched foreground patches in normal weather, camera–radar cosine similarity is 0.9584 for shared features and 0.6545 for specific features (Table 15). The loss penalizes similarity above 0.1 but does not enforce a hard bound.

Presentation and scope. We will consolidate checkpoint-selection and evaluation details, identify which comparisons use matched protocols, and fix the figure, ResNet-50, LSS and VoD references. We will also shorten the mechanism description and limit our conclusions to the evaluated benchmarks.

AuthorsOpenReview ↗

To Reviewer iSNz

Dear Reviewer iSNz,

As the discussion period ends soon, we would greatly appreciate it if you could let us know whether our responses have addressed your concerns, or if there is anything else we can clarify.

Best regards,

The Authors

Reviewer_iSNzOpenReview ↗

Reply

I thank the authors for a detailed response. The new ablation grid is genuinely informative, and I appreciate the explicit statement that the forward context branches act as training-time regularizers. That clarification, however, it also indicates that the paper's central claim cannot survive in its current form. Q1. The response states that 1,000 frames were reserved and that none appear in the test set, but provides no frame or sequence indices, no construction protocol, and no statement on whether this subset also informed thresholds or post-processing. Because the same 1,000 frames selected checkpoints for the full model and all ablations, the selection bias is shared across every reported comparison. An unverifiable assertion does not allow me to assess leakage, so this concern remains open. Q2. The numbers supplied confirm rather than dispel the concern. At the lower export threshold the 3D AP gap at IoU 0.5 moves from +0.31 to −0.12, the BEV gap at IoU 0.5 from +7.77 to +0.21, and the 3D gap at IoU 0.3 from +8.05 to +0.72; only IoU 0.7 remains stable. The authors' own explanation—that ASF loses many low-confidence true detections at the higher export threshold—describes a shift in confidence distribution, which is precisely the calibration account under dispute. No precision-recall curves or threshold-free AP on a common curve were provided, so the question of detection quality versus score placement is still unanswered. Q3. The interaction between supervision and injection is a useful addition. The remaining gaps are substantive. Seed variance is absent, and the authors acknowledge it. The correspondence test provides essentially no discrimination before weather-mean removal (0.9877 vs. 0.9871) and only a weak effect afterwards (0.1487 vs. −0.0147). Most importantly, the specific-branch cosine similarity of 0.6545 against a regularization margin of 0.1 indicates that the constraint is not binding, which undermines the claimed shared/specific disentanglement independently of any new experiment. Finally, the authors report that disabling both context injections at inference changes AP from 22.0603 to 22.1096. If the architectural contribution is inert at test time, the paper's framing of context injection as an inference-time mechanism is not supported.

Reviewer_89ZXOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

Object learns shared and modality-specific representations within ASF’s local BEV fusion framework. Foreground constraints, gating, and object context regulate multi-sensor fusion. Experiments cover K-Radar, V2X-Radar-V, and VoD, with approximately 1.7% additional parameters. The design is coherent and improves some metrics, but the consistency of the comparisons and the evidence for the proposed mechanisms remain insufficient.

Critical strengths
  1. Coherent design with modest overhead. Foreground-supervised representation learning is connected to local fusion control. Parameters increase from 78.13M to 79.47M, and paired eager inference latency increases from 84.00 to 84.69 ms.

  2. Useful ablations and interventions. Removing representation supervision while retaining branch capacity, spatially shuffling gates, and evaluating multiple datasets provide useful evidence. The paper also reports threshold sensitivity and failure cases.

Critical weaknesses
  1. Key results and evaluation protocols are not reconciled. On V2X, ASF’s mean 3D AP is 79.96 in Tables 4/21 but 81.30 in Table 20, while ObjDec remains at 81.38; the gain therefore changes from 1.42 to 0.08 points. On K-Radar v2, ASF’s BEV [email protected] is 58.77 in Table 3, but its class results in Table 10 average to 62.455, reversing its ranking relative to ObjDec (60.93). In Table 9, lowering the export threshold from 0.3 to 0.0 reduces the gains in 3D [email protected] and BEV [email protected] from approximately 8.05/7.77 to 0.72/0.21 points. However, the 3D [email protected] gain remains about 3.17 points. Score filtering and detection quality therefore need to be disentangled; the evidence does not show that all gains are threshold artifacts. The provenance and test-set independence of the 1,000-frame checkpoint-selection subset also remain unclear.

  2. The contributions of decoupling and context are not sufficiently isolated. Cosine constraints and frame-averaged similarities do not establish useful semantic separation. Local correspondence tests, branch-level task probes, and capacity- and supervision-matched controls are needed. Removing the entire context component during retraining reduces 3D [email protected] by 8.22 points (Table 2), whereas disabling its query/output increments at fixed weights reduces it by only about 0.0146 points (Table 14). Since retraining also removes auxiliary supervision, these results cannot isolate supervision from forward injection, nor rule out a training-time role for context.Furthermore, Table 18 shows that replacing camera inputs with black frames does not affect AP and full-modality [email protected] (67.50) trails the camera-free L+R subset (71.68), calling for clarification on the actual contribution of the camera branch.

Questions for authors
  1. Please reconcile the ASF results, identify each checkpoint and evaluation source, and establish whether the 1,000-frame selection subset is independent of the test set.

  2. Under a common, lower head-score prefilter, what are the PR curves, maximum recall, and strict-IoU results for ASF, ObjDec, and key ablations? Do small gains exceed training variability?

  3. What does each representation loss contribute? Can direct token-based control achieve similar results with matched capacity and foreground supervision? Please provide local representation validation.

  4. What happens when training retains context supervision but disables injection, or retains injection but removes context supervision? These controls are central to the mechanism claim.

3 comments
AuthorsOpenReview ↗

Thanks and Response to Reviewer 89ZX

Thank you for cross-checking the baseline results and for asking how much of the gain persists under stricter matching.

Q1. ASF results and checkpoint selection

V2X-Radar-V test AP, 0.16 m (Table 4b).

MethodVehicle 3DPed. 3DCyclist 3DMean 3DMean BEV
ASF83.5474.1782.1879.9685.72
ObjDec84.0675.9484.1381.3886.27
ObjDec − ASF+0.52+1.77+1.95+1.42+0.55

The ASF class scores average to 79.96 (epoch 80), giving a 1.42-point gain for ObjDec (epoch 70). Table 20’s 81.30 came from a different checkpoint. We will correct the test mean and check the validation entry against its own log.

K-Radar v2 two-class mean AP at IoU 0.5 (Table 3).

MethodMean 3DMean BEV
ASF39.7058.77
ObjDec43.0560.93

The ASF BEV entries averaging 62.455 in Table 10 came from a different checkpoint. We will regenerate the ASF rows in Tables 10 and 12 from the Table 3 checkpoint. On v1, Tables 1 and 9 use the official ASF checkpoint model_10.pt and archive exp_250303_200024_A2F_v1_0. Table 1 uses export threshold 0.3. Table 9 repeats 0.3 and adds 0.0. We will correct Table 1’s caption. The 1,000 fixed v1 validation frames used to select ObjDec and its ablations are disjoint from the 10,065 test frames.

Q2. Confidence filtering and strict IoU

K-Radar v1 AP differences in points, calculated from Table 9 (ObjDec − ASF).

Export threshold[email protected][email protected][email protected][email protected]
0.3+8.05+0.31+3.19+7.77
0.0+0.72−0.12+3.17+0.21

The 3D gain at IoU 0.3 shrinks to 0.72 points, while the gain at IoU 0.7 remains above 3 points. Both exports retain the 0.1 head prefilter. Q3. Representation losses and local evidence

R2. Fusion ablation with task supervision retained. Scores are 3D AP.

VariantParams (M)IoU 0.3IoU 0.5IoU 0.7
Simple fusion78.4681.3566.5717.48
Raw-token control79.4780.7965.8416.63
Two branches, losses off79.4780.2366.1616.49
Full ObjDec79.4788.3667.5022.04

At 79.47M parameters, full ObjDec exceeds direct raw-token control by 5.41 points at IoU 0.7. With the losses off, the two branches score 16.49, close to raw-token control (16.63). Enabling the losses in the same architecture adds 5.55 points.

R3. Loss ablation with both branches active. Scores are 3D AP.

Loss settingIoU 0.3IoU 0.5IoU 0.7
All three on88.3667.5022.04
Remove Lc86.9266.3020.15
Remove Lu88.1367.8221.46
Remove Lsep87.5967.0819.74
Remove all three80.2366.1616.49

At IoU 0.7, removing within-sensor shared/specific separation (Lsep) gives the largest single-loss drop (2.30), followed by shared alignment (Lc, 1.89) and the cross-sensor specific-feature penalty (Lu, 0.58). Removing Lu raises AP by 0.32 at IoU 0.5. On matched foreground patches in normal weather, camera–radar cosine is 0.9584 for shared features and 0.6545 for specific features (Table 15).

Q4. Context supervision and forward injection

R1. All four settings were retrained with the same budget, other modules and losses. Injection toggles both query and output increments. Scores are 3D AP.

SettingContext lossInjectionIoU 0.3IoU 0.5IoU 0.7
AOnOn88.3667.5022.04
BOnOff87.0166.3518.53
COffOn86.2566.0217.24
DOffOff80.1466.2317.10

At IoU 0.3, supervision alone adds 6.87 points over D (B−D), injection alone adds 6.11 (C−D), and enabling both adds 8.22 (A−D). With supervision active, enabling injection during training adds 1.35 (A−B). At the same IoU, disabling both increments only at inference changes AP from 88.3599 to 88.3453 (−0.0146, Table 14). Both choices affect the trained result, while this checkpoint shows little direct dependence on the residuals at inference.

Camera contribution

Fixed weights and export threshold 0.3 on v1 (Table 18).

Camera input[email protected][email protected][email protected]
Real images88.3667.5088.10
Black images88.3667.4188.09
Camera removed88.0671.6885.45

Black images barely change AP here. Removing the camera recomputes fusion over LiDAR and radar. At IoU 0.5, 3D AP rises to 71.68 while BEV AP falls to 85.45.

AuthorsOpenReview ↗

To Reviewer 89ZX

Dear Reviewer 89ZX,

As the discussion period ends soon, we would greatly appreciate it if you could let us know whether our responses have addressed your concerns, or if there is anything else we can clarify.

Best regards,

The Authors

AuthorsOpenReview ↗

To Reviewer 89ZX

Dear Reviewer 89ZX, as the discussion deadline is approaching, we would sincerely appreciate your feedback on whether our responses have addressed your concerns and whether you would consider updating your assessment accordingly.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.