acceptodds
Under review as a conference paper at ICLR 2027

Where Is Post-Degradation Repair Limited? Evidence from Camera-Based 3D Occupancy

Abstract

In camera-based 3D occupancy, post-deployment repair aims to recover task performance when observation quality degrades, and its effectiveness is commonly evaluated by the amount of performance recovered. Under a fixed model, a small final task gain indicates limited recovery but does not reveal where the repair process is constrained. Motivated by this ambiguity, we separate the analysis into local corrective evidence, privileged task gain, and inference-time task gain. We evaluate their relations using matched interventions across four nuScenes occupancy anchors and a controlled CARLA setting. This evaluation leads to three main observations. First, local corrective evidence can persist through finite-objective improvement while a positive task-metric advantage over the matched control remains unestablished. Second, a positive privileged task gain can correspond to substantially smaller or negative inference-time task gain. Third, inference-time task gain varies with allocation policy and prescribed intervention capacity, and can also change sign across semantic objectives even when the policy is fixed. In summary, these findings motivate examining how corrective evidence translates into finite task gain.

Primary area:
applications to robotics, autonomy, planning
Keywords:
3D Occupancy, Autonomous Driving, Robust Perception, Computer Vision

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_svzEOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The authors propose a diagnostic analysis framework that decomposes the repair process into local corrective evidence, privileged task gain, and inference-time task gain, and systematically evaluate it through matched intervention experiments across multiple nuScenes occupancy models and a CARLA simulation environment. Experimental results demonstrate its effectiveness.

Critical strengths

The research question is novel. It decomposes limited repair gains into three tiers—local evidence, privileged gains, and inference-time gains—and highlights that a small final gain does not mean the method is ineffective.

The controls and reporting are relatively standardized. Total intervention strength is precisely matched, with Uniform and Shuffled controls, scene-cluster-level confidence intervals, and numerical checks using FP64 and finite differences. Non-significant results are also faithfully reported as unestablished, avoiding any forced positive conclusions.

Critical weaknesses

The core question is insufficiently answered, and the main conclusions approach common sense. Although the paper discusses repair limitations, it merely lists phenomena such as local evidence not equaling finite-step gains. Since the privileged allocation uses ground truth from the same record, its superiority over inference-time allocation is expected, and the paper evaluates no actual repair methods, providing little guidance for improvement.

The negative-gain conclusion suffers from insufficient attribution, as it stems from a small-scale learning policy. In PanoOcc, the negative effect comes from a specific policy trained on 70 development scenes, while alternative allocation policies under the same direction yield positive gains, suggesting poor policy training rather than a general rule.

The experimental setup is heterogeneous with small sample sizes and tiny effect sizes, raising doubts about comparability and robustness. Intervention contracts vary across models, overall gains are mostly within fractions of a percentage point, and some subsets rely on very few scenes or near-failing baselines.

The text is overly burdened with jargon and has poor readability. The extensive use of abbreviations, qualifiers, and hidden key details in the appendices makes the core findings difficult to distill and evaluate.

Questions for authors

How sensitive are the observed negative inference-time task gains in PanoOcc to the specific small-scale training setup of the policy, and would alternative optimization strategies or architectures alter the conclusion that privileged task gain does not determine inference-time gain?

2 comments

We thank the reviewer for their careful reading and constructive feedback. Below, we address the concerns and questions raised, providing additional evidence and clarification of our findings.

W1

Our main claim is not that privileged allocation outperforms inference-time policies, as it can be predicted. Instead, we use privileged information to establish whether a beneficial correction is possible under the tested conditions, and then examine how much of this gain an inference-time policy achieves.

Our experiments go beyond observing this gap. In the original SparseOcc experiment, replacing the inference-time ranking with the privileged ranking increased the task gain from +0.0361 to +0.5480 pp. In an additional PanoOcc experiment, changing only the allocation ranking improved mIoU by 1.7396 pp, even though the first-order advantage decreased by 0.0134 while the finite-loss advantage increased by 0.0522.

These findings help answer our central question: repair performance depends on both the availability of useful corrections and how effectively a policy applies them. Our framework uses controlled comparisons to identify which part of the repair process deserves further investigation and to guide subsequent policy improvements.

W2 & Q

We clarify that the 70 development scenes refer to the original evaluation selection, not policy training. The allocator was trained on 2,048 records covering 553 GT-identified scenes. The new comparison uses 1,221 development records from 66 scenes and three seeds.

To test whether the result depends on training choices, we compared two ways of training the same policy. One learns from local gradient scores (Gradient-trained), while the other minimises the prediction loss after applying corrections (Loss-trained). We tested both using AdamW and SGD with momentum, with two scoring heads.

Optimizer / HeadGradient-trainedLoss-trainedGain
AdamW / Original−1.3059+0.4908+1.7967
SGD / Original−1.4071+0.4763+1.8835
AdamW / Residual−1.3219+0.4828+1.8046
SGD / Residual−1.3461+0.4857+1.8318

The original training method reduces mIoU in all four settings, while training on the loss after correction consistently improves it. Therefore, changing the optimizer or scoring head does not change our main finding: the gain achieved at inference time depends on how the policy is trained, even when the available correction remains the same.

We also extended the original Utility policy from three to six epochs for one seed, obtaining a change of +0.0227 pp (95% CI [−0.0089, +0.0554]). This provides an additional check on training duration.

Regarding evaluation scale, our comparisons use matched controls within each model. We report the sample counts and uncertainty separately for each setting rather than combining results across different models. We will make these details and the magnitude of the observed effects clearer in the revised manuscript.

W3

We report results within each model using matched controls rather than combining results across different settings. FlashOCC covers 150 validation scenes, while the new PanoOcc evaluation includes 1,221 records from 66 development scenes. The two external datasets contain 815 and 1,812 frames, respectively, each from one driving sequence. We account for this temporal dependence when estimating uncertainty.

We also report the size of the observed improvements. In PanoOcc, the revised policy improves mIoU by +0.5735 pp over Degraded (95% CI [0.3616, 0.8208]), recovering 16.74% of the matched degradation gap. In contrast, the negative inference-time effects in KITTI-360 are very small, approximately 0.001–0.003 pp. We use these results to study the separation between available repair gains and actual policy performance, rather than claiming substantial practical degradation.

We also corrected the FB-OCC task-metric evaluation. The aligned Uniform mIoU is 49.1146% on the original 190 records, rather than the previously reported 3.2039% obtained from mismatched coordinates.

W4

We will submit a revised manuscript during the rebuttal period to improve clarity and readability. Specifically, we will move the key experimental results from the appendix to the main text, including absolute mIoU, the PanoOcc policy comparisons, and external validation. We will also simplify the presentation of our three research questions, reduce unnecessary jargon and abbreviations, and clarify the main findings and their implications for repair design. Detailed experimental settings and additional results will remain in the appendix.

Dear Reviewer,

We have uploaded a revised manuscript and provided additional clarification on the statistical uncertainty and evaluation protocols in the main discussion.

As the discussion period is approaching its end, we would appreciate any further feedback on our rebuttal and revisions, particularly if any of your concerns remain unresolved.

Thank you for your time and consideration.

Reviewer_K4iYOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper studies repair under frozen camera-based 3D occupancy models using four nuScenes anchors and a controlled CARLA setting. It separates local corrective evidence, finite-objective improvement, privileged task gain, and inference-time task gain through matched exact-capacity interventions. Task gains are measured against capacity-matched Uniform. FB-OCC shows finite-loss improvement without an established positive mIoU advantage. At reference capacity, PanoOcc shows positive privileged gain but negative learned-policy gain, while entropy and low-confidence allocations yield positive gains. Capacity responses vary with allocation policy, and fixed-policy effects vary with annotation objective. The contribution is a controlled diagnostic protocol and empirical evidence that these measures of repair can diverge.

As regards the rebuttal, thank the authors for addressing my concerns, most of which have been addressed with meaningful additional experiments. The matched PanoOcc comparisons provide useful evidence that allocation ranking, rather than coefficient values alone, plays an important role in post-correction performance. The revised FB-OCC evaluation also resolves the previously reported uncertainty. The SemanticKITTI and KITTI-360 results further strengthen the paper, although the use of only one validation sequence per dataset limits the generality of the conclusions. I also encourage the authors to clarify whether the reported confidence intervals account for training variability and to ensure consistent evaluation across datasets following the FB-OCC correction. Overall, the rebuttal has improved my confidence in the findings. I am inclined to raise my rating from Weak Reject to Weak Accept.

Critical strengths
  1. The diagnostic framework makes limited recovery more informative. Separating local evidence, finite-loss improvement, and task gains helps distinguish stages that a single performance number obscures. This provides a useful way to evaluate repair and identify where further analysis is needed.
  2. The allocation comparisons fix intervention sites, units, directions, and capacity. Local derivative checks and direct finite-loss measurements, where retained, support the diagnostics. Table 7 specifies which contracts retain finite-loss results. The PanoOcc comparison is particularly informative: changing only the allocation policy can reverse the task effect. This strengthens the evidence for policy-dependent repair outcomes.
Critical weaknesses
  1. Q1 provides limited mechanistic explanation. FB-OCC's weighted task loss and reported mIoU use different aggregations, and the paper acknowledges this mismatch. The observed loss improvement without an established positive mIoU advantage therefore does not isolate a repair-specific mechanism. The confidence interval crossing zero also establishes neither harm nor equivalence to zero. The result is a useful diagnostic observation, but its deeper explanation remains open.
  2. External validity is limited to camera-based occupancy on nuScenes and a controlled CARLA setting. The annotation experiments change evaluation targets within nuScenes, rather than test an independent real-world dataset. Although acknowledged in Section 4, this scope limits how broadly the findings can guide repair design.
Questions for authors

Q1. What explains the learned allocation policy's negative effect? PanoOcc's entropy and low-confidence allocations yield positive gains at reference capacity. Appendix D.1 already reports a matched single-seed utility-supervised/Task-only pair on an auxiliary package, separate from the primary analysis. Is an equivalent comparison available under the primary matched protocol? If not, what can the existing evidence distinguish among supervision, optimization, and available information? A new primary-protocol comparison would be additional validation. Q2. Which conclusions are general properties of the diagnostic framework? The definitions in Section 2 are not occupancy-specific, but the observed separations are empirical. What assumptions are needed to apply the diagnostic to another task, and which conclusions should remain specific to the tested contracts? Cross-task experiments would extend this scope and are suggestions for future work. Cross-dataset validation is a request rather than a suggestion, since external validity bears on my assessment. If the same separations hold on an independent dataset under a matched protocol, I would consider raising my rating.

1 comment

We thank the reviewer for their careful reading and constructive feedback. Below, we address the concerns and questions raised, providing additional evidence and clarification of our findings.

W1

We further examined whether the observed separation is merely due to different loss and mIoU aggregations.

First, we identified a coordinate mismatch in the FB-OCC task-metric evaluator. Re-evaluating the same saved predictions with aligned ground truth gives 49.1146% mIoU for Uniform and 50.2392% for GT-guided allocation, an improvement of 1.1246 pp (95% CI [0.7240, 1.8523]). The evaluation uses the original 190 records from 38 scenes with the same semantic classes and valid mask. The native loss measurements remain unchanged. The corrected result replaces the previous FB-OCC example of finite-loss improvement without an established positive task gain. We will update the affected Q1 results, figures, and interpretation in the revised manuscript, including the evaluation scope.

More importantly, in an additional PanoOcc experiment, we kept the same coefficient values and changed only their allocation ranking. The first-order advantage decreased by 0.0134 (95% CI [−0.0227, −0.0041]), while the finite-loss advantage increased by 0.0522 (95% CI [0.0442, 0.0610]) and mIoU improved by 1.7396 pp (95% CI [1.3917, 2.0224]). This shows that local and finite evaluations can favour different allocations even under the same loss function.

Q1

Yes. We completed the requested comparison under the primary PanoOcc protocol. We changed the training objective from local utility supervision to minimising the loss of corrected predictions, while keeping the task model, correction direction, scoring head, training data, and update budget fixed.

We further tested the role of allocation ranking. Specifically, we replaced the original Utility policy's ranking with the revised policy's ranking while preserving the same coefficient values, eligible locations, correction direction, and total capacity. We separately tested changing the coefficient values while keeping the original ranking. The results are:

ChangeOriginal (pp)Modified (pp)Gain (pp)
Training objective−1.3059+0.4908+1.7967
Allocation ranking only−1.3059+0.4337+1.7396
Coefficient values only−1.3059−1.1882+0.1176

Changing the training objective improves mIoU by 1.7967 pp (95% CI [1.4440, 2.0881]). In the controlled comparisons, replacing the ranking improves mIoU by 1.7396 pp, compared with 0.1176 pp from replacing the coefficient values alone. These results identify how corrections are assigned as an important factor in the observed performance difference. The benefit of the revised training objective also remains positive with both tested optimizers and scoring heads.

W2 & Q2

We extended our evaluation to two independent real-world datasets, SemanticKITTI (815 frames) and KITTI-360 (1,812 frames), using matched interventions within each dataset. The following table shows the results:

DatasetCapacity (CC)PrivilegedEntropyGap
SemanticKITTILow (0.010)+0.909+0.010+0.899
SemanticKITTIHigh (0.020)+1.046+0.008+1.038
KITTI-360Low (0.00718)+0.430−0.001+0.431
KITTI-360High (0.02155)+0.497−0.003+0.500

Both datasets show positive privileged gains but substantially smaller inference-time gains at both tested capacities. The gap between Privileged and Entropy is larger at the higher capacity in both datasets. Each evaluation covers one validation sequence, with uncertainty assessed using paired temporal-block resampling.

More generally, our framework is not limited to occupancy. Applying it to other tasks requires a frozen model, matched correction directions and budgets, a differentiable loss with valid gradients, and a task-specific evaluation metric. Privileged information must also be separated from information available at inference time. The observed gains and their signs remain specific to each task and policy.

Reviewer_kdrcOpenReview ↗
Rating
3(+1)
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper studies why repair interventions provide limited gains after camera degradation in 3D occupancy models. It compares local loss improvement, privileged allocation gains, and inference-time allocation gains under matched intervention settings, using four nuScenes models and CARLA.

Critical strengths
  1. Holding intervention directions and prescribed strength fixed helps isolate the effect of allocation, making the comparisons more informative than final-performance comparisons alone.

  2. Component-replacement diagnostics, such as the PanoOcc vertical allocation and SparseOcc ranking experiments, provide concrete evidence about where particular policies lose performance.

Critical weaknesses
  1. The explanatory contribution is limited. Loss improvement need not improve mIoU, and privileged information need not translate into inference-time gains. The paper demonstrates these gaps but provides limited explanation of when they arise or how the diagnosis improves repair design.

  2. In PanoOcc, the learned allocation underperforms uniform repair while simple confidence-based policies remain beneficial. This makes it difficult to separate a general repair limitation from a limitation of the particular learned allocator.

  3. The connection to actual recovery is indirect. The main gains are measured against capacity-matched uniform repair rather than the unrepaired degraded model. Reporting these alongside clean, degraded, and uniformly repaired performance would clarify how much lost capability is actually recovered.

Questions for authors
  1. Can the identified bottlenecks guide a modified repair policy that produces measurable improvements?

  2. How robust is the PanoOcc reversal to stronger allocation policies or alternative training objectives?

  3. How do the reported allocation gains compare with the total degradation gap and the recovery already achieved by uniform repair?

4 comments

We thank the reviewer for their careful reading and constructive feedback. Below, we address the concerns and questions raised, providing additional evidence and clarification of our findings.

W1 & Q1

Yes. Our diagnosis not only identifies the gap but also helps to guide future policy design. In PanoOcc, the utility-supervised policy (Learned in the paper) yields a negative mIoU gain relative to matched Uniform, while Low-confidence achieves a positive gain under the same correction direction and budget. This points to how corrections are distributed as an actionable design factor. We therefore changed the allocator's training objective from predicting local gradient scores to minimising the occupancy loss of the corrected predictions. The table reports mIoU on the same 1,221-record development evaluation.

PolicymIoU (%)Δ vs. Degraded (pp)
Degraded56.9384—
Learned (Utility)55.7152−1.2232
Uniform57.0211+0.0827
Low-confidence (LC)57.2010+0.2626
Revised (finite-loss)57.5119+0.5735

Across three seeds, the revised policy turns the original performance degradation into a positive repair gain of 0.5735 pp over Degraded, while outperforming Learned by 1.7967 pp (95% CI [1.4440, 2.0881]).

W2 & Q2

The negative PanoOcc result is consistent across the tested optimizer and scoring-head configurations, but it is not an inherent limitation of inference-time repair. We tested two training objectives using AdamW and SGD with momentum, each combined with the original scoring head and a residual-bottleneck scoring head. All comparisons use three seeds and matched training budgets.

Optimizer / HeadUtility vs. Uniform (pp)Finite-loss vs. Uniform (pp)Finite-loss vs. Utility (pp)
AdamW / Original−1.3059+0.4908+1.7967
SGD / Original−1.4071+0.4763+1.8835
AdamW / Residual−1.3219+0.4828+1.8046
SGD / Residual−1.3461+0.4857+1.8318

The Utility policy has a negative mIoU difference relative to matched Uniform in all four tested configuration means, whereas the revised policy achieves positive gains in all four. Thus, changing the optimizer or scoring head preserves the observed contrast at the reference capacity, while changing the training objective reverses its sign.

W3 & Q3

Appendix B.4 (Table 17) already reports absolute mIoU for the original comparisons. To clarify how much degraded performance is actually recovered, we additionally evaluated a matched Clean reference for PanoOcc and calculated recovery relative to the unrepaired model. We will move the following summary to the revised main text.

ModelPolicyClean (%)Degraded (%)mIoU (%)Gain (pp)Recovered (%)
FlashOCCUniform32.058221.835921.8730+0.03700.36
FlashOCCInference-time32.058221.835921.9665+0.13061.28
PanoOccUniform60.363556.938457.0211+0.08272.41
PanoOccLearned60.363556.938455.7152−1.2232−35.71
PanoOccRevised60.363556.938457.5119+0.573516.74

For FlashOCC, inference-time repair recovers 1.28% of the degradation gap, compared with 0.36% from Uniform. For PanoOcc, the revised policy recovers 16.74%, compared with 2.41% from Uniform. This directly distinguishes actual performance recovery from improvements measured only against Uniform.

Dear Reviewer,

We have uploaded a revised manuscript and provided additional clarification on the statistical uncertainty and evaluation protocols in the main discussion.

As the discussion period is approaching its end, we would appreciate any further feedback on our rebuttal and revisions, particularly if any of your concerns remain unresolved.

Thank you for your time and consideration.

Thanks for the additional experiments and clarifications. The results show that the diagnosis can guide useful improvements, and the absolute performance results clarify how much performance is actually recovered. These address my main concerns, and I will raise my score.

Dear Reviewer,

Thank you very much for your reply and follow-up.

We are glad that the new results have helped clarify both the practical value of our diagnosis and the extent of performance recovery.

We sincerely appreciate your feedback and consideration throughout the discussion.

Comments on the paper

2 comments
AuthorsOpenReview ↗

Revised version of our submission

Summary of Revisions

We thank the reviewers for their constructive feedback. We have uploaded a revised manuscript that retains the original research questions and controlled intervention framework. The main changes are:

  1. Corrected the FB-OCC evaluation. We fixed a coordinate-alignment issue in the task-metric evaluation and updated the corresponding Q1 interpretation. The finite-loss results remain unchanged.
  2. Strengthened the evidence for Q1. We added a matched PanoOcc comparison showing that first-order and finite-loss evaluations can favor different allocation rankings.
  3. Evaluated training objectives and actual recovery. We compared Gradient-trained and Loss-trained policies. In the matched PanoOcc development evaluation, Loss-trained improves mIoU by 1.80 pp and recovers 16.74% of the measured Clean–Degraded gap. We also added optimizer and scoring-head comparisons.
  4. Extended the evaluation and clarified the protocols. We added frozen MonoScene evaluations on SemanticKITTI and KITTI-360, and clarified the evaluation scopes, statistical procedures, and limitations.
  5. Improved readability throughout. We reduced unnecessary nominalizations and dense noun phrases, simplified sentences, and clarified the presentation of the research questions, findings, and implications, particularly in the Abstract, Introduction, Discussion, and Conclusion.

We appreciate the reviewers' and Area Chair's consideration of these revisions.

AuthorsOpenReview ↗

Additional Clarification on Statistical Uncertainty and Evaluation

We would like to clarify two points regarding the revised experiments.

First, the reported confidence intervals are based on paired resampling of evaluation scenes or temporal blocks, conditional on fixed checkpoints. The PanoOcc comparisons include three training seeds, but the intervals do not fully capture variability from independent retraining.

Second, the FB-OCC correction addresses prediction–ground-truth coordinate alignment. Each matched comparison uses a consistent evaluation protocol and task metric within its own setting. Different datasets retain their respective native evaluation protocols, and we do not directly compare their absolute performance.

The external MonoScene evaluations cover one validation sequence per dataset, and our conclusions are limited to these settings.

We welcome any further questions or remaining concerns regarding the revised manuscript.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.