- Rating
- 3(+1)
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper investigates Shortcut Relocation in prior-aware debiasing for 3D large multimodal models (3D-LMMs). To address this issue, the authors introduce Corrective Function Allocation and propose Orthogonal Interaction Fine-Tuning (OIF). Experiments on Real-3DQA, viewpoint-rotation variants, and 3D-POPE, using Chat-Scene and 3D-LLaVA backbones, demonstrate improvements in several question-answering metrics and reduced answer-policy drift, although some robustness metrics exhibit limited improvements or degradation.
Critical strengths
- A meaningful conceptual perspective supported by a principled formulation. The paper identifies an important distinction between reducing language-prior dependence and ensuring that corrective adaptation relies on appropriate scene evidence. Corrective Function Allocation provides a principled framework for examining this distinction, and OIF offers a mathematically well-defined implementation through centered bilinear interactions.
- Comprehensive controlled comparisons and encouraging cross-backbone results. The experimental evaluation covers debiased question answering, viewpoint consistency, and object hallucination, with structural ablations, matched risk-selection controls, multiple training seeds, and efficiency measurements.
Critical weaknesses
- The central mechanism evidence does not sufficiently establish improved scene grounding. The zero scene-neutralized retention reported for OIF in Figure 4 follows directly from its multiplicative architecture and the intervention defined in Appendix C.3, which explicitly sets the centered scene summary to zero. Consequently, this result verifies a structural identity rather than independently demonstrating that the learned corrections rely on the intended 3D evidence. Furthermore, the orthogonality guarantee holds under an artificial product reference measure, not necessarily under the observed joint distribution or distribution shifts. The linguistic input also contains situation descriptions that may convey scene-specific information. Although the paper acknowledges some of these limitations, its central claim would benefit from more direct counterfactual evaluations that distinguish genuine scene grounding from alternative shortcut-compatible interactions.
- The empirical improvements are mixed, and the main baseline behavior raises concerns about the strength of the conclusions. On Chat-Scene, OIF improves Real-3DQA EM and EM-R, but overall VRS decreases from 11.90 to 11.57, while 3D-POPE accuracy improves only marginally from 72.01% to 72.19% and F1 decreases from 77.38% to 77.20% (Table 1). Similar F1 trade-offs occur on 3D-LLaVA (Table 4). Moreover, the central Shortcut Relocation diagnosis relies heavily on 3DR-FT, whose 3D-POPE affirmative rate reaches 97.78% and whose ground-truth likelihood shifts are uniformly negative in Appendix D.5. This extreme degradation warrants additional checks to determine whether the observed behavior represents a general debiasing failure mode or is particularly associated with this baseline.
Questions for authors
- Can the authors provide counterfactual scene-intervention experiments that do not explicitly force the OIF residual to zero, such as scene swapping, object-level perturbations, or spatial-relation changes? It would also be valuable to separate the information contributed by situation descriptions from that supplied by explicit 3D scene inputs. These results would help establish whether OIF improves dependence on the intended scene evidence.
- Can the authors further validate the 3DR-FT implementation and explain its extreme affirmative prediction bias and uniformly negative ground-truth likelihood shifts? Does Shortcut Relocation remain evident under alternative debiasing baselines or less severely degraded checkpoints?
Thank you for the detailed response and additional experiments. The newly introduced counterfactual scene-intervention experiments substantially address my primary concern regarding whether OIF improves dependence on explicit 3D evidence beyond its structural orthogonality guarantee. The additional results on MainEffect and Full-BM also alleviate my concern that Shortcut Relocation is merely an artifact of the severely degraded 3DR-FT baseline. Furthermore, the multi-seed and cross-backbone evaluations strengthen the empirical evidence for the proposed method.
Nevertheless, some concerns remain. The absolute spatial-relation counterfactual accuracy is still relatively low, and the contribution of situation descriptions has not been fully isolated. Moreover, the additional structural controls do not completely establish the generality of Shortcut Relocation across independent debiasing approaches, and some robustness trade-offs persist.
Overall, I believe the authors have addressed most of my major concerns. Considering the additional evidence and the paper's conceptual and technical contributions, I have decided to increase my rating from 2 (Weak Rejection) to 3 (Weak Acceptance).