- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper proposes SABLE, a framework for learning interpretable 3D animal behavior representations from sparsely overlapping camera views. Built upon E-RayZer, SABLE incorporates monocular depth estimates and pose-based geometric priors to prevent degenerate 3D reconstruction without ground-truth 3D annotations. The learned representations are evaluated on the IBL and Cheese3D datasets, demonstrating improved sparse-view reconstruction and downstream neural encoding and decoding performance. The method also demonstrates generalization to held-out animals without additional model fine-tuning.
Critical strengths
- Practical significance of sparse-view 3D reconstruction. The paper addresses an important practical challenge in animal behavior analysis, where only two camera views with limited overlap are available. The proposed framework extends 3D representation learning to realistic neuroscience settings where dense multi-camera configurations are impractical.
- Effective geometric regularization. The integration of monocular depth and pose-based geometric priors provides a reasonable approach to preventing degenerate reconstruction. The qualitative comparisons with VGGT and E-RayZer demonstrate more coherent 3D structure, while the geometric loss ablation supports the effectiveness of the proposed regularization.
- Meaningful downstream evaluation. The paper goes beyond 3D reconstruction by evaluating learned representations on neural encoding and decoding tasks. The experiments on IBL and Cheese3D demonstrate potential benefits for neuroscience applications, and the evaluation on held-out animals provides encouraging evidence of generalization.
Critical weaknesses
- Limited methodological novelty. The proposed framework largely builds upon existing components, including E-RayZer, DINOv3, Video Depth Anything, and pose estimation models. The main modifications involve patch-level masking and geometric regularization using pseudo-point clouds. Although effective, the technical contribution appears relatively incremental, and the paper does not sufficiently isolate the contributions of the individual components.
- Insufficient quantitative evaluation of 3D reconstruction. The 3D reconstruction results are primarily supported by qualitative visualizations. While the Cheese3D novel-view synthesis experiment provides indirect quantitative evidence, its absolute performance remains limited (12.74 PSNR and 0.362 SSIM on the held-out view). More direct geometric evaluation is needed to establish the accuracy and interpretability of the reconstructed 3D representations.
- Limited evidence that 3D representations improve neural encoding. The neural encoding improvements are modest. On IBL, SABLE achieves 0.168 BPS compared with 0.165 for BEAST, without a clear statistical significance analysis across independent animals or sessions. Furthermore, the neural encoding experiments use DINOv3 [CLS] features, making it unclear whether the improvements actually result from explicit 3D geometric modeling rather than stronger visual features.
Questions for authors
- What is the individual contribution of each proposed component? Could the authors provide controlled ablations comparing the original E-RayZer, E-RayZer with patch-level masking, and the full SABLE model with geometric regularization? In particular, how much improvement comes from the geometric prior versus the architectural modifications?
- Can the authors quantitatively validate the accuracy of the reconstructed 3D geometry? Given the calibrated multi-view recordings available in Cheese3D, could the authors evaluate geometric consistency using triangulated 3D keypoints, camera pose errors, or additional held-out camera views? This would provide stronger evidence of genuine 3D reconstruction accuracy.
- Are the improvements in neural encoding statistically significant, and are they attributable to 3D representations? Could the authors report session-level significance tests and provide a comparison between DINOv3-only embeddings, SABLE without geometric regularization, and full SABLE on neural encoding and decoding tasks? This would help establish whether explicit 3D modeling contributes meaningfully to the downstream performance.
2 comments
W2. Quantitative evaluation of 3D reconstruction quality
We evaluate novel view reconstruction on the Cheese3D dataset to quantify 3D reconstruction quality. SABLE recovers meaningful 3D animal shape from just two input views (Appendix Figure 6C) and produces clear held-out-view reconstructions when provided with fixed camera calibration during training (Appendix Figure 6B). To measure the role of camera pose estimation, we compared the original two-view supervision model with a variant (“additional calibration supervision”) given fixed top-center camera calibration during training. Both variants receive only top-left and top-right image tokens; top-center image tokens are never seen by the model. Both are evaluated on held-out top-center frames.
With the same two input views, additional calibration improves PSNR by 6.43 and SSIM by 0.281, resulting in a clear reconstruction of the held-out view. These results suggest that the original modest scores partly reflect difficulties in camera-pose estimation, and SABLE is able to recover meaningful 3D structure.
| Model / Variant | Frames | PSNR Mean | PSNR SE | SSIM Mean | SSIM SE |
|---|---|---|---|---|---|
| Two view supervision | 520 | 12.093 | 0.031 | 0.3821 | 0.0010 |
| Additional Calibration Supervision | 520 | 18.519 | 0.032 | 0.6633 | 0.0008 |
W3. Encoding Improvement from SABLE
We conducted a new patch-embedding-based encoding analysis comparing full SABLE, SABLE without geometric loss, and DINOv3-only embeddings using the same encoding pipeline. SABLE outperforms both alternatives, supporting the contribution of 3D geometric learning to improving neural prediction (see our response to W1).
In the table below, we report results separately for the 3 held-out sessions, showing consistent improvements in neural prediction across sessions when the representations encode 3D behavioral information in addition to 2D semantic features. Figure 4C has also been updated to show the per-neuron improvements across the 3 held-out sessions.
| Held-Out Session | DINOv3 | SABLE |
|---|---|---|
| 1 | 0.14 ± 0.02 | 0.17 ± 0.02 |
| 2 | 0.19 ± 0.02 | 0.23 ± 0.02 |
| 3 | 0.12 ± 0.02 | 0.14 ± 0.02 |
We thank the reviewer for recognizing the problem’s importance, the effectiveness of 3D geometric regularization, and SABLE’s meaningful downstream evaluation and generalization ability. We address the concerns through additional experiments detailed below. Revisions are highlighted in blue in the updated paper.
Summary: Our revisions clarify two novel contributions:
Please see updated Sec. 5.3 and Fig. 4 for details.
W1. Novelty and component ablations
Novelty: We argue that novelty should be assessed by the new functionality enabled and the challenges resolved, rather than merely the components introduced. SABLE’s central insight is that explicit 3D geometric supervision can prevent 3D reconstruction collapse under extremely sparse-view conditions. Without it, the reconstructed point clouds collapse to 2D surfaces (Figure 1). This conceptually simple but consequential insight enables SABLE to recover meaningful 3D animal structure in a setting where all existing methods fail.
Ablation: We agree that the contributions of individual components should be better isolated. We therefore added comparisons of 3 feature representations using the same neural encoding pipeline:
Our original encoding analysis used DINOv3 [CLS] tokens projected into the latent space of SABLE’s multi-view transformer. These tokens capture frame-level 2D semantic information, whereas SABLE reconstructs 3D Gaussian point clouds from the multi-view transformer’s patch embeddings. Thus, the original analysis does not fully capture the contribution of the 3D information encoded in the patch embeddings. We therefore evaluated neural encoding using these patch embeddings.
Encoding results: Full SABLE outperforms both the variant without geometric loss and DINOv3 alone. The former comparison supports the contribution of geometric supervision to neural prediction; the latter shows that visual pretraining alone does not fully explain SABLE’s performance. Together, these results support using 3D behavioral information in addition to 2D behavioral semantics for neural encoding. See details in revised Section 5.3 and Figure 4.