acceptodds
Under review as a conference paper at ICLR 2027

GenTri: Generative Triangulation for Occlusion-Robust 3D Mouse Pose Estimation

Abstract

Markerless 3D mouse pose estimation enables scalable behavioral analysis in neuroscience, but occlusion remains a major challenge to reliable reconstruction across experimental environments. Limited annotated data make it difficult to handle diverse occlusion patterns through training alone. Applying pose estimation to hours-long behavioral recordings also requires efficient inference. We introduce GenTri, an inference-time framework combining multi-view geometry with a flow-matching pose prior. To reduce the influence of confidently incorrect detections under occlusion, GenTri replaces the reprojection objective with a redescending Cauchy objective. For efficiency, geometry-aware Gauss-Newton guidance determines each correction's direction and magnitude. This enables GenTri to match its first-order variant with (mouse) to (human) less guidance time. Across multiple 2D detectors and three mouse datasets spanning in-domain and out-of-domain environments, GenTri reduces reconstruction error under synthetic occlusion by 45% on average relative to the reconstruction method paired with each detector. The same guidance formulation extends to human pose benchmarks, demonstrating applicability beyond mouse pose estimation. See our Project page: [https://gentri-3dpose.github.io/](https://gentri-3dpose.github.io/).

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
3D pose estimation, multi-view geometry, flow matching, generative modeling, occlusion robustness, animal pose estimation

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_oHvgOpenReview ↗
Rating
3
Confidence
4
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes GenTri, an inference-time framework for occlusion-robust multi-view 3D pose estimation that combines a frozen, unconditional flow-matching pose prior with calibrated multi-view geometry. The method introduces an adaptive Cauchy reprojection objective to suppress unreliable 2D detections and a damped Gauss–Newton correction at the predicted clean endpoint to improve guidance efficiency without backpropagating through the flow network. An identity approximation of the endpoint Jacobian enables efficient per-joint optimization, while multiple generated hypotheses are selected according to their final robust reprojection energy. Experiments across three mouse domains, multiple 2D detection settings, and Human3.6M demonstrate substantial improvements under synthetic occlusion.

Critical strengths
  1. Clear technical motivation. The paper identifies two important limitations of existing observation-guided inference methods: sensitivity to incorrect 2D detections under L2 reprojection and inefficient first-order corrections that do not exploit the directional constraints of multi-view geometry.

  2. Well-controlled experimental comparisons. The authors adapt several existing guidance methods to the same pose prior and multi-view observation model, allowing a relatively fair assessment of the proposed guidance mechanism.

  3. Evaluation across multiple domains and detectors. Experiments cover three mouse domains, including out-of-domain recordings and a newly collected treadmill dataset, as well as transfer to human pose estimation without retuning inference hyperparameters.

Critical weaknesses
  1. Limited real-world occlusion validation. The occlusion benchmarks rely on synthetic occluders, and the paper does not establish whether the reported improvements generalize to naturally occurring occlusions in real experimental environments.

  2. Insufficient analysis of hypothesis selection. Although Figure 8 compares different hypothesis aggregation strategies, the paper does not quantify the gap between lowest-energy selection and oracle selection based on ground-truth 3D error.

  3. Dependence on prior quality. The method requires a separately trained species-specific flow prior, but its sensitivity to prior quality, training-data availability, and transfer to previously unseen species remains unexplored.

  4. Limited analysis of multi-view robustness. The paper does not systematically evaluate how performance changes with fewer available cameras or increasing numbers of simultaneously corrupted views, despite relying on cross-view consistency to reject incorrect observations.

  5. Insufficient detector diversity on Human3.6M. The human experiments rely exclusively on ResNet-152-based detectors, without evaluating whether the proposed guidance generalizes to other commonly used 2D pose estimators such as HRNet or CPN.

  6. Incomplete validation of the Jacobian approximation. Although Figure 5(b) demonstrates the efficiency advantages of avoiding the full endpoint Jacobian, the approximation-induced accuracy difference is not explicitly isolated under identical sampling configurations.

  7. Unexplained differences across species. The improvement over strong competing methods is considerably smaller on Human3.6M-Occluded than in several mouse settings, without sufficient analysis of whether this difference arises from the occlusion protocol, prior quality, or skeletal structure.

Questions for authors
  1. Real-world occlusion: Can the authors provide results on naturally occurring occlusions, or quantitatively compare the synthetic occlusion patterns with those observed in real animal experiments?

  2. Hypothesis selection: How reliable is the best-of-N selection strategy under severe occlusion, and what is the gap between the selected hypothesis and the oracle hypothesis with the lowest ground-truth 3D error?

  3. Prior quality: How does GenTri perform when the flow prior is trained using substantially less 3D data, or when the prior is transferred to a new species with limited annotations?

  4. Multi-view robustness: How does reconstruction accuracy change when the number of available camera views decreases or when multiple views contain confidently incorrect detections simultaneously?

  5. 2D detector generalization: Why is ResNet-152 exclusively adopted for Human3.6M, while DINOv2 is used for mouse experiments, and can the authors evaluate GenTri with HRNet or CPN to demonstrate robustness across different 2D detection error distributions?

  6. Jacobian approximation: Can the authors report the accuracy difference between the identity-Jacobian approximation and the full endpoint Jacobian using identical guidance steps and hypotheses, in addition to the existing accuracy–latency comparison?

  7. Cross-species performance: Why are the improvements over strong competing methods smaller on Human3.6M-Occluded than in several mouse experiments, and can the authors disentangle the effects of occlusion severity, skeletal structure, and prior quality?

6 comments
AuthorsOpenReview ↗

Response to Reviewer oHvg (Part 1/3)

Thank you for your constructive feedback and for recognizing the technical motivation and controlled comparisons. Your comments helped us broaden the evaluation and clarify the method's behavior, as detailed below.

[oHvg] asked whether the gains hold under naturally occurring occlusion.

Thank you for this question. To test GenTri under real occlusion, we recorded four additional open-field sessions with a physical occluder placed in front of one of the four cameras and annotated 100 frames (eight keypoints visible in the unobstructed views). Reconstruction uses the occluded view and two unobstructed views, with no retraining or retuning. As shown in Table R14, GenTri more than halves the error of triangulation, RANSAC and Anipose, is more accurate than multi-view EKS, and improves every paired detector baseline, so its robustness carries over from synthetic to real occlusion. We additionally compare GenTri, RANSAC and multi-view EKS under naturally occurring occlusions in a real cage environment, including a failure case (Figure 4(d) and Figure 7; videos on the anonymous project page).

Table R14. Real occlusion in open-field (OoD). * Mean excludes non-finite predictions or those with errors ≥1 m.

2D detector3D reconstructionTemporalMPJPE (mm)↓
DINOv2Triangulation✗14.7
DINOv2RANSAC✗15.0*
DINOv2Anipose✓12.9
DINOv2Multi-view EKS✓7.3*
DINOv2Ours✗6.1
LTLT✗14.7
LTOurs✗6.0
AdaFuseAdaFuse✗10.7
AdaFuseOurs✗6.2

Revision: Table 2 adds the real-occlusion column, Section 4.1 and Appendix B.1 describe the protocol, and Appendix A.1 adds the cage comparisons.

[oHvg] asked how far lowest-energy selection is from oracle selection, especially under severe occlusion.

Thank you for this question. To measure how reliable lowest-energy selection is, we compared it with an oracle that picks the lowest-error hypothesis among the same four candidates, on all frames and on the hardest ones (Table R15). The selected hypothesis stays within 0.2 mm of the oracle on mouse and 1.1 mm on human, and the gap grows only slightly on the hardest frames, far less than the differences between methods. The reconstruction energy thus identifies a near-best hypothesis without ground truth, even under severe occlusion.

Table R15. Lowest-energy vs. oracle selection. MPJPE (mm), N=4.

MouseHuman
Frame subsetSelectedOracleGapSelectedOracleGap
All frames10.210.00.230.729.61.1
Hardest 50%11.411.10.336.935.51.4
Hardest 25%12.912.60.343.942.01.9
Hardest 10%15.515.10.454.852.02.8

Revision: Figure 9 and Appendix B.2 (Table 18).

AuthorsOpenReview ↗

Response to Reviewer oHvg (Part 2/3)

[oHvg] asked how GenTri depends on prior quality, training-data size and transfer to a new species.

Thank you for this question. To test how GenTri depends on its prior, we ran three experiments that vary the amount of prior data, the species, and the quality of the training labels.

(i) Amount of training data. We trained priors on 50%, 25% and 10% of the training poses (Table R16). Even with 10%, the error rises by only 0.5 mm on mouse and 2.0 mm on human, so GenTri needs far less prior data than we used.

Table R16. Prior training size. Overall 3D error (mm) on Mouse-Occluded and Human3.6M-Occluded.

Training dataMouseHuman
100%10.230.7
50%10.430.7
25%10.430.9
10%10.732.7

(ii) Transfer to a new species. Because keypoint definitions differ across species, a prior is trained per species, and what transfers is the procedure for building it. On Rat7M [1], a prior learned from 106 poses pseudo-labelled by robust triangulation, without 3D GT, lowers the error from 15.2 to 13.3 mm, close to 12.8 mm with a motion-capture prior (Table R17). A useful prior for a new species can thus be built from a small number of unannotated recordings.

Table R17. Prior training on Rat7M [1]. Pseudo-labels are obtained by prior-free robust triangulation.

Prior training dataPosesMPJPE (mm)
Without prior–15.2
Pseudo-labels10613.3
3D GT10612.8

(iii) Quality of the training labels. We trained mouse priors on 99 pseudo-labels triangulated from occluded treadmill images, on 99 from clean images, and on 99 GT poses (Table R18). The occluded-image prior is within 0.2 mm of the clean-image prior on treadmill and nearly matches the others on open-field, so noisy labels from occluded images are sufficient.

Table R18. Prior training with occluded observations. MPJPE (mm) on Mouse-Occluded. †Not evaluated: these poses were used to train the GT prior.

Open-fieldOpen-fieldTreadmill
Prior training dataInDOoDOoD
3D GT, treadmill (99 poses)5.015.2–†
Pseudo-labels, clean images5.215.29.4
Pseudo-labels, occluded images5.415.39.6

Overall, GenTri is robust to both the amount and the quality of prior data, and its prior can be built for a new species without 3D annotation.

Revision: Section 4.3 (Table 6) and Appendix B.4 (Tables 22 and 24).

[oHvg] asked how accuracy changes with fewer cameras and when several views are confidently wrong.

Thank you for this question. To examine both cases, we evaluated GenTri and RANSAC on randomly reduced camera subsets, grouped joints by the number of correct views, and compared low-confidence with confidently wrong detections. GenTri is more accurate than RANSAC at every camera count, and the margin widens as cameras are removed: with two cameras it halves RANSAC's error on in-domain open-field and treadmill (Table R19). When several views are wrong at once, the gain is likewise largest where few correct views remain: with only 0 to 2 correct views, GenTri lowers RANSAC's error by a quarter to over a third (Table 12).

Table R19. Fewer cameras. Absolute MPJPE (mm) on Mouse-Occluded.

Open-field InDOpen-field OoDTreadmill OoD
CamerasRANSACOursRANSACOursRANSACOurs
226.513.3136.635.941.920.6
317.68.4266.954.023.012.0
48.86.537.636.214.09.5
57.25.724.223.4––
66.45.120.416.1––

Detector confidence alone cannot handle confidently wrong detections, which we define as wrong detections (2D error above 10 mm) whose confidence is at least the median confidence of correct ones. Confidence weighting leaves these joints at 47.2 mm, whereas GenTri's Cauchy weighting, which uses reprojection residuals rather than confidence, reduces them to 18.3 mm, and the whole-pose prior to 15.0 mm (Table R20). GenTri's robustness therefore comes from geometric consistency across views, not from the detector's confidence, which is why it holds when the detector is confidently wrong.

Table R20. Wrong detections by confidence. Mean 3D error (mm).

Wrong detectionsConfidence+ Cauchy+ Prior (Ours)
Low-confidence wrong10.05.65.3
Confidently wrong47.218.315.0

Revision: Appendix A.4 (Tables 9 and 10).

AuthorsOpenReview ↗

Response to Reviewer oHvg (Part 3/3)

[oHvg] asked why ResNet-152 is used for Human3.6M and whether the gains hold with other detectors such as HRNet.

Thank you for this suggestion. To show that GenTri's gains do not depend on one detector, we replaced ResNet-152 with HRNet-W48 on Human3.6M-Occluded, keeping GenTri's settings unchanged. HRNet's 2D errors are almost three times larger, which raises the error of every method, but GenTri remains the most accurate and its margin grows: it nearly halves RANSAC's error and improves on NRDF-MV by over a fifth (Table R21). GenTri therefore generalizes across detectors and their error distributions, and helps most when the detector is worse. For mouse, we use DINOv2, a pretrained foundation model, for its transferable features and strong out-of-domain detection with limited mouse training data.

Table R21. ResNet-152 vs. HRNet-W48 on Human3.6M-Occluded. Absolute MPJPE (mm) ↓.

2D detectorResNet-152HRNet-W48
Mean 2D error (px)9.526.1
2D error >50 px3.5%15.3%
Triangulation57.0354.7
RANSAC38.8101.2
NRDF-MV34.772.2
Ours30.756.4

Revision: Appendix A.5 (Table 11).

[oHvg] asked for the accuracy difference between the identity and full Jacobian at matched steps and hypotheses.

Thank you for this question. To isolate the effect of the approximation, we compared identity and full-Jacobian guidance at the same numbers of steps and hypotheses, and compared their update directions at every flow time. The identity approximation is more accurate than the full Jacobian at the deployed 10 steps and stays within 0.7 mm of it with more steps, while each step is 19 to 24× faster (Table R22). Its update is also a descent direction for the full-Jacobian objective at every flow time (positive cosine in 99.9% of states; Table R23). It therefore keeps the accuracy of the full Jacobian at a small fraction of its cost.

Table R22. Identity vs. full endpoint Jacobian at the same number of steps S. MPJPE (mm) on Mouse-Occluded and Human3.6M-Occluded; last row: guidance time per step. Deployed setting: S=10, N=4.

MouseHuman
Steps SIdentityFull JacobianIdentityFull Jacobian
511.013.834.453.0
1010.210.530.738.1
2510.19.729.531.3
5010.19.429.529.7
1009.99.630.029.3
ms / step8.6207.48.7162.9

Table R23. Gradient alignment of identity and full-Jacobian guidance (Mouse-Occluded).

tMean cosineCosine >0
[0.0, 0.2)0.4799.8%
[0.2, 0.4)0.43100.0%
[0.4, 0.6)0.40100.0%
[0.6, 0.8)0.4699.9%
[0.8, 1.0)0.69100.0%
All0.4999.9%

Revision: Section 4.3 (Table 5) and Appendix D (Table 27).

[oHvg] asked why the margins over strong baselines are smaller on Human3.6M-Occluded than on mouse.

Thank you for this question. To find where the difference comes from, we compared the quality of the 2D inputs across species, degraded the human input with a less accurate detector, and varied the prior's training data. Relative to body size, mouse detections are about four times less accurate than human ones (Table R24), and Mouse-Occluded occludes every view, whereas Human3.6M-Occluded [2] leaves one of four views unoccluded. Degrading the human input confirms this: with the less accurate HRNet detector, GenTri's gain over RANSAC grows from 21% to 44% (Table R21). Prior quality, by contrast, matters little, since shrinking the prior's training data changes the error only slightly (Table R16). The margins therefore differ mainly because the 2D inputs differ in quality: the worse the detections, the more robust reconstruction can recover, and the smaller human margins reflect cleaner human inputs.

Table R24. 2D input quality relative to body size. Mean back-projected 2D error divided by mean bone length.

BenchmarkMean 2D error / bone length
Mouse-Occluded0.68
Human3.6M-Occluded0.16

Revision: Appendix A.7 (Table 15).

References

[1] Timothy W. Dunn, Jesse D. Marshall, Kyle S. Severson, et al. Geometric deep learning enables 3D kinematic profiling across species and environments. Nature Methods, 18(5):564–573, 2021.

[2] Laura Bragagnolo, Matteo Terreran, Davide Allegro, and Stefano Ghidoni. Multi-view pose fusion for occlusion-aware 3D human pose estimation. In Computer Vision, ECCV 2024 Workshops, LNCS, pages 117–133. Springer, 2025.

Reviewer_sHtUOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
1
Presentation
1
Soundness
2
Read review
Paper summary

GenTri is an inference-time framework for combining multi-view geometry with a flow-matching pose prior to triangulate noisy 2D predictions from multiple views into a single robust 3D prediction. The pose prior is learned from clean observations, then used on noisy observations. The authors show, in both mouse and human datasets, that GenTri can successfully infer 3D pose even in the presence of strong occlusions.

Critical strengths

Results are good, especially in the low-guidance-time-per-frame regime (Fig 5).

I appreciate that the authors use a single set of hyperparameters for their method across species and detectors, rather than searching for the optimal set in each condition - this provides greater confidence that the method is robust.

Critical weaknesses

From what I can tell, this method appears to require clean 3D poses to train the prior of GenTri. So if I have a new dataset, how am I supposed to get the clean 3D poses in the first place? Isn't that the goal of GenTri? Unless there is a clear way for someone with a new (noisy) dataset to successfully use GenTri this seems to just be a toy approach that requires high quality benchmark datasets and artificial occlusions rather than a tool that will work generically on real-world data. For example, in L350, "Occlusion enters only at evaluation." In the real world, occlusion is present from the very beginning in the raw data, making it difficult to extract the clean 3D poses this method seems to depend on (but also asserts it provides?). If this is meant to be a method that works on human data, where clean benchmarks exist and the field more or less uses the same skeleton, this is fine. If this is meant to be a tool adopted by animal behavioral scientists, where benchmarks are sparse, each dataset has its own skeleton, and the data are noisy, I'm not sure how useful this method is.

I found this paper very difficult to understand. There is a lot of dense information without much effort at building intuition for the problem or the solution. Many sentences lack any kind of information whatsoever, perhaps due to unedited LLM outputs. For example, L351: "Single mouse entries are macro averages over the three domains." What is an entry? What is a macro average? What are the three domains? None of this is clear.

Questions for authors

Regarding Table 2, GenTri performs substantially better than the alternatives, especially in open-field OOD. What do the authors think is the difference between triangulation/Anipose vs RANSAC, which shows a large decrease in error? Where are the additional gains from GenTri coming from? What types of gains are these - are all keypoints slightly better, or is this the result of a smaller number of large outliers being fixed? What do these results look like per keypoint, are there obvious trends? The qualitative results in Fig. 6 point to the latter, but hard to tell from a small number of examples.

One of the strengths of GenTri is that it is robust to outliers, which might explain some of its performance gains in Tables 2+3. It also addresses the pose wholistically at each time step. It would be interesting to include a baseline like Multi-view EKS (Aharon et al, 2026; the Lightning Pose 3D paper should also be included Section 2) that is robust to outliers due to its use of ensemble variance as observation noise, but processes each keypoint independently rather than processing the entire skeleton. Such a comparison could partially disentangle performance improvements coming from the two features of GenTri.

Regarding my comment above about the generality of this approach to new datasets: It would be instructive to see if there is a way to get this to work, for example, if only given Mouse-Occluded, for forming the prior, training the detector, and evaluating GenTri. In other words, if I want to use GenTri on my own data, which does not have a "clean" version for prior training, how would I use it?

6 comments
AuthorsOpenReview ↗

Response to Reviewer sHtU (Part 1/2)

Thank you for your careful reading and feedback on the practical applicability and clarity of our work. Your comments helped us improve the presentation and add experiments clarifying the prior requirements and the sources of GenTri's gains, as detailed below.

[sHtU] asked how GenTri can be used on a new dataset that has no clean 3D poses to train the prior.

Thank you for this central question. A new dataset typically lacks two things that our original prior used: 3D ground-truth poses, and occlusion-free images from which clean poses could be obtained. To test whether GenTri still works without them, we replaced the 3D GT with pseudo-labels, that is, 3D poses triangulated from our 2D detections with RANSAC and filtered for geometric consistency (Appendix B.4), and removed the two ingredients one at a time.

(i) Without 3D GT. We trained priors on pseudo-labels from two other recordings, THM open-field and treadmill, and evaluated each on the recordings it was not trained on (Table R7). Both priors perform close to, or better than, the original GT-trained prior on the unseen recordings, so the prior does not need 3D annotation.

Table R7. Cross-recording pseudo-label priors. MPJPE (mm) on Mouse-Occluded. (source): recording the prior is trained on.

Open-fieldOpen-fieldTreadmill
Prior training dataInDOoDOoD
3D GT, open-field (990 poses)5.116.19.5
Pseudo-labels, open-field (OoD)4.4(source)10.2
Pseudo-labels, treadmill (OoD)5.215.2(source)

(ii) Without 3D GT and with occluded inputs. To mimic recordings in which occlusion is present from the start, we triangulated pseudo-labels from synthetically occluded treadmill images, without using their clean counterparts, and compared the resulting prior with priors trained on clean-image pseudo-labels and on 3D GT (Table R8). The occluded-image prior reaches 9.6 mm on treadmill against 9.4 mm for the clean-image prior and nearly matches the others on open-field, so pseudo-labels from occluded recordings are sufficient.

Table R8. Prior training with occluded observations. MPJPE (mm) on Mouse-Occluded. †Not evaluated: these poses were used to train the GT prior.

Open-fieldOpen-fieldTreadmill
Prior training dataInDOoDOoD
3D GT, treadmill (99 poses)5.015.2–†
Pseudo-labels, clean images5.215.29.4
Pseudo-labels, occluded images5.415.39.6

(iii) A new skeleton. Since animal datasets each define their own keypoints, we applied the same procedure to Rat7M [1], which uses a different rat skeleton. A prior trained on 106 pseudo-labelled poses lowers the error from 15.2 to 13.3 mm, close to 12.8 mm for a prior trained on 106 motion-capture poses (Table R9), so the procedure carries over to a new species.

Table R9. Prior training on Rat7M [1]. Pseudo-labels are obtained by prior-free robust triangulation.

Prior training dataPosesMPJPE (mm)
Without prior–15.2
Pseudo-labels10613.3
3D GT10612.8

Together, these experiments show that GenTri's prior needs neither 3D ground truth nor occlusion-free images. For a new dataset, one annotates visible 2D keypoints to train a detector (e.g., with DeepLabCut [2]), triangulates the detections robustly to obtain pseudo-labels, and trains the prior on them.

Revision: Section 4.3 (Table 6) and Appendix B.4 (Tables 23 and 24).

[sHtU] found the paper dense and hard to follow, and asked, for example, what an "entry," a "macro average" and the "three domains" in L351 mean.

Thank you for this feedback. To make the paper easier to follow, we rewrote ambiguous sentences and defined terms where they first appear, such as "clean" (no added synthetic occlusion), open-field versus treadmill recordings (free movement in an arena versus locomotion on a moving belt), and the per-species prior used with one shared set of inference settings. For the example in L351, an "entry" is the error in a table cell, the "three domains" are DANNCE open-field, THM open-field and treadmill, and a "macro average" weights the three equally regardless of their size; Section 4.1 now states this directly. To build intuition, the introduction now also explains why each component is needed and how they fit together, each step addressing the difficulty left by the previous one.

Revision: Introduction, Section 4.1, Section 4.2 and the Conclusion.

AuthorsOpenReview ↗

Response to Reviewer sHtU (Part 2/2)

[sHtU] asked what makes RANSAC so much more accurate than triangulation and Anipose in Table 2, and where GenTri's additional gains come from.

Thank you for this question. To locate where each method gains, we grouped joints by the number of views with correct detections (2D error of at most 10 mm; Table R10). With three or more correct views, RANSAC halves the error of triangulation and Anipose, and GenTri matches it; with only 0 to 2 correct views, GenTri lowers RANSAC's error by a quarter to over a third. RANSAC thus gains by rejecting views that disagree with the consensus, which works when enough views are correct, and GenTri gains further where they are not, because its whole-pose prior constrains joints whose views are mostly wrong.

Table R10. Mean 3D joint error (mm) by correct-view count. DANNCE open-field, THM open-field, and treadmill.

Correct viewsTriangulationAniposeRANSACOurs
0107.2183.940.430.7
161.539.530.320.9
228.218.917.810.8
≥310.29.95.04.8

Revision: Appendix A.6 (Table 12).

[sHtU] asked whether the gains come from all keypoints improving slightly or from a few large errors being fixed, and how they vary across keypoints.

Thank you for this question. To answer this, we measured how much of the improvement comes from large errors and broke the errors down by body part and keypoint (Table R11). RANSAC errs by 20 mm or more on about 15% of joint predictions, and these account for about 90% of GenTri's total improvement, as the reviewer observed in Figure 6. Every body part improves, the limbs most; at the keypoint level, the ankles and elbows improve most and the tail tip least. GenTri thus corrects the joints that occlusion corrupts rather than shifting every keypoint slightly.

Table R11. Mean 3D joint error (mm) by body part. Same evaluation as Table R10. Forelimb: paw, wrist, elbow, shoulder; hindlimb: paw, ankle, knee.

Body partTriangulationAniposeRANSACOurs
Head27.213.17.76.7
Spine15.413.78.57.5
Tail26.829.810.18.4
Forelimb16.310.910.87.9
Hindlimb23.429.614.09.6

Revision: Appendix A.6 (Table 13).

[sHtU] suggested comparing with multi-view EKS, which is robust to outliers but processes each keypoint independently, to separate the benefit of outlier robustness from that of modeling the whole skeleton.

Thank you for this suggestion. To separate the two factors, we ran two comparisons. First, we added multi-view EKS from LP3D [3], with three independently trained DINOv2 detectors, to the main comparison (Table R12). Without temporal smoothing, GenTri is comparable to EKS in-domain and more accurate on treadmill, and with simple smoothing it is the most accurate on both (EKS's treadmill mean covers only its finite predictions).

Table R12. Comparison with temporal baselines. MPJPE (mm) on Mouse-Occluded. * Mean excludes non-finite predictions or those with errors ≥1 m.

Open-fieldTreadmill
MethodTemporalInDOoD
Anipose✓5.213.2
Multi-view EKS (LP3D)✓4.79.9*
Ours✗5.19.5
Ours + smoothing (σ=20 ms)✓4.47.7

Second, because EKS also differs from GenTri in smoothing and in its detector ensemble, we isolated whole-skeleton modeling inside GenTri: with the same robust weighting, each joint is refined independently instead of through the prior (Table R13). Whole-pose updates lower the out-of-domain error on both datasets while the in-domain error stays within 0.1 mm, so modeling the whole skeleton helps beyond down-weighting outliers, most where observations are poor.

Table R13. Effect of the whole-pose prior. Absolute MPJPE (mm) on Mouse-Occluded.

Open-fieldOpen-fieldTreadmill
VariantInDOoDOoD
Per-joint refinement5.018.013.0
Ours (whole-pose prior)5.116.19.5

Revision: Table 2, Section 2.1 and Appendix A.6 (Table 14).

References

[1] Timothy W. Dunn, Jesse D. Marshall, Kyle S. Severson, et al. Geometric deep learning enables 3D kinematic profiling across species and environments. Nature Methods, 18(5):564–573, 2021.

[2] Tanmay Nath, Alexander Mathis, An Chi Chen, et al. Using DeepLabCut for 3D markerless pose estimation across species and behaviors. Nature Protocols, 14(7):2152–2176, 2019.

[3] Lenny Aharon, Keemin Lee, Karan Sikka, et al. An uncertainty-aware framework for data-efficient multi-view animal pose estimation. arXiv preprint arXiv:2510.09903, 2025.

AuthorsOpenReview ↗

A Kind Reminder for Comments

We sincerely thank Reviewer sHtU for the constructive feedback on practical applicability, clarity and the sources of GenTri's gains. Our responses and revised paper address these concerns through the following evidence and clarifications:

  • No clean 3D poses: Showed that the prior needs neither 3D ground truth nor occlusion-free images, including on Rat7M with a new skeleton, where a pseudo-label prior nearly matches a motion-capture prior (Section 4.3, Appendix B.4).
  • Clarity: Defined terms where they first appear, explained the terms in L351, and added to the introduction why each component is needed (Introduction, Section 4.1).
  • Where the gains come from: RANSAC gains by rejecting inconsistent views when enough views are correct, and GenTri lowers RANSAC's error by a quarter to over a third where only 0 to 2 views are correct (Appendix A.6, Table 12).
  • Large errors vs. uniform gains: Large RANSAC errors (about 15% of joints) account for about 90% of the improvement, with the limbs improving most (Appendix A.6, Table 13).
  • Comparison with multi-view EKS: Added EKS to the main comparison, where GenTri with simple smoothing is the most accurate, and isolated whole-skeleton modeling, which lowers out-of-domain error beyond outlier down-weighting (Table 2, Appendix A.6).

We appreciate the reviewer's time and would welcome any remaining questions about these revisions. Changes in the revised paper are highlighted in blue.

Reviewer_jcWVOpenReview ↗
Rating
2
Confidence
3
Informativeness
1
Presentation
3
Soundness
2
Read review
Paper summary

This is an inference-time method for multi-view 3D pose estimation under occlusion. It guides an unconditional flow-matching pose prior with the 2D keypoints detected in each calibrated view. At each sampling step, the pose predicted by the prior is corrected by one damped Gauss–Newton step per joint on a Cauchy-robust reprojection error, with the robustness scale set from the median residual; a separate Gauss–Newton step updates the global translation. The network Jacobian is approximated by the identity, so no backpropagation through the prior is needed, and the correction is carried back to the current sampling state. Four hypotheses are sampled, and the one with the lowest final energy is returned. Under synthetic occlusion, it improves on the reconstruction paired with every one of the three 2D detectors across all three mouse datasets. Without retuning, it also achieves the lowest error among the compared multi-view and prior-based methods on occluded Human3.6M.

Critical strengths
  1. Consistent, detector-agnostic robustness gains with one fixed configuration.It improves every detector–dataset combination under occlusion, with the largest gains out of domain. The same inference settings transfer to human pose, and clean accuracy stays within 0.4 mm of the baseline where it does not improve. Working on top of existing detectors without retraining is also valuable for behavioral neuroscience.

  2. Careful efficiency analysis is also a strong point. Guidance rules are compared on the same prior, detections and translation solver, with DPS and CLAMP also given the same Cauchy objective. Ablations separate the Gauss–Newton step from first-order steps and the identity approximation from partial and full Jacobians. Together these make the accuracy-per-time claim credible.

Critical weaknesses
  1. I think the technical novelty is limited. The method largely combines known components: Cauchy-robust Levenberg–Marquardt steps from bundle adjustment, the identity-Jacobian approximation from QGF, and transport from MPGD. The paper offers no analysis of when the identity approximation holds. It also does not examine how the median-based scale behaves when most residuals are outliers, which is exactly the heavy-occlusion regime it targets.

  2. Also the evaluation is somewhat narrow. All occlusions are synthetic, with object occluders placed around the joints. The mouse test sets are small (200, 99 and 99 frames). And the prior is trained on 990 poses from the same recording as the in-domain test set. The headline 45% gain comes mostly from baseline failures on THM; against RANSAC with the same detections, the gain is about 25%. Despite the motivation of hours-long recordings, the method is per-frame, has no temporal baseline beyond Anipose, and needs about 0.09 s per frame for guidance.

Questions for authors
  1. Table 4(a) shows robustness on frames with high initial reprojection error, but how many detections per frame are actually wrong on Mouse-Occluded? How does accuracy hold up as this fraction approaches half, where the median-based Cauchy scale would no longer separate inliers from outliers?

  2. Have you tested on real occlusion, even a small annotated set with actual cage bars or bedding? The current occlusions are synthetic, with object occluders placed around the joints.

  3. Since the prior is trained on 990 poses from a single recording, how does accuracy change for poses far from that training set, such as those in treadmill locomotion?

5 comments
AuthorsOpenReview ↗

Response to Reviewer jcWV (Part 1/3)

Thank you for your detailed feedback and for recognizing the consistent robustness gains and efficiency analysis. Your comments helped us clarify the technical rationale, examine the method's behavior under challenging observations, and broaden the evaluation, as detailed below.

[jcWV] noted that GenTri largely combines known components: robust Levenberg–Marquardt steps from bundle adjustment, the identity Jacobian of QGF, and the transport of MPGD.

Thank you for this point. Our contribution is the chain that links these components to solve one problem: reconstructing 3D pose from multi-view detections, many of which are wrong under occlusion, within a few sampling steps. Each link fixes the difficulty that the previous one leaves, where the existing method stops.

(i) Robust weighting. Wrong detections are the first difficulty: L2 updates, as in DPS and CLAMP, are dominated by them. GenTri down-weights them with Cauchy weights and a median-based scale (Table 4(b)).

(ii) Geometry-aware step. The remaining correct views constrain some directions far more than others, so QGF's first-order step, with one step size for all directions, needs many steps. GenTri takes a per-joint damped Gauss–Newton step, reaching the same accuracy with 2× and 17× less guidance time on mouse and human (Figure 5(b)).

(iii) Prior at every step. A per-joint step cannot recover joints with too few correct views, which is where per-joint robust refinement, as in bundle adjustment, stops. GenTri lets the prior re-predict the whole pose at every step, lowering the out-of-domain error from 18.0 to 16.1 mm and from 13.0 to 9.5 mm (Table 14).

(iv) Endpoint correction. Using the prior at every step requires passing each correction back to the flow state, and differentiating through the prior, as in CLAMP, is costly. GenTri applies the Gauss–Newton correction to the predicted endpoint and transports it to the flow state without backpropagation, extending MPGD's transport from a first-order step, and improves on CLAMP and DPS under the same Cauchy objective (Figure 5(a)).

Each link is validated by its own ablation, and removing any one of them costs accuracy or time; the chain as a whole, rather than any single component, is our contribution.

Revision: Section 4.3 and Appendix A.6 (Table 14).

[jcWV] noted that the paper offers no analysis of when the identity approximation holds.

Thank you for this question. To examine when the identity approximation holds, we compared its update direction with that of the full Jacobian at every flow time, and its accuracy and cost at the same numbers of steps and hypotheses. Its update is a descent direction for the full-Jacobian objective at every flow time (positive cosine in 99.9% of states; Table R1). It is more accurate than the full Jacobian at the deployed 10 steps and stays within 0.7 mm of it with more steps, while each step is 19 to 24× faster (Table R2). The identity approximation therefore holds throughout sampling and keeps the accuracy of the full Jacobian at a small fraction of its cost.

Table R1. Gradient alignment of identity and full-Jacobian guidance (Mouse-Occluded).

tMean cosineCosine >0
[0.0, 0.2)0.4799.8%
[0.2, 0.4)0.43100.0%
[0.4, 0.6)0.40100.0%
[0.6, 0.8)0.4699.9%
[0.8, 1.0)0.69100.0%
All0.4999.9%

Table R2. Identity vs. full endpoint Jacobian at the same number of steps S. MPJPE (mm) on Mouse-Occluded and Human3.6M-Occluded; last row: guidance time per step. Deployed setting: S=10, N=4.

MouseHuman
Steps SIdentityFull JacobianIdentityFull Jacobian
511.013.834.453.0
1010.210.530.738.1
2510.19.729.531.3
5010.19.429.529.7
1009.99.630.029.3
ms / step8.6207.48.7162.9

Revision: Section 4.3 (Table 5) and Appendix D (Table 27).

AuthorsOpenReview ↗

Response to Reviewer jcWV (Part 2/3)

[jcWV] asked how many detections per frame are wrong on Mouse-Occluded and how accuracy holds up as this fraction approaches half.

Thank you for this question. To examine this regime, we counted the wrong detections in each frame (2D error above 10 mm), compared accuracy across wrong-detection fractions, and checked which wrong detections the median-based scale admits. A quarter of Mouse-Occluded frames, and over a third of the hardest ones, have at least half of their detections wrong. Accuracy degrades only gradually as the wrong fraction passes one half, and GenTri stays below RANSAC in every bin (Figure 10). In these frames the scale admits more wrong detections, but mostly the small ones (Table R3), so the large errors are still down-weighted when most detections are wrong.

Table R3. Wrong detections under median-based scaling. Accepted: final reprojection residual ≤ Cₜ. Errors are mean back-projected 2D errors.

Wrong detectionsMean 2D error (mm)
Frames# framesacceptedAccepted wrongRejected wrong
<50% wrong30012%15.129.8
≥50% wrong9831%22.455.1

Revision: Appendix B.3 (Figure 10).

[jcWV] noted that all occlusions are synthetic and asked whether GenTri was tested on real occlusion.

Thank you for this question. To test GenTri under real occlusion, we recorded four additional open-field sessions with a physical occluder placed in front of one of the four cameras and annotated 100 frames (eight keypoints visible in the unobstructed views). Reconstruction uses the occluded view and two unobstructed views, with no retraining or retuning. As shown in Table R4, GenTri more than halves the error of triangulation, RANSAC and Anipose, is more accurate than multi-view EKS, and improves every paired detector baseline, so its robustness carries over from synthetic to real occlusion. We additionally compare GenTri, RANSAC and multi-view EKS under naturally occurring occlusions in a real cage environment, including a failure case (Figure 4(d) and Figure 7; videos on the anonymous project page).

Table R4. Real occlusion in open-field (OoD). * Mean excludes non-finite predictions or those with errors ≥1 m.

2D detector3D reconstructionTemporalMPJPE (mm)↓
DINOv2Triangulation✗14.7
DINOv2RANSAC✗15.0*
DINOv2Anipose✓12.9
DINOv2Multi-view EKS✓7.3*
DINOv2Ours✗6.1
LTLT✗14.7
LTOurs✗6.0
AdaFuseAdaFuse✗10.7
AdaFuseOurs✗6.2

Revision: Table 2 adds the real-occlusion column, Section 4.1 and Appendix B.1 describe the protocol, and Appendix A.1 adds the cage comparisons.

[jcWV] noted that the headline 45% gain comes mostly from baseline failures on THM, and is about 25% against RANSAC.

Thank you for this point. To make the scope of each number clear, we toned down the headline: the abstract now reports the reduction against RANSAC with identical detections (about 25%), and the introduction defines the 45% as the average reduction against each detector's own reconstruction baseline. To locate where the gain arises, we grouped joints by the number of views with correct detections (Table 12): GenTri matches RANSAC where three or more views are correct and lowers its error by a quarter to over a third where only 0 to 2 are. The gain therefore comes from the joints that occlusion corrupts.

Revision: Abstract, Introduction, Section 4.2 and Appendix A.6.

AuthorsOpenReview ↗

Response to Reviewer jcWV (Part 3/3)

[jcWV] noted that the method is per-frame, has no temporal baseline beyond Anipose, and needs about 0.09 s per frame.

Thank you for this point. To evaluate GenTri on long recordings, we compared it with temporal baselines, Anipose and multi-view EKS from Lightning Pose 3D (LP3D) [1], and measured sequence-level processing time. Because GenTri reconstructs each frame independently, any temporal smoother can be added on top of it, so for a fair comparison we also report GenTri with simple Gaussian smoothing. As shown in Table R5, GenTri without smoothing is comparable to EKS in-domain and more accurate on treadmill, and with smoothing it is the most accurate on both datasets (EKS diverges on part of the treadmill sequence, so its mean covers only its finite predictions).

Table R5. Temporal baselines: accuracy and time. MPJPE (mm) on Mouse-Occluded. †CPU; other timings use GPU. * Mean excludes non-finite predictions or those with errors ≥1 m.

TimeOpen-fieldTreadmill
MethodTemporal(ms/frame)InDOoD
Anipose✓23.4†5.213.2
Multi-view EKS (LP3D)✓192.5†4.79.9*
Ours✗2.95.19.5
Ours + smoothing (σ=10 ms)✓2.94.57.9
Ours + smoothing (σ=20 ms)✓2.94.47.7
Ours + smoothing (σ=40 ms)✓2.94.37.9

The 0.09 s/frame in the paper is the cost of one frame processed alone. Because frames are independent, a whole sequence is processed in parallel GPU batches at 2.9 ms per frame, excluding 2D detection, which makes hours-long recordings practical. Anipose and EKS are timed on CPU, since Anipose's temporal optimizer runs only on CPU and EKS ran faster on CPU than on GPU.

Revision: Table 2 and Appendix D (Table 28).

[jcWV] noted that the prior is trained on 990 poses from the same recording as the in-domain test set, and asked how accuracy changes for poses far from them, such as treadmill locomotion.

Thank you for these points. To address both, we checked the data split, evaluated the original prior on treadmill, and trained priors on other recordings. We would kindly refer the reviewer to Appendix B.1: the in-domain test frames come from a later, non-overlapping part of the DANNCE sequence and are excluded from prior training. Treadmill locomotion is far from these open-field training poses, and with the original prior GenTri already improves on every baseline there (Table 2). To test generalization further, we trained priors on two other recordings, THM open-field and treadmill, using RANSAC pseudo-labels instead of 3D GT, and evaluated each on the recordings it was not trained on (Table R6). Each prior performs close to the original GT-trained prior in both directions, and the THM prior is even slightly more accurate in-domain, so the prior generalizes across recordings and behaviors rather than fitting the test recording.

Table R6. Cross-recording priors. Absolute MPJPE (mm) on Mouse-Occluded with DINOv2 detections. (source): recording the prior is trained on. InD/OoD follow the original setup.

Open-fieldOpen-fieldTreadmill
Prior training dataInDOoDOoD
Open-field (InD), GT, 990 poses5.116.19.5
Open-field (OoD, THM), pseudo4.4(source)10.2
Treadmill (OoD), pseudo5.215.2(source)

Revision: Appendix B.4 (Table 23).

References

[1] Lenny Aharon, Keemin Lee, Karan Sikka, et al. An uncertainty-aware framework for data-efficient multi-view animal pose estimation. arXiv preprint arXiv:2510.09903, 2025.

Comments on the paper

3 comments
AuthorsOpenReview ↗

General Response to All Reviewers

We thank all reviewers for their careful reading and constructive feedback. The reviewers found that GenTri improves every detector and dataset with one fixed configuration (jcWV), which gives confidence that the method is robust (sHtU); that the guidance rules are compared fairly on a shared prior and observation model (jcWV, oHvg), making the accuracy-per-time claim credible (jcWV) with strong results at low guidance time (sHtU); and that the technical motivation is clear (oHvg).

Below, we summarize the main concerns and the corresponding revisions, followed by detailed responses to each reviewer.

  1. Evaluation under real occlusion (jcWV, oHvg). We show that GenTri's robustness carries over to real occlusion: on 100 newly annotated frames with physical occluders, it improves every detector baseline without retraining (Table 2). We also add qualitative comparisons in a real cage environment (Appendix A.1, Figure 7), with videos on our anonymous project page.

  2. Technical rationale and component analysis (jcWV, sHtU, oHvg). We explain how the components form one chain, each addressing the difficulty left by the previous one, and validate each link with analyses of identity-Jacobian guidance (Section 4.3; Appendix D), median-based scaling (Appendix B.3), hypothesis selection (Appendix B.2), and whole-pose updates versus robust per-joint refinement (Appendix A.6).

  3. Prior requirements with limited or noisy data (jcWV, sHtU, oHvg). We show that the prior needs neither 3D ground truth nor occlusion-free images: priors trained on pseudo-labels, including those triangulated from occluded recordings, perform close to the GT-trained prior (Section 4.3, Table 6); a tenth of the training data changes the error only slightly; and a rat prior can be built in the same way (Appendix B.4).

  4. Additional baselines and robustness evaluations (jcWV, sHtU, oHvg). We show that GenTri remains robust against stronger baselines and in harder settings: with simple smoothing it outperforms multi-view EKS (Table 2), it processes long sequences at 2.9 ms per frame (Appendix D), it stays ahead of RANSAC with fewer cameras (Appendix A.4), and it generalizes to another human 2D detector (Appendix A.5).

  5. Clearer presentation and scope of claims (jcWV, sHtU). To make the paper easier to follow, we defined the evaluation protocol and dataset aggregation (Section 4.1), explained in the introduction why each component is needed, and revised the headline numbers in the Abstract and Introduction to name the comparison behind each.

We hope that these responses and revisions address the reviewers' concerns. The revised PDF marks all changes in blue.

AuthorsOpenReview ↗

A Kind Reminder for Comments

Dear Area Chair and Reviewers,

We sincerely thank the reviewers for their constructive feedback and careful evaluation of our work. We have provided detailed point-by-point responses and updated the manuscript with additional experiments and clarifications addressing the reviewers' concerns, including evaluation under real occlusion, priors built without 3D ground truth, and comparisons with multi-view EKS. The key changes are summarized in our individual reviewer responses, and all revisions in the manuscript are highlighted in blue.

We would greatly appreciate the Area Chair's help in inviting reviewers to engage in further discussion and consider our responses. We welcome any remaining questions and would be happy to provide further clarification.

Thank you again for your time and consideration.

Area_Chair_B7vHOpenReview ↗

Please review the authors' responses and engage in the discussion

Dear Reviewers,

The authors have now responded to your concerns. It is time to review their responses and assess whether they adequately address your concerns or whether any further questions remain.

I would kindly ask you to read the authors' responses and actively engage in the discussion, as there are only a few days remaining.

Thank you for your time and contribution.

Best regards, Your AC

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.