acceptodds
Withdrawn from ICLR 2027

Scale from Priors, Location from Pixels: The Limits of Monocular Pitch Reconstruction

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Professional baseball measures pitch trajectories with multi-camera tracking systems. Most games have only one broadcast camera behind the pitcher. We ask how much of a pitch's 3D trajectory this single view can recover, and where the accuracy of a working system comes from. We build a leakage-audited benchmark of 12,140 MLB pitches from OpenCommand, with solved camera poses and Statcast-derived reference trajectories. For the constant-acceleration trajectory models used in practice, metric depth has an exact scale ambiguity from a fixed camera. Gravity does not resolve it, because pitched baseballs are not gravity-only projectiles. For these models, absolute scale must come from prior information. Anchoring depth at the release and plate planes brings median 3D error from tens of meters to about one meter and plate-crossing error to 7 cm. We then measure what the observed ball track adds once calibration, temporal support, and priors are held fixed. A learned population prior with no pixel coordinates already beats the anchored physics fit in depth. In matched ablations tested once in each of the 31 MLB venues, adding the track cuts plate error from 30 to 5.4 cm and speed error from 3.6 to 1.9 mph. Depth error falls from 71 to 47 cm. Every venue improves on all three. These figures use the camera calibration supplied with the dataset. For the anchored physics fit, a reference-free audit with calibration and cohort built from inputs alone gives 8.7 cm median plate error. Priors set the scale, and the pixels set the location.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
Monocular 3D Reconstruction, Observability, Baseball Pitch Tracking

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_qdPNOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
3
Read review
Paper summary

This paper studies the limits of reconstructing 3D baseball pitch trajectories from a single broadcast camera and introduces a benchmark of 12,140 MLB pitches from OpenCommand. Experiments show that gravity-only fitting mismatches baseball dynamics, causing large depth errors. The authors argue that priors must supply metric scale and show that depth anchors substantially improve reconstruction. Once scale is established, ablations show that tracks primarily improve plate-crossing localization while refining speed and depth.

Critical strengths

Benchmark construction. The paper builds a 12,140-pitch MLB benchmark from OpenCommand with defined evaluation protocols and explicit audits of reference dependencies in calibration, alignment, and sample selection. Analysis of prior information. The paper examines how geometric and learned priors support reconstruction. Depth-anchor experiments and matched ablations clarify how priors establish metric scale and what additional information observed tracks provide.

Critical weaknesses
  1. Unclear added value of the benchmark. The paper does not sufficiently justify the significance of introducing an MLB-specific monocular pitch trajectory reconstruction benchmark. Monocular trajectory reconstruction is a classical computer vision problem, and neither metric-scale ambiguity nor the limitations of gravity-only dynamics are unique to baseball. The paper does not clarify what distinct challenges or insights this benchmark offers beyond existing sports trajectory reconstruction benchmarks.
  2. Limited novelty in the discussion of gravity-based scale recovery. The limitations of fixed-gravity models in capturing real ball dynamics have already been discussed in [1]. The paper should clarify what additional insight its analysis provides beyond this observation.
  3. Limited novelty of the geometric-prior analysis. Geometric priors are widely used to improve monocular reconstruction. The paper should clarify what new insight its depth-anchor experiments provide beyond this expected benefit.

[1] Grad L, Czajkowski K M, Varashylau A. Physics-Based 3D Ball Trajectory Reconstruction from Monocular Soccer Video: A Multi-Model Benchmark[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2026: 9940-9948.

Questions for authors

NA

1 comment

We have uploaded a revised PDF with changes highlighted in blue. It adds the viewing-geometry numbers that motivate an MLB-specific benchmark (§3.1), an expanded comparison with Grad et al. (§2), and the endpoint decomposition that goes beyond "geometric priors help" (§6.2 and Table 3), and we would appreciate it if you could take a look.

Reviewer_uJz2OpenReview ↗
Rating
1
Confidence
2
Informativeness
1
Presentation
1
Soundness
1
Read review
Paper summary

The paper addresses the task of 3D trajectory recovery of the ball for baseball pitches in a monocular setup. The authors compare several fitting methods to retrieve the 3D trajectory of the ball from the image detection of the ball, timestamps, and camera parameters. They establish that leveraging the distance between the release plane and the plate plane combined with a free acceleration parameter gives the best accuracy. Then the paper investigates how learned estimators benefit from different modalities (2D track of the ball, camera parameters, duration of the throw, camera parameters) to retrieve the actual trajectory parameters (position, speed, and acceleration). Their results suggest that the physic fit of the ball trajectory performs worse than learned variants.

The paper also proposes a benchmark built on an existing dataset by combining it with Statcast ground truth for 3D ball trajectory.

Critical strengths

The problem is challenging, it is a good contribution to provide a benchmark to enable new research on this.

Critical weaknesses

[W1] The paper is extremely difficult to follow. The structure of the paper needs to be completely revised to make its claims understandable. It would be much easier to follow if the explanations were not scattered across all sections and appendices.

[W2] Every experiment or task solved could benefit from a clearer problem statement; for instance, learned regressors N0 and N1 are mentioned to obtain the best precision, yet there is no formalization of their input and outputs in the main part of the paper.

[w3] The start and the end of the trajectory from the release plane to the plate plane seem to be known to enable the scheduled depth anchor; this looks like a major prior that should be more explicit.

Questions for authors

The fact that the physical fit works so poorly is disappointing and makes me wonder whether the optimization could be further constrained. The ballistic model is fitted with different types of acceleration constraints, but were there any tests done regarding the possible addition of constraints to the optimization, like reprojection errors in the image and the size of the ball in the images constraining the depth of the ball?

2 comments

We have uploaded a revised PDF, with changes highlighted in blue, that restructures the paper around a single problem statement (§3.1 and Table 1), formalizes N0 and N1 in §4, and makes the endpoint prior explicit in §3.1 and §6.2. The ball-size experiment you suggested is now in Appendix O, and we would be grateful if you could read the revised version.

Reviewer_uJz2OpenReview ↗

Response to the authors

I thank the authors for addressing the various shortcomings mentioned and also for their thorough response to my question.

The main problem that remains is the difficulty in reading the text, as—despite a major revision—it is still fundamentally structured as a series of points presented one after another. Another lingering issue is whether ICLR is the best venue for publishing an application benchmark.

For these reasons, I am maintaining my score.

Reviewer_zB39OpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
1
Soundness
2
Read review
Paper summary

The paper investigates what info enables 3d pitch recognition specifically for baseball from a broadcast feed. It also separates the contribution of prior works and observed ball tracks.

Critical strengths
  1. Useful questions are addressed and findings were clearly detailed.
  2. Ablations were well designed and helped isolate the contributions beyond camera and timing meta information.
Critical weaknesses
  1. Limited conceptual novelty, though the paper offers useful baseball-specific analysis.
  2. The no-pixel baseline receives track duration and observation count, which already contain information from the observation process. The experiments isolate the additional value of track coordinates, not a complete separation of visual evidence from priors.
Questions for authors
  1. How widely may the primary conclusions be applied? Is it possible to provide the previous vs. observation analysis using a different motion class, capture regime, or significantly different learning estimator?
  2. To what extent does the trained model's benefit come from fixing detector-window/anchor mismatch instead of retrieving more trajectory data? Is it possible to distinguish between these effects?
  3. Given that the synthetic alignment test does not account for linked detection bias, motion blur, and model mismatch, how sensitive are the stated gains?
  4. To what extent does detector-specific temporal support contribute to the correctness of the no-coordinate baseline?
1 comment

We have uploaded a revised PDF with changes highlighted in blue. Your points are addressed in §7 and Table 4 (the new cam and cam+pix rungs, W2/Q4), §6.2 and Table 3 (endpoint oracle, Q2), Appendix E and Table 5 (realistic alignment errors, Q3), and §8 Generality (Q1), and we would appreciate it if you could take a look.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.