- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 1
Read review
Paper summary
UniPitch reconstructs soccer matches from broadcast video. For each visible player it predicts an identity slot and a 2D pitch position. It builds on a decoupled baseline (MOTIP for tracking, PnLCalib for calibration) and adds three modules, trained in three stages:
- PSIM: a persistent appearance memory (4 prototypes per identity slot) added to MOTIP’s ID logits, to recover players who leave and re-enter the view.
- TCSM: a temporal camera model supervised through player pitch positions via the homography projection. At inference it is only used to fill frames where PnLCalib is unreliable.
- SFTR: a bounded residual correction of the projected player positions, using geometry, visual, trajectory and calibration-history features.
On SoccerNet-GSR it improves P-HOTA by 4.8 points over the best decoupled baseline, but stays below rule-augmented methods that use jersey and team cues.
Critical strengths
- Coupling tracking and calibration is a relevant problem for game state reconstruction. The gains over decoupled baselines are consistent across all metrics, mainly in detection and identity association.
- The ablations are systematic: each module adds to the result (Table 2), and the SFTR input ablation shows which cues matter (Table 4).
- The related work covers tracking, field registration and game state reconstruction well.
Critical weaknesses
- The main claim (unified, end-to-end, differentiable geometric mapping) is not supported.
- The “differentiable geometric mapping” is the standard camera → homography projection, used as a loss in TCSM. Reprojection losses on camera parameters are well known (TVCalib)
- Training is staged, not end-to-end. MOTIP is trained in Stage I and then frozen. PnLCalib uses released weights and is never trained. The gradient from player positions never reaches the calibration that actually places the players.
- At inference, players are projected with PnLCalib. TCSM only fills unreliable frames, using hand-set rules. Its position supervision adds only +0.56 P-HOTA (Table 3). Most gains come from SFTR and PSIM, which are independent refinements on top of the decoupled pipeline.
- Only visible players are reconstructed, so “reconstruction of soccer matches” is overstated. Off-screen players, studied in the trajectory-imputation literature [A,B,C], are not addressed.
- What remains is a set of add-on modules on a decoupled pipeline, with limited novelty (e.g., the memory is close to DeepSORT and MeMOTR).
- SFTR operation is not properly explained. Which are the four branch features?
- The experimental evaluation is not convincing, and key details are missing.
- UniPitch is below existing methods (73.30 vs 78.96 P-HOTA for KISTGSR). Not using jersey or team cues is a design choice, and adding them is never tested.
- It uses its own metric (P-HOTA), and the other methods’ numbers were recomputed without explaining how.
- The Table 2 baseline (66.86) doesn’t match MOTIP + PnLCalib in Table 1 (68.50).
- Simple controls are missing: MOTIP with a longer history instead of PSIM, direct camera supervision using the available ground truth, and an ablation on the number of prototypes (K = 4).
- The method is hard to follow. Fig. 1 blocks are not defined in the text, and the paper does not explain what q_i is, how the memory is initialized and updated, or how temporal processing works (autoregressive or sliding window). The FPN is not cited.
[A] Omidshafiei, Shayegan, et al. "Multiagent off-screen behavior prediction in football." Scientific reports 12.1 (2022): 8638. [B] Xu, Yi, and Yun Fu. "Sports-traj: A unified trajectory generation model for multi-agent movement in sports." International Conference on Learning Representations. 2025. [C] Capellera, Guillem, et al. "Jointdiff: Bridging continuous and discrete in multi-agent trajectory generation." International Conference on Learning Representations. 2026.
Questions for authors
- Q1. SoccerNet-GSR provides ground-truth camera annotations. Why supervise TCSM only through player positions, and how does this compare with direct camera supervision?
- Q2. In what fraction of test frames is PnLCalib flagged as unreliable, so that TCSM is actually used? Are TCSM’s camera estimates more accurate than PnLCalib’s on those frames?
- Q3. How were the rule-augmented results in Table 1 (taken from other papers) obtained under P-HOTA, and on which split?
- Q4. Why does the Table 2 baseline (66.86) differ from MOTIP + PnLCalib in Table 1 (68.50)?
- Q5. What is q_i? How is the memory initialized and updated, and how does performance change with K? Does PSIM beat MOTIP with a longer history window?
- Q6. Is inference online or based on sliding windows? Can you report runtime and results over several seeds?
3 comments
Response to rnbj - Part 2
W1 and Q5 · Appearance memory and SFTR
We agree that appearance memory is established.
-
PSIM links identities and trajectories. DeepSORT uses a descriptor gallery, while MeMOTR updates track embeddings via memory attention. Persistent slot identity memory (PSIM) adjusts MOTIP's short-term identity logits via slot-indexed prototypes. These identities organize SFTR histories.
-
Q5: role of and memory initialization and updating. As detailed in the revised paper, is the Deformable DETR feature, which is served for ID prediction, prototype matching and memory updating. A slot's first accepted features initialize its prototypes. Later features update the closest prototype by cosine similarity via EMA.
-
Q5: Ablations support persistent memory. With a fixed 60-frame window, PSIM raises P-AssA from 58.04 to 61.57 (Table 2). For ablation studies, gives the highest P-HOTA, P-AssA, and P-IDF1 as illustrated in the table below. We have not tested memory-matched longer-history MOTIP.
P-HOTA P-DetA P-AssA P-LocA P-IDF1 1 72.92 82.73 64.31 93.40 79.03 2 73.06 82.90 64.43 93.43 79.14 3 73.06 82.64 64.63 93.41 79.24 4 73.30 82.67 65.04 93.36 79.32 5 72.70 81.81 64.65 93.22 79.08 -
SFTR/Figure 1: Four branches guide refinement. We agree clarification was needed, so we expanded the caption and added the FPN citation. Four branches encode box/projection geometry, pooled DETR features, same-slot position history, and calibration history. Their concatenation predicts a pitch position residual.
-
Visibility. Yes, we reconstruct visible players, as clarified in the abstract and limitations.
W2, Q3 and Q4 · Evaluation
-
Q3: All methods share the same protocol. We reproduce the compared methods with their public Github implementations and evaluate them on the official SoccerNet-GSR test split using the same annotations and our P-HOTA evaluator. We agree that KISTGSR scores higher (78.96 vs. 73.30), while UniPitch gains 4.80 points over the strongest decoupled baseline. Explanation have beend added in the paper.
-
Q4: Baseline configurations differ. Both share same PnLCalib with different configurations in tracker. The tracker in Table 1(MOTIP + PnLCalib, 68.50) uses default SportMOT config, with 50 identity classes plus a newborn class, whereas Table 2(66.86) uses 25 fixed slots and no newborn class with all proposed modules removed . We have clarified this in the revised paper.
Q6 · Inference, runtime and seeds
Per-frame prediction is online with a causal sliding window, whereas correction is offline because it uses past and future anchors. We have clarified the inference details in Appendix C.
The full pipeline averages 230.88 seconds per 750-frame clip (3.25 FPS) over 49 clips on one A100. We report point estimates because multi-seed evaluation has not been run.
I want to thank the authors for the effort in the rebuttal, for revising the end-to-end claims, and for resolving my doubts about the camera annotations, the baseline configuration and the evaluation protocol. However, I still see this approach as a well-engineered pipeline for GSR with limited methodological novelty. The most original idea, supervising the camera model with player positions, is only used to correct frames where PnLCalib fails, and the approach remains below the rule-augmented methods. I keep my score.
Response to rnbj - Part 1
Thank you for your careful review.
W1 · Parametric trajectory reconstruction
We would like to clarify that UniPitch is not a set of independent add-on modules.
Unified parametric modeling is the contribution. Persistent identities link pitch positions over time, and camera states connect them to image observations. This geometry lets pitch-position labels supervise temporal camera state modeling (TCSM), whose motion guides correction of unreliable projections.
Staged training retains geometric supervision. We agree that training is staged and projection is standard, and have corrected the end-to-end description. However, position errors update TCSM through projection with the tracker and PnLCalib fixed.
TCSM improves reconstruction through camera correction and position supervision. Table 2 shows that adding TCSM to PSIM+SFTR improves P-HOTA from 71.61 to 73.30, demonstrating the benefit of TCSM-guided correction. Furthermore, Table 3 shows that enabling pitch-position supervision for TCSM improves P-HOTA from 72.74 to 73.30.
Q1 and Q2 · Camera supervision and calibration
Q1: Position supervision directly optimizes reconstruction. SoccerNet-GSR derives player pitch positions by projecting annotated bounding-box bottom centers using estimated camera geometry. However, the publicly available
Labels-GameState.jsonfile, available for downloading from the official website, provides player pitch positions and pitch-line annotations but does not include explicit camera parameters or homography matrices. Therefore, direct camera supervision requires additional target construction.For comparison, we fitted camera targets from box-bottom centers and pitch labels via RANSAC, homography decomposition, and nonlinear refinement. Pinhole fitting assumes a centered principal point, equal focal lengths, and negligible distortion. With other settings fixed, we averaged Smooth L1 loss over seven fitted parameters. The results are reported in the table below.
Q2: Correction affects 21.12% of frames (7,761/36,750). To evaluate the effectiveness of this correction, we follow the camera calibration evaluation protocol of Magera et al. and report results on these unreliable frames. UniPitch correction reduces mean reprojection error by 45.5% (31.97 to 17.42 pixels) and raises JaC@10 from 44.50 to 50.24.