- Rating
- 2
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
The paper jointly generates football trajectories, possession, and narration, and proposes LAG to mitigate text under-generation in few-step insertion sampling.
Critical strengths
The analysis of under-generation is informative, and the controlled ablations demonstrate improvements from LAG.
Critical weaknesses
-
The benefit of joint generation is insufficiently established. On identical generated plays, joint generation improves carrier-consistency F1 by only 0.02, while CIDEr favors post-hoc narration on League and joint generation on World Cup (Appendix E.2). Trajectory quality also does not consistently improve. These results provide limited justification for coupling the two generation processes instead of generating trajectories first and narrating them afterward.
-
LAG leaves the motivating problem substantially unresolved. It changes training corruption but preserves the sampler’s structural growth constraint (§3.5). Narrations still reach only 66–67% of reference length at ten calls, and fifty calls do not close this gap. Shorter text alone need not indicate failure, but the residual under-generation is also associated with reduced ball motion (§4.3). LAG improves its baseline without establishing sufficiently complete narration that avoids degrading the simulated scene.
Questions for authors
-
What clear quality or measured efficiency advantage does joint generation offer over a competitive two-stage approach?
-
Can a growth-aware schedule or multi-token insertion address the remaining failure more directly than LAG?
6 comments
(2/2)
W2 — Under-generation and reduced ball motion. The 66–67% (Table 3) is measured in words. In events the gap is smaller, and closing it further does not move the ball.
- At ten steps our narrations write 76% / 85% as many complete events as the reference. At fifty steps complete events reach 85% / 86%, with 67% / 61% of the words (Appendix E.5). Much of the missing length is qualifiers (pressure, zones).
- For a narration given as input, ball travel follows events, not words. The reference without its qualifiers (66–71% of the words) gives the same ball path as the full reference (0.98 / 0.94 vs 0.98 / 0.92 of real, the latter in Table 3); cutting events drops it (0.78 / 0.67).
- For our own narrations, more events do not move the ball. From 13% to 85% as many complete events as the reference (WC; 5 to 50 steps, or a doubled insertion rate), the ball path stays at 0.89–0.91 (0.83–0.86 on League). Timing does move it on WC: pinning the same final narration from the first step raises it from 0.91 to 0.96, close to the full reference's 0.98 (0.91 and 0.98 are in Table 3).
These results refine the tempo analysis of Section 4.3: a narration given as input steers the play through its events, not its length in words. The revision says "events" where the abstract, introduction, Section 4.3, Appendix D.4 and the conclusion say "length", and reports the timing result in Section 4.3. LAG already helps the scene: without it the ball path is 0.80 / 0.65 (Table 11), against 0.91 / 0.86 (Table 3). The limitations now report the timing effect and the remaining ball-path gap.
Q2 — Growth-aware schedule or multi-token insertion? Both act on the growth limit, which is not the main cause of the short narrations: the model's length inference is, and that is what LAG fixes. We traced the sampler step by step and tested a growth-aware schedule at sampling (starting values in this answer are from Table 1):
- Where the length is lost. The growth limit (one insertion per site, so the text at most doubles per step) leaves the text behind schedule in the first two to three of 10 steps. Without LAG, the model then infers its target length from its own short text (Eq. 3) and stays behind, ending at about a third of the reference tokens. With LAG it catches up most of the way, and the limit binds again only at the last step, where it costs 0.09 / 0.08 of the reference length.
- Growth-aware schedule. At sampling, we spaced the 10 steps so that the length the schedule asks for never more than doubles from one step to the next (more steps early, fewer at the end). For the model without LAG, this raises CIDEr only from 0.375 to 0.446 (WC) and from 0.213 to 0.293 (League), far from LAG's 0.785 / 0.587. On our model it lowers CIDEr (0.785 → 0.567 on WC). LAG instead improves narration and trajectories together (ES 2.82 → 2.71, 3.46 → 3.25).
- Next steps. Training with a growth-aware reveal schedule and multi-token insertion would act on the remaining deficit and are complementary to LAG; our limitations (Appendix F) already list both. On top of LAG, doubling the insertion rate at sampling already helps: CIDEr 0.785 → 0.854 / 0.587 → 0.629, at unchanged ES.
Thank you for the additional experiments and explanations. The new results strengthen the evidence that LAG reduces text under-generation. However, I still have concerns about the benefit of jointly generating the play and its narration.
If the goal is to describe a generated play, generating trajectories first and then using a dedicated captioner already achieves higher CIDEr. The joint model also does not clearly improve trajectory quality over JointDiff alone, and the authors do not claim an efficiency advantage. Editing the play through text is therefore an important reason to use the joint model.
The editing results do not yet convince me. Simple edits can succeed, but harder edits remain unreliable. The new local-editing experiments show some positive results, but they do not clearly establish how much better the joint model is at making the requested change while preserving the parts of the play that should remain unchanged.
Meanwhile, the main method, LAG, addresses short narrations. The additional experiments support its effectiveness, but they do not show that more complete narrations lead to better editing. This leaves my central question unanswered: if the editing benefits are still limited, why generate the play and narration jointly rather than generate the play first and describe it afterward?
The authors have explained more clearly how to improve the proposed model, but have not sufficiently shown why we should choose this model over the simpler alternative.
(1/2)
We thank the reviewer for a precise review. Each answer starts with the short version. Numbers cited with a table, section or appendix are from the paper; all others are new. Pairs of numbers are World Cup / League (the paper's tables list League first), and "steps" are the paper's denoiser calls.
W1/Q1 — What does joint generation offer over a competitive two-stage approach? A pipeline that simulates first and narrates afterwards cannot steer the play with text: its simulator ignores the narration. Coupling the two makes the narration a control. Against pipelines whose simulator does take text, we measure two quality advantages: at equal change to the rest of the play, local edits (resumed mid-sampling) succeed 12–14 points more often (point 1), and the play is simulated better than by a pipeline that narrates first, on every seed (point 2). Narrations also agree slightly more with their play (point 3). A post-hoc captioner narrates finished plays closer to the references (Table 1, point 4).
Edit a sample and keep the rest of it. A user may like a generated play but want one change. In this experiment, the change is that the last pass goes to the receiver's nearest teammate, swapped throughout the narration. Sampling takes 10 steps from noise, and we store the state after each step. To edit, we pin the sample's own narration, edited, and re-run its last steps from a stored state: "resumed after step 3" keeps the first 3 steps and re-runs the other 7; step 0 regenerates the whole sample from its initial noise. Resuming earlier follows the edit more often but changes more of the play. A pipeline can do the same if its simulator accepts text: text-guided JointDiff (JointDiff+Guid) generates the play, a T5 captioner trained on our data narrates it, and JointDiff+Guid re-runs the same steps with the edited narration as its prompt. "Play moves" is how much the procedure itself changes the play: we re-run the same steps with the narration unchanged and take the mean distance to the original sample, over all players and the ball in the generated frames. Each cell gives the share of samples where the new receiver gets the ball (higher is better) and, in brackets, how far the play moves (lower is better):
At every step ours changes the play less. At equal change (ours resumed two steps earlier: 3 against 5, 5 against 7), ours follows the edit 12–14 points more often, and 6–9 points more after subtracting how often the new receiver gets the ball without the edit (paired bootstrap over test windows; all 95% intervals exclude zero). Regenerating the whole sample, ours follows more and changes less on League, while the pipeline follows more on WC (0.74 vs 0.69). An edited play has no ground truth, so ES is measured with the narration unchanged: equal or lower for ours at every step. Our model does this zero-shot; JointDiff+Guid is trained for text-guided simulation.
Simulate without inheriting the errors of a narration written first. Narrations from an autoregressive T5 forecaster make JointDiff+Guid worse than an empty prompt (ES 2.93 vs 2.80 on WC, 3.35 vs 3.29 on League; empty prompt: Table 9). Pinned into our model in place of its own narration, they hurt it too (2.77–2.84 vs 2.71; 3.28–3.33 vs 3.25 in Table 1; given whole or revealed along the schedule). Generating narration and play together avoids this: ours beats the forecaster pipeline on every seed (WC 2.71–2.83 over 5 seeds, League 3.24–3.27 over 3).
Coherence. On identical plays, co-generation adds one to two points of cF1 (0.83 vs 0.81 on WC, 0.81 vs 0.79 on League; Appendix E.2). It holds for cF1 and order on both corpora.
Where the pipeline is better or equal. Autoregressive T5 narrators reach higher CIDEr: 1.41 / 1.23 for the captioner on finished plays and 1.26 / 1.13 for the forecaster of point 2, against our 0.79 / 0.59 in Table 1. The forecaster's narrations agree less with the real play's ball carriers (cF1_gt 0.48 / 0.45, against our 0.51 / 0.47 in Table 1). We do not claim an efficiency advantage. Trajectories are on par: JointDiff alone is slightly better (seed means, ES 2.72 vs 2.77 on WC, 3.17 vs 3.25 on League; Section 4.1), and initialising our model from JointDiff matches it on WC (mean ES 2.73 over three seeds, against 2.74 for JointDiff on the same seeds and 2.71 when trained for the same total number of epochs) with the same narration quality.