- Rating
- 1
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 1
Read review
Paper summary
The paper studies video event prediction and introduces PGE-RL, a reinforcement learning method for generating timestamped event chains that are both consistent with the observed video and useful for predicting subsequent events. A shared policy operates in two passes: it first describes the observed history without access to the question or candidate answers, then predicts using the original video, candidates, and an immutable copy of the chain. During training, frozen visual and text verifiers provide visual-consistency and predictive-sufficiency rewards, while perception-aware credit routing (PACR) assigns separate learning signals to the chain and prediction stages based on answer correctness and verifier assessments.
Critical strengths
-
The paper clearly distinguishes visual consistency from predictive usefulness, recognizing that an accurate but generic description and a discriminative but unsupported description are both insufficient for reliable prediction. Jointly optimizing these properties is well motivated by the task.
-
Generating an option-blind history and preserving it as fixed context for prediction makes the information boundaries between the two stages explicit. Stage-specific credit assignment is also well motivated, seeking to avoid rewarding incorrect observations after a correct answer or suppressing correct observations after a prediction failure.
Critical weaknesses
-
The main experimental results contain unresolved aggregation discrepancies. Appendix A.6 identifies FutureBench averages that cannot be reconstructed using the official category weights, VEPBench rates that do not correspond to the stated sample count, and AVEP metrics and split labels requiring reconciliation; without per-example predictions and the actual aggregation rules, the reported gains and comparisons cannot be reliably verified.
-
The training rewards and primary chain diagnostics rely on the same frozen verifiers, with limited evidence of robustness under alternative verifiers or independent evaluation. The 500-example human audit reports only overall agreement above 90%, without class distributions, classwise errors, or inter-annotator agreement, leaving uncertainty about whether the policy learns reliable visual evidence extraction or adapts to the particular verifiers’ preferences.
Questions for authors
-
Please explain in detail the metric-aggregation and evaluation-record inconsistencies discussed in Appendix A.6, including why the FutureBench averages cannot be reconstructed using the official category weights, why the VEPBench rates do not correspond to the stated sample count, and which AVEP metrics and split labels require reconciliation. Please clarify the specific discrepancies, their causes, and their implications for the experimental conclusions.
-
How does an answer-only reward baseline perform under the same two-pass interface? Could the authors also compare event chains without timestamps and length-matched free-form reasoning, and report realized generated-token counts and training/inference costs to distinguish the effects of the representation, reward design, and additional computation?
-
Do the full method’s visual-consistency and predictive-sufficiency advantages over the single-verifier variants persist under independent verifiers or human evaluation? Please provide the audit sampling procedure, positive/negative class distributions, classwise error rates, and inter-annotator agreement, and clarify the sensitivity of PACR to changes in verifier thresholds.