- Rating
- 2
- Confidence
- 4
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper proposes Perception-Question On-Policy Distillation (PQ-OPD) for detailed video captioning. An external video model generates timestamped perception questions (PQs) for training videos. A frozen teacher receives these questions as privileged context and provides token-level supervision along student-generated trajectories, including reasoning, seek/zoom calls, and captions. The student learns through a top-k approximation to Jensen–Shannon divergence and requires no PQs at inference. Experiments cover three captioning benchmarks, two caption-to-QA benchmarks, multiple teacher–student configurations, and different frame budgets. Additional analyses examine temporal exploration and the effects of providing PQ answers during training.
Critical strengths
- The problem is well-motivated. Generic captioning instructions provide limited guidance about which evidence to acquire. Using training-time questions to teach active perception is a sensible approach, and removing the question-generation dependency at inference is practically appealing.
- The training framework is coherent. Supervising the teacher on student-generated interaction histories directly addresses the situations encountered by the student. Applying the same distribution-matching objective to tool calls and caption tokens provides a straightforward way to train the complete agent.
- The empirical evaluation has useful breadth. Improvements appear across multiple benchmarks and model scales. For example, the 9B→9B model improves DREAM-1K event F1 from 31.3 to 33.3 and MotionBench accuracy from 57.0 to 59.0 over the untrained agent. Table 4 also compares against vanilla OPD in cross-scale settings, providing evidence that PQ conditioning contributes beyond ordinary distillation.
- The behavioral analyses go beyond aggregate scores. The frame-budget experiments, temporal-alignment measurements, and answer-leakage study help characterize the learned behavior. In particular, the increase in zero-observation captions under answer-conditioned training is an informative failure mode.
Critical weaknesses
- Limited novelty. The main idea augments an established OPD framework with training-only perception questions, while retaining a standard token-level distillation objective. The contribution therefore appears incremental. Although the vanilla-OPD and shuffled-PQ comparisons are useful, they do not establish whether video-specific questions are necessary. Shuffled questions may contain irrelevant content or mismatched timestamps, making them a disadvantaged control. A generic inspection checklist with valid, duration-normalized intervals would better distinguish video-specific guidance from simply encouraging systematic exploration.
- Potential content leakage. In Figure 1, the question about which object a person “pick[s] up and carr[ies] away” already reveals a concrete action, even though the object identity is withheld. Questions can therefore supply caption-relevant semantic information beyond specifying where to look. This complicates the interpretation that PQs primarily teach evidence acquisition. The answer-leakage experiment measures the effect of adding explicit answers, but does not assess information already embedded in the questions. An audit of generated PQs and an ablation using strictly neutral wording would strengthen this claim.
- Unclear exploration benefits. PQ-OPD also improves captioning with fixed observations, showing that some benefits can arise without changing evidence acquisition. The temporal-alignment analysis provides useful evidence, but better overlap with PQ intervals does not establish how much it contributes to caption quality. Evaluating the base and trained models on identical observation histories, or separating distillation on tool-call and caption tokens, would clarify the mechanism. Moreover, the frame-budget experiments control maximum image counts, while actual visual-token consumption and latency remain unreported, leaving practical efficiency unclear.
- Marginal gains. For 9B→9B training, improvements over the untrained agent include 0.1 points in CaReBench Action F1, 1.1 points in ShortVidBench accuracy, and 2.0 points in DREAM-1K event F1. The consistency across benchmarks is encouraging, but some differences are small enough that uncertainty matters. Although Figure 2 includes bootstrap intervals, the main comparisons lack paired confidence intervals and training-seed variability. The replacement model judges are also not validated against human judgments or alternative evaluators, making the reliability of smaller gains difficult to assess.
Questions for authors
- How does PQ-OPD compare with a teacher given a generic object/action/temporal inspection checklist and valid duration-normalized intervals? Can you separately ablate question wording and video-specific timestamps?
- Can you evaluate the base and trained captioners on identical observation histories, or separate the distillation loss on tool-call tokens from that on caption tokens? An off-policy soft-target distillation baseline would also help isolate the contribution of on-policy training.
- Does the method apply to other video tasks, e.g., video QA and temporal grounding.
6 comments
Response to Reviewer 19Ju (Part 2)
Q1 / W2: Are video-specific PQs necessary, and do they leak content?
We thank the reviewer for suggesting these controls, which make the role of the PQs much clearer. All variants below are trained with the same recipe; only the teacher questions differ.
| Teacher questions | Wording | Timestamps | CaReBench | DREAM-1K |
|---|---|---|---|---|
| Generic checklist (strictly neutral) | same for all videos | uniform, valid | 40.2 | 32.8 |
| PQs without timestamps | video-specific | none | 40.6 | 31.9 |
| L1 (leakier PQs) | video-specific (leakier) | video-specific | 40.3 | 32.3 |
| L2 (leakier PQs) | video-specific (more leaky) | video-specific | 39.9 | 32.5 |
| L3 (leakiest PQs) | video-specific (leakiest) | video-specific | 37.3 | 29.4 |
| PQ-OPD (ours) | video-specific | video-specific | 40.6 | 33.3 |
To ensure a fair control, we designed the generic checklist to match both the question count and the type distribution of our training PQs (~7.3 PQs per video, split 33% / 36% / 31% across object, action, and temporal types). Specifically, every video of duration T receives the same eight strictly neutral questions—three on static visual/scene elements (Q1, Q3, Q5), three on events/actions (Q2, Q4, Q6), and two on temporal change and overall progression (Q7, Q8)—with no video-specific object, action, or attribute words, splitting the video into quarters:
- What scene setting and visible elements appear between 0 and T/4 seconds?
- What happens in the video between 0 and T/4 seconds?
- What visual details and subject appearance are shown between T/4 and T/2 seconds?
- What happens in the video between T/4 and T/2 seconds?
- How are the scene layout, framing, or camera perspective shown between T/2 and 3T/4 seconds?
- What happens in the video between T/2 and 3T/4 seconds?
- What happens or changes in the video between 3T/4 and T seconds?
- How does the video progress overall from 0 to T seconds?
- Video-specific PQs beat a systematic checklist. Valid whole-video intervals alone are not enough. The same holds for the teacher itself: 40.0 / 32.9 with the checklist vs. 40.6 / 33.4 with our PQs.
- Wording and timestamps both matter. Removing timestamps leaves CaReBench unchanged (40.6) but costs 1.4 points on the event-centric DREAM-1K (33.3 → 31.9). The questions guide what to inspect; the timestamps guide when.
- The gains do not come from leaked content. L1–L3 rewrite our PQs at three increasing levels of content leakage, with the same timestamps. We quantify how many caption facts each question set leaks: on a 1,000-video subset of our training set, a text-only model writes a caption from the questions alone, without the video, and the caption F1 of this question-only caption measures the leaked content. From our PQs it reaches just 20.8 / 14.9 Action / Object F1, versus 37.0 / 36.4 for L1, 43.6 / 46.5 for L2, and 52.7 / 52.9 for L3; an agent watching the video reaches 48.7 / 55.8. If our gains came from leaked content, leakier questions should train better students. Instead, L1–L3 are all worse, down to 37.3 / 29.4. This is fully consistent with what we observe in Figure 4.
Response to Reviewer 19Ju (Part 3)
W3 / Q2: Exploration benefits and on-policy training
Identical observation histories. We froze the trajectories of the untrained agent (Table 3, #3) and of PQ-OPD (#8) on all videos, and swapped them across the two captioners so each model writes the caption immediately from the other's collected observations:
| Captioner ← trajectory | CaReBench | DREAM-1K |
|---|---|---|
| Base ← Base (Table 3, #3) | 39.8 | 31.3 |
| Base ← PQ-OPD | 40.3 | 32.6 |
| PQ-OPD ← Base | 40.3 | 32.2 |
| PQ-OPD ← PQ-OPD (Table 3, #8) | 40.6 | 33.3 |
- Exploration helps by itself. PQ-OPD's trajectories improve both captioners (+0.5 / +1.3 for Base, and +0.3 / +1.1 for PQ-OPD). On DREAM-1K, giving PQ-OPD's trajectories to Base alone recovers +1.3 of the +2.0 total gain.
- Caption writing improves too. Under the same trajectory, the PQ-OPD captioner consistently outperforms Base (+0.5 / +0.9 on Base's trajectories, +0.3 / +0.7 on PQ-OPD's), consistent with Table 3 #5, and combining both yields the best performance.
Off-policy soft-target distillation. This baseline uses the same PQ-conditioned teacher, objective, and hyperparameters, but distills on the fixed teacher trajectories of #6:
| Training | CaReBench | DREAM-1K |
|---|---|---|
| SFT on PQ-conditioned teacher trajectories (#6) | 39.7 | 32.6 |
| Off-policy distillation (soft targets) | 40.2 | 32.9 |
| PQ-OPD (on-policy) | 40.6 | 33.3 |
Soft targets help over SFT, and on-policy training helps further.
Visual tokens and latency. Per-video inference statistics of the two agents (full benchmarks, 1,000 videos each):
| Benchmark | Returned frames | Tool calls | Tool rounds | Visual tokens | Latency (median) | |
|---|---|---|---|---|---|---|
| Qwen3.5 9B Agent | CaReBench | 34.5 | 11.1 | 3.7 | 11.4K | 30 s |
| PQ-OPD | CaReBench | 54.6 | 18.4 | 4.1 | 17.6K | 41 s |
| Qwen3.5 9B Agent | DREAM-1K | 31.3 | 9.6 | 3.3 | 12.7K | 31 s |
| PQ-OPD | DREAM-1K | 48.8 | 14.6 | 3.5 | 19.8K | 42 s |
Visual tokens count the image tokens of all returned observations; each observation is capped at 460,800 pixels. Latency is end-to-end wall-clock time per video under our batched evaluation setting.
To check that the gain does not simply come from extra frame input, Figure 2 compares all systems under the same cap on returned frames:
| Frame budget | Direct input (uniform) | Qwen3.5 9B Agent | PQ-OPD |
|---|---|---|---|
| 8 | 35.8 / 25.8 | 38.5 / 29.2 | 39.1 / 30.4 |
| 16 | 37.1 / 29.7 | 39.6 / 30.8 | 40.0 / 31.1 |
| 32 | 38.4 / 30.7 | 39.6 / 30.4 | 40.3 / 33.0 |
At the 32-frame budget of the fixed-input models, PQ-OPD still leads both uniformly sampled direct input and the untrained agent. The gain comes from choosing better evidence, not from consuming more of it.
W4: Size and reliability of the gains
The gains are consistent across every axis we evaluate:
- Benchmarks and scales. All three PQ-OPD configurations improve over the untrained 9B agent on DREAM-1K, CaReBench Object F1, ShortVidBench, and MotionBench (e.g., MotionBench 57.0 → 57.9 / 59.0 / 59.6). Even the 4B student beats the untrained 9B agent on these.
- Training methods. PQ-OPD beats vanilla OPD in both cross-scale settings (+1.1 / +2.2 and +0.8 / +1.8).
- Budgets and controls. PQ-OPD beats direct input at every frame budget (Figure 2), and every new control above falls below PQ-OPD on DREAM-1K while none exceeds it on CaReBench.
Judge. CaReBench (ICLR 2026) shows in its judge-stability analysis that caption scores from GPT-4.1, Gemini 2.5 Pro, and DeepSeek-V3 are strongly and significantly correlated. This supports using GPT-4.1 as a reliable replacement evaluator, applied uniformly to all models.
Seeds. We fully agree that seed variability is valuable to report. Unfortunately, each PQ-OPD run involves on-policy rollouts and teacher scoring on multiple GPU nodes, so repeating every configuration over several seeds is costly and, regrettably, difficult to be completed within the limited rebuttal period. We hope the consistent gains across the many independently trained models above offer reasonable evidence that the effect is reliable.
Response to Reviewer 19Ju (Part 1)
We thank the reviewer for the detailed and constructive review. Following the reviewer's suggestions, we ran the requested controls with the exact PQ-OPD recipe (9B → 9B, 372 steps, same objective and hyperparameters, full benchmarks); only the teacher-side questions or the training data source change. Scores are CaReBench F1 / DREAM-1K F1.
W1: Novelty — Addressing an open-ended exploration challenge distinct from QA
The problem is new. PQ-OPD addresses the exploration policy of an open-ended captioning agent. In video QA, the question itself points to the evidence that must be found, so tool-using QA agents and privileged OPD methods (Vision-OPD, ViCuR, StreamOPD) can tie their guidance to that question. Detailed captioning only receives "Describe the video in detail." Nothing says which objects, actions, or moments matter, or when enough has been seen. Prior captioning methods do not face this question, since they perceive a fixed set of pre-sampled frames. To our knowledge, PQ-OPD is the first method that teaches a captioning agent what evidence to acquire under such an unspecified instruction. Unlike query-anchored QA, this open-ended exploration challenge has, to the best of our knowledge, received little attention in video captioning.
Existing ingredients do not solve it; our design does. Our ablations show that combining OPD with privileged information is not sufficient by itself:
What works is a specific form of guidance: answer-free, video-specific, timestamped questions that turn a captioning teacher into an exploration teacher. We keep the standard OPD objective on purpose, so that the gains are attributable to this guidance. It also makes PQ-OPD a drop-in addition to any OPD pipeline, with no PQ generation or extra modules at inference.
The impact is substantial. Without PQs at inference, the 9B student matches its PQ-conditioned teacher (40.6 / 33.3 vs. 40.6 / 33.4). A 4B student trained with PQ-OPD beats the untrained 9B agent on DREAM-1K, CaReBench Object F1, ShortVidBench, and MotionBench. PQ-OPD achieves the strongest primary-metric results among the compared open-source models on all five benchmarks (Tables 1–2). The learned exploration is also transferable: giving PQ-OPD's trajectories to the untrained Base captioner recovers nearly two-thirds of the DREAM-1K gain (+1.3 out of +2.0; Q2). We therefore view PQ-OPD as more than an incremental add-on to OPD: it formulates and addresses the exploration-policy problem in open-ended agentic captioning, a challenge that typically does not arise in query-anchored QA settings.