acceptodds
Under review as a conference paper at ICLR 2027

Teaching What to Look For: Privileged Perception Questions for Agentic On-Policy Distillation in Video Captioning

Abstract

Detailed video captioning requires comprehensive and faithful descriptions of objects, actions, and temporal events. Fixed-frame sampling, however, can miss brief motions, small objects, and other fine-grained evidence, motivating captioning agents that actively seek relevant moments and inspect local regions. Training such behavior is challenging because, unlike video question answering, generic captioning instructions do not specify what visual evidence should be acquired. We propose Perception-Question On-Policy Distillation (PQ-OPD), which uses training-only Perception Questions (PQs) to guide active video perception. PQs specify what should be inspected without revealing the corresponding answers, encouraging the teacher to explore the video more comprehensively and focus on relevant visual evidence. Along trajectories generated by the student, on-policy distillation transfers this guidance to the student's reasoning, tool calls, and captions; no PQs are required at inference. Across three detailed-captioning and two caption-to-QA benchmarks, PQ-OPD consistently improves the corresponding untrained agents across teacher-student scales and input-frame budgets. Under our evaluation protocol, PQ-OPD achieves the strongest primary-metric results among the compared open-source models on DREAM-1K, CaReBench, ShortVidBench, MotionBench, and VidCapBench-AE. Our ablations, together with analysis of temporal coverage and answer leakage, further indicates that the student internalizes the training-time guidance as a more effective video exploration policy.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
Video Captioning, On-Policy Distillation, Video Agent

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_19JuOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes Perception-Question On-Policy Distillation (PQ-OPD) for detailed video captioning. An external video model generates timestamped perception questions (PQs) for training videos. A frozen teacher receives these questions as privileged context and provides token-level supervision along student-generated trajectories, including reasoning, seek/zoom calls, and captions. The student learns through a top-k approximation to Jensen–Shannon divergence and requires no PQs at inference. Experiments cover three captioning benchmarks, two caption-to-QA benchmarks, multiple teacher–student configurations, and different frame budgets. Additional analyses examine temporal exploration and the effects of providing PQ answers during training.

Critical strengths
  1. The problem is well-motivated. Generic captioning instructions provide limited guidance about which evidence to acquire. Using training-time questions to teach active perception is a sensible approach, and removing the question-generation dependency at inference is practically appealing.
  2. The training framework is coherent. Supervising the teacher on student-generated interaction histories directly addresses the situations encountered by the student. Applying the same distribution-matching objective to tool calls and caption tokens provides a straightforward way to train the complete agent.
  3. The empirical evaluation has useful breadth. Improvements appear across multiple benchmarks and model scales. For example, the 9B→9B model improves DREAM-1K event F1 from 31.3 to 33.3 and MotionBench accuracy from 57.0 to 59.0 over the untrained agent. Table 4 also compares against vanilla OPD in cross-scale settings, providing evidence that PQ conditioning contributes beyond ordinary distillation.
  4. The behavioral analyses go beyond aggregate scores. The frame-budget experiments, temporal-alignment measurements, and answer-leakage study help characterize the learned behavior. In particular, the increase in zero-observation captions under answer-conditioned training is an informative failure mode.
Critical weaknesses
  1. Limited novelty. The main idea augments an established OPD framework with training-only perception questions, while retaining a standard token-level distillation objective. The contribution therefore appears incremental. Although the vanilla-OPD and shuffled-PQ comparisons are useful, they do not establish whether video-specific questions are necessary. Shuffled questions may contain irrelevant content or mismatched timestamps, making them a disadvantaged control. A generic inspection checklist with valid, duration-normalized intervals would better distinguish video-specific guidance from simply encouraging systematic exploration.
  2. Potential content leakage. In Figure 1, the question about which object a person “pick[s] up and carr[ies] away” already reveals a concrete action, even though the object identity is withheld. Questions can therefore supply caption-relevant semantic information beyond specifying where to look. This complicates the interpretation that PQs primarily teach evidence acquisition. The answer-leakage experiment measures the effect of adding explicit answers, but does not assess information already embedded in the questions. An audit of generated PQs and an ablation using strictly neutral wording would strengthen this claim.
  3. Unclear exploration benefits. PQ-OPD also improves captioning with fixed observations, showing that some benefits can arise without changing evidence acquisition. The temporal-alignment analysis provides useful evidence, but better overlap with PQ intervals does not establish how much it contributes to caption quality. Evaluating the base and trained models on identical observation histories, or separating distillation on tool-call and caption tokens, would clarify the mechanism. Moreover, the frame-budget experiments control maximum image counts, while actual visual-token consumption and latency remain unreported, leaving practical efficiency unclear.
  4. Marginal gains. For 9B→9B training, improvements over the untrained agent include 0.1 points in CaReBench Action F1, 1.1 points in ShortVidBench accuracy, and 2.0 points in DREAM-1K event F1. The consistency across benchmarks is encouraging, but some differences are small enough that uncertainty matters. Although Figure 2 includes bootstrap intervals, the main comparisons lack paired confidence intervals and training-seed variability. The replacement model judges are also not validated against human judgments or alternative evaluators, making the reliability of smaller gains difficult to assess.
Questions for authors
  1. How does PQ-OPD compare with a teacher given a generic object/action/temporal inspection checklist and valid duration-normalized intervals? Can you separately ablate question wording and video-specific timestamps?
  2. Can you evaluate the base and trained captioners on identical observation histories, or separate the distillation loss on tool-call tokens from that on caption tokens? An off-policy soft-target distillation baseline would also help isolate the contribution of on-policy training.
  3. Does the method apply to other video tasks, e.g., video QA and temporal grounding.
6 comments
AuthorsOpenReview ↗

Response to Reviewer 19Ju (Part 1)

We thank the reviewer for the detailed and constructive review. Following the reviewer's suggestions, we ran the requested controls with the exact PQ-OPD recipe (9B → 9B, 372 steps, same objective and hyperparameters, full benchmarks); only the teacher-side questions or the training data source change. Scores are CaReBench F1 / DREAM-1K F1.

W1: Novelty — Addressing an open-ended exploration challenge distinct from QA

The problem is new. PQ-OPD addresses the exploration policy of an open-ended captioning agent. In video QA, the question itself points to the evidence that must be found, so tool-using QA agents and privileged OPD methods (Vision-OPD, ViCuR, StreamOPD) can tie their guidance to that question. Detailed captioning only receives "Describe the video in detail." Nothing says which objects, actions, or moments matter, or when enough has been seen. Prior captioning methods do not face this question, since they perceive a fixed set of pre-sampled frames. To our knowledge, PQ-OPD is the first method that teaches a captioning agent what evidence to acquire under such an unspecified instruction. Unlike query-anchored QA, this open-ended exploration challenge has, to the best of our knowledge, received little attention in video captioning.

Existing ingredients do not solve it; our design does. Our ablations show that combining OPD with privileged information is not sufficient by itself:

  • Plain OPD fails to teach exploration. Without PQs, the same on-policy distillation is clearly weaker (vanilla OPD, Table 4: 40.1 / 31.2 vs. 41.2 / 33.4).
  • Answer-side privilege backfires. OPSD (Zhao et al., 2026) conditions the teacher on reference solutions, and ViCuR identifies this answer-side privilege as a source of shortcut learning. In open-ended captioning the problem becomes severe: giving our teacher the PQ answers cuts tool use by more than half and produces captions written without any observation (Figure 4).
  • Generic guidance falls short. A systematic checklist with valid intervals does not reach video-specific PQs (below).
  • Imitation falls short. SFT and off-policy distillation on PQ-guided teacher trajectories are both weaker than on-policy training (Q2).

What works is a specific form of guidance: answer-free, video-specific, timestamped questions that turn a captioning teacher into an exploration teacher. We keep the standard OPD objective on purpose, so that the gains are attributable to this guidance. It also makes PQ-OPD a drop-in addition to any OPD pipeline, with no PQ generation or extra modules at inference.

The impact is substantial. Without PQs at inference, the 9B student matches its PQ-conditioned teacher (40.6 / 33.3 vs. 40.6 / 33.4). A 4B student trained with PQ-OPD beats the untrained 9B agent on DREAM-1K, CaReBench Object F1, ShortVidBench, and MotionBench. PQ-OPD achieves the strongest primary-metric results among the compared open-source models on all five benchmarks (Tables 1–2). The learned exploration is also transferable: giving PQ-OPD's trajectories to the untrained Base captioner recovers nearly two-thirds of the DREAM-1K gain (+1.3 out of +2.0; Q2). We therefore view PQ-OPD as more than an incremental add-on to OPD: it formulates and addresses the exploration-policy problem in open-ended agentic captioning, a challenge that typically does not arise in query-anchored QA settings.

AuthorsOpenReview ↗

Response to Reviewer 19Ju (Part 2)

Q1 / W2: Are video-specific PQs necessary, and do they leak content?

We thank the reviewer for suggesting these controls, which make the role of the PQs much clearer. All variants below are trained with the same recipe; only the teacher questions differ.

Teacher questionsWordingTimestampsCaReBenchDREAM-1K
Generic checklist (strictly neutral)same for all videosuniform, valid40.232.8
PQs without timestampsvideo-specificnone40.631.9
L1 (leakier PQs)video-specific (leakier)video-specific40.332.3
L2 (leakier PQs)video-specific (more leaky)video-specific39.932.5
L3 (leakiest PQs)video-specific (leakiest)video-specific37.329.4
PQ-OPD (ours)video-specificvideo-specific40.633.3

To ensure a fair control, we designed the generic checklist to match both the question count and the type distribution of our training PQs (~7.3 PQs per video, split 33% / 36% / 31% across object, action, and temporal types). Specifically, every video of duration T receives the same eight strictly neutral questions—three on static visual/scene elements (Q1, Q3, Q5), three on events/actions (Q2, Q4, Q6), and two on temporal change and overall progression (Q7, Q8)—with no video-specific object, action, or attribute words, splitting the video into quarters:

  1. What scene setting and visible elements appear between 0 and T/4 seconds?
  2. What happens in the video between 0 and T/4 seconds?
  3. What visual details and subject appearance are shown between T/4 and T/2 seconds?
  4. What happens in the video between T/4 and T/2 seconds?
  5. How are the scene layout, framing, or camera perspective shown between T/2 and 3T/4 seconds?
  6. What happens in the video between T/2 and 3T/4 seconds?
  7. What happens or changes in the video between 3T/4 and T seconds?
  8. How does the video progress overall from 0 to T seconds?
  • Video-specific PQs beat a systematic checklist. Valid whole-video intervals alone are not enough. The same holds for the teacher itself: 40.0 / 32.9 with the checklist vs. 40.6 / 33.4 with our PQs.
  • Wording and timestamps both matter. Removing timestamps leaves CaReBench unchanged (40.6) but costs 1.4 points on the event-centric DREAM-1K (33.3 → 31.9). The questions guide what to inspect; the timestamps guide when.
  • The gains do not come from leaked content. L1–L3 rewrite our PQs at three increasing levels of content leakage, with the same timestamps. We quantify how many caption facts each question set leaks: on a 1,000-video subset of our training set, a text-only model writes a caption from the questions alone, without the video, and the caption F1 of this question-only caption measures the leaked content. From our PQs it reaches just 20.8 / 14.9 Action / Object F1, versus 37.0 / 36.4 for L1, 43.6 / 46.5 for L2, and 52.7 / 52.9 for L3; an agent watching the video reaches 48.7 / 55.8. If our gains came from leaked content, leakier questions should train better students. Instead, L1–L3 are all worse, down to 37.3 / 29.4. This is fully consistent with what we observe in Figure 4.
AuthorsOpenReview ↗

Response to Reviewer 19Ju (Part 3)

W3 / Q2: Exploration benefits and on-policy training

Identical observation histories. We froze the trajectories of the untrained agent (Table 3, #3) and of PQ-OPD (#8) on all videos, and swapped them across the two captioners so each model writes the caption immediately from the other's collected observations:

Captioner ← trajectoryCaReBenchDREAM-1K
Base ← Base (Table 3, #3)39.831.3
Base ← PQ-OPD40.332.6
PQ-OPD ← Base40.332.2
PQ-OPD ← PQ-OPD (Table 3, #8)40.633.3
  • Exploration helps by itself. PQ-OPD's trajectories improve both captioners (+0.5 / +1.3 for Base, and +0.3 / +1.1 for PQ-OPD). On DREAM-1K, giving PQ-OPD's trajectories to Base alone recovers +1.3 of the +2.0 total gain.
  • Caption writing improves too. Under the same trajectory, the PQ-OPD captioner consistently outperforms Base (+0.5 / +0.9 on Base's trajectories, +0.3 / +0.7 on PQ-OPD's), consistent with Table 3 #5, and combining both yields the best performance.

Off-policy soft-target distillation. This baseline uses the same PQ-conditioned teacher, objective, and hyperparameters, but distills on the fixed teacher trajectories of #6:

TrainingCaReBenchDREAM-1K
SFT on PQ-conditioned teacher trajectories (#6)39.732.6
Off-policy distillation (soft targets)40.232.9
PQ-OPD (on-policy)40.633.3

Soft targets help over SFT, and on-policy training helps further.

Visual tokens and latency. Per-video inference statistics of the two agents (full benchmarks, 1,000 videos each):

BenchmarkReturned framesTool callsTool roundsVisual tokensLatency (median)
Qwen3.5 9B AgentCaReBench34.511.13.711.4K30 s
PQ-OPDCaReBench54.618.44.117.6K41 s
Qwen3.5 9B AgentDREAM-1K31.39.63.312.7K31 s
PQ-OPDDREAM-1K48.814.63.519.8K42 s

Visual tokens count the image tokens of all returned observations; each observation is capped at 460,800 pixels. Latency is end-to-end wall-clock time per video under our batched evaluation setting.

To check that the gain does not simply come from extra frame input, Figure 2 compares all systems under the same cap on returned frames:

Frame budgetDirect input (uniform)Qwen3.5 9B AgentPQ-OPD
835.8 / 25.838.5 / 29.239.1 / 30.4
1637.1 / 29.739.6 / 30.840.0 / 31.1
3238.4 / 30.739.6 / 30.440.3 / 33.0

At the 32-frame budget of the fixed-input models, PQ-OPD still leads both uniformly sampled direct input and the untrained agent. The gain comes from choosing better evidence, not from consuming more of it.

W4: Size and reliability of the gains

The gains are consistent across every axis we evaluate:

  • Benchmarks and scales. All three PQ-OPD configurations improve over the untrained 9B agent on DREAM-1K, CaReBench Object F1, ShortVidBench, and MotionBench (e.g., MotionBench 57.0 → 57.9 / 59.0 / 59.6). Even the 4B student beats the untrained 9B agent on these.
  • Training methods. PQ-OPD beats vanilla OPD in both cross-scale settings (+1.1 / +2.2 and +0.8 / +1.8).
  • Budgets and controls. PQ-OPD beats direct input at every frame budget (Figure 2), and every new control above falls below PQ-OPD on DREAM-1K while none exceeds it on CaReBench.

Judge. CaReBench (ICLR 2026) shows in its judge-stability analysis that caption scores from GPT-4.1, Gemini 2.5 Pro, and DeepSeek-V3 are strongly and significantly correlated. This supports using GPT-4.1 as a reliable replacement evaluator, applied uniformly to all models.

Seeds. We fully agree that seed variability is valuable to report. Unfortunately, each PQ-OPD run involves on-policy rollouts and teacher scoring on multiple GPU nodes, so repeating every configuration over several seeds is costly and, regrettably, difficult to be completed within the limited rebuttal period. We hope the consistent gains across the many independently trained models above offer reasonable evidence that the effect is reliable.

Reviewer_UCtKOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This work uses training-only Perception Questions (PQs) and on-policy distillation to teach the model what and when to observe in video captioning. The student generates its own trajectory using seek/zoom, while a frozen teacher conditioned on additional PQs provides token-level supervision along that trajectory. This supervision is transferred through On-Policy Distillation (OPD), jointly improving tool calls and caption generation. At inference time, the model autonomously explores important temporal and spatial evidence without relying on PQs.

Critical strengths

1. Effective exploration guidance via training-only PQs: By conditioning the teacher on PQs to focus on key objects, actions, and temporal intervals, the framework successfully trains the student to autonomously explore important visual evidence at inference time without relying on PQs.

2. Learning from student-generated trajectories via OPD: Unlike SFT, which follows fixed teacher trajectories, the proposed method provides token-level supervision directly on the seek/zoom trajectories generated by the student. This allows the model to learn an exploration policy that better aligns with its actual inference behavior.

3. Thorough empirical validation through diverse ablations and analyses: Through extensive analyses—including PQ trace SFT, Shuffled-PQ, Vanilla OPD, varying frame budgets, temporal coverage, and answer leakage—the authors convincingly demonstrate that PQ-OPD contributes to both effective tool use and caption quality.

Critical weaknesses

1. Insufficient direct validation of PQ quality: The paper states that 43,477 PQs are automatically generated from 5,979 videos using Gemini 3.1 Pro, with answers and evidence descriptions retained for quality control and analysis. However, there is a lack of direct quality analysis verifying how accurate and reliable these generated PQs actually are.

2. Lack of direct evidence that the PQ-conditioned teacher provides superior supervision: In PQ-OPD, the teacher receives the same interaction history as the student but is additionally conditioned on PQs to generate token-level supervision. Yet, the paper lacks a direct analysis showing whether conditioning on PQs genuinely improves the teacher's own caption quality or the quality of its supervision.

3. Absence of an explicit training criterion for stopping behavior: Although the paper defines the exploration policy as deciding "what to inspect next and when to stop," the training process lacks an explicit stopping criterion or reward signal to teach the model when to terminate exploration.

Questions for authors

Q1. How did the authors verify the actual accuracy and reliability of the 43,477 automatically generated PQs?

Q2. Could the authors provide a direct comparison of the teacher's captioning performance with and without PQ conditioning to directly validate the quality of its supervision?

Q3. What specific mechanism guides the student model on "when to stop" exploring during inference, given the absence of an explicit stopping signal in training?

4 comments
AuthorsOpenReview ↗

Response to Reviewer UCtK

We thank the reviewer for recognizing the training-only PQ guidance, the on-policy supervision, and the breadth of our ablations, and for the thoughtful questions. We address each of them below.

W1 / Q1: Quality and reliability of the generated PQs

  1. Automatic quality control. Each PQ is generated with a concise answer and a visual-evidence statement for its interval (Section 5.1, Appendix B.1). An automatic validator rejects and regenerates outputs with invalid or mismatched time ranges, empty answers or evidence, answers leaked into the question, duplicates, or event-premise clauses. All 43,477 PQs pass these checks, and their intervals cover 97.6% of each video on average.
  2. Human verification. Two human experts cross-verified 50 randomly sampled training videos (344 PQs) against the source frames. For the questions used in training, 99.7% (343/344) ask about visually present content without hallucinated premises, and 99.4% (342/344) have time intervals covering the target evidence. For the generated answers (excluded from training), 90.1% (310/344) are also consistent with the video—and because PQ-OPD never uses these answers in training, our method is unaffected by any errors in them.
  3. Robust by design. Receiving only answer-free questions, the teacher still describes the real frames returned by seek/zoom, and its prompt asks only for visually supported information (Appendix B.4). PQs are also clearly video-specific: replacing them with PQs from other videos (Shuffled-PQ) lowers the student from 40.6 / 33.3 to 40.1 / 31.5.

W2 / Q2: Does PQ conditioning improve the teacher and its supervision?

Thank you for this important question. Yes, it does. Our teachers are the frozen, untrained Qwen3.5 agents themselves: the 9B agent is the teacher for 9B → 9B and 9B → 4B, and the 27B agent is the teacher for the cross-scale 27B → 9B setting. Running each agent with and without PQs therefore provides the teacher comparison the reviewer suggests (for 9B, this is Table 3 #3 → #4):

TeacherWithout PQsWith PQs
Qwen3.5 9B agent (teacher of 9B → 9B, 9B → 4B)39.8 / 31.340.6 / 33.4
Qwen3.5 27B agent (teacher of 27B → 9B)40.2 / 32.342.0 / 35.9

Supervision quality is shown by Table 4, where the same teacher supervises the same student and only the PQs differ:

Teacher → StudentVanilla OPDPQ-OPD
27B → 9B40.1 / 31.241.2 / 33.4
9B → 4B39.4 / 30.840.2 / 32.6

(CaReBench F1 / DREAM-1K F1.) In 9B → 9B, the student without PQs (40.6 / 33.3) matches the PQ-conditioned teacher (40.6 / 33.4).

W3 / Q3: How does the student learn when to stop?

How an agent stops. We thank the reviewer for raising this point and clarify the mechanism here. Like standard tool-using LLM agents, our agent works in a loop of assistant turns. In each turn, the model generates its reasoning and then either (a) a tool call, which the environment executes and whose returned frames are appended to the context before the next turn, or (b) the final caption with no tool call, which ends the loop. Stopping is therefore not a separate module or signal: it is the model's choice to write the caption instead of another tool call. The 16-round limit is only a safety cap on cost.

How this choice is trained. The choice between (a) and (b) is made by ordinary output tokens, and the JSD objective (Eq. 5) supervises these tokens like all others. While the questioned evidence is still missing, the PQ-conditioned teacher puts probability on another seek/zoom; once it has been observed, the teacher favors writing the caption. The student learns when to stop from this signal. PQ-OPD stops on its own after 4.1 / 3.5 rounds on average; only 2.8% / 1.1% of videos reach the limit. The answer-leakage study (Figure 4) confirms that the teacher controls stopping. When the teacher is also given the PQ answers, it can already describe the questioned content from text alone, so at each turn it no longer needs more frames and favors writing the caption right away. The student, trained on this signal, learns to stop early: average tool calls fall from 18.4 to 8.4 on CaReBench, and the share of videos captioned before any observation rises from 0% to 13.7% (16.8% on DREAM-1K). Only the teacher's input changed, not the stopping rule, so the stopping behavior is learned from the teacher. With answer-free PQs, the teacher cannot answer from text and keeps favoring inspection until the questioned evidence has been observed. This answer-free design is precisely our key contribution: it teaches PQ-OPD to explore better and, in turn, to write better captions.

We sincerely thank the reviewer for the constructive feedback. We hope that our responses have addressed the reviewer's concerns. If so, we would be grateful if the reviewer would kindly consider raising the score. We would be happy to answer any further questions.

Dear reviewer UCtK,

We appreciate your constructive comments and the time and effort spent on reviewing our submission! We would like to kindly confirm whether we have sufficiently addressed all of your concerns (or at least part of them). Should there be any remaining questions or areas requiring further clarification, please do not hesitate to let us know. If you are satisfied with our responses, we would greatly appreciate your consideration in adjusting the scores accordingly.

We sincerely look forward to your feedback.

Best regards,

Authors of Submission 833

Thank you for your diligent responses to my questions. I have received the following clarifications:

  • W1: Effectively resolved. The quality of the automatically generated PQs has been verified through human evaluation, yielding high scores.

  • W2: Mostly resolved. While not a perfectly "same-history controlled" analysis, it does provide the direct teacher comparison requested in my question.

  • W3: Effectively resolved. It has been clarified—with supporting empirical evidence—that the stopping mechanism is learned as a decision between a tool call and a final answer token, rather than as a separate reward.

Most of my concerns have been sufficiently addressed. Consequently, I will reconsider the score I previously assigned.

Thank you very much for your feedback on our response and for your willingness to reconsider the score! Regarding your second point concerning a "same-history controlled" analysis, we have conducted experiments using identical observation histories (as mentioned in the response to reviewer 19Ju).

We froze the trajectories of the untrained agent (Table 3, #3) and of PQ-OPD (#8) on all videos, and swapped them across the two captioners so each model writes the caption immediately from the other's collected observations:

Captioner ← trajectoryCaReBenchDREAM-1K
Base ← Base (Table 3, #3)39.831.3
Base ← PQ-OPD40.332.6
PQ-OPD ← Base40.332.2
PQ-OPD ← PQ-OPD (Table 3, #8)40.633.3
  • Exploration helps by itself. PQ-OPD's trajectories improve both captioners (+0.5 / +1.3 for Base, and +0.3 / +1.1 for PQ-OPD). On DREAM-1K, giving PQ-OPD's trajectories to Base alone recovers +1.3 of the +2.0 total gain.
  • Caption writing improves too. Under the same trajectory, the PQ-OPD captioner consistently outperforms Base (+0.5 / +0.9 on Base's trajectories, +0.3 / +0.7 on PQ-OPD's), consistent with Table 3 #5, and combining both yields the best performance.

We hope these experiments help address your concerns.

Reviewer_32tkOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes Perception-Question On-Policy Distillation (PQ-OPD) for agentic detailed video captioning. During training, a frozen teacher receives video-specific Perception Questions that indicate what visual evidence to inspect and when, without revealing the answers. The teacher then provides token-level supervision along student-generated trajectories involving temporal seek , spatial zoom , and caption generation. The student receives no Perception Questions at inference. The authors construct 43,477 questions for 5,979 videos and evaluate the method on three video-captioning and two caption-to-QA benchmarks. Across multiple teacher–student scales and frame budgets, PQ-OPD consistently improves over the untrained agent, vanilla on-policy distillation, and SFT on PQ-conditioned teacher trajectories. Additional analyses suggest that it improves temporal evidence acquisition, while exposing the teacher to question answers reduces tool use and degrades performance.

Critical strengths
  1. The answer-free PQ design is the paper’s strongest contribution. PQs tell the teacher what and when to inspect without revealing the answer, encouraging visual evidence acquisition rather than textual shortcuts. The answer-leakage results in Figure 4 are particularly convincing: exposing answers reduces tool use, increases zero-observation responses, and substantially degrades captioning performance.
  2. The empirical study is thorough and goes beyond the main benchmark results. The authors compare PQ-OPD against several relevant training baselines and evaluate its robustness across model scales and frame budgets. Additional analyses of temporal coverage and tool use support the claim that PQ guidance affects both caption quality and evidence acquisition. Although the gains are moderate, they are reasonably consistent across settings.
Critical weaknesses
  1. The distinction from closely related OPD methods could be clearer. PQ-OPD combines active video perception, privileged teacher context, and on-policy distillation, but these components are individually established. The paper does not sufficiently distinguish its contribution from related methods such as ViCuR, Vision-OPD, and StreamOPD, making it difficult to assess the extent of its methodological novelty beyond the video-captioning setting.
  2. PQ-OPD convincingly improves over the same-interface agent baselines, and Figure 2 shows that this advantage persists under matched frame budgets. However, the headline comparisons in Tables 1 and 2 are not fully compute-matched: the fixed-input models use 32 frames, whereas the agent setting permits up to 128 input frames and 16 adaptive seek/zoom rounds with iterative reasoning. Thus, the controlled results support the effectiveness of PQ-OPD, but the best-open-source comparisons should be interpreted with some caution.
Questions for authors
  1. How does PQ-OPD differ from closely related privileged multimodal OPD methods such as ViCuR, Vision-OPD, and StreamOPD? A more explicit comparison of the privileged information, supervision targets, trajectory generation, and inference-time requirements would help establish the specific novelty of PQ-OPD.
  2. Since the objective in Equation 5 distills reasoning, tool-call, and caption tokens jointly, it is not yet clear how much of the improvement comes from better evidence acquisition versus better caption generation. Ablations that distill only tool-related tokens or only caption tokens, or a trajectory-swapping analysis, would help separate these effects.
  3. Given the different inference settings in Tables 1 and 2, what are the average number of returned frames, tool calls, visual tokens, and inference latency of the agent models? How would the comparison change when total computation or latency is also taken into account?
4 comments
AuthorsOpenReview ↗

Response to Reviewer 32tK

We thank the reviewer for the positive assessment of the answer-free PQ design and of the breadth of the empirical study. We address each point below.

W1 / Q1: Relation to ViCuR, Vision-OPD, and StreamOPD

All four methods give privileged information to the teacher only, and the student does not receive it at inference. They differ in what the privilege is, what is supervised, and how the student obtains its visual input:

Privileged teacher informationSupervised tokensStudent trajectoryInference-time input
Vision-OPDCrop-conditioned view of the same image (self-distillation)Single-turn response to the given query (no tool calls)On-policy, single turn, fixed full imageFull image, no tools
ViCuRQuery-related visual cues derived from the same input (replacing answer-side privilege)Single-turn response to the given query (no tool calls)On-policy, single turn, fixed inputSame input, with a student-side cue-recovery module; no tools
StreamOPDSpatio-temporal cues; OPD reweighted by cue vs. no-cue teacher likelihood ratiosSingle-turn response to the given query (no tool calls)On-policy, single turn, fixed streaming windowFixed streaming window, no tools
PQ-OPDAnswer-free Perception Questions: what to inspect and when, never the answerThe whole interaction: reasoning, seek/zoom calls, and the final captionOn-policy multi-turn trajectory; the student decides where to look and acquires its own observations through toolsVideo tools only; no PQs

PQ-OPD differs in two ways:

  1. It solves a problem specific to open-ended tasks. In QA, the question already tells the model what evidence to look for, so prior methods can tie privileged information to it. Open-ended captioning has no such target: "Describe the video in detail" leaves the model to decide what to observe, when, and when to stop. A teacher with the same instruction has no better idea of what to look for, and giving it the content to describe creates a textual shortcut (Figure 4). PQs fill this gap by turning the open-ended goal into video-specific, answer-free sub-goals for the teacher only. Without PQs, the same distillation is clearly weaker (vanilla OPD, Table 4: 40.1 / 31.2 vs. 41.2 / 33.4 for 27B → 9B).
  2. The student acquires its own evidence. These three OPD methods, like prior captioning methods, work on a fixed visual input. PQ-OPD performs captioning in an agentic loop: the student gathers observations through seek/zoom, and the teacher supervises this acquisition process, not only the final caption.

We will add this comparison to the related-work section.

Q2: Evidence acquisition vs. caption generation

We thank the reviewer for this helpful suggestion and ran the trajectory-swapping analysis accordingly. We freeze the full exploration trajectories of the untrained agent (Table 3, #3) and of PQ-OPD (#8) on all 1,000 videos of each benchmark. For the cross-model swaps, we give each trajectory to the other model and ask it to write the caption immediately from the observations already collected, without further exploration:

Captioner ← trajectoryCaReBench F1DREAM-1K F1
Base ← Base (Table 3, #3)39.831.3
Base ← PQ-OPD40.332.6
PQ-OPD ← Base40.332.2
PQ-OPD ← PQ-OPD (Table 3, #8)40.633.3

Both components contribute, and combining them works best:

  • Better evidence acquisition. Giving PQ-OPD's trajectories to the untrained Base model improves it on both benchmarks (+0.5 on CaReBench, +1.3 on DREAM-1K), recovering nearly two-thirds of the total DREAM-1K gain from better observations alone.
  • Better caption generation. Under the same trajectory, the PQ-OPD captioner consistently outperforms Base (+0.5 / +0.9 on Base's trajectories, and +0.3 / +0.7 on PQ-OPD's trajectories).
AuthorsOpenReview ↗

Response to Reviewer 32tK (Continue)

W2 / Q3: Compute, visual tokens, and latency

We agree that the headline comparisons with fixed-input models in Tables 1 and 2 are not compute-matched, and we will state the input settings explicitly next to these tables. The table below gives the per-video inference statistics of the two agents (full benchmarks, 1,000 videos each):

BenchmarkReturned framesTool callsTool roundsVisual tokensLatency (median)
Qwen3.5 9B AgentCaReBench34.511.13.711.4K30 s
PQ-OPDCaReBench54.618.44.117.6K41 s
Qwen3.5 9B AgentDREAM-1K31.39.63.312.7K31 s
PQ-OPDDREAM-1K48.814.63.519.8K42 s

Visual tokens count the image tokens of all returned observations; each observation is capped at 460,800 pixels. Latency is end-to-end wall-clock time per video under our batched evaluation setting.

PQ-OPD looks at more of the video, using about 1.5× the frames and visual tokens of the untrained agent and about 1.35× its latency. To check that the gain does not simply come from extra frame input, Figure 2 compares all systems under the same cap on returned frames. At the 32-frame budget, which equals the fixed-input setting of Tables 1 and 2, PQ-OPD still leads:

Frame budgetFixed input (uniform)Qwen3.5 9B AgentPQ-OPD
835.8 / 25.838.5 / 29.239.1 / 30.4
1637.1 / 29.739.6 / 30.840.0 / 31.1
3238.4 / 30.739.6 / 30.440.3 / 33.0

(CaReBench F1 / DREAM-1K F1.)

Under equal frame budgets, PQ-OPD outperforms both models with fixed frames and the untrained agent. The advantage over fixed frame input holds even at 8 frames. So PQ-OPD's improvement comes from choosing better evidence, not from consuming more of it.

We thank the reviewer again for the encouraging and constructive review. We hope these new analyses address the remaining questions, and we would be happy to clarify any further points.

Dear reviewer 32tk,

We appreciate your constructive comments and the time and effort spent on reviewing our submission! We would like to kindly confirm whether we have sufficiently addressed all of your concerns (or at least part of them). Should there be any remaining questions or areas requiring further clarification, please do not hesitate to let us know.

We sincerely look forward to your feedback.

Best regards,

Authors of Submission 833

Dear Reviewer 32tk,

We apologize for following up once again. As the discussion period is approaching its end, we would be very grateful if you could let us know whether our responses have addressed your main concerns, or whether there are any remaining points that would benefit from further clarification.

We fully understand that you may have a busy schedule, and we sincerely appreciate the time and effort you have already devoted to reviewing our submission.

Thank you again for your valuable feedback and consideration.

Best regards,

Authors of Submission 833

Comments on the paper

1 comment
AuthorsOpenReview ↗

Global Responses

Dear Area Chair and Reviewers,

Thank you very much for your time and for handling and reviewing our submission. Here we provide a concise summary of our responses. Scores are CaReBench F1 / DREAM-1K F1.

Novelty

(1) PQ-OPD addresses a problem that prior OPD and captioning methods do not face: the exploration policy of an open-ended captioning agent. In video QA, the question points to the evidence to find. "Describe the video in detail" does not say what to inspect, when, or when to stop. To our knowledge, PQ-OPD is the first method that teaches a captioning agent what evidence to acquire under such an instruction. (32tk-W1, 19Ju-W1)

(2) PQ-OPD differs from Vision-OPD, ViCuR, and StreamOPD in both the privilege and what is supervised. Only the teacher receives answer-free, video-specific, timestamped Perception Questions, which say what to inspect and when but never the answer. The teacher supervises the whole multi-turn trajectory, including the seek/zoom calls through which the student gathers its own observations, while the three prior methods supervise a single-turn response over a fixed input. The student uses no PQs at inference. We provide a comparison table against these methods. (32tk-Q1, 19Ju-W1)

(3) Answer-free PQ is better than generic privileged information. Vanilla OPD without PQs is clearly weaker (27B → 9B: 40.1 / 31.2 vs. 41.2 / 33.4; 9B → 4B: 39.4 / 30.8 vs. 40.2 / 32.6). Answer-side privilege backfires: giving the teacher the PQ answers cuts tool calls from 18.4 to 8.4 and makes 13.7% of CaReBench captions be written before any observation (Figure 4). With PQs used only in training, the 9B student matches its PQ-conditioned teacher (40.6 / 33.3 vs. 40.6 / 33.4). (19Ju-W1, UCtK-Q2)

Where the improvements come from

(4) The gains come from both better evidence acquisition and better caption writing. In a new trajectory-swap experiment, giving PQ-OPD's frozen trajectories to the untrained captioner raises it from 39.8 / 31.3 to 40.3 / 32.6, which recovers +1.3 of the +2.0 DREAM-1K gain from better observations alone. Under the same trajectory, the PQ-OPD captioner is consistently better (+0.3 to +0.9). (32tk-Q2, 19Ju-W3)

(5) On-policy training is necessary. With the same PQ-conditioned teacher, SFT (39.7 / 32.6) and an off-policy soft-target distillation baseline (40.2 / 32.9) both fall below PQ-OPD (40.6 / 33.3). (19Ju-Q2)

Necessity of PQs and content leakage

(6) Both the video-specific wording and the timestamps of PQs are needed. We conduct an experiment with a generic question list instead of video-specific PQs and the results show that it reaches only 40.2 / 32.8. Removing timestamps from our PQs costs 1.4 DREAM-1K F1 (33.3 → 31.9). (19Ju-Q1/W2)

(7) The gains do not come from leaked caption content. We rewrote our PQs at three increasing levels of leakage (L1–L3) and measured leakage by writing captions from the questions alone with a text-only model: Action / Object F1 is 20.8 / 14.9 for our PQs and rises to 52.7 / 52.9 for L3. Students trained with leakier questions are all worse, down to 37.3 / 29.4. (19Ju-W2)

PQ quality and teacher supervision

(8) The PQs are reliable. All 43,477 PQs pass automatic validation. Two human experts cross-verified 344 PQs from 50 random training videos: 99.7% ask about visually present content and 99.4% have intervals covering the target evidence. The generated answers are never used in training. (UCtK-W1/Q1)

(9) PQs improve the teacher, and the student learns from it when to stop. PQs raise the 9B teacher from 39.8 / 31.3 to 40.6 / 33.4 and the 27B teacher from 40.2 / 32.3 to 42.0 / 35.9. Stopping is the agent's choice to write the caption instead of another tool call, and the same JSD objective supervises it. PQ-OPD stops on its own after 4.1 / 3.5 rounds on average, and only 2.8% / 1.1% of videos reach the round cap. (UCtK-W2/W3)

Efficiency and fair comparison

(10) The extra cost is moderate, and the gains hold under matched frame budgets. PQ-OPD uses about 1.5× the frames and visual tokens of the untrained agent and about 1.35× its median latency (41 s vs. 30 s per video on CaReBench). At the 32-frame budget of the fixed-input models, PQ-OPD reaches 40.3 / 33.0, above the untrained agent (39.6 / 30.4) and uniform sampling (38.4 / 30.7), and it also leads at 8 and 16 frames. It means the gains of PQ-OPD are not only from observing more frames. (32tk-W2/Q3, 19Ju-W3)

Consistency of results

(11) The gains are consistent across benchmarks and model scales. All three PQ-OPD configurations improve over the untrained 9B agent on DREAM-1K, CaReBench Object F1, ShortVidBench, and MotionBench, and even the 4B student beats it on these benchmarks. CaReBench's judge-stability analysis supports GPT-4.1 as the evaluator, applied uniformly to all models. (19Ju-W4)

All remaining points are clarified in the per-reviewer responses.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.