acceptodds
Under review as a conference paper at ICLR 2027

Less Data, Better Timing: Student–Curriculum Coupling for VLM On-Policy Distillation in Temporal Video Grounding

Abstract

On-policy distillation (OPD) provides dense supervision directly on student-generated trajectories, making it an effective post-training strategy for vision-language models in temporal video grounding (TVG). However, existing pipelines typically construct the training curriculum from a fixed teacher and the initial student state, implicitly assuming that selected examples retain positive supervision value throughout optimization. We show that supervision trustworthiness and supervision necessity are distinct yet coupled: the former concerns target credibility, while the latter varies with the student's current task competence; together, they shape supervision value. Building on this coupled view, we introduce Student–Curriculum Coupling (SCC), a closed-loop framework in which a compact Anchor–Frontier curriculum defines the candidate supervision space and the evolving student dynamically determines its active subset. Supervision can therefore be activated, suspended, or reactivated as competence changes, concentrating teacher computation and optimization on current task-level deficits. Across three TVG benchmarks, SCC achieves a 5.1% relative improvement in mean recall over Video-OPD on its original curriculum, while using 60.0% fewer training examples and reducing training time by 50.4%. Ablations support the complementary roles of capability-structured curriculum design and student-dependent supervision in achieving these gains. Together, these results establish SCC as a data- and compute-efficient framework for TVG post-training, delivering stronger temporal grounding by aligning trustworthy supervision with the student's evolving learning needs.

Primary area:
foundation or frontier models, including LLMs
Keywords:
Temporal Video Grounding, Vision-Language Models, On-Policy Distillation, Curriculum Learning, Efficient Post-Training

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_G7E7OpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes Student-Curriculum Coupling (SCC) for on-policy distillation in temporal video grounding. It selects 100 teacher-qualified Anchor examples that the initial student handles well and 900 Frontier examples that it handles poorly. During training, a sampled prediction receives teacher scoring and distillation only when its temporal IoU is below 0.7. The paper evaluates a Qwen3-VL-8B student on three TimeLens benchmarks. It reports a 5.1% relative mean-recall improvement over Video-OPD with 60.0% fewer selected examples and 50.4% shorter training time. The appendices examine curriculum composition, routing, normalization, teacher choice, and rollout count.

Critical strengths
  1. The paper demonstrates a useful accuracy-efficiency trade-off with a simple implementation. Holding the AF curriculum fixed, Table 6 improves mean recall at every threshold while reducing training time from 1:11:20 to 1:00:26. Table 2 explains the saving through 492 teacher-scored routes instead of 1,004. This comparison gives more direct evidence for routing than the headline comparison against the larger TVDF curriculum.

  2. The experimental decomposition is substantial. Tables 3, 11, 14, 15, and 16 examine curriculum composition, the competence threshold, the Anchor/Frontier ratio, curriculum size, and normalization. These details make the practical contribution easier to assess and avoid overstating an unbiased reverse-KL interpretation for SCC itself.

Critical weaknesses
  1. The claimed stabilization mechanism is not demonstrated by the reported schedule. Appendix B.2 and Tables 4-5 present each unique example once, apart from four final Frontier repeats. All 100 Anchors therefore have only one training presentation. An Anchor can fail its initial competence criterion at that presentation, as Section 3.3 defines, but the experiment does not then revisit it to establish recovery. Figure 4 also does not state whether successive competence values evaluate a fixed set or different scheduled examples. With one stochastic rollout per example, changes can reflect both sampling variation and changes in the evaluated cohort. The endpoint gains are useful, but they do not establish the stronger acquisition, suspension, and restoration account in Section 5.

  2. The static control does not fully isolate the value of current-student routing. Appendix B.12 matches the total and per-step supervision counts, but selects 49 Anchors and 443 Frontiers. SCC instead supervises 11 Anchors and 481 Frontier presentations in Table 14. The methods thus differ in their active curriculum composition as well as their routing rule. A static gate based on initial student competence, or a random mask matching SCC's active region counts, would be more diagnostic. Repeated seeds are also absent. This matters because some gains over uniform AF training are small, including 0.21 points on Charades [email protected] in Table 11. The current results support the overall recipe more strongly than the specific causal role assigned to evolving supervision necessity.

Questions for authors
  1. How are the competence curves in Figure 4 computed and smoothed? Are the same examples evaluated at successive checkpoints? Is there evidence that an Anchor's grounding performance recovers after its corrective update, beyond the initial-to-current threshold crossing?

  2. Does SCC still outperform a static initial-student gate and a random routing mask that match its per-step counts and active Anchor/Frontier composition? Please report variation across independent runs for this comparison and uniform OPD on AF.

  3. How many candidate examples require teacher and initial-student inference to construct AF? Table 6 measures training after curriculum construction. What are the shared and additional screening costs relative to TVDF, and how do they affect the total cost of obtaining a trained model?

7 comments

Dear Reviewer G7E7,

We appreciate your careful review and constructive feedback. Below, we clarify the interpretation of online training dynamics and Anchor reactivation, and present additional evidence from controlled evaluations of capability retention, matched static routing, multi-seed robustness, and construction-inclusive efficiency.

W1 & Q1: Anchor Stabilization and Competence Measurements

Response: Figure 4 summarizes online training rollouts on the examples presented at each step. We clarify its computation and interpretation below and provide an Anchor-muted control evaluated on both the TVG benchmarks and a fixed Anchor set to assess performance retention.

1. Computation and Smoothing of Figure 4

At each training step, we compute the current student's rollout tIoU for all presented examples, including both supervised and suspended presentations. Each competence point pools presentations from the current step and the preceding four steps. The Overall, Anchor, and Frontier curves report mean tIoU over their corresponding presentations within this window. Each presentation contributes equally, so steps contribute in proportion to their presentation counts. We provide the unsmoothed curves in Appendix B.17 (Figure 6, lines 1471–1489).

The curves do not repeatedly evaluate a fixed cohort, and the checkpoint guides are visual reference marks only. Figure 4 describes online grounding scores and supervision activity; changes can reflect student evolution, differences in the examples presented, and rollout variability.

2. Meaning of Anchor Reactivation

In our approximately single-pass implementation, reactivation is defined relative to the offline initial assessment. Each Anchor has one training presentation. It initially meets the competence criterion and receives teacher scoring and OPD if its current rollout falls below τS=0.7\tau_S=0.7 at that presentation. Reactivation therefore denotes an initial-to-current threshold crossing and does not require an earlier online presentation. This definition is stated in Section 3.3 (lines 242–247) and Appendix A.3 (lines 877–882).

A single sampled failure does not establish capability loss. The criterion identifies an observed task deficit that triggers supervision. Appendix B.14 (Figure 5, lines 1377–1400) compares this single-rollout decision with three- and five-rollout assessments using mean tIoU. Additional rollouts provide no consistent accuracy advantage across benchmarks and increase training time, supporting the practical single-rollout choice.

[Continuation of W1 & Q1]

3. Controlled Evaluation of Anchor Supervision

We added an Anchor-muted control that retains the same 100 Anchors, 900 Frontiers, and presentation schedule but disables teacher scoring and OPD for all Anchors. Frontier routing continues to use the control model's current predictions. Student initialization, teacher, normalization, and the remaining training settings are unchanged.

Downstream grounding performance. Table 1 reports mean mIoU across the three benchmarks and mean recall across the nine recall metrics. Values are percentages; bold marks the higher score at each checkpoint.

Table 1. Effect of disabling Anchor supervision on downstream grounding performance.

CheckpointSettingMean mIoU (%)Mean recall (%)
20SCC53.4854.14
Anchor-muted SCC53.6454.25
40SCC55.7155.65
Anchor-muted SCC54.8154.57
59SCC55.6655.64
Anchor-muted SCC54.5454.22
79SCC55.7155.80
Anchor-muted SCC54.3854.04

Disabling Anchor supervision weakens later-stage grounding performance. SCC maintains higher aggregate performance at checkpoints 40, 59, and 79 and outperforms the control on all 12 benchmark metrics at the final checkpoint.

Retention on the fixed Anchor set. We additionally evaluate the same 100 Anchors from the AF curriculum at checkpoints 20, 40, 59, and 79, using five fresh rollouts per example under unchanged sampling settings. Table 2 reports mean tIoU and the percentage satisfying tIoU≥0.7\mathrm{tIoU}\geq0.7, computed over 500 evaluation rollouts per checkpoint for each model. These evaluations do not contribute to training.

Table 2. Grounding performance on the fixed set of 100 Anchors.

CheckpointSettingMean tIoUSuccess rate (%)
20SCC0.937395.2
Anchor-muted SCC0.925893.4
40SCC0.938894.0
Anchor-muted SCC0.914891.4
59SCC0.932494.2
Anchor-muted SCC0.910891.0
79SCC0.932694.4
Anchor-muted SCC0.908990.8

SCC achieves higher mean tIoU and success rate at all four checkpoints. From checkpoints 20 to 79, its success rate decreases by 0.8 percentage points, compared with 2.6 points for the Anchor-muted control. At the final checkpoint, SCC achieves 94.4% versus 90.8% success.

Together, these comparisons support the contribution of Anchor supervision to sustained downstream performance and retention on the selected Anchor set during continued training. The fixed-Anchor evaluation holds cohort composition constant and tracks performance across checkpoints, extending the evidence beyond the threshold crossings used for routing.

These analyses assess performance retention across checkpoints but do not fully isolate the contribution of corrective supervision on individual Anchors to subsequent performance recovery. We have clarified this scope in Section 7 (lines 507–515). We have also specified the computation and smoothing procedure in the Figure 4 caption (lines 428–431), clarified the interpretation of the training curves in Section 5 (lines 409–416), and incorporated the controlled evaluations in Section 5 (Table 4, lines 433–454) and Appendix B.16 (Tables 25–26, lines 1418–1468).

W2 & Q2: Matched Static Controls and Multi-Seed Robustness

Response: The original static control did not fully separate supervision composition from the routing rule. We therefore added both requested static controls and evaluated SCC, the two controls, and uniform Video-OPD on AF across three random seeds.

1. Controls with Matched Supervision Allocation

Both controls use the same 1,000 AF examples and 1,004-presentation schedule as SCC. They match the total supervision budget and the number of supervised Anchor and Frontier presentations at each step. The reference SCC allocation contains 492 supervised presentations: 11 Anchors and 481 Frontiers. Student initialization, teacher, optimization settings, and loss normalization are unchanged.

  • Matched Random Routing (AF) randomly selects the required number of presentations within each step and curriculum region.
  • Initial-Score Routing (AF) selects the required number of presentations with the lowest offline initial-student tIoU scores qS,0q_{S,0} within each step and region, implementing static selection based on initial competence under the same supervision quotas.

Both routing masks are fixed before training. Only SCC's aggregate supervision counts are reused; neither control uses its example-level routing decisions or current-student scores. Both controls generate fresh on-policy trajectories during training.

These comparisons hold supervision budget, timing, and Anchor–Frontier composition fixed while testing whether current-student assessment improves the selection of supervised presentations.

2. Three-Seed Results

All AF results in Table 3 report mean ± sample standard deviation across three independent training runs at checkpoint 79. Published single-run Video-OPD (TVDF) results are included for reference. Values are percentages; bold indicates the highest mean for each metric.

Table 3. Matched static controls and three-seed TVG performance.

BenchmarkMethodmIoU[email protected][email protected][email protected]
Charades-TimeLensVideo-OPD (TVDF)52.073.145.832.4
Video-OPD (AF)52.09 ± 0.3373.26 ± 0.0647.00 ± 0.4232.30 ± 0.60
Matched Random Routing (AF)51.80 ± 0.3372.64 ± 0.3946.48 ± 0.5631.96 ± 0.39
Initial-Score Routing (AF)52.00 ± 0.3873.17 ± 0.5746.79 ± 0.7032.07 ± 0.68
SCC (AF)52.78 ± 0.1173.82 ± 0.1248.15 ± 0.2533.09 ± 0.25
ActivityNet-TimeLensVideo-OPD (TVDF)47.360.545.635.8
Video-OPD (AF)49.86 ± 0.5163.61 ± 0.6747.96 ± 0.4037.58 ± 0.51
Matched Random Routing (AF)49.33 ± 0.7263.08 ± 0.9447.25 ± 0.8937.02 ± 0.78
Initial-Score Routing (AF)49.64 ± 0.5263.30 ± 0.7947.64 ± 0.3937.34 ± 0.59
SCC (AF)50.81 ± 0.1764.88 ± 0.3849.15 ± 0.0738.65 ± 0.36
QVHighlights-TimeLensVideo-OPD (TVDF)61.073.860.350.4
Video-OPD (AF)62.34 ± 0.6875.25 ± 0.7262.41 ± 1.0652.17 ± 0.81
Matched Random Routing (AF)61.80 ± 0.7174.89 ± 1.0861.41 ± 0.6751.61 ± 0.84
Initial-Score Routing (AF)62.00 ± 0.8675.12 ± 0.7961.86 ± 1.0251.31 ± 1.07
SCC (AF)63.79 ± 0.0976.99 ± 0.3764.18 ± 0.3354.27 ± 0.36

SCC achieves higher mean performance on all 12 metrics than both matched static controls and uniform Video-OPD on AF. Averaged across the nine recall metrics, SCC improves mean recall by 1.87 percentage points over Matched Random Routing, 1.62 points over Initial-Score Routing, and 1.29 points over uniform Video-OPD (AF).

For the highlighted Charades-TimeLens [email protected], SCC achieves 33.09 ± 0.25, compared with 32.30 ± 0.60 for uniform Video-OPD (AF), a mean improvement of 0.79 percentage points.

3. Interpretation and Manuscript Revision

The gains persist when supervision budget, timing, and Anchor–Frontier composition are matched. The random control addresses the alternative explanation that SCC benefits simply from allocating more supervision to Frontiers. The initial-score control additionally tests whether initial difficulty can provide an equally effective fixed selection. SCC's stronger results in both comparisons support the value of using current student performance to determine which presentations receive supervision.

We have incorporated the control protocols and three-seed results into Appendix B.13 (Table 23, lines 1326–1376).

Reviewer_adFSOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes Student–Curriculum Coupling (SCC), a framework for improving the efficiency of on-policy distillation (OPD) in temporal video grounding. The authors argue that the value of supervision changes as the student learns, and that examples selected at the beginning of training may not always require further supervision. To address this, SCC constructs an Anchor–Frontier curriculum based on teacher reliability and initial student performance, and selectively applies OPD according to the student's current tIoU. Experiments on three temporal grounding benchmarks show consistent improvements over Video-OPD while reducing the number of training examples and the overall training time.

Critical strengths
  1. The paper addresses a meaningful problem. Reducing unnecessary teacher supervision is particularly relevant for OPD, where teacher computation can be expensive. The proposed approach is simple and can be incorporated into existing training pipelines without additional trainable components.
  2. The experimental results are promising. SCC achieves consistent improvements over Video-OPD across three benchmarks while using substantially fewer training examples. I appreciate that the authors also evaluate Video-OPD on the same Anchor–Frontier curriculum, which provides a more direct comparison of the student-dependent supervision strategy.
  3. The paper is generally clear and well organized. The authors provide a fairly comprehensive set of ablations, including curriculum composition, supervision thresholds, teacher choice, and training data size. These experiments help explain the behavior of the proposed method.
Critical weaknesses
  1. The methodological novelty could be better justified. The idea of adapting supervision according to the student's current performance is closely related to existing work on self-paced learning and online hard example mining. In SCC, the dynamic supervision strategy is essentially implemented through a tIoU-based threshold. While applying this idea to OPD is reasonable and potentially useful, the paper does not sufficiently clarify what is particularly new about this formulation compared with existing adaptive data selection strategies. A more explicit discussion of its distinction from prior work, especially Video-OPD, would strengthen the contribution.
  2. The role of Anchor reactivation is not entirely clear. According to the curriculum statistics in Appendix B.2, each Anchor example is presented only once during training. Therefore, reactivation appears to refer to an example that was initially handled correctly but receives supervision when encountered later. This is somewhat different from the broader motivation of suspending and reactivating supervision as the student's capabilities evolve. Moreover, since the routing decision is based on a single sampled prediction, a lower tIoU does not necessarily indicate that the student has forgotten a previously acquired capability. I would like to see stronger evidence that Anchor supervision actually helps preserve or recover learned capabilities.
  3. The static routing comparison does not fully isolate the benefit of dynamic supervision. In Appendix B.12, the static routing baseline matches the total supervision budget but uses a different distribution of Anchor and Frontier examples. Specifically, static routing supervises 49 Anchors and 443 Frontiers, while SCC supervises 11 Anchors and 481 Frontiers. Since Frontier examples are initially more difficult, part of the improvement may come from allocating more supervision to these examples rather than from adapting the routing decisions to the current student. A comparison that also matches the supervision distribution would help clarify this point.
  4. Some aspects of the experimental evaluation could be strengthened. Although SCC consistently improves performance, the gains over uniform OPD on the same curriculum are relatively small on several metrics. The paper does not report variance across random seeds, making it difficult to assess the stability of these improvements. Also, the reported training efficiency does not appear to include the offline computation required for curriculum construction. Since this stage involves evaluating both the teacher and the initial student, it would be useful to include this cost when discussing the overall computational savings.
Questions for authors
  1. Could the authors clarify the intended meaning of Anchor reactivation? Since each Anchor is presented only once during training, is there evidence that these examples actually help recover or preserve previously learned capabilities, rather than simply providing additional supervision?
  2. Could the authors provide a static routing comparison that matches both the total supervision budget and the Anchor–Frontier supervision distribution of SCC? This would help establish whether the gains primarily come from dynamic routing or from the resulting allocation of supervision.
  3. How consistent are the improvements across different random seeds? In particular, are the gains over Video-OPD on the same Anchor–Frontier curriculum stable across repeated runs?
  4. What is the computational overhead of constructing the Anchor–Frontier curriculum? How would the efficiency comparison change if the offline data selection costs of both SCC and Video-OPD were included?
7 comments

Dear Reviewer adFS,

We appreciate your careful review and constructive feedback. Below, we clarify SCC's methodological contributions and the role of Anchor supervision, and present additional evidence from matched static controls, multi-seed evaluations, and construction-inclusive efficiency analysis.

W1: Methodological Novelty and Relationship to Prior Work

Response: SCC builds on established adaptive data-selection principles. Its contribution is a supervision-allocation design for TVG OPD that combines a teacher-qualified Anchor–Frontier candidate space with task-level assessment of each current student rollout.

Selection criterion and computational placement. Classical self-paced learning selects low-loss examples, while online hard example mining and Selective-Backprop prioritize high-loss examples. SCC assesses the current rollout's grounding outcome before obtaining teacher token scores. Task success and teacher–student distributional agreement are distinct: a response can meet the grounding criterion while retaining a nonzero distillation loss (Appendix A.1). SCC therefore uses task-level assessment to determine whether teacher scoring and OPD should be performed, reusing the rollout already required by OPD. Selective computation itself is not our novelty claim; the contribution concerns how teacher supervision is efficiently allocated within the OPD process.

Relationship to Video-OPD and the AF design. Video-OPD combines teacher validation with teacher–student disagreement for candidate selection and applies OPD to every presentation. SCC jointly designs which supervision opportunities to retain and whether to use them as the student evolves. Its teacher-qualified AF space includes Frontiers for capability acquisition and Anchors that preserve corrective supervision opportunities for initially supported capabilities. The task-level decision governs their use as the student evolves, separating fixed candidate membership from active supervision. Together, these components align trustworthy supervision with evolving learning needs to support data- and compute-efficient TVG post-training.

Controlled evidence. We added Matched Random Routing, a static control that randomly preselects 492 of the 1,004 presentations (11 Anchor and 481 Frontier presentations) from the same AF curriculum and fixes these selections before training. The control also matches the reference SCC run's per-step supervision counts within each region. Averaged over three training seeds, SCC improves mean recall by 1.87 percentage points over this control, supporting the value of current-response assessment beyond matched supervision budget, composition, and timing. Detailed results are provided in our response to W3 and Q2.

We therefore position SCC as an empirically supported supervision-allocation design for TVG OPD. The tIoU threshold provides a simple implementation of the task-level decision within this coupled design. We have clarified the methodological distinctions from self-paced learning, online hard example mining, and Video-OPD in the revised manuscript (Section 6, lines 480–495; Appendix A.1, lines 790–809).

W2 & Q1: Anchor Reactivation and Capability Stabilization

Response: We clarify the meaning of Anchor reactivation and provide controlled evaluations of its contribution to downstream grounding performance and retention on the fixed Anchor set.

1. Meaning of Anchor Reactivation

Anchors retain corrective supervision opportunities for examples the initial student already handles well. Reactivation is defined relative to the initial competence assessment. An Anchor initially satisfies the competence criterion but receives supervision if its current rollout falls below τS=0.7\tau_S=0.7 at its training presentation. This definition is stated in Section 3.3 (lines 242–247) and Appendix A.3 (lines 877–882).

In the approximately single-pass setting, SCC performs dynamic, one-shot supervision routing at each presentation. The design does not repeatedly revisit individual examples to cycle through activation, suspension, and reactivation. The feedback loop operates across batches, with earlier updates shaping supervision decisions for later examples. We have clarified this design in the revised manuscript (Sections 3.3–3.4, lines 242–247 and 291–294).

2. Single-Rollout Assessment and Sampling Variability

A single low-scoring rollout does not, by itself, establish forgetting. The routing criterion identifies an observed task deficit that triggers supervision.

To examine sensitivity to this assessment, Appendix B.14 compares the default single-rollout decision with three- and five-rollout assessments using mean tIoU. Additional rollouts provide no consistent accuracy advantage across benchmarks and increase training time, supporting the practical single-rollout choice. Performance retention is evaluated separately through the fixed-Anchor comparison below.

[Continuation of W2 & Q1]

3. Controlled Evaluation of Anchor Supervision

We added an Anchor-muted control that retains the same 100 Anchors, 900 Frontiers, and presentation schedule but disables teacher scoring and OPD for all Anchors. Frontier routing continues to use the control model's current predictions. Student initialization, teacher, normalization, and the remaining training settings are unchanged.

Downstream grounding performance. Table 1 compares the two settings across training checkpoints. Mean mIoU is averaged across the three benchmarks, and mean recall across the nine recall metrics. Values are percentages; bold marks the higher score at each checkpoint.

Table 1. Effect of disabling Anchor supervision on downstream grounding performance.

CheckpointSettingMean mIoU (%)Mean recall (%)
20SCC53.4854.14
Anchor-muted SCC53.6454.25
40SCC55.7155.65
Anchor-muted SCC54.8154.57
59SCC55.6655.64
Anchor-muted SCC54.5454.22
79SCC55.7155.80
Anchor-muted SCC54.3854.04

With the curriculum and presentation schedule held fixed, disabling Anchor supervision weakens later-stage grounding performance. At the final checkpoint, SCC outperforms the control on all 12 benchmark metrics, improving mean mIoU by 1.33 percentage points and mean recall by 1.76 percentage points.

Retention on the fixed Anchor set. To assess retention on initially competent examples, we evaluate the same 100 Anchors from the AF curriculum at checkpoints 20, 40, 59, and 79, generating five fresh responses per Anchor. Table 2 reports mean tIoU and the percentage of rollouts satisfying tIoU≥0.7\mathrm{tIoU}\geq0.7, computed over 500 evaluation rollouts per checkpoint for each model. These evaluations do not contribute to training.

Table 2. Grounding performance on the fixed set of 100 Anchors.

CheckpointSettingMean tIoUSuccess rate (%)
20SCC0.937395.2
Anchor-muted SCC0.925893.4
40SCC0.938894.0
Anchor-muted SCC0.914891.4
59SCC0.932494.2
Anchor-muted SCC0.910891.0
79SCC0.932694.4
Anchor-muted SCC0.908990.8

SCC achieves higher mean tIoU and success rate at every evaluated checkpoint. From checkpoints 20 to 79, its success rate decreases by 0.8 percentage points, compared with 2.6 points for the Anchor-muted control. At the final checkpoint, SCC achieves 94.4% versus 90.8% success.

The fixed-Anchor comparison holds the evaluated examples constant and measures their performance beyond the initial-to-current threshold crossings used for routing. Together, the two analyses support the contribution of Anchor supervision to sustained downstream performance and retention on the selected Anchor set, consistent with its intended stabilizing role.

We have incorporated the benchmark comparison in Section 5 (Table 4, lines 433–454), the detailed TVG and fixed-Anchor evaluations in Appendix B.16 (Tables 25–26, lines 1418–1468), and clarified Anchor reactivation in Section 3.3 (lines 242–247) and Appendix A.3 (lines 877–882).

Reviewer_FZxKOpenReview ↗
Rating
2
Confidence
2
Informativeness
1
Presentation
2
Soundness
2
Read review
Paper summary

The authors propose Student–Curriculum Coupling (SCC), a framework for temporal video grounding. The method distinguishes supervision trustworthiness from supervision necessity, constructs a compact Anchor–Frontier candidate curriculum, and dynamically determines which samples receive teacher supervision based on the student’s current task performance. Experimental results show that the framework improves grounding performance while reducing training data requirements and computational costs.

Critical strengths

The improvements in performance and training efficiency brought about by dynamically adjusting the training process may be a strength of the paper.

Critical weaknesses

I currently still have questions about some implementation details of the training setup and the dynamic supervision mechanism. Since the dynamic supervision mechanism is a core contribution of the paper, these details are my main concern at this stage.

Questions for authors

I have a question about how many times each sample is checked in the paper. After looking at Tables 4 and 5 in the appendix, my understanding is that the experiments involve 1,000 unique samples with 1,004 presentations in total, with each of the 100 Anchors presented only once. Is this correct? If so, does this mean that training is approximately a single pass through the dataset? How, then, are activate, suspend, and reactivate carried out? Or is the same sample repeatedly revisited during training, with its supervision status switching? If my understanding is incorrect, could you further explain how sample visits and evaluations are scheduled?

Under the logic of the current algorithm, could the learning process get stuck in a cycle? For example, could a particular image repeatedly go through a cycle of activation, suspension, activation, and so on?

Appendix B.7 examines sensitivity to (\tau_S), but I could not find corresponding analyses for (\tau_T), (\tau_A), and (\tau_F). Could the authors further justify these threshold choices and assess how sensitive the accuracy and efficiency gains are to changes in them?

6 comments

Dear Reviewer FZxK,

We appreciate your careful review and constructive questions. Below, we clarify the training procedure and supervision-routing mechanism, and present additional analyses of curriculum-construction thresholds, including their effects on grounding performance and computational efficiency.

Q1: Clarification of Single-Pass Training and Supervision Routing

Response: Your understanding is correct. The main experiment uses 1,000 unique examples and 1,004 presentations, with each of the 100 Anchors presented once and four additional Frontier presentations in the final step. Training is therefore approximately a single pass.

The initial student is assessed offline to construct AF. At each presentation, a single rollout from the current student determines whether supervision is activated or suspended. Trajectories with tIoU below τS=0.7\tau_S=0.7 receive teacher scoring and OPD; otherwise, supervision is skipped. Therefore, for an Anchor, reactivation is defined relative to the initial assessment. The example initially met the competence criterion but receives supervision when its current rollout falls below the threshold. This definition is stated in Section 3.3 (lines 242–247) and further clarified in Appendix A.3 (lines 877–882).

Accordingly, SCC performs dynamic, one-shot supervision routing at each presentation. These supervision states do not imply repeated activation, suspension, and reactivation of individual examples. The feedback loop operates across batches, with earlier updates shaping supervision decisions for later examples. We have clarified this distinction in the revised manuscript (Sections 3.3–3.4, lines 229–295).

Q2: Clarification of Potential Supervision Cycles

Response: As clarified in Q1, the main experiment is approximately single-pass, and activation or suspension does not trigger additional sample visits. Training follows a fixed, finite schedule that advances after each batch, even when no OPD update is performed. Thus, supervision decisions cannot trap the reported training procedure in a cycle of repeated visits.

Q3: Construction-Threshold Rationale and Sensitivity

Response: We have expanded the threshold rationale and added a curriculum audit and controlled sensitivity analysis to the revised manuscript (Appendix B.2, lines 985–1043; Tables 7–9).

1. Rationale for the Chosen Thresholds

The following examples use an annotated interval of [0,10][0,10] seconds. Here, qT(x)q_T(x) and qS,0(x)q_{S,0}(x) denote the teacher's and initial student's tIoU scores against the annotation.

Teacher qualification (τT=0.7\tau_T=0.7). All 1,000 AF examples satisfy qT(x)≥0.7q_T(x)\geq0.7, requiring substantial temporal agreement before an example enters the candidate curriculum. Predictions of [0,7][0,7] and [0,8][0,8] score 0.7 and 0.8, respectively. Both capture most of the event without extending beyond it. Although the latter has a more accurate end boundary, the former already provides substantial overlap, motivating 0.7 as a practical qualification criterion.

Anchor selection (τA=0.7\tau_A=0.7). The 100 Anchors additionally satisfy qS,0(x)≥0.7q_{S,0}(x)\geq0.7, preserving corrective supervision opportunities for initially competent examples. Predictions of [0,7][0,7] and [0,8][0,8] both meet the online competence criterion τS=0.7\tau_S=0.7. Setting τA=τS\tau_A=\tau_S aligns initial Anchor qualification with the standard used during training, so supervision remains available when a current prediction falls below that standard.

Frontier selection (τF=0.3\tau_F=0.3). The 900 Frontiers satisfy qS,0(x)<0.3q_{S,0}(x)<0.3, providing acquisition opportunities where the teacher qualifies but the initial student exhibits substantial localization deficits. Predictions of [0,2.5][0,2.5] and [0,1.5][0,1.5] score 0.25 and 0.15, respectively; both miss most of the event. Lowering τF\tau_F to 0.2 would exclude the former despite its clear deficit. The chosen threshold therefore identifies substantial learning needs without restricting selection to the most extreme initial errors.

A further motivation is quicker curriculum curation. Compared with stricter filters, the chosen thresholds retain a broader eligible pool, potentially reducing the additional screening needed to fill source-specific selection quotas. We quantify this trade-off below.

2. Curriculum Audit and Controlled Variations

The AF curriculum used in the main experiments exhibits the intended separation in initial-student performance, with strong teacher grounding in both regions. Scores below are tIoU values on a 0–1 scale.

CohortUnique examplesMean qS,0q_{S,0}qS,0q_{S,0} rangeMean qTq_TqTq_T range
Overall1,0000.11790–10.93750.7–1
Anchor1000.96060.8390–10.97420.8–1
Frontier9000.02430–0.29950.93350.7–1

For each controlled variation, we retained original examples that remained eligible and replaced only those excluded by the revised criterion, preserving the selection budgets and source quotas.

All 100 selected Anchors already satisfy qS,0(x)>0.8q_{S,0}(x)>0.8. Under this protocol, raising τA\tau_A to 0.8 leaves the curriculum and training configuration unchanged. We therefore reuse the main-run results for this setting without additional training (Table 1).

For the other two settings, lowering τF\tau_F to 0.2 and raising τT\tau_T to 0.8 require replacing 52 and 92 examples, respectively. We trained both modified curricula while keeping the remaining thresholds, selection budgets, source quotas, and training protocol fixed. Each configuration retains 100 Anchors, 900 Frontiers, and τS=0.7\tau_S=0.7.

Table 1. Curriculum changes under stricter construction thresholds.

Setting(τT,τA,τF)(\tau_T,\tau_A,\tau_F)Examples replacedTraining comparison
Main configuration(0.7, 0.7, 0.3)—Reference SCC run
Stricter Anchor selection(0.7, 0.8, 0.3)0Unchanged configuration; main-run results reused
Lower Frontier threshold(0.7, 0.7, 0.2)52Additional training run
Stricter teacher qualification(0.8, 0.7, 0.3)92Additional training run

[Continuation of Q3]

3. Grounding Performance

Table 2 reports final-checkpoint results. All values are percentages; bold marks the best value for each metric within each benchmark.

Table 2. Grounding performance under the main and modified curricula.

BenchmarkSettingmIoU[email protected][email protected][email protected]
Charades-TimeLensMain configuration52.7973.9548.1433.01
Lower Frontier threshold, τF=0.2\tau_F=0.252.5874.0147.4932.86
Stricter teacher, τT=0.8\tau_T=0.852.9373.6248.2033.30
ActivityNet-TimeLensMain configuration50.6364.7849.1838.24
Lower Frontier threshold, τF=0.2\tau_F=0.250.4264.1648.7338.02
Stricter teacher, τT=0.8\tau_T=0.850.7764.7649.1638.62
QVHighlights-TimeLensMain configuration63.7076.7763.8554.25
Lower Frontier threshold, τF=0.2\tau_F=0.262.5675.5462.1752.43
Stricter teacher, τT=0.8\tau_T=0.863.5176.1263.6053.73

The results suggest that more restrictive curriculum selection does not necessarily yield more effective supervision. Lowering τF\tau_F to 0.2 reduces aggregate recall, suggesting that examples with less extreme but still substantial initial deficits remain useful learning opportunities. Raising τT\tau_T to 0.8 yields similar aggregate accuracy, with no consistent advantage across benchmarks.

4. Construction Cost and Training Efficiency

Table 3 reports candidate-screening requirements, construction time, and online training workload. Times are reported as hours:minutes:seconds.

Table 3. Curriculum-construction costs and online training workload.

SettingCandidates screenedConstruction timeTeacher-scored OPD routesTraining time
Main configuration6,5003:52:434921:00:26
Lower Frontier threshold, τF=0.2\tau_F=0.27,0004:07:024890:58:17
Stricter teacher, τT=0.8\tau_T=0.89,0005:21:304751:05:23

Relative to the main configuration, stricter filtering increases construction time by 6.2% for τF=0.2\tau_F=0.2 and 38.2% for τT=0.8\tau_T=0.8, supporting the practical curation advantage of the chosen thresholds. Online supervision remains sparse across the tested settings. Relative to scoring all 1,004 presentations, the main and modified curricula reduce teacher-scored OPD routes by 51.0%, 51.3%, and 52.7%, respectively. Training time varies from 0:58:17 to 1:05:23, with no consistent reduction from stricter filtering.

Together, these controlled comparisons support the main configuration as a practical balance between grounding accuracy, selective supervision, and curation cost. Raising τA\tau_A to 0.8 leaves the selected curriculum unchanged, while the tested teacher and Frontier restrictions require additional screening without improving aggregate recall over the main configuration.

Comments on the paper

2 comments

General Response

Dear Reviewers,

We appreciate your careful evaluations and constructive feedback. Your comments have helped us clarify the methodological contributions of Student–Curriculum Coupling (SCC) and strengthen the experimental evidence. We have revised the manuscript accordingly and prepared detailed, point-by-point responses to each review.

The principal revisions, highlighted in blue in the revised manuscript, are summarized below.

  • Methodological Contributions and Relationship to Prior Work. Clarified SCC's distinctions from self-paced learning, online hard example mining, and Video-OPD, emphasizing the joint design of a teacher-qualified Anchor–Frontier (AF) candidate space and student-dependent supervision realization (Section 6).

  • Training Procedure and Supervision Routing. Clarified the approximately single-pass training schedule, the meanings of supervision activation, suspension, and Anchor reactivation, and how the feedback loop operates across successive training batches (Sections 3.3–3.4 and 4.1).

  • Training Dynamics and Anchor Stabilization. Clarified the computation and interpretation of online grounding scores in Figure 4 and added unsmoothed training dynamics. Introduced an Anchor-muted control and fixed-Anchor retention evaluation to assess the contribution of Anchor supervision to sustained TVG performance and its intended role in capability stabilization, while clarifying the scope of this evidence (Sections 5 and 7; Appendices B.16–B.17, Tables 25–26, Figure 6).

  • Matched Static Controls and Multi-Seed Robustness. Introduced matched random routing and initial-score routing controls that preserve the reference SCC run's supervision budget, timing, and Anchor–Frontier allocation. We further evaluated SCC and the corresponding baselines across three independent training seeds, reporting mean performance and sample standard deviations (Section 4.3; Appendix B.13, Table 23).

  • Curriculum-Construction Threshold Sensitivity. Expanded the rationale for the AF construction thresholds and added a curriculum-score audit, controlled threshold variations, and analyses of their effects on grounding accuracy and computational cost (Section 4.1; Appendix B.2, Tables 7–9).

  • Offline Construction and Overall Efficiency. Quantified the screening and inference costs of AF and KL-based TVDF curriculum construction, and expanded the efficiency comparison to include both offline construction and training. The analysis distinguishes savings from curriculum construction and student-dependent supervision routing (Appendix B.3, Tables 10–12).

Together, these revisions provide a clearer account of SCC's design, a more precise interpretation of its training behavior, and additional controlled evidence supporting its effectiveness and efficiency. We believe the revised manuscript and our detailed responses address the reviewers' principal concerns.

Best regards,
The Authors

General Response

Dear Reviewers,

We appreciate your time and consideration. We have submitted detailed responses to your comments and revised the manuscript accordingly.

With approximately 3 days remaining before the author–reviewer discussion period ends, we would be grateful for any further feedback or remaining concerns you may have. Thank you again for your thoughtful reviews.

Best regards,
The Authors

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.