- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper proposes Flux-OPD, an iterative on-policy distillation framework that uses evolving contexts extracted from student trajectories as in-training supervision. It anchors the distillation target to a context-free teacher, introduces a correction derived from multiple context-conditioned teachers, and adjusts its strength using a reverse-KL-derived conflict score. Experiments on prompt optimization and medical question answering show improved aggregate performance over the evaluated OPD variants.
Critical strengths
- The evaluation covers distinct open-ended tasks, multiple student scales and downstream generators, together with relatively comprehensive ablations of the main components.
- The paper presents an interesting empirical finding: contextual supervision is more effective when applied as a controlled correction to a stable teacher and attenuated when contextual teachers disagree.
- The paper is clearly written, with well-organized and informative figures and tables.
Critical weaknesses
- The evaluation is limited to Qwen-family students of at most 4B, leaving scalability and broader generalizability unclear.
- The context-extraction procedure relies on a very small set of randomly sampled trajectories, raising concerns about the representativeness and robustness of the extracted supervision.
- The additional computational cost appears substantial relative to the reported gains, while the resulting cost–performance trade-off is not sufficiently evaluated.
Questions for authors
- Given the greater degradation at 4B than at 2B, can the authors evaluate Flux-OPD on students of 8B or larger?
- With only eight randomly sampled trajectories and no explicit selection for failures, is there sufficient error coverage to derive representative and comprehensive guidance? Could context extraction be improved by jointly synthesizing insights from all eight trajectories, with deduplication and conflict resolution, rather than accumulating them sequentially?
- How sensitive are the results to extraction-set size, update frequency, and trajectory order?
- How do the baselines perform when given the same additional compute as Flux-OPD? Do the modest gains justify the added cost, and could alternative configurations offer a better performance–compute trade-off?
3 comments
Thanks for the additional experiments. They address several of my concerns. I still have two reservations: the 8B results only cover VBench, without VideoVerse or the number of independent runs; and the compute comparison excludes the additional context extraction cost, which makes the overall cost relatively high for modest gains. Overall, the rebuttal is helpful, but these concerns still prevent me from increasing my score.
Thank you for your follow-up. We would like to further clarify two points:
1. 8B Results
We additionally evaluate the 8B model on VideoVerse, with the following results:
| Method | VACE | CogVideoX |
|---|---|---|
| Student | 1123 | 1125 |
| + OPD | 1102 | 1118 |
| + Flux-OPD | 1132 | 1129 |
The 8B experiments follow the same setting as Table 1, with one independent run.
2. Cost and Performance
As reported in our Compute-Matched OPD Comparison, even including context extraction (~25 GPU-hours in total), Flux-OPD outperforms OPD with 4× teacher forwards (~23 GPU-hours). This demonstrates that the improvements cannot be attributed solely to additional teacher computation.
Moreover, VBench is a relatively saturated benchmark where absolute score differences are typically small. Our method achieves performance competitive with or even surpassing the teacher in most settings, while other baselines often exhibit degradation. In comparison, VideoVerse shows larger absolute gains (~10–25 points over the OPD baseline), further demonstrating the effectiveness of our method.
Response to Reviewer 3yqQ
We appreciate your constructive feedback and positive assessment of our empirical findings, comprehensive ablations, and presentation. We address the concerns below.
W1/Q1. Student Scalability
Thank you for your valuable suggestions. We evaluate an 8B student with a 32B teacher on VBench across two video generators:
Flux-OPD improves the 8B student on both generators and outperforms OPD, showing scalability to larger students.
W2/Q2/Q3. Context Extraction
Thank you for your thoughtful question. Our core contribution is how to use evolving contexts for distillation instead of how to extract them. Our extraction procedure largely follows prior works (OPCD and OEL), with minimal task-specific engineering. We provide additional analyses using a 2B student on VBench with CogVideoX.
1. Trajectory Sampling and Extraction-Set Size
We clarify that 8 fresh student trajectories are sampled at each context update instead of reusing the same trajectories throughout training. We also evaluate extraction-set size:
All extraction-set sizes improve over the initial student, while sizes beyond 8 provide no further benefit. This supports our default choice as a favorable performance–cost trade-off.
2. Sequential Accumulation vs. Joint Synthesis
We clarify that sequential accumulation is not simple concatenation. Our extraction prompt (
Fig.6) explicitly instructs the teacher to avoid redundant or conflicting experience items when incorporating new items.Joint synthesis has a limitation: processing all trajectories in a single input is challenging, while sequential accumulation scales better with the number of trajectories. This is particularly useful for multimodal tasks such as video prompt optimization, where jointly processing eight video trajectories is difficult.
3. Sensitivity to Context Update Frequency
We evaluate different context update intervals:
Flux-OPD improves over the initial student at all update frequencies. Updating every 600 steps still yields +0.60, suggesting frequent updates are unnecessary for efficient training.
4. Sensitivity to Trajectory Order
Starting from the same initial student, we extract contexts from an identical trajectory set in three orders, then train each variant for 300 steps.
All three orders yield improvements, suggesting limited sensitivity to trajectory ordering in the evaluated initial-context setting.
Overall, these results support the effectiveness and practical robustness of our simple extraction procedure, without requiring larger extraction sets, frequent updates, or carefully optimized trajectory orders.
W3/Q4. Computational Cost and Trade-Off
Thank you for your valuable suggestions. We provide additional cost analysis and compute-matched comparisons to evaluate the performance–compute trade-off.
1. Training and Context Extraction Overhead
Although Flux-OPD requires 4 teacher forward passes per step, its training GPU-hour overhead is considerably smaller than 4×:
Context extraction additionally adds ~14 and ~9 GPU-hours, respectively. The higher extraction cost for video prompt optimization mainly stems from video generation and understanding.
2. Compute-Matched OPD Comparison
We evaluate OPD under larger compute budgets on VBench (2B, CogVideoX):
Flux-OPD outperforms OPD at matched training GPU-hours. Even including context extraction (~25 total GPU-hours), Flux-OPD surpasses OPD with 4× teacher forwards (~23 GPU-hours).
Thus, Flux-OPD's gains are not solely due to additional computation, but also benefit from more effective use of contexts.
3. More Efficient Configurations
Our additional analyses indicate that less frequent context updates can reduce extraction overhead, while smaller context histories (
Appendix C.4) can reduce computation while maintaining competitive performance, enabling flexible trade-offs.