acceptodds
Under review as a conference paper at ICLR 2027

VideoTeach: Do Video and Instructional Quality Translate into Learning Transfer?

Abstract

Generative models increasingly solve complex multimodal reasoning problems and present their solutions as instructional videos. Yet evaluating video-based teaching requires tracing the full path from the generated video, through teaching, to changes in learner performance. Existing benchmarks typically examine only one part of this path. We introduce VideoTeach, a benchmark of 1,176 teaching cases across five academic domains that evaluates all three layers at the problem level. It measures video artifact quality, instructional process quality, and a simulated student’s learning outcomes on an anchor problem, a surface-transfer problem and a structural-transfer problem. Evaluation across four representative systems yields two findings. First, video and instructional quality are almost unrelated to learning gains (|r|≤0.05). Second, transfer remains weak: anchor-problem gains reach +8.3 percentage points, whereas surface- and structural-transfer gains peak at +4.5 and +1.2 points and can be negative. This transfer gap persists with stronger teacher backbones, including GPT-5.5, Gemini-3-Pro, and Claude-Opus-4.7. Additional teaching turns mainly improve anchor-problem performance without producing consistent surface- or structural-transfer gains. Overall, high-quality instructional videos do not guarantee transferable learning.

Primary area:
datasets and benchmarks
Keywords:
instructional video generation, multimodal evaluation, learning transfer, simulated student agents, multi-agent evaluation

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_kugWOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper presents VideoTeach, a benchmark with 1,176 teaching cases across five domains. The benchmark evaluates instructional videos on three layers: video quality, teaching quality, and learning outcomes. Each case has an anchor problem plus a surface transfer problem and a structural transfer problem. The authors evaluate four video generation systems and report two findings. First, video quality and teaching quality scores are almost uncorrelated with learning gains (|r| <= 0.05). Second, all systems improve performance on the anchor problem (up to +8.3 pp), but transfer is weak: surface transfer peaks at +4.5 pp and structural transfer at +1.2 pp. This gap persists when using stronger teacher models (GPT-5.5, Gemini-3-Pro, Claude-Opus-4.7) and more teaching turns.

Critical strengths
  1. The three layer evaluation is a methodological advance. Previous benchmarks measure video quality, teaching quality, and learning separately, so they cannot test whether upstream quality actually leads to learning. The controlled transfer distances also let the authors separate memorizing the taught solution from understanding the underlying concept.
  2. The protocol includes several thoughtful controls: the NO-VIDEO baseline rules out retest effects, and the common-success subset separates generation reliability from output quality.
  3. The main finding is replicated across systems, teacher backbones, and interaction lengths. This consistency across independent experimental axes makes it unlikely to be an artifact of one particular setup.
Critical weaknesses
  1. My main concern is that all learning outcomes come from a single simulated student (Qwen2.5-VL-7B). The near-zero structural transfer gains may reflect the student model's limited ability rather than bad teaching. The 12 person human study is too small to settle this. I would suggest rerunning the main analysis with one or two additional student models of different capability (e.g., a 72B-class VLM), which should be feasible without changing the benchmark itself. Alternatively, the claims could be scoped more explicitly to simulated learners.
  2. The headline gains have no confidence intervals or significance tests. Some effects are only 1 to 4 pp, so it is hard to tell real effects from noise. Since case-level scores should already be logged, adding bootstrap confidence intervals or a paired test over cases would strengthen the paper at low cost. The correlation analysis would also be more convincing if computed per transfer problem instead of on averaged gains.
  3. GPT-5.5 serves as both evaluator and, in some experiments, teacher. This raises a self-preference concern for the absolute L1/L2 scores, although it does not affect the L3 gains and is therefore unlikely to change the main conclusions. A small human spot check of L1/L2 scores on a sample of cases would help calibrate the absolute values.
Questions for authors
  1. Were the main results obtained from a single run of the teacher, student, and evaluator models, or from multiple repetitions? If multiple, how much do the L3 gains vary across runs?
  2. How sensitive are the L1 scores to the two-second frame sampling used by the evaluator?
4 comments
AuthorsOpenReview ↗

Response to Reviewer kugW (Part 1/3)

Dear Reviewer kugW,

Thank you for your feedback, we address your concerns as follows:

W1: Limited transfer may reflect the single student model’s capabilities rather than teaching limitations.

First, We add two student models, Gemma-3-4B-IT and LLaVA-OneVision-7B, alongside the original Qwen2.5-VL-7B-Instruct. All three were evaluated on the same 300 VideoTeach-Lite cases using MONO-generated teaching videos.

Student modelAnchor: T0pre→T0postT_0^{pre} \rightarrow T_0^{post} (ΔT0\Delta T_0)Surface transfer: T1pre→T1postT_1^{pre} \rightarrow T_1^{post} (ΔT1\Delta T_1)Structural transfer: T2pre→T2postT_2^{pre} \rightarrow T_2^{post} (ΔT2\Delta T_2)
Gemma-3-4B-IT48.67% → 56.00%(+7.33 pp)65.00% → 63.33%(−1.67 pp)59.33% → 58.00%(−1.33 pp)
Qwen2.5-VL-7B-Instruct47.67% → 51.67%(+4.00 pp)62.33% → 60.67%(−1.67 pp)62.67% → 59.67%(−3.00 pp)
LLaVA-OneVision-7B25.00% → 32.00%(+7.00 pp)36.33% → 36.00%(−0.33 pp)37.00% → 37.00%(0.00 pp)

Gemma-3-4B-IT, Qwen2.5-VL-7B-Instruct, and LLaVA-OneVision-7B show anchor gains of +4.00 to +7.33 pp but no positive transfer gains despite substantial headroom. Thus, limited transfer is not unique to the original student.

Second, we test whether the original student can achieve transfer under improved teaching conditions.

We develop a human-guided video refinement framework. A Claude Opus-5.5 teacher explains the anchor solution and two worked examples using newly generated problem variants, separate from the benchmark’s transfer problems. Qwen2.5-VL-7B-Instruct watches the video and then answers these practice questions. A separate Claude Opus-5.5 critic reviews the video’s visual quality and the student’s answers, providing feedback to a Codex modifier using GPT-6.1-sol. The modifier revises the video-generation code over three rounds. A human then selects the best video from the initial version and the three revisions. Human feedback informed the framework’s design, while the final T1T_1/T2T_2 questions remained hidden throughout refinement and selection.

With Qwen2.5-VL-7B-Instruct fixed, results on the same 100 cases are:

ConditionL1L2ΔT0\Delta T_0 (pp)ΔT1\Delta T_1 (pp)ΔT2\Delta T_2 (pp)
Original MONO72.3452.89+9+3+3
Human-guided refined MONO74.7057.90+11+10+9

Refinement increases surface-transfer gains by 7 pp and structural-transfer gains by 6 pp. The student therefore achieves greater transfer when the teaching condition is improved, indicating that the original videos’ limited gains cannot be explained solely by a fixed inability to learn from video.

Together, the multi-student evaluation and the fixed-student refinement comparison provide complementary evidence: limited transfer is neither unique to Qwen2.5-VL-7B-Instruct nor explained solely by its capabilities. We have included the experiments in Section 4.3 and Appendix H.9.

AuthorsOpenReview ↗

Response to Reviewer kugW (Part 2/3)

W2: Uncertainty in learning gains and correlations by transfer target.

We added pointwise 95% paired-bootstrap confidence intervals for learning gains on the common-success subset, using 20,000 case-level resamples and preserving pre/post pairing.

MethodΔT0\Delta T_0 (pp)ΔT1\Delta T_1 (pp)ΔT2\Delta T_2 (pp)
MONO+8.32 [5.90, 10.53]+2.84 [−1.14, 4.64]−0.95 [−4.74, 2.85]
FREE+5.48 [3.02, 8.13]+0.95 [−3.04, 4.93]+0.76 [−3.04, 4.55]
SCAF+3.97 [1.51, 5.43]+4.54 [0.76, 6.33]−0.76 [−4.36, 2.84]
T2V+0.76 [−0.95, 2.46]−0.76 [−4.55, 3.03]+1.13 pp [−1.14, 4.82]

These results support our main finding: current code-based methods improve anchor performance, but transfer gains remain limited. All three code-based methods show anchor gains whose 95% confidence intervals exclude zero. Only SCAF shows a modest surface-transfer gain (+4.54 pp, 95% CI [0.76, 6.33]); no structural-transfer interval excludes zero.

We also compute within-system, item-level correlations separately for each target, without averaging gains:

MethodQuality metricrr with ΔT0\Delta T_0rr with ΔT1\Delta T_1rr with ΔT2\Delta T_2
MONOL1: Video artifact−0.0070.042−0.033
MONOL2: Instructional process0.078−0.0110.084
FREEL1: Video artifact0.009−0.0200.066
FREEL2: Instructional process−0.068−0.085−0.050
SCAFL1: Video artifact−0.006−0.0010.016
SCAFL2: Instructional process0.150-0.029-0.013
T2VL1: Video artifact0.0060.034−0.078
T2VL2: Instructional process−0.004−0.0620.027

Across all the quality–learning comparisons, Pearson correlations remain close to zero (|r| ≤ 0.15). The weak quality–learning association persists when each learning target is analyzed separately.

We report the confidence intervals in Table 16 and target-specific correlations in Table 17 of Appendix H.

W3: Potential self-preference bias in GPT-5.5’s L1/L2 evaluations requires human calibration.

We randomly sample four cases from each of the five domains, yielding 20 cases and 60 videos generated by the three teacher backbones. Three human evaluators independently scored L1 and L2 using the same rubrics.

Teacher backboneL1: EvaluatorL1: HumanL2: EvaluatorL2: Human
GPT-5.567817570
Claude Opus-4.779906563
Gemini-3-Pro80936862

For L1, both evaluations support Gemini ≈ Claude > GPT-5.5. For L2, both support GPT-5.5 > Claude ≈ Gemini, with GPT-5.5 leading by 7–8 points under human assessment and 7–10 points under automatic evaluation. Thus, GPT-5.5’s discrepancy is not uniquely large, and this spot check provides no clear evidence that self-preference drives its leading L2 score.

Human L1 scores are consistently 11–14 points higher. This reflects different interpretations of rubric severity—for example, humans may assign smaller penalties to minor rendering defects. Importantly, the L1 ordering is unchanged. We have included the results in Appendix H.11.

Q1: Were the main results obtained from a single run of the teacher, student, and evaluator models, or from multiple repetitions? If multiple, how much do the L3 gains vary across runs?

The original results were obtained from a single run of the teacher–student–evaluator pipeline. We additionally performed three repetitions of the student pre-test and post-test evaluations on the same cases, keeping the generated teaching videos fixed. The table reports mean gains ± sample standard deviation across these three repetitions, in percentage points.

MethodΔT0\Delta T_0 (pp)ΔT1\Delta T_1 (pp)ΔT2\Delta T_2 (pp)
MONO+6.21 ± 2.93+2.82 ± 2.59+1.69 ± 1.69
FREE+5.04 ± 0.98+2.52 ± 3.18+1.13 ± 2.59
SCAF+3.39 ± 2.94+5.34 ± 2.59−2.82 ± 2.06
T2V+2.82 ± 2.18+1.52 ± 2.59−0.56 ± 0.98

Across methods, mean structural-transfer gains range from −2.82 to +1.69 pp, compared with positive anchor gains of +2.82 to +6.21 pp. Thus, the repeated evaluations retain the observed pattern of limited mean structural-transfer gains. We have included the results in Table 20 and Appendix H.12.

AuthorsOpenReview ↗

Response to Reviewer kugW (Part 3/3)

Q2: Sensitivity to frame sampling.

We rescore the same videos at 1-, 2-, and 4-second intervals, keeping the evaluator and rubric fixed. L1 scores are on a 0–100 scale.

Sampling intervalMONOFREESCAFT2VMean L1
1 second49.3455.2046.1340.5447.80
2 seconds50.2758.5347.7340.9249.36
4 seconds53.3460.2751.4742.2651.83

Relative to 2-second sampling, mean L1 changes by −1.56 points at 1-second intervals and +2.47 points at 4-second intervals. The method ranking remains FREE > MONO > SCAF > T2V under all three settings. Thus, sampling affects absolute scores, but the observed system ranking is stable across the tested intervals. We have included this sensitivity analysis in the revised paper. We have included the results in Appendix H.13.

If the response addresses your concerns, could you consider increasing the score?

AuthorsOpenReview ↗

Response to Reviewer kugW (Follow up)

Dear Reviewer kugW,

Thank you for confirming that your concerns have been addressed. In light of this, would you consider raising your score?

Best regards,

The Authors

Reviewer_6tRbOpenReview ↗
Rating
1(−1)
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper proposes VideoTeach, a benchmark that brings together video quality, teaching quality, and learning outcomes. It measures improvement on the original question, a similar question with changed details, and a question that requires applying the same knowledge differently. These ideas build on earlier work, so the main contribution is bringing them together in one evaluation. The main takeaway is that better video and teaching-quality scores do not consistently lead to better learning. In their experiments, improvement is mostly on the original question, with limited gains on the other two, even with stronger teachers or more teaching rounds.

Critical strengths
  1. Bringing video quality, teaching quality, and learning outcomes together in the same cases is a good contribution, as it allows to test whether better video and teaching quality scores actually lead to better student performance.

  2. The comparison across generation methods is useful because it considers both how well the videos teach and how reliably each method produces them.

  3. The ablations on with different teacher models, and the experiments with more teaching rounds, are also useful.

Critical weaknesses
  1. The main concern is that this is a benchmark paper proposing an evaluation method, so its main message that "better video and teaching quality do not consistently lead to better learning on new questions", needs stronger evidence and evaluation. The paper uses only one model for its AI-student experiments: Qwen2.5-VL-7B-Instruct. Improvement depends not only on how useful the lesson is, but also on the student model's video understanding capability, its ability to apply the explanation, and its existing world or concept knowledge. The authors should show whether this trend holds across at least 3 to 5 student models, including newer and larger models with different reasoning capabilities, such as Qwen3-VL, Gemini 3 Pro, or Flash. Otherwise, it is unclear whether the findings reflect a general limitation of the teaching or the limitations of this particular student model.

  2. What is the relevance of the benchmark for stronger models? How many questions can frontier models already solve without teaching, using their existing knowledge and reasoning abilities, and how much do they improve afterward? I do not see a smaller improvement with stronger models as a problem. But if a model already answers most questions correctly before teaching, a small gain afterward says little about the quality of the lesson and the benchmark. The paper should report these results to show how much room the benchmark provides to measure learning from instruction.

  3. Using only videos without audio seems like a major limitation, especially for a benchmark targeting a real-world use case like teaching.

Questions for authors

Please address the questions raised in the weaknesses above, particularly why only one student model is evaluated and whether the benchmark remains a meaningful measure of teaching effectiveness for stronger models.

8 comments
AuthorsOpenReview ↗

Response to Reviewer 6tRb (Part 1/2)

Dear Reviewer 6tRb,

Thank you for your feedback, we address your concerns as follows:

W1: Limited evaluation across student models raises concerns about the generalizability of the findings.

We add two experiments to test whether limited transfer persists across student models and whether better teaching improves transfer in Qwen2.5-VL-7B-Instruct.

First, we evaluate learning gains on four student models to test whether limited transfer is specific to Qwen2.5-VL-7B-Instruct. All students were evaluated on VideoTeach-Lite using MONO-generated videos. We report pre-test accuracy, post-test accuracy, and gains separately for each problem type:

Student modelAnchor: T0pre→T0postT_0^{pre} \rightarrow T_0^{post} (ΔT0\Delta T_0)Surface transfer: T1pre→T1postT_1^{pre} \rightarrow T_1^{post} (ΔT1\Delta T_1)Structural transfer: T2pre→T2postT_2^{pre} \rightarrow T_2^{post} (ΔT2\Delta T_2)
Qwen3-VL-Plus83.00% → 82.67%(−0.33 pp)95.00% → 91.33%(−3.67 pp)90.67% → 88.00%(−2.67 pp)
Gemma-3-4B-IT48.67% → 56.00%(+7.33 pp)65.00% → 63.33%(−1.67 pp)59.33% → 58.00%(−1.33 pp)
Qwen2.5-VL-7B-Instruct47.67% → 51.67%(+4.00 pp)62.33% → 60.67%(−1.67 pp)62.67% → 59.67%(−3.00 pp)
LLaVA-OneVision-7B25.00% → 32.00%(+7.00 pp)36.33% → 36.00%(−0.33 pp)37.00% → 37.00%(0.00 pp)

Gemma-3-4B-IT, Qwen2.5-VL-7B-Instruct, and LLaVA-OneVision-7B all improve on the anchor problem but show no positive transfer gains despite substantial headroom. This recurring pattern across different model families suggests that limited transfer is not specific to Qwen2.5-VL-7B-Instruct, but is a shared challenge for video teaching. Qwen3-VL-Plus also shows no positive gains, although its near-saturated transfer baselines make it less informative for distinguishing teaching limitations from student limitations.

Second, we improved the teaching videos to test whether Qwen2.5-VL-7B-Instruct’s limited transfer reflects its own capabilities or shortcomings in the instruction. We develop a human-guided video refinement framework. A Claude Opus-5.5 teacher explains the anchor solution and two worked examples using newly generated problem variants, separate from the benchmark’s transfer problems. Qwen2.5-VL-7B-Instruct watches the video and then answers these practice questions. A separate Claude Opus-5.5 critic reviews the video’s visual quality and the student’s answers, providing feedback to a Codex modifier using GPT-6.1-sol. The modifier revises the video-generation code over three rounds. A human then selects the best video from the initial version and the three revisions. Human feedback informed the framework’s design, while the final T1T_1/T2T_2 questions remained hidden throughout refinement and selection.

With Qwen2.5-VL-7B-Instruct fixed, results on 100 randomly sampled cases are:

ConditionL1L2ΔT0\Delta T_0 (pp)ΔT1\Delta T_1 (pp)ΔT2\Delta T_2 (pp)
Original MONO72.3452.89+9+3+3
Human-guided refined MONO74.7057.90+11+10+9

With the student model fixed, refinement increases surface-transfer gains by 7 pp and structural-transfer gains by 6 pp. These improvements demonstrate that Qwen2.5-VL-7B-Instruct can learn transferable knowledge from video, so the original results cannot be attributed solely to the student’s limitations.

Together, these experiments suggest that limited transfer is neither unique to Qwen2.5-VL-7B-Instruct nor explained solely by its capabilities. We have included the results in section 4.3 and Appendix H.10.

AuthorsOpenReview ↗

Response to Reviewer 6tRb (Part 2/2)

W2: The benchmark’s relevance to frontier models is unclear, as near-saturated pre-test performance could leave little room to measure learning.

We agree that learning gains should be interpreted relative to pre-test performance. We evaluate strong student model Qwen3-VL-Plus on all 300 VideoTeach-Lite cases and report its pre/post results below:

Problem typeCorrect before instruction (TipreT_i^{pre})Correct after instruction (TipostT_i^{post})Net change (ΔTi\Delta T_i)
Anchor (T0T_0)249/300 (83.00%)248/300 (82.67%)−1 question (−0.33 pp)
Surface transfer (T1T_1)285/300 (95.00%)274/300 (91.33%)−11 questions (−3.67 pp)
Structural transfer (T2T_2)272/300 (90.67%)264/300 (88.00%)−8 questions (−2.67 pp)

Qwen3-VL-Plus already solves 83.00–95.00% of the questions before instruction, leaving limited room for improvement, particularly on transfer problems. We therefore do not use its small negative gains alone as evidence of ineffective teaching. We have included the results in Appendix H.7.

VideoTeach evaluates video teaching effectiveness, which requires students with sufficient pre-test headroom. As reported in W1, three students with lower baselines consistently show anchor gains without positive transfer gains. This agreement suggests that the observed transfer limitation is not confined to a single student model.

W3: Using only videos without audio seems like a major limitation, especially for a benchmark targeting a real-world use case like teaching.

We agree that real-world teaching combines visuals and narration. VideoTeach currently evaluates visual instruction separately to identify how effectively the video itself supports learning.

To examine whether narration affects our findings, we use teacher-generated transcripts as a proxy for its verbal content, approximating a setting where spoken narration is transcribed into text before being passed to the student model. We compare video-only and video+text instruction for MONO and T2V, keeping the teacher and student models fixed.

ConditionAnchor (ΔT0\Delta T_0)Surface transfer (ΔT1\Delta T_1)Structural transfer (ΔT2\Delta T_2)
MONO7.83.4−0.3
MONO + text15.84.50.0
T2V1.20.2−0.3
T2V + text15.44.30.1

Adding text reduces the MONO–T2V gap from 6.6 to 0.4 pp on anchor problems and from 3.2 to 0.2 pp on surface transfer. T2V often produces garbled text and mathematical symbols, while MONO renders notation more legibly (Section 4.2 and Table 9). The narrowing gap suggests that accompanying text may compensate for visual weaknesses, making them less apparent in combined-input evaluation.

These results support evaluating visual instruction separately: similar combined gains do not imply equally effective videos. Full audiovisual evaluation is a promising direction for future work. A necessary first step is to understand the strengths, limitations, and challenges of visual instruction itself, which VideoTeach is designed to assess. We have included this discussion in Appendix H.9.

If the response addresses your concerns, could you consider increasing the score?

Thank you for the detailed responses and additional experiments. I appreciate the effort, but I still have some concerns. In W1, the multi-student evaluation supports the original finding of limited transfer. However, the human-refined videos show a different result, with substantial improvements on T1 and T2. This raises questions about the central message that "better video and teaching quality do not consistently lead to better learning". If better teaching can substantially reduce the transfer gap, would the main conclusion still hold on the full benchmark? Another concern is that these experiments use different subsets and settings (1,176 vs. 300 vs. 100 cases), and it is unclear how well these findings generalize. For a benchmark paper, I think this is important to establish.

Regarding W2, the Qwen3-VL-Plus results actually confirm my concern, with pre-test accuracy already reaching 95% and 90.67% on T1/T2. Testing on SOTA/frontier video-understanding models is still missing (eg Gemini-3-Pro/Flash). Therefore, it remains unclear whether the benchmark would also be saturated for these stronger models, leaving little room to measure learning improvements. Regarding W3, the video+text experiment is useful, but it does not address my concern about missing audio in real-world teaching videos. Overall, I appreciate the additional experiments, but I don't think my main concerns are fully resolved. I will keep my original score.

Reviewer_9LNROpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes VideoTeach, a benchmark with 1176 test cases. It evaluates AI generated video quality, instructional quality and learning gains on anchor and transfer problems. Using Qwen 2.5-VL-7B as the student, the experiments show that the learning gains is poorly correlated with the video visual quality. Student's context learning ability of transfering to new questions is also limited.

Critical strengths
  1. The work is interesting, presenting a VideoTeach benchmark, testing the video context learning ability of a LLM.
  2. The question setting is reasonable. The benchmark includes anchor, surface-transfer and structural transfer questions, allowing it to distinguish performance gains on taught problems form generalization to new problems
Critical weaknesses
  1. The paper reports pre-to-post gains. My concern is that could high pre-test accuracy leave limited room for improvement for both human and model? The human structural-transfer results are zero in Figure 8 across four systems.
  2. In the multi-round teaching experiments, limited learning gains could come from two sources: the teacher’s explanation may be insufficient, or the generated video may fail to convey it clearly. With the student model fixed, comparing the same teaching content in text and video form would help identify where the problem arises.
  3. Including a few benchmark cases in the main body would improve readability. For example, a case on anchor problem, surface-transfer problem and structural transfer problem, and the teaching AI video example.
  4. Human validation of the video artifact quality scores would help establish how well they align with human judgments.
  5. A useful addition would be a teaching-video baseline that is iteratively refined through human feedback. If these videos produce clear transfer gains in the student model, this would demonstrate that effective video teaching is achievable under this benchmark setup, even if current generation methods fall short. Such a baseline could also provide concrete guidance for improving AI-generated teaching videos.
Questions for authors

Please address the weakness mentioned above

10 comments
AuthorsOpenReview ↗

Response to Reviewer 9LNR (Part 1/2)

Dear Reviewer 9LNR,

Thank you for your feedback, we address your concerns as follows:

W1: Risk of limited improvement due to high pre-test accuracy.

Thank you for raising this concern. The pre-test accuracies below indicate substantial room for improvement for both humans and the student model:

StudentAnchor (T0preT_0^{pre})Surface transfer (T1preT_1^{pre})Structural transfer (T2preT_2^{pre})
Qwen2.5-VL-7B-Instruct44.33%56.67%56.33%
Human participants49.67%51.33%53%

These results suggest that a ceiling effect is unlikely to explain the zero human structural-transfer gains in Figure 8. One explanation is that the videos focus on the anchor solution without sufficiently explaining how to apply the underlying principles to differently structured problems. They highlight why VideoTeach distinguishes anchor improvement from transfer: success on the taught problem does not necessarily demonstrate transferable learning. We have added these pre-test accuracies to Table 11 and the corresponding discussion to Appendix H.3.

W2: Comparing the same teaching content in text and video form to distinguish inadequate explanations from ineffective video presentation.

We compare text- and video-based instruction on VideoTeach-Lite, keeping the student model fixed and using GPT-5.5 as the teacher. After generating each video, the teacher produces a corresponding textual transcript, which serves as the text-only instruction to align the teaching content across formats. Pre-to-post gains are reported below in percentage points.

ConditionProblem typeTurn 1Turn 2Turn 3Turn 4Turn 5
VideoAnchor (ΔT0\Delta T_0)7.811.111.112.815.2
TextAnchor (ΔT0\Delta T_0)15.213.410.213.79.5
VideoSurface transfer (ΔT1\Delta T_1)3.42.413.42.7
TextSurface transfer (ΔT1\Delta T_1)6.16.67.411.23.2
VideoStructural transfer (ΔT2\Delta T_2)−0.31.41.7−3.0−1.7
TextStructural transfer (ΔT2\Delta T_2)0−1.23−2.9−0.5

Text outperforms video in three of five turns for anchor problems (T0T_0) and all five turns for surface transfer (T1T_1). Because the text is derived from the corresponding video, this consistent advantage suggests that video presentation fails to explain the teaching content clearly. Structural-transfer (T2T_2) gains remain small in both formats, suggesting that the teacher’s explanation itself provides insufficient support for generalization. We have added these results to Table 14 and the corresponding discussions to Appendix H.8.

This comparison strengthens VideoTeach’s diagnostic value. Text–video comparisons of L3 help identify presentation barriers and challenges shared by both formats. Together, they guide improvements in visual clarity and teaching for generalization.

W3: Include benchmark examples in the main text to improve readability.

We have added an illustrative case study to Section 4.3 (Analysis). Figure 7 presents an anchor problem, its surface- and structural-transfer variants, two frames from the AI-generated teaching video, and the student’s pre- and post-test answers. The student corrects the anchor answer after instruction, remains correct on the surface-transfer problem, and still answers the structural-transfer problem incorrectly.

For convenience, the problem triplet is reproduced below.

Problem typeExample
Anchor (p0p_0)Find the area of quadrilateral ABCD with vertices A(3, 7), B(3, 5), C(−1, 1), and D(−1, 3).
Surface transfer (p1p_1)Find the area of quadrilateral ABCD with vertices A(5, 10), B(5, 7), C(−1, 1), and D(−1, 4).
Structural transfer (p2p_2)Find the area of parallelogram ABCD with vertices A(2, 8), B(5, 4), C(1, 0), and D(−2, 4).

The surface-transfer problem p1p_1 changes the coordinates while preserving the configuration with vertical opposite sides, so the demonstrated base–height calculation remains directly applicable. The structural-transfer problem p2p_2 uses oblique parallel sides, requiring the learner to determine the perpendicular height rather than directly reuse the coordinate differences illustrated in the lesson. This case illustrates the distinction between correcting the taught solution and adapting the underlying principle to a new configuration. Additional problem triplets appear in Appendix B.1, and further video frames with corresponding learning outcomes appear in Figure 11.

AuthorsOpenReview ↗

Response to Reviewer 9LNR (Part 2/2)

W4: How aligned are the video artifact quality scores with the human judgements?

We conduct a human validation study on 200 videos generated by MONO, FREE, SCAF, and T2V for the same 50 benchmark cases. Three human raters independently assessed each video using the L1 video quality rubric. Their ratings were averaged to obtain a human reference score for each video. The mean scores on the 0–100 scale are reported below:

SystemEvaluator modelHuman
MONO6278.33
FREE6476.67
SCAF5872
T2V4050.33

Human scores are higher on average, but both assessments support the ordering MONO ≈ FREE > SCAF > T2V. These results show that the evaluator’s rankings broadly align with human judgments (video-level Kendall’s τ-b = 0.62). We have included the results in Table 18 and Appendix H.11.

W5: Add a human-feedback-refined teaching-video baseline to test whether VideoTeach can capture successful transfer and identify actionable improvements to current generation methods.

Following this suggestion, we develop a human-guided video-refinement framework on 100 cases. We first generate baseline videos using MONO with Claude Opus-5.5. Through repeated cycles of human video inspection, framework revision, and video regeneration, we developed the following procedure:

  1. Teaching-video generation. A Claude Opus-5.5 teacher generates a video explaining the anchor solution and two additional worked examples that it devises from the anchor problem.

  2. Critic feedback. A separate Claude Opus-5.5 critic inspects the video for issues such as overlapping elements and discontinuous presentation. It also calls Qwen2.5-VL-7B-Instruct as a student to answer the anchor problem and the two additional examples after viewing the video. The critic uses these responses and its visual inspection to provide revision suggestions.

  3. Code revision. A modifier implemented with Codex using GPT-6.1-sol revises the video-generation code according to the critic’s feedback and renders a new video. Critique and code revision are repeated for three rounds.

  4. Human selection. Each case yields four candidates: the initial video and three revised versions. A human selects the best candidate as the final teaching video.

Human feedback therefore guides the framework design and final video selection, while the critic–revision loop performs case-specific refinement. Compared with baseline MONO on the same 100 cases, the refined videos achieve:

ConditionL1L2ΔT0\Delta T_0 (pp)ΔT1\Delta T_1 (pp)ΔT2\Delta T_2 (pp)
Original MONO72.3452.89+9+3+3
Human-guided refined MONO74.7057.90+11+10+9

Surface-transfer gains increase from +3 to +10 pp, and structural-transfer gains from +3 to +9 pp. These improvements suggest that effective video teaching is achievable under this benchmark setup. We have included these results in section 4.3 and the details in Appendix H.10.

If the response addresses your concerns, could you consider increasing the score?

Reviewer_9LNROpenReview ↗

Reviewer Response

Hello, authors.

Thank you for the careful responses.

The responses have addressed most of my concerns.

To W2, I think the setup is a little bit different from what I have expected. What I have expected is that in a video generation scenario, text to video generation, denoted as T2V, we JUST DIRECTLY use the T here, instead of using the generated video (V) with a diffusion model to teach the student. The author's rebuttal setup is to firstly use the diffusion model to make videos through T2V,and a vision model to make texts through V2T,DIRECTLY USE THE text in T2V would be more reasonable, I think.

Another unresolved concern is that in W1, the pre-test acc does not has a ceiling effect, which is clear. Human participants' transfer gain is minimal due to bad teaching video's quality, which sounds fair. In W5, the author added the baseline that use human-in-the-loop to make a better teaching videos to teach the students, which is also clear.

However, this is where things are a bit odd, human participants know how to make better teaching videos to teach student models to answer these transfer questions, but they cannot answer the transfer questions with a low-quality teaching video.

A more reasonable and consistent experiment results are like this: on anchor and transfer questions, human participants have high accuracy, the models have low accuracy. The transfer gains for both of them are small, but the reasons are different. For the model, it is because the teaching video is low quality, for the human participant, it is because of the ceiling effect, regardless of the quality of the video, human participants can answer them correctly. For human in the loop creation of teaching video. A better video is created, model's transfer gains are larger.

I hope I have made myself understood.

Best regards,

Comments on the paper

1 comment
AuthorsOpenReview ↗

Kind Reminder of the Discussion Period End

Dear Reviewers,

Thank you for your thoughtful feedback and the discussion so far. We appreciate the acknowledgments that our responses and additional experiments have addressed the concerns raised.

As the discussion period is drawing to a close, we would appreciate your review of our latest clarifications and results. If any concerns remain that affect your assessment, please let us know so that we can address them before the discussion ends. We also kindly invite you to reconsider your scores in light of the additional evidence and the concerns resolved during the discussion.

Thank you again for your time and effort in reviewing our work.

Best regards, The Authors

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.