acceptodds
Withdrawn from ICLR 2027

LessonStudio: Agentic Environment for Interactive Educational Content Generation

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Agentic AI systems are increasingly used to generate educational content such as figures, slides, and videos, yet most existing systems still treat lessons as static artifacts that learners passively consume after generation. We introduce LessonWorld, an environment for human-AI collaborative authoring of interactive, executable educational content. LessonWorld represents lessons as dynamic state machines in which students can manipulate 3D objects, write mathematical expressions, interact with visualizations, and ask questions, with the lesson pathway adapting to their actions. LessonWorld consists of three components: (1) LessonEngine, a TypeScript-based authoring platform with a stateful runtime and programmatic primitives for constructing and modifying interactive lessons; (2) LessonAgent, a distributed agentic pipeline that grounds lessons in instructional resources, generates interactive figures, composes lessons, and iteratively validates and adapts them; and (3) LessonBench, a benchmark spanning challenging scientific demonstrations and spatially complex figures, with automated VLM-based pairwise evaluation and human evaluation of over 2,000 figure/demo pairs and 960 minutes of educational videos and interactive lessons. Across benchmark tasks, LessonAgent outperforms existing methods on figure generation and interactive content generation in both automated and human evaluations. Together, LessonWorld establishes interactive, executable lessons as a new paradigm for agentic educational content creation.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
Interactive content generation, agentic AI, educational AI

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_ZvyuOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper introduces LessonWorld, an environment for generating interactive educational lessons rather than static videos or slides. It has three parts: 1 LessonEngine, a TypeScript DSL that represents a lesson as a state machine of typed beats with learner-driven transitions; 2 LessonAgent, a multi-stage pipeline built on Claude Opus 5 (planner, figure agent, parallel act writers, deterministic checks, editor, review agent, and a runtime tutor); 3 LessonBench, with 250 figure-level tasks across four STEM domains and a lesson-level comparison against video and slide generators, evaluated with Gemini 3.1 Pro pairwise judging and two small human studies.

Critical strengths

1 The move from generating educational media to generating an executable, stateful lesson is a clear and useful framing. Representing lessons as typed state machines makes interactivity, remediation and branching explicit, and also lets agents work on lessons in parallel. 2 The deterministic checks are a nice idea. They turn pedagogical rules into automatically verifiable constraints: every checkpoint has remediation, no beat is a dead end, layers are revealed in order, and so on. The paper is also very transparent, releasing all prompts, the evaluation protocol and the human-study design.

Critical weaknesses

1 The evaluation does not establish educational value, and the lesson-level comparison is not fair. Most baselines produce static videos or slides, and the VLM judge watched them as silent videos, so a large part of the 91% win rate reflects the difference in format rather than lesson quality. 2 The gains seem to come mostly from the backbone model rather than from the proposed pipeline. For example, in Table 3, adding the planner lowers performance for all four backbones (e.g., Gemini drops from 63.5 to 41.5), and the full pipeline is worse than the single-call baseline for Gemini. 3 Overall, the contribution is mainly an engineering system and prompt engineering. So I think the novelty is limited.

Questions for authors

See weakness.

Reviewer_2R5SOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper introduces LessonWorld, a framework for generating interactive educational content through a stateful lesson engine and a multi-agent authoring pipeline. It also presents LessonBench, a benchmark for evaluating generated figures and lessons, with experimental results based on automated and human assessments.

Critical strengths
  1. The paper addresses a meaningful direction by moving beyond static educational content toward interactive, adaptive learning environments.

  2. The proposed system integrates a programmable lesson engine with an agentic generation pipeline, providing a structured approach to educational content authoring.

  3. The evaluation covers multiple STEM domains and includes both automated and human assessments, along with comparisons against existing content-generation systems.

Critical weaknesses
  1. Fairness of Baseline Comparisons. The lesson-level evaluation compares interactive lessons against static videos and slides, which differ fundamentally in their capabilities and presentation formats. This makes it difficult to isolate improvements attributable to the proposed framework rather than the inherent advantages of interactivity.

  2. Limited Evidence of Learning Effectiveness. The human evaluation primarily relies on subjective ratings from a small participant pool. While these results reflect perceived instructional quality, they do not establish whether the generated lessons lead to measurable improvements in student learning outcomes.

  3. Inconsistent Ablation Results. Table 3 shows that introducing the planner alone substantially degrades performance across multiple backbones. Moreover, the full pipeline does not consistently outperform direct generation, raising concerns about the robustness and generalizability of the proposed agentic design.

  4. Insufficient Evaluation of Runtime Adaptivity. Although stateful interaction and adaptive lesson execution are central contributions, the evaluation focuses primarily on generated content quality. More direct evidence is needed to demonstrate the reliability and effectiveness of adaptive instructional behaviors across different learner trajectories.

Questions for authors
  1. How do the authors ensure fair comparisons between interactive lessons and static educational content? Can the benefits of the proposed framework be isolated from those of the output modality itself?

  2. Can the authors provide objective measurements of learning outcomes, rather than relying primarily on subjective assessments of instructional quality?

Reviewer_uqTfOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
3
Soundness
1
Read review
Paper summary

This paper introduces LessonWorld, a framework for generating interactive educational lessons rather than static learning materials. It consists of three components: LessonEngine, a stateful runtime for executable lessons; LessonAgent, a multi-agent pipeline for structured content generation and iterative refinement; and LessonBench, a benchmark evaluating interactive figures and complete lessons across four STEM domains. The system supports learner interactions and adaptive tutoring during lesson execution. Automated and human evaluations demonstrate improvements over existing educational content generation methods in both generation quality and educational effectiveness.

Critical strengths
  1. Runtime-Editable Lesson Representation. Unlike existing systems that generate largely fixed educational content, LessonWorld enables tutor agents to dynamically modify instructional nodes, interaction states, and learning pathways during execution. This represents a meaningful extension beyond conventional offline content generation.

  2. Constraint-Aware Agentic Generation. The framework integrates a typed intermediate representation, explicit editing permissions, and deterministic verification into a distributed agentic pipeline. This design enables structured coordination and targeted error correction, reducing reliance on unconstrained LLM generation and self-evaluation.

Critical weaknesses
  1. Limited Methodological Novelty. The framework largely combines established techniques, including state-machine representations, multi-agent orchestration, and verification-guided refinement. The paper does not clearly establish a novel technical mechanism beyond their system-level integration.
  2. Insufficient Validation of Core Claims. Despite emphasizing runtime adaptivity, the paper lacks systematic evaluation of dynamic instructional modifications and their effectiveness. The small-scale human study also relies primarily on subjective ratings rather than objective learning outcomes.
  3. Incomplete Baseline Comparisons. Most baselines generate static content, limiting the fairness of comparisons. The human evaluations omit important competitors, including EduVisAgent and the general-purpose Coding Agent, while generation budgets are insufficiently controlled.
  4. Inconsistent Ablation Results. Introducing the Planner alone degrades performance across all four backbone models, and the full pipeline underperforms direct generation with Gemini 3.1 Pro. Component-level ablations for complete lesson generation are also missing.
Questions for authors
  1. What specific technical innovations distinguish LessonEngine from existing state-machine frameworks combined with LLM-based tutoring, beyond system-level integration?
  2. How reliably can the Tutor Agent modify instructional content and learning pathways at runtime? Is there empirical evidence that such adaptations improve learning outcomes compared with fixed lessons?
  3. How are backbone models, tool access, and generation budgets controlled across baselines? Why are EduVisAgent and the general-purpose Coding Agent excluded from the respective human evaluations?
  4. What explains the consistent performance degradation introduced by the Planner and the weaker full-pipeline results with Gemini 3.1 Pro? Are similar effects observed in complete lesson generation?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.