acceptodds
Withdrawn from ICLR 2027

C⁴-EDS:Planning Before Prompting for Educational Dialogue Synthesis

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Synthetic-data systems can now specify how individual examples are generated in considerable detail, while the data space formed by those examples may remain implicit. In educational dialogue synthesis, rich source, task, role, cognitive, and pedagogical conditions do not by themselves expose which semantic combinations are legal or how dataset composition should be planned. We formulate this gap as a generation-context representation problem and introduce C⁴-EDS. Before a complete context is assembled, C⁴-EDS decouples Content and Task into independently addressable semantic objects, makes their legal relations explicit, and thereby forms an enumerable and allocatable planning space. Selected relations are compiled into executable Jobs and integer candidate workloads. Representation, execution, and quality governance retain distinct responsibilities, while planning identities persist through synthesis and filtering so that retained data can be audited against the original plan. In K–12 Classical Chinese tutoring, 5,628 theoretical Content–Task combinations yield 2,807 legal Pairs and 5,614 executable Jobs. Under a matched workload, explicit reasoning improves structural realization without monotonically improving pedagogical quality; full Job-level planning reaches an automatic hard-pass rate of 88.28% and the highest mean continuous human rating. Downstream training further exhibits a target-dependent Process–Result trade-off. High-quality synthetic data therefore depends on how the generation space is represented, planned, and governed, as well as on how each example is generated.

Primary area:
foundation or frontier models, including LLMs
Keywords:
synthetic data generation, large language models, controllable generation, data curation, data quality, instruction tuning

Reviews

Among:
Shape:
1234this paper, Bottom 2%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.33 over 3 reviews, above 2% of the others.
Reviewer_cSyvOpenReview ↗
Rating
2(−1)
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes C4-EDS, a framework for planning synthetic educational dialogue datasets before generation. The central idea is to represent Content and Task as independently addressable semantic objects, explicitly define their admissible relations, and compile legal Content–Task Pairs into executable Jobs with integer candidate Slot allocations. By preserving Pair, Job, and Slot identities throughout generation and quality filtering, the framework enables dataset composition to be planned beforehand and audited afterward.

The paper evaluates C4-EDS in K–12 Classical Chinese tutoring through three experiments. E1 demonstrates the construction of an enumerable planning space containing 2,807 legal Content–Task Pairs and 5,614 executable Jobs, along with exact materialization of integer allocation targets. E2 compares four progressively structured generation conditions under a matched candidate budget and reports that full Job-level planning (CTSP) achieves the highest automatic semantic–pedagogical hard-pass rate (88.28%) and the highest mean continuous human quality score. The experiments also reveal that explicit reasoning requirements can improve structural realization while reducing pedagogical quality. E3 investigates downstream supervised fine-tuning using different curated training views and observes a trade-off between Process and Result performance.

Overall, the paper presents a systematic approach to dataset-level planning and quality governance. Its main contribution is a structured representation and workflow for controlling and auditing synthetic-data composition rather than a new generation model or optimization algorithm.

Critical strengths
  1. A useful dataset-level perspective on synthetic-data generation. The paper addresses a meaningful limitation of example-centric synthetic-data pipelines: specifying detailed conditions for individual examples does not necessarily provide an explicit representation of the feasible dataset space or its target composition. By separating Content and Task, defining admissible Pairs, and maintaining stable Job/Slot identities, C4-EDS offers a practical mechanism for planning and auditing dataset composition. This perspective is potentially valuable for applications that require systematic coverage and traceability.

  2. A reasonably careful empirical analysis with human evaluation. The paper separates structural validity from semantic–pedagogical quality, compares four generation conditions using the same candidate identities and budget, and reports both automated and independent human evaluations. In particular, the finding that explicit reasoning improves early structural validity while substantially reducing semantic hard-pass rates is interesting and practically relevant. The authors also report inter-evaluator agreement, automatic–human calibration, and important limitations of their experimental design, which strengthens the credibility of the descriptive findings.

Critical weaknesses
  1. Insufficient evidence for the benefits of explicit planning. Although the paper compares B0, B1, B1-CoT, and CTSP, these experiments mainly evaluate different generator-facing instructions under the same upstream planning space. It remains unclear what additional benefits the explicit Content–Task/Job/Slot planning framework provides over a simpler workflow with equivalent prompting instructions.

  2. Lack of component-level ablation studies. CTSP simultaneously introduces role relations, turn/message planning, cognitive progression, pedagogical actions, and closing requirements. Without individual ablations, it is difficult to identify which components contribute to the observed improvements or whether all these design choices are necessary.

Overall Assessment: I find the proposed framework promising and have given the paper a relatively favorable initial rating. However, the above concerns are important for establishing the paper's core contributions. I would be particularly interested in seeing additional experimental evidence addressing these points, as it would help me better understand and assess the proposed method.

Questions for authors

Q1. Could the authors compare CTSP with a baseline that uses identical generation prompts and quality filtering but samples Content–Task combinations without explicit Pair/Job/Slot allocation, under the same candidate budget?

Q2. Could the authors provide component-level ablations of CTSP to identify which execution-planning components contribute most to the improvements over B1-CoT?

Q3. The prompts provided in the appendix are helpful for understanding the implementation. Could the authors also provide English translations alongside the original Chinese prompts? This would improve accessibility for non-Chinese-speaking readers and facilitate reproducibility. Whether or not the authors address this suggestion will not affect my overall rating.

Reviewer_iiDoOpenReview ↗
Rating
1
Confidence
3
Informativeness
2
Presentation
1
Soundness
1
Read review
Paper summary

This paper proposes C⁴-EDS, a framework for educational dialogue synthesis. It represents Content and Task as independent objects, forms legal combinations, and compiles them into execution plans and generation quotas. Planning identities are preserved throughout generation, quality filtering, and data auditing. In K-12 Classical Chinese tutoring, the paper examines the framework's executability, generation quality, and changes in dataset composition after filtering through planning-space construction, comparisons of generation conditions, human evaluation, and downstream fine-tuning experiments.

Critical strengths

The planning and tracking mechanisms have practical value. Pair, Job, and Slot identities link the intended dataset composition to generation attempts and the samples ultimately retained. This helps identify distribution shifts caused by filtering and supports the management and auditing of synthetic data. The paper includes human evaluation and acknowledges some limits of the evidence. Two evaluators assess 256 outputs from 64 matched cases, and the paper reports inter-evaluator agreement and calibration between automatic and human judgments.

Critical weaknesses
  1. The method's novelty and value beyond integration are not sufficiently established.

The framework mainly integrates Content and Task structuring, compatibility filtering, execution planning, quota allocation, quality checks, and sample tracking. However, it does not clearly explain which mechanisms offer novelty or benefits beyond combining existing components into a workflow. The exact realization of quotas in E1 mainly verifies implementation correctness. E2 changes several requirements at once, including role relations, turn structure, cognitive progression, and pedagogical actions. The examples in the appendix also differ in interaction roles and the number of turns. As a result, the current experiments cannot clearly distinguish whether the gains come from the proposed representation and planning mechanisms or from more detailed prompts and different generation tasks. Although the paper acknowledges that this is a block-level comparison, the core contribution still lacks direct validation.

  1. There are important gaps in evaluation validity and experimental verifiability.

The main text and appendices do not disclose the specific identities and versions of the generator, automatic evaluator, or downstream base model. They also lack sufficient details about the training settings, downstream test set, and scoring criteria. Appendix D states that the strict training view was reconstructed from archived records. However, this is not enough to verify how the actual training data and model runs correspond to the scores in Table 2. In addition, Table 8 and Appendix B.8 mark all the displayed examples as passing all LQC2 hard gates. However, the user message in the CTSP output contains instructional text from the generation input, which clearly conflicts with the prompt non-leakage requirement. The B0 output also fails to follow the generation requirements that prohibit answering on behalf of the student and embedding role labels in the content. These discrepancies need to be explained with item-level evaluation records and clear decision rules. In E3, the Final score changes only from 62.1 to 62.3. The paper does not report enough uncertainty information to assess the stability of this difference, and the training views differ in both sample size and composition. The downstream results are therefore mainly exploratory and do not yet establish consistent gains.

Questions for authors
  1. What is distinctive about the mechanisms in C⁴-EDS compared with a template-based generation approach that provides the same semantic information, roles, and execution requirements? Are there controlled experiments that isolate the contribution of the representation and planning mechanisms?
  2. Why do the examples discussed above in the appendix pass all LQC2 hard gates? Please clarify the criteria for judging task alignment and prompt leakage, and whether the evaluation covers user messages, reasoning text, and final answers.
  3. Please provide the identities and versions of all models, key generation and training parameters, strict filtering rules, and the downstream evaluation protocol. Please also provide per-sample scores, statistical analysis scripts, and training and evaluation records that can be checked.
Reviewer_ua8XOpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
2
Soundness
2
Read review
Paper summary

The paper investigate dataset-level planning for educational dialogue synthesis. Its core idea is to separate the dialogue content and educational tasks. It constructs content–task pairs through compatibility rules, compiles them into executable Jobs with role, turn, cognitive, and pedagogical requirements. The experiment is conduct on classical Chinese tutoring educational task.

Critical strengths
  1. The dataset construction workflow is clear. It separates planning, execution, and quality governance and create diverse educational dialogues.
  2. The evaluation stage includes two human raters assessing 256 responses from 64 matched cases across four conditions, which is helpful for understanding the quality of the dataset.
Critical weaknesses
  1. The proposed dataset construction workflow is not sufficiently distinguished from a domain-specific data-engineering workflow, which is very common in the past a few years.
  2. Lack of comparison between the other dialogue construction workflow. The experiment only focuses on basic approaches. It also does not isolate the contribution of the planning mechanism.
Questions for authors

See weaknesses.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.