acceptodds
Under review as a conference paper at ICLR 2027

DataStamp: Domain-Specific Data Synthesis via Structural Projection

Abstract

Large language models (LLMs) achieve strong performance on general-purpose tasks through instruction tuning, yet high-quality instruction–response pairs remain scarce in specialized domains such as humanities and medicine, where typically only raw, unlabeled documents are available. The central challenge is synthesizing pairs that are both grounded in target documents and diverse in reasoning patterns. Existing approaches either bootstrap from seed examples via LLM generation, defaulting to shallow and repetitive questions, or rely on manually engineered templates that offer limited diversity and fail to transfer across domains. We observe that high-quality QA data spanning diverse domains has already been produced by prior research and that each pair implicitly encodes a domain-free reasoning pattern. Inspired by meta-learning, we propose DataStamp, which treats these patterns as transferable prototypes and adapts them to new target domains. Each document is abstracted into a logic graph that retains only its logical skeleton; from existing cross-domain QA data, DataStamp extracts transfer graphs—typed-graph prototypes encoding how a logical structure should be transformed into an instruction–response pair. Given a target document, the most structurally compatible transfer graphs are retrieved via a relational GCN and projected onto the target content, producing grounded and diverse pairs without domain-specific engineering. Evaluated on twelve benchmarks across two domains, DataStamp outperforms open instruction corpora and domain-specific baselines, matching or exceeding Qwen3-8B-Instruct trained on proprietary data, while producing data that is substantially harder and more diverse than existing alternatives.

Primary area:
foundation or frontier models, including LLMs
Keywords:
Data Synthesis, Instruction Tuning, LLMs

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_Ug4pOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes DATASTAMP, a domain-specific instruction data synthesis framework that uses logic graphs and transfer graphs to reuse reasoning patterns across domains. Experiments on medical and humanities benchmarks show competitive performance, with ablations supporting the benefits of structural retrieval and graph guidance.

Critical strengths
  1. Generating diverse, high-quality instruction data from unlabeled domain documents addresses an important challenge in adapting LLMs to specialized domains.

  2. The method achieves competitive performance across two domains. The expanded ablations in Appendix F.2 provide useful evidence that structural retrieval and explicit graph guidance contribute to downstream improvements.

Critical weaknesses
  1. The authors use a randomly initialized, frozen R-GCN for structural retrieval, without showing that its embeddings reliably capture structural compatibility. Moreover, removing domain-specific vocabulary does not necessarily make the extracted reasoning patterns domain-independent. The downstream improvements are encouraging, but they do not directly validate the claimed structural transfer mechanism.

  2. Previous AMR and semantic-graph-based question generation methods have already explored structural abstraction and graph-guided generation. DESIGNER also extracts and transfers question-design patterns across domains. The proposed dual-graph representation and retrieval pipeline appear to be an extension of these ideas, but the paper does not clearly demonstrate what fundamentally new capability they provide.

Questions for authors
  1. Why should a randomly initialized, frozen R-GCN capture meaningful structural similarity? Have the authors compared it with standard graph similarity methods or evaluated its stability across different random seeds?

  2. How do the authors verify that the extracted graphs are genuinely domain-independent, rather than simply removing domain-specific terminology? For example, can the source domain still be predicted from the transfer graphs?

  3. The node correspondence is constructed through greedy type alignment, while the final projection relies on LLM prompting. How do the authors ensure that the retrieved reasoning structure is actually preserved? Would perturbing the graph structure or node mapping affect the results?

  4. What is the key difference between DATASTAMP and existing AMR/semantic-graph-based question generation methods, particularly DESIGNER? Can the authors clarify which part of the proposed framework provides a new capability beyond prior work?

  5. How does the method compare with direct document-based QA generation or exemplar-guided generation using the same teacher model, documents, and generation budget? This seems necessary to isolate the contribution of the proposed graph-based pipeline.

Reviewer_ksb7OpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
1
Read review
Paper summary

This paper proposes DATASTAMP, a method for creating domain-specific instruction data from raw documents. The method represents a document with a logic graph and represents a question-generation pattern with a transfer graph. It builds a library of about 189K transfer prototypes from existing web QA data.

For a new target document, DATASTAMP extracts a logic graph and retrieves similar prototypes with a relational GCN. A teacher LLM then uses the target document, transfer graph, node mapping, and source QA example to generate a new question-answer pair. The pairs are checked by a verifier and filtered for diversity. The paper creates medical and humanities/social-science datasets and fine-tunes Qwen3-8B-Base on them.

Critical strengths
  1. The high-level idea is interesting. A question-generation operation may be reusable across domains even when the vocabulary and content are different. This is a reasonable direction for domain data synthesis.

  2. The paper includes several useful experiments.The medical results are stronger than the humanities/social-science results. The ablations suggest that retrieval, graph guidance, examples, and verification all have some effect. I also appreciate that the appendix discusses several limitations directly.

Critical weaknesses
  1. The structural retrieval space is not well justified. The R-GCN is randomly initialized once and then frozen. There is no objective that trains graph distance to represent structural compatibility or successful projection. With six node types, small graphs, skewed type frequencies, and mean pooling, the embedding may mainly represent simple type statistics. Since structural retrieval is a central contribution, I see this as a major issue.

  2. The experiments do not isolate the contribution of structural transfer. The paper does not include a baseline where the same teacher directly reads the target document and generates the same number of examples without a graph or retrieved QA. The comparison with older public datasets also mixes the method effect with the stronger teacher model. Finally, the high cross-domain retrieval percentage is not compared with the domain distribution of the library, and the paper does not compare same-domain-only and cross-domain-only retrieval.

Questions for authors
  1. Why should an untrained and randomly initialized R-GCN produce a useful structural similarity space? Can the authors compare it with node-type histograms, a Weisfeiler-Lehman kernel, graph edit distance, and a trained graph encoder? Please also report variation across random initialization seeds.

  2. Can the authors add a baseline where the same Qwen teacher directly reads each target document and generates the same number of verified QA pairs, without graph guidance or retrieved examples?

  3. Can the authors run two settings using only same-domain prototypes and only cross-domain prototypes? This seems necessary to support the cross-domain transfer claim.

  4. Can the authors report the full data-generation process, including the number of candidates per document, verification pass rate, final selected count, and total LLM cost?

  5. Does the mapping from an abstract node such as Entity_1,identify any specific span or entity in the target document? If not, what additional information does the mapping provide to the teacher model?

1 comment

Dear Reviewer ksb7,

Thank you again for your thoughtful feedback. We have provided clarifications and additional experiments addressing your initial concerns. We would appreciate your thoughts on our responses.

Best regards, Authors

Reviewer_8HASOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper introduces DataStamp, an instruction synthesis framework for specialized domains that decouples reasoning structures from domain content. DataStamp abstracts documents into domain-free logic graphs and extracts reusable transfer graph prototypes from cross-domain QA data. For unlabeled target documents, an R-GCN retrieves structurally compatible prototypes to guide verified QA generation. Evaluated on 12 medical and humanities benchmarks, DataStamp outperforms open instruction baselines, generating data with superior difficulty, diversity, and accuracy.

Critical strengths

1.The paper introduces an insightful dual-graph representation that decouples domain-specific content from general reasoning patterns. By establishing a reusable prototype library from existing datasets, it reduces reliance on manually designed domain-specific templates and enables flexible instruction generation across heterogeneous domains.

2.The framework demonstrates strong performance across 12 benchmarks in medical and humanities domains. The ablation studies further support the effectiveness of structural retrieval and explicit graph guidance.

Critical weaknesses
  1. The R-GCN encoder is initialized once and frozen. Although the paper argues that retrieval only needs to generate candidates before verification, it does not directly evaluate retrieval quality or compare with simpler typed-graph similarity methods. The greedy same-type node alignment is also not isolated against alternative correspondence strategies, despite the author providing useful end-to-end faithfulness results.

  2. The paper lacks uncertainty estimates across independent training seeds. In HSS, DATASTAMP only marginally outperforms Qwen3-8B-Instruct (53.7 vs. 53.4). Without variance estimates or statistical significance tests, it remains unclear whether this small improvement reflects a robust advantage.

Questions for authors
  1. What motivated the choice of six node types and seven edge types for logic graphs? Were alternative type vocabularies considered?

  2. What proportion of generated QA pairs pass the answerability and factual-consistency checks, and what are the main reasons for rejection?

  3. How is the number of retained QA pairs per document determined? How does the diversity selection strategy balance reasoning diversity against the quantity of training data?

  4. Since the prototype library is constructed from DLR-Web, how sensitive is DataStamp to the quality and composition of the source QA dataset? Would the method remain effective with a different, potentially lower-quality source corpus?

1 comment

Thank you for the detailed response and additional experiments. The authors have addressed most of my concerns and provided useful evidence supporting the proposed approach. Some uncertainties remain regarding statistical robustness and source data quality. Overall, I appreciate the authors' efforts, and my overall assessment remains unchanged.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.