acceptodds
Under review as a conference paper at ICLR 2027

Rich-Node Graphs for Corpus-Grounded Long-Form SFT Data Synthesis

Abstract

Domain-specific supervised fine-tuning (SFT) requires training data that cover related subtopics while remaining grounded in source evidence. Existing pipelines often rely on parametric knowledge, individual passages, or flat retrieved contexts, limiting their ability to organize distributed evidence into coherent supervision. We present RiNoGen, a framework that transforms domain corpora into mixed-length question–answer pairs using provenance-aware Rich-Node graphs. RiNoGen selects corpus-supported seed entities, expands them with retrieved evidence, and applies Map–Reduce synthesis to compose local subtopics into globally coherent answers. The target Qwen3-8B model generates the grounded answers and is then fine-tuned on them, without requiring a larger teacher for answer generation. On a corpus of 670k Chinese educational papers, RiNoGen constructs 30,000 SFT examples. Under a fixed 3,000-example budget, it achieves 87.59% closed-book accuracy on a 427-item evaluation subset, outperforming the strongest baseline by 1.14 percentage points. Ablations confirm the contributions of graph expansion and Map–Reduce synthesis. An optional web branch further improves responses to time-sensitive topics.

Primary area:
transfer learning, meta learning, and lifelong learning
Keywords:
Large Language Models, Synthetic Data Generation, Retrieval-Augmented Generation, Supervised Fine-Tuning, Domain Adaptation

Reviews

Among:
Shape:
1234this paper, Bottom 9%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.50 over 4 reviews, above 9% of the others.
Reviewer_94USOpenReview ↗
Rating
1
Confidence
2
Informativeness
2
Presentation
3
Soundness
1
Read review
Paper summary

The paper presents RiNoGen, a pipeline to build long-form SFT data from a domain corpus. Seed entities are selected with clustering plus an LLM filter, expanded into "Rich-Node" graphs grounded on retrieved chunks, and the answers are synthesized with a Map-Reduce step, then the question is regenerated from the answer. Exps. on a large Chinese educational corpus with Qwen3-8B report a gain of about 1 pp over several corpus-to-data baselines on a 427-item MCQ subset.

Critical strengths

The problem (turning domain corpora into grounded SFT data) is relevant and the pipeline is described clearly. The internal controls (parametric, single chunk, flat RAG, graph) are reasonable.

Critical weaknesses
  1. Weak positioning w.r.t. SOTA. The main building blocks are presented as new but they are quite established. Generating long SFT answers by decomposing them into sections is close to LongWriter/AgentWrite, retrieval + outline for long grounded text is close to STORM, and regenerating the question from the answer is basically instruction backtranslation (Humpback). Map-Reduce over summaries is the core of GraphRAG global search, which the paper cites only for the community summaries. None of these is discussed. Also the paper assumes that SFT transfers corpus knowledge to the model, but there is a known line of work showing that SFT mostly teaches the model to use what it already knows and that new facts in SFT data can increase hallucinations (e.g. LIMA, Gekhman et al. 2024). Using the same model as generator and student is related to self-distillation fine-tuning, and the data selection/curation literature is not considered either. As it is, it is hard to see what is really new apart from the combination of known pieces.

  2. Relevant works omitted from the comparisons. The baselines are mostly generic text transformation methods, and some of them (EntiGraph, WRAP) are designed for (continued) pretraining, so a 3k-example SFT budget is not their natural setting. The most direct competitors for adapting a model from unlabeled domain corpora are missing, e.g. AdaptLLM (reading comprehension), Instruction Pre-Training and Bonito. Simple but strong baselines are missing too: a plan-then-write long-form generator (see AgentWrite), and the fine-tuned model used with RAG on the same index. For the web part, the obvious comparison RiNoGen-Web vs naive web RAG is not reported.

  3. Experimental analysis is not fully convincing. The main gain (1.14 pp on 427 items, i.e. about 5 questions) is within the statistical noise. No significance test is given, and repeating the eval 20 times at temperature 0 does not measure any variance. The same method gets about the same difference as the claimed gain in two different settings. The external comparison is confounded, since RiNoGen uses the full corpus index while the baselines get a 7k chunk slice, and RiNoGen answers are much longer so token budgets are not matched (length alone is known to affect SFT results). The benchmark is generated by a same-family LLM from the same corpus without human validation. Long-form quality and provenance, which are the main selling points, are never evaluated directly. The web eval relies on an unspecified LLM judge and the result (web access helps on 2025-26 topics) is expected.

Questions for authors
Reviewer_FjbnOpenReview ↗
Rating
1
Confidence
3
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

RiNoGen is a framework for automatic generation of multi-subtopic, mixed-length SFT question-answer data from a domain corpus,which builds a rich-node graph containing entity summaries, expansion cues and source information from the retrieved evidence, and then synthesizes answers using map-reduce. Qwen3-8B is used as both the answer generator and the fine-tuning target to create 30,000 training samples from Chinese education papers. In the experiment, RiNoGen achieves an accuracy of 87.59% on the evaluation set of 427 multiple-choice questions. This paper also provides internal comparisons, component ablations and time-sensitive experiments.

Critical strengths
  1. The research problem is valuable in real-world applications. Scattered domain evidence needs to be converted into training samples covering sub-topics for the purpose of automated SFT data construction. This paper presents a processing pipeline for this problem and the feasibility of answer generation.

  2. Rich-Node collects the summaries and origins of expansion cues, while Map and Reduce handle local subtopics and global integration, respectively. This architecture is reasonable.

Critical weaknesses
  1. The evaluation results do not directly verify the data-quality claims. The method focuses on long answers, multi-subtopic coverage, and source traceability; however, the main experiments are conducted using closed-book four-option multiple-choice accuracy and thus fail to evaluate the factual correctness, coverage, and evidence faithfulness of the synthesized long answers.

  2. The current comparisons do not attribute the improvements to the Rich-Node design. The six external baselines use the same set of 7,000 text chunks and RiNoGen employs the whole corpus index; only the number of samples is the same, but the training tokens and construction costs are not. Therefore, the results in Section 4.3 can only compare the full pipelines under these specific configurations and cannot isolate the impact of graph structure from that of extended corpus coverage. The independent gain of the new design proposed in this paper requires stronger support from evidence.

  3. There are no paired statistical analyses or variations of training seeds in the paper, and it does not explain why only 427 of the 10,000 questions were used.

Questions for authors

1 .Are there already comparison results of RiNoGen restricted to the same set of 7,000 text chunks? Please provide the actual number of training tokens for each method, the answer-length distribution and data construction cost.

  1. Why were only a fixed number of 427 questions selected from the total of 10,000 for assessment, and what sampling method was employed?

  2. The format of this paper shoud be polished.

Reviewer_G3D9OpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes RiNoGen, a framework for constructing corpus-grounded, mixed-length SFT QA pairs. It selects corpus-supported seed entities and expands them into Rich-Node graphs that retain retrieved evidence and expansion clues, then uses Map–Reduce synthesis to generate answers and corresponding questions. The same Qwen3-8B model is used for data generation and downstream fine-tuning. The authors build a large SFT set from a Chinese educational paper corpus and evaluate on a fixed multiple-choice subset, where RiNoGen outperforms internal controls and external data-construction baselines.

Critical strengths
  1. The paper addresses a clearly defined and practically relevant problem: how to organize distributed evidence from large document collections into coherent training examples. The proposed Rich-Node representation provides a useful way to organize retrieved evidence, expansion relationships, and local subtopics before answer generation.
  2. The framework uses the same 8B model for both data generation and downstream fine-tuning, without relying on a larger teacher model for answer generation. This makes the approach potentially practical in resource-constrained settings.
  3. The paper includes internal controls, comparisons with several existing data-construction methods, and ablation studies targeting different components of the framework. These experiments provide an initial empirical assessment of the proposed pipeline.
Critical weaknesses
  1. The comparison with external baselines is not entirely fair. Unlike the baselines, which use a fixed corpus subset, RiNoGen employs its own retrieval pipeline. Thus, it is unclear whether the gains come from the Rich-Node representation or broader access to source evidence.
  2. The contribution of Rich-Node graphs is not sufficiently established. The evaluation relies on automatically generated questions without independent human verification, and the performance gains are modest. Moreover, the ablations lack comparisons with simpler hierarchical summarization methods, leaving the benefit of the graph structure unclear.
Questions for authors
  1. Would the performance advantage of RiNoGen remain if it and the strong baselines were given access to the same candidate corpus and comparable training-token budgets? Ideally, the amount of retrieved evidence and the data construction costs should also be controlled.
  2. Could the authors provide more details about the sampling procedure and random seed used to select the 427 evaluation questions, together with results from independent human verification of the questions and answer labels? Additionally, could the authors clarify whether the evaluation is intended to measure knowledge acquisition from the indexed corpus or generalization to previously unseen source documents?
  3. Compared with alternative methods that generate answers of matched lengths, can RiNoGen demonstrate improvements in factual faithfulness, evidence coverage, or textual coherence in long-form generation tasks?
Reviewer_fEvGOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes RiNoGen, a framework for generating corpus-grounded, long-form SFT data through Rich-Node graphs and Map–Reduce synthesis. The method selects corpus-supported seed entities, recursively organizes retrieved evidence into hierarchical graphs, and synthesizes local subtopics into coherent QA pairs.

Critical strengths

The paper addresses an important practical problem: constructing coherent, evidence-grounded SFT data from large domain corpora without relying on a stronger teacher model. The proposed Rich-Node representation provides a clear mechanism for connecting retrieval, topic expansion, and local-to-global synthesis. Importantly, the authors provide meaningful internal controls and component ablations rather than relying solely on external benchmark comparisons.

Critical weaknesses
  1. The main comparison does not adequately isolate the proposed method's contribution. External baselines are restricted to a 7,000-chunk corpus subset, while RiNoGen accesses its native retrieval index over the larger corpus. Consequently, the reported 1.14-point improvement may reflect greater evidence coverage rather than the effectiveness of Rich-Node graphs. The internal comparisons are more informative, but the gain over the simpler Cut-Train baseline is only 0.8 points, with token budgets and generation costs not fully controlled. Furthermore, the ablations primarily support staged synthesis rather than establishing the independent necessity of the graph representation.

  2. The evaluation provides limited evidence for the central long-form grounding claim. The primary results rely on a fixed 427-question, corpus-derived multiple-choice benchmark, which does not directly assess long-form factuality, evidence integration, or cross-document coverage. The reported gains are relatively small, and the existing leakage audit checks textual and semantic similarity rather than source-document overlap. Although this does not establish actual leakage, it leaves uncertainty about whether the improvements reflect robust knowledge transfer and the intended benefits of structured synthesis.

Questions for authors
  1. Comparison fairness: Can the authors provide results under matched upstream corpus access and comparable training-token budgets? In particular, does RiNoGen outperform a non-graph hierarchical Map–Reduce baseline using the same retrieved evidence?

  2. Evaluation independence: Were the source documents used to construct the 427 evaluation questions excluded from the corpus accessed during training-data generation? Can the authors clarify the extent of source-document overlap?

  3. Long-form effectiveness: Is there direct evidence that Rich-Node synthesis improves factual grounding and cross-document integration in generated long-form answers, beyond downstream multiple-choice accuracy?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.