acceptodds
Under review as a conference paper at ICLR 2027

Group-Balanced Supervised Fine-Tuning For Knowledge-Intensive Domains

Abstract

Supervised fine-tuning (SFT) weights every supervised token equally, so on knowledge-dense corpora the loss budget follows token count rather than importance: on a legal QA benchmark, citation strings absorb 81% of the supervised loss weight budget while the answers being taught absorb 5%. We present Group-Balanced SFT (GB-SFT), which equalizes the loss budget across meaningful groups of tokens, as a two-parameter objective family that contains standard SFT, class-level balancing, and span-level balancing as corners, whose geometry we characterize exactly. Across four benchmarks in law and medicine on two model families, we show that: (i) GB-SFT injects new knowledge better than uniform SFT, pointwise token weighting, memory editing, and matched-budget data mixing; (ii) it matches the injection of masking out everything but the knowledge payload while retaining the citation recall that masking destroys (0.98 vs. 0.38); and (iii) role-aware grouping carries the gain: label-free partitions recover at most a third of it, while an LLM tagger recovers nearly all, but only when the groups are lexically distinctive. We release two new contamination-controlled knowledge-injection benchmarks with validated ground truth.

Primary area:
foundation or frontier models, including LLMs
Keywords:
supervised fine-tuning, large language models, knowledge injection, loss reweighting, token-level weighting, knowledge preservation, catastrophic forgetting, domain adaptation, benchmarks

Reviews

Among:
Shape:
1234this paper, Top 32%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.50 over 2 reviews, above 68% of the others.
Reviewer_nV7YOpenReview ↗
Rating
3
Confidence
3
Informativeness
3
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes Group-Balanced Supervised Fine-Tuning (GB-SFT), a simple yet general framework that redistributes the supervised loss budget across semantically meaningful token groups to emphasize knowledge payload (e.g., answers/values) over scaffolding tokens (e.g., citations, template prose) in knowledge-intensive domains. The authors unify standard SFT, class-balanced, and span-balanced objectives in a two-parameter family that controls class- and span-level budget splits, provide a concise analysis characterizing how the loss moves with these parameters, and validate the approach on four legal/medical benchmarks with new, contamination-controlled datasets. Empirically, GB-SFT consistently improves knowledge injection over strong token-level reweighting baselines and group-robust optimization, while preserving scaffold knowledge (e.g., citation recall) better than masking-based controls.

Critical strengths
  1. The paper clearly reveals that standard token-level SFT can allocate an excessive portion of the loss budget to long scaffold tokens, which is an interesting observation that may attract further research attention.
  2. The paper proposes GB-SFT, a simple yet effective method. Experiments across multiple datasets and models show that it consistently improves performance over strong baseline methods.
  3. The authors conduct experiments from multiple perspectives and demonstrate significant performance improvements over the baselines.
Critical weaknesses
  1. GB-SFT achieves clear improvements, but its core loss formulation is relatively similar to existing class-balanced weighting methods. The authors are encouraged to further clarify the fundamental differences between GB-SFT and traditional class-balanced loss.
  2. The experiments suggest that the method depends on accurate semantic grouping. For example, the performance of automatic grouping drops noticeably on NC-REAL, which may limit its applicability to general SFT data.
  3. The main results are still largely based on ranking metrics, while the accuracy under free-form generation is much lower. It would be helpful for the authors to further discuss this gap.
Questions for authors
  1. If the training data does not contain explicit payload/scaffold annotations or semantic group labels, how can GB-SFT be applied in practice?
  2. Why does GB-SFT achieve very high performance on the ranking metrics, while the success rate under free-form generation remains relatively low?
4 comments
AuthorsOpenReview ↗

Part 1/2: The observation generalizes further 🙂 ; clarified relation to class-balanced loss

We thank the reviewer for the careful summary, which states the contribution exactly as we intend it. We are glad the budget-allocation observation reads as "an interesting observation that may attract further research attention": that is the framing we hoped for, since we regard inspiring further work on the problem as the durable half of the contribution and the objective as one way of acting on it. We are glad too that the method read as simple and effective across datasets and models.

The observation generalizes further than the paper claimed. We think this is the most useful thing we can report back, because it speaks directly to the framing you gave the work.

If budget misallocation is a general property of token-summed cross-entropy rather than an artifact of the corpora we built, it should govern outcomes we never set out to measure. We tested that during the discussion period on GSM8K, a math corpus we did not construct, whose grouping is free because the payload boundary is the dataset's own delimiter. We found that GB-SFT improved a behavior that is not knowledge at all: the model's tendency to refuse harmful requests.

Ordinary SFT drops harmful-prompt refusal from 95.5% to 81.0% (LLM-judged, five seeds, 420 prompts). The erosion is monotone in the budget parameter (81.0 / 82.9 / 85.7 / 89.2 as α goes 1.0 → −0.5, Cochran-Armitage trend z = 7.48) and α = 0 retains about a third of what SFT loses, at matched task accuracy (67.4% against 68.4%). This is not a uniform rise in reluctance: false refusal on 350 benign prompts rises far less than refusal on harmful ones, so the separation between the two improves from 73.7 under SFT to 77.9 which is above the aligned checkpoint fine-tuning started from. So the loss budget governs what fine-tuning damages as well as what it acquires, on a corpus, a grouping source and a metric all outside the ones the paper was built around. We think that is the strongest available evidence that the budget-allocation observation is a durable part of the contribution. The conditions bind in both directions: Alpaca's 98/2 payload split would have an equal-budget objective upweight boilerplate instead, and a corpus that erodes refusal in one model need not erode it in another. App. E.7 of the revised paper gives the full scope analysis.

The three weaknesses raised are three points we would ourselves flag as worth clarifying. We take W1 here, and W2 and W3 in Part 2.

On the relation to class-balanced loss (W1). The family is deliberately in that lineage and we will make the inheritance explicit, including that Cui et al.'s own β is already a continuous knob that corresponds structurally to our α. The substantive difference is that classification admits an equivalent post-hoc correction and we do not. Logit adjustment, balanced softmax and Elkan-style thresholding exist because classification has a final argmax over the grouped variable and a test-time group prior to correct. Neither holds here: a token's semantic class is not the prediction target and is unknown at inference. A model emitting 2% citation tokens at test time is correct, not biased.

We will also pre-empt Byrd & Lipton (2019): in an autoregressive loss the terms share not only parameters but conditioning, so a down-weighted span is still generated at inference and still conditions every token after it.

AuthorsOpenReview ↗

Part 2/2: Grouping accuracy; the practical importance of our problems and groupings; ranking vs. generation

On dependence on accurate grouping (W2, Q1). The NC-Real drop in automatic labeling accuracy is instructive to explain, because its mechanism is not what the error rate suggests. Relocating a fraction of gold payload spans under random corruption costs almost nothing out to p = 0.5, where the arm still captures 95–100% of the gold gain. A prompted tagger erring at p = .293, a rate the corruption curve says is free, nevertheless lands at ungrouped SFT.

The difference is the structure of the error. The tagger returns a well-formed value of the right type in the right position ("ten days" where gold reads "$12,480"), which the model takes seriously; random relocation produces incoherent groups it ignores. The operative condition is not label accuracy but whether a wrong guess is also plausible. We will state that as a testable precondition.

On availability more broadly, which all three reviews raise: in law and medicine the output format is mandated, not chosen (e.g. published citation grammars, regulation-structured drug labels, required clinical schemas) and the same regulation that creates the scaffold is what makes the payload locatable. The grouping is a property of the domains where the pathology is worst rather than a convenience we selected for. We did not write these templates: Housing-QA is an external published benchmark of real U.S. landlord–tenant law, so the 81.4%/5.1% split measures how those documents are written, and NC-Real and OpenFDA substitute counterfactual payload values into real statutory and real FDA-label carriers as contamination control requires.

One half of grouping selection is free: at α = 0 each class takes 1/K regardless of class masses, so granularity fixes the budget with no labels. Testing a labeler's error structure, though, required gold labels, and we have no gold-free prospective test to offer. Where groups come from without annotation: Housing-QA's citations from a citation grammar, OpenFDA's dose fields from structured source labels, Tulu's classes from Python's tokenize module, and GSM8K's payload boundary from the dataset's own #### delimiter.

On ranking versus free-form generation (W3, Q2). Teacher-forced ranking is the standard instrument in the knowledge-editing literature. We report both ranking and free generation, which is why the gap is visible in our paper.

On NC-Real the teacher-forced overwrite rate is .9544 against .190 in free continuations: roughly five-fold overstatement. We emphasize, however, that the ordering is preserved, and GB-SFT still emits the taught value about 4× as often as SFT.

We can now say why the generation rate is low, which we think is the more useful answer to Q2. The gap is knowledge-limited, not decoding-limited: constraining decoding to emit a well-formed citation does not raise the rate (it scores .560 against .580 for ordinary greedy decoding). If the model were failing to express a payload it had learned, forcing the surface form would help; it does not. The payload is not yet learned robustly enough to be retrieved under the model's own decoding.

That points to a corpus-level remedy rather than a loss-level one, of the kind Allen-Zhu & Li (2024) identify: a fact becomes extractable once it appears across sufficient paraphrase and format diversity. It is orthogonal to how the loss is weighted, and we would not expect a different weighting to close it.

Closing the gap plausibly needs a corpus-level intervention of training on contexts that elicit the payload, not only on the payload itself. We leave this as future work and note that it is a particularly exciting direction. As is, our ranking improvements are measured on the instrument the memory-editing literature uses to establish its own claims, and the generation results point the same way.

We thank the reviewer for their careful attention and valuable feedback. For convenience, all substantial new content from the discussion period is in App. E of our updated paper.

Reviewer_nV7YOpenReview ↗

Response to the Authors

I appreciate the authors' thorough response. The provided explanations have resolved my initial concerns.

AuthorsOpenReview ↗

Follow-up: additional findings support your positive framing

Thank you! We’re glad the explanations resolved your concerns. We also appreciated the framing in your initial review: describing the budget-allocation observation as something that “may attract further research attention” is exactly how we hope the work is read.

The revision adds a result that speaks to that generality. On the grade-school math benchmark GSM8K, the loss budget turns out to be a monotone lever on safety-guardrail erosion, a behavior that is not part of the training objective at all (App. E.6). We think you’ll appreciate the greater applicability of our work already being apparent.

Your review noted minor or fixable concerns about soundness, which we believe the revision now resolves. If you agree, we would be grateful if you would consider whether a stronger assessment is warranted.

Reviewer_oBBtOpenReview ↗
Rating
2
Confidence
2
Informativeness
1
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes Group-Balanced Supervised Fine-Tuning, which reallocates supervision across token classes and spans to prevent long supporting text from dominating short factual payloads. A two-parameter formulation unifies standard SFT, class balancing, and span balancing. Experiments on four knowledge-injection benchmarks, together with grouping and optimization controls, report improved factual acquisition while examining knowledge preservation.

Critical strengths
  • well-written paper
  • theoretical characterization is included
Critical weaknesses
  • The method depends on reliable semantic groups, but guidance for selecting and validating C and S without gold payload boundaries remains limited.
  • SFT-GO with the same role labels is already competitive, leaving the additional benefit of the proposed balancing objective insufficiently isolated.
  • Evaluation remains concentrated in structured factual tasks, with limited evidence across diverse domains, response formats, and model families.
  • The novelty of GB-SFT needs clearer positioning relative to prior role-wise loss balancing and token-group reweighting, including [1], and [2], which are not discussed. The authors should clarify what the within-response grouping and two-parameter formulation add beyond these existing approaches.

[1] Hsieh, Cheng-Yu, et al. "Distilling step-by-step! outperforming larger language models with less training data and smaller model sizes." Findings of the association for computational linguistics: ACL 2023. 2023.

[2] Ye, Ziang, et al. "Disentangling reasoning tokens and boilerplate tokens for language model fine-tuning." Findings of the Association for Computational Linguistics: ACL 2025. 2025.

Questions for authors
  • How should we select and validate C and S without gold payload boundaries, and which diagnostics can identify systematically harmful grouping errors?
  • Under identical role labels and matched optimization conditions, when does GB-SFT consistently improve over SFT-GO, and what explains those differences?
  • Section 3.3 analyzes within-class pooling at a fixed class budget. How should this result be interpreted when the complete class- and span-balanced objectives assign different class budgets?
  • Do the gains persist on real-fact benchmarks under free generation, with unseen phrasings and joint evaluation of answer and citation correctness?
  • Can authors try more datasets and models?
5 comments
AuthorsOpenReview ↗

Part 1/3: grouping provenance and practical value of our problem setting

We thank the reviewer for the two references, which were the most directly useful pointers we received and are now discussed in the paper. We are glad the paper read as well-written and that the theoretical characterization registered as a strength: Lemma 1 does real work in our answer to Q3 below.

We take the informativeness rating seriously, and have used the discussion period to add what we think was really missing rather than to re-argue what was already there: a corpus whose grouping requires no labeling apparatus at all, and an application outside knowledge injection. We answer in two parts.

Part 1: where the groupings come from, and why that setting is critical to many real-world domains, especially ones where regulation mandates boilerplate.

Part 2: related work connecting our formulation to the two works you identify, and to the wider reweighting literature.

Part 3: the objective and the evaluation. What the objective contributes once grouping is held fixed; how to read §3.3; breadth of domains, formats and models, including a sixth corpus (math) and a new task outside knowledge injection (preserving safety guardrails); and free generation, including a measurement that inspires future work.

We now proceed with part 1.

On grouping availability and diagnostics (W1, Q1). The short form of our answer: in law and medicine the output format is mandated, not chosen, e.g. published citation grammars, regulation-structured drug labels, required clinical schemas, and the same regulation that creates the scaffold is what makes the payload locatable. The grouping is a property of the regulated, knowledge-intensive domains where loss-budget misallocation does the most damage, not a convenience we selected for; the practical complaint in these settings is exactly that the content you want to learn is a small part of a mostly mandated document.

We did not write these templates: Housing-QA is an external published benchmark of real U.S. landlord–tenant law, and NC-Real and OpenFDA take their carriers from real statutes and real FDA structured product labels, substituting plausible but counterfactual payload values because a contamination-controlled test of injection requires it. The scaffold is real throughout; only the fact being taught is synthetic. Indeed, we chose corpora whose format is fixed externally so that we were not building the structure we then exploit. The cases outside that setting where the grouping is still free are Tulu, whose seven classes come from Python's own tokenize module, and GSM8K, where the payload boundary is the dataset's own #### delimiter (no parser, citation grammar, tagger or annotation).

A budget-based grouping diagnostic is available for free: at α = 0 each class takes 1/K regardless of class masses, so granularity fixes the split with no labels and no runs. A four-class Housing-QA grouping scores .7536 against .7721 for three, a drop the dose-response predicts before the run because splitting the citation moves its share from .333 to .500. Granularity sets the budget; it is not a direction of improvement. A gold-free test of a labeler is a question we think worth posing and cannot yet answer.

AuthorsOpenReview ↗

Part 2/3: comparing to requested related works

On missing related work (W4). Both are relevant and now discussed in App. E.8; they stand in different relations to our contribution.

Hsieh et al. (2023). A multi-task decomposition, not a token-level reweighting. Their λ weights two separate output sequences, selected by [label] and [rationale] prefixes; it is never assigned a value or ablated. It is nonetheless useful to us, and App. E.8 now cites it in motivation rather than only in contrast: their Eq. 5 ablation concatenates rationale and label into one sequence under uniform cross-entropy and finds it can fall below standard finetuning, independent evidence for the failure mode we address in the single-sequence regime they set aside.

Ye et al. (2025). Much closer, and we should have cited it: they likewise argue that tokens serving different roles should not be weighted equally, identifying two groups with a shuffle-aware discriminator and reweighting the per-token cross-entropy.

The distinction is what the weight is a function of: their ω_g ∝ exp(L̄_g/τ) tracks each group's current mean loss and multiplies its summed loss, so gradient mass stays proportional to token count, whereas ours tracks group cardinality. Equal budget is a corner of our family and a per-corpus tuning target in theirs, which also stops at a binary split with no span level to refine.

We ran α-FT on Housing-QA and NC-Real and RFT on NC-Real under our own grouper (App. E.4). The objectives coincide exactly when their coefficient equals the corpus payload share, which we read as evidence for the parameterization: equal budget is what GB-SFT’s α = 0 means, untuned, on any corpus.

Crediting the pooling-unit axis. Token-mean versus sequence-mean is established vocabulary in RL post-training, and λ-GRPO already presents those variants as one exponent-parameterized family; on the class axis, WIT and prompt-loss weighting place non-uniform weights on semantic token classes. §2 now credits the pooling-unit axis (Dr. GRPO) explicitly.

None expresses what we claim: a budget over semantic classes within a single target sequence whose weight is a function of class mass. That is what makes equal-budget-per-class reachable. A free per-class coefficient leaves a 2000-token prose class dominating a 20-token answer class at any fixed value.

AuthorsOpenReview ↗

Part 3/3: Value added of our objective as well as our grouping; insights about fixed class budget; extended evaluation

On isolating the objective from the grouping (W2, Q2). For an empirical comparison, see the full SFT-GO_oracle versus GB-SFT table across every corpus and model where we ran both (App. E of the revised paper). GB-SFT leads on injection in five of six cells, by margins up to .194; the oracle-grouped SFT-GO arm leads the sixth and both scaffold-preservation cells. One cell cited in review belongs to Anchor-Only rather than to SFT-GO, and we give that correction in our response to that review. On the mechanism: SFT-GO optimizes a mixture of worst-group and average loss, so it adaptively emphasizes whichever group is currently hardest. On fact-editing corpora that is typically the long scaffold the model has least reason to fit, which is the opposite of where supervision should go. Our budget is fixed by the grouping rather than chased during training. That single axis is why the ordering between the two depends on whether the metric rewards injection or preservation.

On §3.3 at fixed class budget (Q3). A fair question. The two analyses are complementary halves of one decomposition.

Lemma 1 holds the class budget pcp_c fixed and isolates the within-class pooling difference. That is why it explains the otherwise surprising result that class- and span-balancing perform comparably: at matched budget the allocations differ only by the within-class covariance between span length and span-mean loss, and the coefficient of variation of span lengths is small on our corpora.

Fig. 9 then varies pcp_c and shows the between-class effect is what moves the metric. Neither alone accounts for the result; together, they say the budget is the active variable and the pooling unit is not. We will make that linkage explicit at the end of §3.3.

On more datasets and models (W3, Q5). The evaluation covers five corpora across law, medicine and code, two model families, four sizes from 0.8B to 27B (App. D.3), five LoRA ranks (D.4) and four learning rates (D.5). Tulu is deliberately the case with no payload/scaffold decomposition and no human labeling. We accept that response-format diversity is thinner than domain diversity, and have partly addressed it.

We added a sixth corpus, GSM8K, which differs on every axis you raise: a new domain (math), a free-form chat response format, and a task that is not knowledge injection at all. SFT drops harmful-prompt refusal 95.5% → 81.0%; erosion is monotone in α; GB-SFT at α = 0 retains about a third of that at matched task accuracy, and the harmful/benign separation improves rather than refusal rising uniformly. Group balancing benefits a new task and setting (App. E.7).

On generation, phrasing, and joint correctness (Q4). We reported promising results in the paper that partially address this but acknowledge a remaining gap. App. D.2 retrains against a held-out phrasing whose 126 questions share 0.336 of their content words with the original; the relative advantage is unchanged (+.1056 vs +.1058), so the gains are not phrasing memorization. App. D.8 reports free generation, where the ordering is preserved but absolute rates are far lower.

Per request, joint answer-and-citation correctness under free generation on Housing-QA held-in (one seed, 150 items). GB-SFT leads on both components (answer .813 against .780, exact citation .407 against .360) but the joint rate is .493 against .487, a negligible improvement: the component gains do not compound, suggesting the two error types are anti-correlated rather than independent. Held-out joint correctness is ~0 for every arm, since held-out citations are not taught.

This is a corpus limitation rather than a loss limitation, which shapes the future work that would close it. Constrained decoding does not help (forcing a well-formed citation scores .560 against .580 for greedy), so the bottleneck is retrieving the payload under the model's own decoding, not emitting the right surface form. That points to the corpus-level intervention Allen-Zhu & Li (2024) identify, namely that a fact becomes extractable once it appears across sufficient paraphrase and format diversity. Such an intervention is orthogonal to how the loss is weighted.

Teacher-forced ranking is the instrument on which the methods we compare against establish their own claims, so our improvements are measured on the same footing, and reporting free generation alongside it is more than the comparison requires.

We thank the reviewer for their careful attention and valuable feedback. For convenience, all substantial new content from the discussion period is in App. E of our updated paper.

Comments on the paper

7 comments
AuthorsOpenReview ↗

Part 1/3: Importance of the problem settings our method benefits; improvement on additional setting & task

We thank the reviewer for an unusually close reading. We are glad the motivation came through as "intuitive and clearly grounded," and we particularly appreciate the characterization of the Fig. 3 partition study as "well designed and relatively honest." That figure is where we most wanted to be held to account, since it tests whether our gains come from role-aware grouping or a particular reweighting.

One weakness (W3) we can simply fix: the noted appendix duplication was caused by a single stray \input. Fixing it resolves corrupted references and saves 2.5 pages that we replace with experiments the reviewer feedback inspired 🙂 We answer the rest in three parts.

Part 1: the setting. That the templated regime is where knowledge-intensive fine-tuning is actually deployed, and that the method extends beyond it, including a new safety guardrail erosion result.

Part 2: where the groupings come from, and the second parameter. That a cheaply obtained grouping does not imply a small gain, and what β is for.

Part 3: the objective. What it contributes once the grouping is held fixed, and what uniform weighting within spans does and does not cost. The objection we found most useful we answer with a new experiment rather than an argument.

We proceed with part 1.

On whether the templated setting is artificial (W2). This is the concern we most want to answer, and we think its premise inverts the situation.

In law and medicine the output format is mandated, not chosen. Citations follow published grammars; FDA drug labels are structured by regulation; clinical documentation, discharge summaries and diagnostic coding follow required schemas. In each, the valuable payload (e.g. the holding, the dose, the obligation) is a small span inside a large body of obligatory scaffold.

One mechanism produces both halves. The regulation that creates the scaffold is what makes the payload locatable. So we would put the conclusion differently from the review: not that the method works only where structure happens to exist, but that the pathology is worst exactly where structure is published. The practical complaint in these domains is precisely this: what you want the model to learn is a small part of a document that is mostly mandated boilerplate, and ordinary fine-tuning spends its budget accordingly.

We did not write these templates. Housing-QA is an external published benchmark derived from the Legal Services Corporation Eviction Laws Database (Zheng et al., 2025) containing real U.S. landlord–tenant law, real citations and real answers. Thus, the 81.4% / 5.1% split in Fig. 1 measures how those documents are written rather than anything we constructed. NC-Real and OpenFDA take their carriers from real statutes and real FDA structured product labels, substituting plausible but counterfactual payload values as contamination control requires: the scaffold is real throughout, only the taught fact is synthetic. We chose corpora whose format is fixed externally, by citation convention or regulation, precisely so that we were not constructing the structure we then exploit.

This is why the setting deserves attention rather than being a corner of the problem: regulated, knowledge-intensive domains are where fine-tuning on new facts is actually deployed, and they are precisely the regime in which Fig. 1's payload-drowning 81%/5% split arises.

On whether the method applies outside templated settings (W2-ii). It does, and we add evidence rather than argument. Tulu's seven classes come from Python's own tokenize module, and GSM8K's payload boundary is the dataset's own #### delimiter (no parser, citation grammar, tagger or annotation) on a general-domain corpus nobody would call templated. The grouping is free: the payload boundary is a delimiter the dataset already ships.

We also use that free grouping to study refusal: an alignment behavior rather than a fact, elicited by prompts sharing nothing with the fine-tuning corpus. The question is whether the loss budget governs collateral damage of that kind. On Qwen2.5-3B-Instruct, SFT on GSM8K drops harmful-prompt refusal from 95.5% to 81.0% (LLM-judged, five seeds, 420 prompts), and the arms order 81.0 / 82.9 / 85.7 / 89.2 as α goes 1.0 → −0.5 (trend z = 7.48). Erosion of safety refusal is monotone in the budget parameter, at matched task accuracy, and GB-SFT with α = 0 retains about a third of what SFT loses (+4.8 pp, t(4) = 3.10, p ≈ 0.036, five of five seeds; 67.4% against 68.4% on GSM8K).

We scope it precisely: where a corpus's payload is a small fraction of its supervised tokens and refusal erodes under fine-tuning, budget correction mitigates the erosion. Both conditions are binding in practice, and App. E.7 gives the full analysis.

AuthorsOpenReview ↗

Part 2/3: Grouping provenance and gains; clarifying the role of GB-SFT's second parameter

On whether cheap groupings imply small gains (W2-iii). This is the circularity worry, and it needs one more premise than the review supplies: that a payload which is easy to label is one SFT already handles. Housing-QA refutes it. Its citations are regex-matchable, yet it is where ungrouped SFT fails worst: .6523 against .7695. Distinctiveness is a property of the payload's form; whether SFT fails is a property of its budget share (5.1% here). Our most labelable corpus is where ungrouped SFT fails worst: provenance and benefit are not entangled.

We concede the converse: automatic labeling is viable only at the distinctive end, which is why the paper already reports the ≈0% NC-Real tagger result. NC-Real's gain under gold labels is large, so what is confined is automatic labeling, not the method.

Where the groupings come from. We agree the anchored arms take roles by construction. Across the paper the sources form a mostly annotation-free cost ladder: a citation grammar (Housing-QA), structured source labels (OpenFDA), a lexer (Tulu), a delimiter (GSM8K). Nor is the template the only route: a prompted LLM grouper retains 98.8% of the gold gain on Housing-QA, and a cross-fit detector ~34% from ~25 labeled datums.

On cost, which you rightly note we omitted: App. E reports tagger ↔ gold agreement per corpus (.9997 token-F1 on Housing-QA, .983 and .707 elsewhere). We did not instrument wall-clock or token spend, so we state the structural cost: one LLM call per datum, a one-off pass that does not recur at training or inference. Genuinely out of scope is free-form prose whose payload is neither delimited nor distinctive, now stated in Limitations.

On β being redundant (W1-i). We agree with the observation, and will change the framing with three notes:

What β is for. The formulation began as one parameter; the second exponent was added to unify SFT, class- and span-balancing as corners of a single object. β is a control, not a knob: its inertness is the result it was there to establish. Without varying the pooling unit, one cannot tell whether a class-level gain is class reweighting or span-length imbalance within classes. Lemma 1 provides that decomposition using two parameters, so β acting only through the budget is a result obtained using β, the same result as Fig. 9 (r = −.945).

A correction to the numbers you quote. Two of the three cells came from a 96-datum subsample, which the caption did not say. Re-run at full corpus, three seeds: .7719 / .7560 / .7721 (Qwen) and .7489 / .7815 / .7460 (Llama). The corners (β = 0 pooling by class, β = 1 by span) differ by .0002 and .0029 against seed spreads of .010–.025. The inertness you identify is cleaner than we reported, and now replicates on a second family. We will note that on Llama, the β = 0.5 midpoint sits ~3.5 pp above both corners (t ≈ 3.2, three seeds, three uncorrected comparisons). We do not read one mid-grid cell at that sample size as evidence for β as a knob. This finding is why we claim equivalence at the corners rather than flatness across the range.

What we changed. We read this as parsimony we can hand practitioners: one dial α, one structural switch β, and have updated our exposition accordingly, with corner recovery as a sanity property rather than a headline claim.

AuthorsOpenReview ↗

Part 3/3: Clarifying mechanisms in the objective that lead to improvement

On SFT-GO with oracle grouping (W1-ii). The review reads the OpenFDA Llama overwrite column as .891 for SFT-GO_oracle against .852 for GB-SFT. In Table 3 the .891 belongs to Anchor-Only, one of our own anchored variants; SFT-GO_oracle reads .747 in that column, so the correct comparison is .747 against .852 leaving GB-SFT ahead by .105. We flag it because it reverses the inference the cell was cited for. Our §4.1 text already notes that Anchor-Only "produces the most overwrite, eking out GB-SFT by a narrow margin" on this benchmark. The remaining cell, Housing-QA Llama cite-out (.815 vs .723), is correct, and it is on the scaffold-preservation axis rather than the injection axis the claim concerns.

To settle this properly rather than cell by cell, we will add a table of SFT-GO_oracle versus GB-SFT across the benchmarks where we ran both, on each one's primary metric. GB-SFT leads on injection in the large majority: NC-Real .954 vs .910 (Qwen), OpenFDA .799 vs .605 (Qwen) and .852 vs .747 (Llama); on RealVal, SFT-GO_oracle falls below ungrouped SFT. We will concede explicitly the cells where SFT-GO_oracle leads, including Housing-QA cite-out on both models, and soften the Table 1 caption. We agree good grouping is a large share of the effect (Fig. 3 makes that point), but the objective is not inert given the grouping.

The mechanism is identifiable: SFT-GO targets the worst group, which on fact-editing corpora is the scaffold, not the payload. It optimizes a mixture of worst-group and average loss, so it emphasizes whichever group is currently hardest. In our fact-editing setting this ends up being the long scaffold the model has least reason to fit, the opposite of where supervision should go. Meanwhile, our budget is fixed by the grouping rather than chased during training, and that difference is also why the ordering between the two flips with the axis measured (knowledge payload injection vs scaffold preservation).

On uniform weighting within spans (W1-iii). This is the strongest objection in the three reviews and we accept the framing: within a span our weights are uniform, and your examples are real structure we do not exploit.

(1) The limitation is in the grouping rather than the framework. C and S are arbitrary maps over supervised tokens, so splitting the citation into identifier and reporter is a change of grouping, not of objective. We ran exactly that four-class Housing-QA grouping: it scores .7536 ± .0054 against .7721 ± .0136 for our three-class partition.

(2) The finer grouping is worse for a reason we can state in advance. Equal-budget-per-class is not invariant to subdividing a class: at α = 0 each of K classes receives 1/K regardless of class masses, so splitting the citation increases that concept's share from .333 to .500. Fig. 9 identifies citation share as the active coordinate (r = −.945), so the four-class drop is predicted by the dose-response before the arm is run: granularity sets the budget, it is not a direction of improvement.

Ordering every Housing-QA grouping by the citation budget share it induces gives a monotone series: at induced shares of .333, .444 and .500, held-in AUC is .7721, .7559 and .7536. A three-class partition tuned to that same .500 share scores .7414, within .0122 of the four-class arm, which the residual answer-share gap accounts for. Granularity and budget move together here, and it is the budget that tracks the metric.

So we accept the description of the method as uniform within spans while disputing that further subdivision is the fix. We note one limit of the control: a three-class partition has one free parameter against the four-class target's two, so each match fixes one coordinate and leaks the other.

(3) The two axes compose rather than compete. Our weights alter the loss's aggregation geometry; Rho-1 and DFT alter its per-token scoring function, and the product is well defined. We ran that product, DFT weights inside our groups, and found it trades acquisition for preservation: held-in AUC rises to .7901 ± .0171 from .7721, while held-out citation recall falls .6043 → .5422. That is the exchange Fig. 9 already describes rather than an escape from it.

We thank the reviewer for their careful attention and valuable feedback. For convenience, all substantial new content from the discussion period is in App. E of our updated paper.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.