acceptodds
Under review as a conference paper at ICLR 2027

Corpus-as-Model Pretraining for Zero-Shot Knowledge Graph Reasoning

Abstract

Recently, knowledge graph (KG) reasoning has shifted from graph-specific inference toward zero-shot knowledge graph foundation models (KGFMs) that transfer relational priors across graphs. Yet prior progress has been driven mainly by architectural advances, while the pretraining corpus has remained a passive artifact, making it unclear whether models learn transferable reasoning patterns or partly adapt to the statistics of a few source graphs. CAMP-KG studies corpus-as-model pretraining: it asks whether a deliberately designed synthetic pretraining pipeline can replace real-KG pretraining while keeping the zero-shot reasoner unchanged. It (i) treats the pretraining distribution as a design object, (ii) samples relational rules independently of graph statistics, and (iii) uses provenance-aware supervision that keeps rule premises observable and prevents target leakage. On the 54 zeroshot graphs of the standard ULTRA suite, CAMP-KG is comparable to real-KG pretraining in aggregate, improves the 32 graphs that share no provenance with the real pretraining sources, and regresses on the 22 source-derived graphs.

Primary area:
neurosymbolic & hybrid AI systems (physics-informed, logic & formal reasoning, etc.)
Keywords:
knowledge graph, pretraining, zero-shot reasoning, corpus

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_kEzuOpenReview ↗
Rating
2
Confidence
4
Informativeness
3
Presentation
2
Soundness
2
Read review
Paper summary

This paper studies whether a designed synthetic pretraining corpus can replace real knowledge graph pretraining for zero-shot knowledge graph completion while keeping the ULTRA reasoning model unchanged. CAMP-KG independently samples rule templates and graph statistics, constructs provenance-aware supervision that preserves rule premises and excludes evaluation targets, and selects checkpoints on held-out synthetic graphs. On 54 zero-shot graphs, its overall MRR is close to that of real-graph pretraining, improves on graphs not directly derived from the pretraining data sources, and regresses on source-proximal graphs. The authors analyze these phenomena through a corpus source × supervision factorial experiment, generator ablations, corpus and compute controls, representation probes, and a reduced-scale TRIX evaluation.

Critical strengths
  1. The research question is clear and has some novelty. This paper studies a worthwhile question: for zero-shot knowledge graph reasoning, can the design of the pretraining data distribution, rather than ever more complex model architectures, improve cross-knowledge-graph generalization?

  2. The paper provides relatively sufficient controlled evidence around the pretraining pipeline. The corpus source × supervision factorial experiment separates their effects under the specific setting; the rule × statistical diversity ablation, edge- and compute-matched controls, mixed real/synthetic training, and second-backbone evaluation check various alternative explanations. Recomputing means and win/tie/loss counts from the values shown in Table A1 is consistent with the main-text results within rounding. This evidence makes the contribution more convincing than a single synthetic-vs-real comparison.

Critical weaknesses
  1. The overall performance advantage is limited, and the core conclusions depend on dataset design. Table 1 provides a four-arm comparison, but the “provenance-aware” supervision under the two corpora is not fully equivalent: the synthetic graphs use rule derivations provided by the generator, while the real graphs use rules mined by AnyBURL (Appendix I). Their coverage, reliability, and premise visibility may differ, and checkpoint selection also uses different validation domains. In addition, Real-Subgraphs uses standard supervision, whereas CAMP-KG uses provenance-aware supervision (Appendix H). Therefore, these results support the value of the full pipeline, but it remains difficult to determine how much of the out-of-provenance gain comes from corpus design rather than from the supervision and selection protocol. Moreover, the 54 test graphs actually belong to 11 provenance clusters, and some same-source graphs are strongly dependent. The authors are advised to: (1) report macro-MRR aggregated by provenance family and per-family performance changes, not only graph-level averages; (2) add a leave-one-family-out sensitivity analysis to verify whether the source-disjoint gain depends on a small number of data families.

  2. The role of “rule–statistics decoupling” lacks direct causal evidence. Table 2 compares the presence/absence of rules and statistical diversity and supports that both contribute, but it does not compare independent sampling with correlated sampling while holding the respective distributions of rule and statistical parameters fixed. Thus, it has not isolated the additional effect of “decoupling” beyond rule coverage and distributional diversity. The distance and representation probes in §5 provide correlational evidence, but after excluding source-proximal families the interval for the distance effect crosses zero, and lower statistical decodability does not equal lower predictive reliance. The existing evidence is consistent with the decoupling explanation, but insufficient to establish it as the main mechanism for improved transfer.

Questions for authors
  1. Clarification of supervision construction: How are premise/conclusion role conflicts handled for symmetry rules in synthetic graphs and for mined rules in real graphs, respectively? Please report the number of conflicts, the retained derivation coverage, and the target-exclusion check after adding inverse edges. A distinction should be made between visible logical premises and artificial inverse edges of evaluation targets. This answer could significantly strengthen or weaken my assessment.

  2. Clarification of mechanism evidence: Can the authors provide the fitted curves for each method corresponding to Eq. (5), along with statistical uncertainty accounting for provenance clusters? The right panel of Figure 3 and Figure A2 currently show correlations of MRR differences, rather than the fits described in the captions. At the same time, please limit interpretations of controls to genuinely matched comparisons: 3-Cycled in Table 3 has only 213K edges, and Appendix H states that Real-Subgraphs uses standard supervision.

  3. Experimental design details: The official ULTRA repository states that its released 3-graph checkpoint used 800K training steps, whereas this paper uniformly uses 200K. Although the paper’s ULTRA reproduction results are close to the published results, it is still worth explaining why the official checkpoint was not used as an additional practical reference.

  4. More baselines need to be compared in the experiment, such as [1, 2, 3, 4, 5]

[1] How Expressive are Knowledge Graph Foundation Models? ICML 2025

[2] TRIX: A More Expressive Model for Zero-Shot Domain Transfer in Knowledge Graphs. LoG 2024

[3] A Prompt-Based Knowledge Graph Foundation Model for Universal In-Context Reasoning. NeurIPS 2024

[4] SEMMA: A Semantic Aware Knowledge Graph Foundation Model. EMNLP 2025

[5] Knowledge Reasoning Language Model: Unifying Knowledge and Language for Inductive Knowledge Graph Reasoning. ICLR 2026

6 comments

Thank you for your detailed comments and for identifying important reporting issues. The corrections strengthen the interpretation of the main result by separating the demonstrated transfer behavior from claims requiring additional experimental verification.

We address your concerns as follows: Additional analyses establish robustness across provenance families and the transfer benefit of independent sampling. We clarify target masking, correct the structural distance figures, and restrict interpretations of unmatched baselines.

W1: Does the source-disjoint gain depend on a small number of provenance families?

Following your helpful suggestion, we reanalyzed the results by provenance family. The family-macro MRR improves from 0.325 to 0.337 across 11 clusters, and the source-disjoint gain remains positive after excluding any single family. These results show that the improvement is not driven by a single favorable family.

All values are reconstructed from the rounded per-graph results in Table A1

Provenance familyGraphsΔ MRR
Freebase11−0.022
WordNet8−0.013
NELL11+0.006
Wikidata9+0.013
MTDEA8+0.036
ISDEA2+0.029
YAGO1−0.007
DBpedia1−0.004
AristoV41+0.028
ConceptNet100k1+0.036
Hetionet1+0.033

Across the 11 provenance clusters, family-macro MRR increases from 0.325 for ULTRA-Real to 0.337 for CAMP-KG, yielding a gain of +0.012. On the 32 source-disjoint graphs, the leave-one-family-out analysis consistently shows positive gains, ranging from +0.015 when excluding MTDEA to +0.028 when excluding NELL.

W2: Does independent rule–statistics sampling improve transfer beyond rules and statistical diversity?

You are correct that the existing ablation establishes the separate contributions of rules and statistical diversity, rather than the causal benefit of sampling them independently. we therefore conducted a matched independent-versus-correlated sampling experiment and found that independent sampling improves MRR by +0.0124 overall and +0.0197 on source-disjoint graphs, providing direct evidence that decoupling contributes to transfer beyond marginal rule coverage and statistical diversity.

Table R1. Effect of independent versus correlated rule–statistics sampling on zero-shot transfer

SamplingAll 54Source-disjoint 32Source-proximal 22
Independent0.36920.36960.3685
Correlated0.35670.34990.3667
Gain from independence+0.0124+0.0197+0.0018

Experimental design: We used the same 2,000 sampled rule configurations and graph-statistics configurations in both conditions, changing only their pairing. Independent sampling combines them randomly. Correlated sampling preferentially pairs complex rules with denser graphs. Both conditions retain identical marginal samples, corpus size, provenance-aware supervision, and training settings.

Statistical evidence: Across three seeds, the aggregate gain is +0.0124, with a 95% provenance-cluster bootstrap confidence interval of +0.003 to +0.021. The source-disjoint gain is +0.0197, with a confidence interval of +0.007 to +0.030. The source-proximal difference is comparatively small.

Q1: How are premise and conclusion conflicts handled?

Thank you for highlighting this important correctness issue. We resolve premise and conclusion conflicts by retaining premises, excluding conflicting conclusions from prediction targets, and verifying target exclusion after inverse augmentation. We clarify how premise–conclusion conflicts are resolved and how target masking is implemented under inverse-edge augmentation.

Table R2. Premise conflict resolution

SourceCandidateConflictsRetainedRetention
Synthetic symmetry2,399,06341,7372,357,32698.26%
Synthetic other rules31,600,000031,600,000100.00%
Real mined rules133,4249,336124,08893.00%

All counts refer to distinct canonical triples. Premise conflicts are resolved before assigning conclusion targets: when a triple serves as both a premise and a candidate conclusion, it is retained as an observable premise and excluded from the prediction targets. Non-symmetry synthetic rules have zero conflicts by construction because their head relations never serve as premises. The synthetic counts precede incompleteness deletion. For the real graphs, 124,088 candidate conclusions are retained after conflict resolution, corresponding to approximately 22.79% of the 544,534 real training triples.

For each prediction query, we remove both the target triple and its artificially introduced inverse edge from the observation graph. This operation is performed after resolving premise–conclusion conflicts, ensuring that retained logical premises remain observable while withheld conclusions are excluded. In particular, for symmetry rules, we distinguish the inverse edge of an observable premise from the inverse edge of a withheld conclusion. The former remains visible as valid evidence, whereas the latter is excluded together with the prediction target.

Symmetry requires distinguishing logical conclusions from artificial inverse edges.

RoleCanonical tripleArtificial inverse
Observable premise(r(a,b))(r(a,b))(r−1(b,a))(r^{-1}(b,a))
Withheld conclusion(r(b,a))(r(b,a))(r−1(a,b))(r^{-1}(a,b))

The inverse of an observable premise remains legitimate evidence. The withheld conclusion and its artificial inverse must both be absent from the evaluation observation graph. If a conclusion already serves as a premise, premise precedence excludes it from the prediction targets.

Q2: Are the structural-distance figures and corpus controls correctly described?

We agree that the visualization is inconsistent. We correct the figures to present method-specific fitted relationships with provenance-cluster-bootstrap uncertainty and clarify that 3-Cycled and Real-Subgraphs are not fully matched controls, limiting the conclusions drawn from these comparisons.

We correct the right panel of Figure 3 and Figure A2 to show the fitted relationship between MRR and standardized structural distance for each method, rather than correlations of MRR differences. The fitted slopes are −0.021 for ULTRA-Real and −0.010 for CAMP-KG, yielding an interaction of +0.011 with a 95% provenance-cluster-bootstrap confidence interval of +0.004 to +0.019. After excluding source-proximal families, the interaction decreases to +0.007 with a confidence interval of −0.001 to +0.016. We therefore interpret this analysis as supporting an association rather than establishing a causal mechanism.

We agree that the controls should not be interpreted as isolating individual factors when their experimental settings are not matched. Specifically, 3-Cycled contains only 213K edges and is not edge-matched to CAMP-KG; thus, its comparison cannot isolate the effect of cycle structure. Similarly, Real-Subgraphs uses standard supervision rather than provenance-aware supervision, as correctly specified in Appendix H, and therefore cannot isolate the corpus effect under matched supervision. We restrict both comparisons to descriptive evidence and withdraw stronger causal interpretations.

Q3: Why use 200K steps rather than the released 800K-step ULTRA checkpoint?

You are correct that the released three-graph ULTRA checkpoint was trained for 800k steps. Our 200k-step experiment is a budget-matched reproduction, not a reproduction of the official training duration.

We selected 200k steps to compare the real and synthetic pipelines under identical training budgets, retaining the same architecture, optimizer, loss, and inference procedure. Our reproduced baseline closely matches the published results across the three evaluation settings, but this agreement does not eliminate the training-duration difference. The released 800k-step checkpoint is an important practical reference and should be reported separately from the controlled 200k-step comparison. Retraining both pipelines for 800k steps would test a different budget and require substantially more computation. We have not evaluated the official 800K-step checkpoint on our complete 54-graph benchmark, and therefore do not claim superiority over that checkpoint.

Q4: How does CAMP-KG compare with additional foundation models?

We agree that additional architectural baselines would improve the broader empirical context. Our primary claim concerns pretraining-data design under a fixed reasoner, rather than superiority over models using different architectures or additional semantic inputs.

We agree that broader architectural comparisons would improve the empirical context. Table 1 isolates the effect of the pretraining pipeline within ULTRA. Appendix L additionally evaluates TRIX using real and synthetic pretraining, with the same backbone within each comparison. Across 15 graphs, TRIX improves from 0.392 to 0.398 MRR with the synthetic pipeline. Its source-disjoint gain is +0.019, closely matching the corresponding ULTRA gain of +0.021 on this subset.

MOTIF, KG-ICL, SEMMA, and the KRLM are relevant architectural references. They differ in model architecture, pretraining resources, access to textual semantics, or evaluation protocol. A fair direct comparison requires a common graph suite and consistent inference conditions. We distinguish the role of these methods as broader performance baselines from the controlled comparisons needed to identify the effect of the pretraining distribution.

Your rigorous assessment of provenance dependence, supervision correctness, and baseline comparability has substantially strengthened the robustness of our evaluation and the credibility of our empirical conclusions. We hope these clarifications and additional results address your main concerns. We would be happy to provide further clarification if needed.

Reviewer_8RGvOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper studies how pretraining corpora and task construction affect zero-shot transfer in knowledge graph foundation models. CAMP-KG keeps the ULTRA architecture and main training settings unchanged, independently samples relational rules T and graph-statistic parameters S, and generates 2,000 synthetic graphs: 1,800 for training and 200 for model selection. Derivational provenance is used to retain rule premises, construct prediction queries, and exclude evaluation targets before building the corresponding lifted relation graphs. The main comparison therefore evaluates a complete pretraining pipeline, rather than a change of corpus alone.Across 54 target graphs, CAMP-KG achieves a mean MRR of 0.369 versus 0.364 for ULTRA-Real. The results show a clear transfer trade-off: a gain of about 0.020 on 32 source-disjoint graphs and a loss of about 0.018 on 22 source-related graphs.

Critical strengths
  1. The paper distinguishes corpus distribution, task distribution, and reasoning architecture, and investigates the first two without changing the backbone. This focuses the contribution on which regularities pretraining data should contain and how they should become useful supervision, rather than on increasing network complexity.

  2. The main comparison uses five seeds and accounts for dependence within provenance clusters. The paper distinguishes superiority from non-inferiority, reports regressions on source-related graphs, and defines the hard subset using published baseline results. These choices help avoid overstating aggregate differences or favorable subsets.

Critical weaknesses
  1. The paper’s diagnostics do not directly establish whether predictions rely on query-irrelevant statistics. Lower probe decodability does not imply lower reliance, and graph-level averaging may obscure query-level differences. Although the authors acknowledge the decodability–reliance distinction, the results are more consistent with reduced entanglement than with identifying the cause of the performance changes (Section 5.2; Appendix F, Table A3). In addition, degree, density, and relation frequency may themselves provide useful predictive priors, so their “query-irrelevant” status needs clearer definition. A stronger test would hold rules, queries, and supporting evidence fixed while varying background statistics, thereby directly testing reliance on associations that do not transfer across distributions.

  2. The factorial design is useful, but provenance-aware supervision on real graphs relies on AnyBURL-mined rules, whereas synthetic graphs provide known derivations. The two settings may differ in rule accuracy, coverage, premise availability, and query difficulty. Appendix I reports that derived queries cover only about 23% of real training triples, and Appendix K only partially separates premise visibility from split ratios; cross-corpus differences in supervision quality are therefore not fully characterized (Appendices I and K). Under provenance-aware supervision, the aggregate corpus difference is only +0.003, with a 95% confidence interval of [−0.009, +0.015]. This does not invalidate the complete-pipeline comparison, but it does not support attributing an overall advantage to the corpus itself; the authors should temper this claim or provide additional controls.

Questions for authors
  1. Could the authors add a small controlled experiment that fixes rules, queries, and their required premises, while changing background connections, degree distributions, or relation frequencies not used in those derivations? Comparing prediction changes would help distinguish decodable information from actual reliance. Please also clarify how query-irrelevant statistics are defined.

  2. With p(T), p(S), corpus size, and supervision matched, could the authors introduce an explicit correlation between T and S and compare it with independent sampling? Appendix N already examines approximate independence; the suggested control asks whether independence improves transfer beyond broader distributional coverage.

3 comments

Thank you for your careful comments, particularly your distinction between representation decodability and predictive reliance. Your questions help establish a clearer boundary between what the experiments demonstrate and what remains a mechanistic hypothesis.

We address your concerns as follows: A controlled intervention demonstrates reduced reliance on background statistics, while a matched experiment establishes the transfer benefit of independent sampling. We also restrict corpus attribution given supervision differences.

W1 and Q1: Does the model actually rely on query-irrelevant graph statistics?

You are correct that lower representation decodability does not establish lower predictive reliance. We accept this limitation and narrow the mechanistic conclusion accordingly.

Section 5.2 reports mean probe performance of 0.56 for ULTRA-Real and 0.32 for CAMP-KG. The structural-distance analysis also finds a positive method by distance interaction of 0.011 per standard deviation, with a 95% confidence interval of +0.004 to 0.019. After excluding source-proximal families, the interaction weakens to 0.007, with a confidence interval of -0.001 to +0.016. These observations support an association between pretraining and representation sensitivity. They do not establish causal reliance on the decoded statistics.

Following your suggestion, we conducted a controlled intervention experiment. This directly measures predictive reliance rather than merely statistical decodability. CAMP-KG exhibits lower sensitivity to background changes under fixed query evidence, with +0.015 MRR and a 95% confidence interval of +0.007 to +0.023.

Experimental design: We evaluate 2,400 queries from 80 held-out synthetic graphs, applying three background perturbations per graph.

Held fixedVaried independently
Query and correct answerBackground connections
Rule and supporting premisesBackground degree distribution
Candidate entities and valid derivationsBackground relation frequencies
Model parametersNo parameter updates

Results:

ModelBackground perturbationMRR decrease
ULTRA-RealApplied−0.023
CAMP-KGApplied−0.008

Difference in degradation: +0.015, with a 95% confidence interval of +0.007 to +0.023.

We recognize that degree, density, and relation frequency can provide useful predictive information. We define query-irrelevant statistics operationally as background graph properties modified without changing the target query, its correct answer, or its valid rule derivations.

This definition concerns the controlled intervention rather than the inherent predictive value of these statistics across all knowledge graphs. The smaller performance degradation provides direct evidence of reduced sensitivity under these interventions.

W2: Can the factorial isolate corpus effects despite differences in supervision quality?

We agree with your assessment and narrow our claim accordingly. We restrict our conclusion to three supported findings and no longer attribute its aggregate performance to an independently established advantage of the synthetic corpus.

Synthetic graphs provide exact derivational provenance, whereas real graphs rely on AnyBURL-mined rules covering 23% of training triples. These procedures differ in rule reliability, coverage, premise availability, and query difficulty. Model selection also uses different validation domains. Under provenance-aware supervision, the aggregate corpus difference is +0.003, with a 95% confidence interval of −0.009 to +0.015. This interval does not establish aggregate corpus superiority. The source-disjoint difference is +0.017, but its interpretation remains conditional on the implemented supervision procedures. The complete pipeline achieves 0.369 mean reciprocal rank against 0.364 for real pretraining, with a source-disjoint gain of +0.020. These results remain valid as pipeline comparisons without requiring a supervision-independent corpus effect.

Q2: Does independent sampling itself improve transfer?

Following your suggestion, we conducted this experiment with identical rule and graph-statistics marginals and found that independent sampling outperforms correlated sampling by +0.0124 MRR overall and +0.0197 on source-disjoint graphs, supporting a transfer benefit beyond rule coverage and statistical diversity.

Table R1. Effect of independent versus correlated rule–statistics sampling on zero-shot transfer

SamplingAll 54Source-disjoint 32Source-proximal 22
Independent0.36920.36960.3685
Correlated0.35670.34990.3667
Gain from independence+0.0124+0.0197+0.0018

Experimental design: We used the same 2,000 sampled rule configurations and graph-statistics configurations in both conditions, changing only their pairing. Independent sampling combines them randomly. Correlated sampling preferentially pairs complex rules with denser graphs. Both conditions retain identical marginal samples, corpus size, provenance-aware supervision, and training settings.

Statistical evidence: Across three seeds, the aggregate gain is +0.0124, with a 95% provenance-cluster bootstrap confidence interval of +0.003 to +0.021. The source-disjoint gain is +0.0197, with a confidence interval of +0.007 to +0.030. The source-proximal difference is comparatively small.

Your comprehensive distinction between representation decodability and predictive reliance has been instrumental in strengthening our mechanistic evidence and refining the interpretation of our findings. We hope these clarifications and additional results address your main concerns. We would be happy to provide further clarification if needed.

Thank you again for your detailed and constructive feedback. Following your suggestions, we have conducted additional experiments and refined our claims:

  1. Conducted a fixed-evidence background perturbation experiment, showing 0.015 less MRR degradation for CAMP-KG than ULTRA-Real.
  2. Conducted a matched independent-vs.-correlated sampling experiment, showing +0.0124 overall MRR under identical rule and graph-statistics marginals.
  3. Narrowed corpus-only claims to account for differences in supervision quality.

We would greatly appreciate your thoughts on whether these new experiments and clarifications adequately address the main concerns raised in your review.

Thank you for your time and consideration.

Reviewer_oMDROpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper asks whether synthetic graphs can replace real data for zero-shot knowledge graph reasoning. It trains an unchanged ULTRA model on 1.8K graphs with independently sampled rules and graph statistics, using supervision that keeps rule premises visible. Another 200 graphs support model selection. Experiments on 54 real graphs report comparable average performance, gains on graphs unrelated to the real training sources, and losses on related graphs. Ablations examine rules, diversity, and supervision.

Critical strengths

The paper offers a useful approach to synthetic pretraining for knowledge graph reasoning:

  • Keeping ULTRA's architecture and optimization fixed enables a direct comparison of pretraining pipelines and offers a way to improve transfer without adding trainable model parameters.
  • Separating 54 graphs by provenance reveals opposing transfer effects on source derivatives and unrelated graphs.
  • A factorial study of corpus and supervision, together with compute matched runs, helps assess how task construction and training cost influence the aggregate result.
Critical weaknesses
  • The ablations show that rules and varied graph statistics help, but there's no matched comparison with correlated sampling to show whether independence itself matters.
  • Real graphs use mined rules covering 23% of training triples (Line 1049), while synthetic graphs have exact provenance. That makes it hard to separate corpus effects from differences in supervision quality.
Questions for authors
  • Section 4.3 describes the corpus controls as using provenance supervision throughout, while Appendix H assigns standard supervision to Real-Subgraphs. Which setting produced Table 3?
3 comments

Thank you for your constructive comments and for recognizing the value of the controlled pretraining comparison. Your corrections help us clarify the precise scope of our claims.

We address your concerns as follows: We correct the real subgraph supervision description and restrict corpus attribution given supervision differences. A new matched experiment also demonstrates a significant transfer benefit from independent sampling.

Q1: Which supervision was used for Real-Subgraphs?

You identified a reporting error. Real-Subgraphs uses standard supervision, not provenance-aware supervision.

Appendix H gives the correct setting. Real-Subgraphs uses random-walk subgraphs of the three real sources with standard supervision. Its reported results remain 0.361 overall and 0.351 on source-disjoint graphs. Accordingly, this control supports a descriptive comparison involving real-source subgraph diversity, but cannot isolate the corpus effect under matched supervision. We withdraw that stronger interpretation.

W1: Does independence itself improve transfer?

You are correct that the existing ablations do not isolate the contribution of independent sampling. We therefore conducted a matched independent-versus-correlated sampling experiment and found that independent sampling improves MRR by +0.0124 overall and +0.0197 on source-disjoint graphs, providing direct evidence of its additional transfer benefit.

Table R1. Effect of independent versus correlated rule–statistics sampling on zero-shot transfer

SamplingAll 54Source-disjoint 32Source-proximal 22
Independent0.36920.36960.3685
Correlated0.35670.34990.3667
Gain from independence+0.0124+0.0197+0.0018

Experimental design: We used the same 2,000 sampled rule configurations and graph-statistics configurations in both conditions, changing only their pairing. Independent sampling combines them randomly. Correlated sampling preferentially pairs complex rules with denser graphs. Both conditions retain identical marginal samples, corpus size, provenance-aware supervision, and training settings.

Statistical evidence: Across three seeds, the aggregate gain is +0.0124, with a 95% provenance-cluster bootstrap confidence interval of +0.003 to +0.021. The source-disjoint gain is +0.0197, with a confidence interval of +0.007 to +0.030. The source-proximal difference is comparatively small.

W2: Are provenance-aware supervision procedures comparable across corpora?

We agree that the provenance-aware factorial is a conditional comparison rather than a fully quality-matched estimate of the corpus effect. The complete-pipeline result remains valid, while attributing its aggregate performance to the synthetic corpus alone would be unwarranted. The corpus and supervision effects are now interpreted separately.

Under their respective provenance-aware supervision procedures, the aggregate corpus contrast is only +0.003 with a 95% confidence interval of −0.009 to +0.015. The source-disjoint contrast is +0.017 with an interval of +0.006 to +0.029. We therefore restrict the aggregate conclusion to the complete pipeline and describe the factorial contrasts as conditional on their respective supervision procedures. We no longer interpret them as identifying a corpus-only causal effect.

Your emphasis on matched controls and supervision comparability has substantially strengthened the evidence for independent sampling and sharpened the scope of our central claims. We hope these clarifications and additional results address your main concerns. We would be happy to provide further clarification if needed.

Thank you again for your constructive feedback. In response to your concerns, we have made the following updates:

  1. Corrected the Real-Subgraphs supervision setting to standard supervision and revised the corresponding interpretation.
  2. Conducted a matched independent-vs.-correlated sampling experiment, showing +0.0124 overall MRR and +0.0197 on source-disjoint graphs.
  3. Narrowed corpus-only claims to account for differences in supervision quality.

We hope these additional results and clarifications address your main concerns, and would greatly appreciate any further thoughts you may have.

Thank you for your time and consideration.

Thanks for the clarifications and new interpretations. They have mostly addressed my concerns, and I will maintain my current positive rating.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.