acceptodds
Under review as a conference paper at ICLR 2027

Co-adaptive Rule Routing and Constraining for Structured Knowledge Retrieval

Abstract

Structured knowledge retrieval aims to acquire task-relevant evidence from structured knowledge sources for downstream task execution. Existing methods either rely on unconstrained retrieval or predefined query-rule supervision (e.g., author–paper for seeking papers in scientific discovery and anatomy–drug for precision medicine in biomedical analysis), which is rarely available in practice. To address this limitation, we propose Co-Adaptive Rule Routing and Constraining Retrieval (CoRE), a framework that alternately determines which rule to route each query to and how to retrieve under that rule without requiring predefined query–rule pairs for training supervision. Specifically, a routing agent assigns each query to a reasoning rule, and the corresponding retriever traverses the structured knowledge base and ranks evidence that satisfies the rule’s constraints. Since the rule-routing agent and rule-constraining retriever are inherently interdependent, we formulate an alternate optimization that couples reinforcement learning for rule routing with contrastive learning for the rule-constraining retriever. This closed-loop learning enables both components to co-adapt without explicit paired query–rule supervision. Experiments on STaRK-QA datasets, spanning scientific discovery, biomedical reasoning, and E-commerce enterprise knowledge management domains, verify the superiority of our proposed CoRE. Ablations validate co-adapting both router and retriever, and generation-based routing demonstrates cross-dataset transferability and benefits from multi-domain pretraining with target-specific fine-tuning. Furthermore, marginalizing the routing policy over all queries of each dataset reveals the distribution of rule usage, informing automatic rule refinement and discovery.

Primary area:
other topics in machine learning (i.e., none of the above)
Keywords:
Structured Knowledge Retrieval, Agentic Routing, Co-adaptation, Reinforcement Learning

Reviews

Among:
Shape:
1234this paper, Top 15%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.00 over 3 reviews, above 85% of the others.
Reviewer_mJLTOpenReview ↗
Rating
3(+1)
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper introduces CoRE, a framework for structured knowledge retrieval that jointly learns a rule-routing policy and rule-constraining retrievers without explicit query-rule supervision. The key idea is to treat the reasoning rule as a latent variable and optimize routing and retrieval through alternating bi-level updates driven by downstream retrieval feedback. Experiments on three STaRK-QA datasets show that CoRE outperforms several structured and agentic retrieval baselines, and ablations highlight the benefit of co-adaptation.

Critical strengths
  • The problem of learning rule routing without query-rule labels is well motivated and practically relevant.
  • The co-adaptive optimization framework is conceptually clean and supported by ablation studies.
  • The case study in Appendix C illustrates the gap between semantic rule plausibility and retrieval effectiveness.
Critical weaknesses
  1. Insufficient treatment of the rule set itself.
    The rules are predefined from the graph schema (6 for AMAZON, 8 for MAG, 65 for PRIME). The paper does not study rule-set quality, rule-number sensitivity, redundancy, or automatic rule discovery/pruning. Appendix B already shows that the rule space is over-complete and contains many tied-optimal or rarely useful rules, yet the main experiments never ablate the rule set. This is a major scalability concern: if rules must be hand-crafted and the method is sensitive to their quality and size, it is difficult to apply to real, evolving knowledge graphs.

  2. Reward design is coarse.
    The routing reward is reciprocal rank of the ground-truth evidence under a given rule, using the best rank among correct answers for multi-answer queries. It does not directly optimize recall, multiple-answer coverage, score margins, or retrieval cost. A more informative reward (e.g., NDCG or a recall-aware composite) could substantially change routing behavior, but this is not explored.

  3. Error analysis is limited.
    Appendix C provides a confusion matrix and two representative case studies, but there is no systematic breakdown of routing errors vs. retrieval errors, no analysis of which query types or rules fail, and no mechanism to detect or recover from routing mistakes. This limits our understanding of the method’s failure modes.

Questions for authors
  • How sensitive is CoRE to the number and quality of rules? Can redundant rules be pruned automatically?
  • Would a richer reward (e.g., NDCG or recall-aware) improve routing?
  • Can the authors provide a quantitative error analysis separating routing errors from retrieval errors?
5 comments

We thank the reviewer for recognizing CoRE's motivation and co-adaptive framework, and for the constructive suggestions on rule-set design, reward choice, and failure analysis. In response, we have added rule-set sensitivity, pruning, and discovery experiments; compared alternative routing rewards; and analyzed routing versus retrieval errors, together with a correction experiment.

[Q1] Sensitivity to rule number, quality, and redundancy - (Part1)

We have added experiments on PRIME examining rule-number sensitivity, rule-quality sensitivity, automatic pruning, and rule discovery.

1. Sensitivity to rule number

We compare randomly sampled subsets of 20, 30, 40, and 50 rules with the full 65-rule set.

Rule setRules retainedH@1MRRRecall@20
Random subset2043.3757.2177.29
Random subset3044.6557.8678.26
Random subset4047.5260.6480.33
Random subset5047.2460.3580.63
Full set6549.3461.2780.81

Performance generally improves as more rules become available. Subsets of 40–50 rules retain much of the full set's answer coverage, while the complete rule set provides the strongest ranking performance. This suggests that additional rules offer useful retrieval alternatives, but their contribution depends on which rules are included.

2. Sensitivity to rule quality

We measure rule utility using the fractional oracle frequency introduced in Appendix B, Figure 6. For each query, a rule receives one unit of credit if it uniquely achieves the highest retrieval reward, or 1/k1/k credit if it ties with k−1k-1 other rules. A rule's utility score is its total Unique + Tied credit, normalized by the number of queries. Thus, the score reflects how frequently a rule provides retrieval-optimal results while accounting for equally effective alternatives.

We rank rules by this score and compare equal-sized subsets from different positions in the ranking.

Rule subsetRules retainedH@1MRRRecall@20
Top 101046.2958.5679.25
Ranks 11–201039.3652.5874.98
Ranks 21–301039.2752.4072.43
Bottom 101034.3546.8069.32

At a fixed rule-set size, higher-utility subsets consistently perform better across all three metrics. The top 10 rules even outperform the random 20- and 30-rule subsets, indicating that rule utility matters beyond rule count. Nevertheless, the full rule set remains stronger than the top 10, suggesting that rules with lower overall oracle frequency can collectively provide complementary value for queries less well served by the highest-ranked rules.

3. Automatic pruning

Motivated by these findings, we evaluated a simple usage-based pruning strategy that retains the 30 most frequently used rules and removes the remaining 35. The following table reports the reduced rule set alongside its full-set reference in this pruning experiment.

Rule setRulesH@1MRRRecall@20
Top 303048.5661.0280.75
Full set6548.7561.5281.46

Removing more than half of the rules retains performance close to the full-set reference. We also quantify the associated changes in model size and computational workload:

ComponentFull 65Top 30Reduction
Number of rules653053.8%
Router parameters109.53M109.51M0.025%
Router training steps7,9507,9500%
Retriever training steps5,9504,70021.0%
Test query–rule pairs scored6,5094,58729.5%
  • Pruning primarily reduces rule-dependent retrieval work. Retriever training steps decrease by 21.0%, and test query–rule scoring decreases by 29.5%.
  • The BERT router uses a shared query encoder followed by a lightweight rule-classification layer. Reducing the number of rules shrinks only the final layer; the dominant encoding computation and number of router training steps remain largely unchanged.
  • Why savings are not proportional to rule count. Rule usage is nonuniform: the retained rules account for a substantial share of retrieval activity. Consequently, removing 53.8% of the inventory reduces scoring workload only by a smaller proportion.

These results demonstrate the benefit of pruning unnecessary rules in maintaining retrieval performance and reducing resource complexity, which provides a practical starting point for future exploration on rule-aware pruning.

[Q1] Sensitivity to rule number, quality, and redundancy - (Part2)

4. Automatic rule discovery

We also implemented an LLM-assisted discovery procedure to investigate whether retrieval feedback can guide discovery of more useful rules, which will subsequently lead to more effective retrieval.

  • Identify queries for exploration. We collect training queries for which none of the existing rule-conditioned retrievers ranks a correct answer as top 1. These failures identify opportunities for rule set expansion.
  • Propose rules. We prompt an LLM with the graph schema, five sampled queries from this set, and their ground-truth answer entity types, asking it to propose one potentially useful retrieval rule.
  • Validate proposals. We check graph validity and reject rules that duplicate the current rule set. We repeat this process until 65 additional rules are collected.
  • Retrain and evaluate. We combine the original and proposed rules and retrain CoRE with the expanded 130-rule set.
Rule-space settingRulesH@1MRR
Original6549.3461.27
Expanded13051.7963.34

The expanded rule set improves both H@1 and MRR. This supports the potential of using training-time retrieval failures to guide rule proposals and of CoRE to exploit the resulting alternatives. It also provides a practical way to expand the rule set without manually designing each additional rule.

Together, these experiments show how retrieval feedback can inform both which rules to retain and where to explore new rules. Pruning offers a more compact rule set with lower retrieval workload, while discovery improves retrieval performance by introducing additional alternatives.

These experiments and their construction details are included in Section 5.4 and Figure 3 of the revised paper.

[Q2] Reward design and multi-answer coverage

Reciprocal rank (RR) emphasizes the earliest correct answer without directly rewarding additional relevant answers. We evaluate three alternative routing rewards on PRIME and MAG.

  • Alternative rewards. We compare the original RR reward with NDCG@20, Recall@20, and 0.5 RR + 0.5 Recall@20, keeping initialization, data splits, training schedules, and budgets consistent.
  • Evaluation. We report H@1 and MRR to assess early ranking quality, Recall@20 to measure answer coverage, and NDCG@20 to assess the ranking of relevant answers within the retrieved list.

PRIME

Routing rewardH@1MRRRecall@20NDCG@20
RR (original)49.3461.2780.8161.84
NDCG@2049.4862.1582.2362.97
Recall@2049.1661.9982.4762.88
0.5 RR + 0.5 Recall@2048.7561.6382.0362.49

MAG

Routing rewardH@1MRRRecall@20NDCG@20
RR (original)63.3774.1783.5171.74
NDCG@2066.2675.6684.7572.38
Recall@2062.1472.6883.7770.04
0.5 RR + 0.5 Recall@2066.3475.5683.8371.87

Overall, richer rewards improve answer coverage on both datasets, with NDCG@20 providing the most consistent gains across ranking and coverage metrics. Recall-only rewards favor coverage but can sacrifice early ranking quality, while combining RR and recall offers a balance whose effectiveness varies by dataset. These results support the reviewer’s suggestion and demonstrate that CoRE can accommodate different retrieval priorities through its reward design.

The reward definitions and comparative results are included in Appendix H, Table 5 of the revised paper.


[Q3] Quantitative error decomposition and failure modes

We have added an error breakdown across all three datasets and a routing-correction experiment on MAG.

1. Separating routing errors from retrieval errors

We divide queries into three outcomes:

  • Successful retrieval: the selected rule ranks a correct answer as top-1.
  • Routing error: the selected rule fails, but another rule ranks a correct answer as top1.
  • Retrieval error: no rule ranks a correct answer as top1, so changing the routing decision alone would not resolve the failure.

Percentages below are calculated over all queries evaluated in this analysis.

DatasetQueriesSuccessful retrievalRouting errorRetrieval error
AMAZON1,71950.77%16.50%32.73%
MAG2,74863.37%8.35%28.28%
PRIME2,89949.52%21.37%29.11%

The results show that both routing and retrieval contribute to the remaining errors. PRIME has the largest proportion of failures recoverable through a different rule choice, while MAG has the smallest. Across all three datasets, however, a substantial portion of failures persists even with the best available rule.

2. Can routing mistakes be recovered?

To investigate this possibility, we introduce a lightweight routing correction model on MAG. Specifically, the model takes the frozen BERT query embedding and CoRE’s predicted routing distribution as input, and learns to produce a corrected routing distribution using retrieval rewards from training queries as supervision.

At inference, we compare the highest-ranked rule from the original and corrected distributions. If they agree, the original routing decision is retained; otherwise, the rule with the highest corrected score is selected. The original data splits and trained retrievers remain unchanged, and only the final selected retriever is executed.

We evaluate the correction mechanism using the original evaluation protocol.

MethodH@1 (%)MRR
CoRE63.370.7417
CoRE + correction64.770.7534

The improvement in both H@1 and MRR demonstrates that routing decisions can be refined using information already available in the query representation and original routing distribution.

Together, these results demonstrate the feasibility of detecting and correcting routing errors without modifying the underlying retrievers. We include these analyses in Appendix E, Figure 10, and Table 3 of the revised paper.

Reviewer_svHxOpenReview ↗
Rating
3(+1)
Confidence
4
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes CoRE, a framework for structured knowledge retrieval that jointly learns a rule-routing policy and rule-constraining retrievers from query–evidence pairs, without explicit query–rule supervision. The router selects a rule from a predefined candidate set, and the corresponding retriever ranks evidence under that rule. Training alternates between adapting the retrievers using the current query–rule assignments and updating the router using retrieval rewards. Experiments on the AMAZON, MAG, and PRIME datasets from STaRK-QA evaluate retrieval performance, component adaptation, and cross-domain routing transfer.

Critical strengths
  1. CoRE connects routing decisions with actual retrieval feedback and allows the retrievers to adapt to their assigned queries. Learning this interaction from query–evidence pairs reduces reliance on explicit query–rule annotations and gives the framework practical value.

  2. The classification-based variants achieve strong average retrieval performance across the three evaluated domains. The component ablations also support the usefulness of adapting both the router and the retrievers.

Critical weaknesses
  1. The evaluation uses predefined candidate sets of 6, 8, and 65 rules for AMAZON, MAG, and PRIME, respectively. These results support effectiveness within the evaluated settings, but applicability to more complex rule spaces remains unclear. In particular, the evaluation does not establish whether co-adaptation remains effective with more diverse relations, more candidate paths, or more complex relational compositions. Since rule selection is central to CoRE, the size and expressiveness of the rule space directly affect routing difficulty and the queries it can accommodate.

  2. The paper does not clearly distinguish selecting the rule that best matches query semantics from selecting the rule–retriever pair that maximizes retrieval performance. A semantically appropriate rule may receive lower rewards because of retriever limitations, while a broader rule may retrieve the answer more successfully. The training objective supports performance-driven routing, but descriptions such as the rule that “best explains” a query suggest semantic faithfulness, which retrieval gains alone cannot establish. Moreover, the prevalence and diversity of this mismatch within the limited rule spaces remain unclear, weakening the evidence connecting the proposed motivation to the observed improvements.

Questions for authors
  1. Figure 5 shows rapid convergence, with limited gains after the second round. Is an alternating update mechanism necessary, or could a simpler sequential training procedure achieve comparable performance?

  2. Is the intended routing target the rule that best reflects query semantics, or the rule–retriever pair that maximizes retrieval performance? Do you consider these objectives distinct, and how common are cases in which the semantically most appropriate rule does not yield the best retrieval performance?

6 comments

We thank the reviewer for recognizing the practical value of learning routing from retrieval feedback and for the constructive suggestions on rule-space complexity, alternating training, and the routing objective. In response, we have expanded the rule inventory on PRIME, added sequential training controls and optimization trajectories, and clarified the distinction between semantic alignment and retrieval effectiveness through additional comparisons and mismatch analysis.

[W1] Applicability to larger and more complex rule spaces

We examine this question from two complementary perspectives: how CoRE’s advantage varies across datasets with different schema and rule-space diversity, and how its performance changes when the rule set expands within the same dataset.

1. Effectiveness across different levels of schema and rule-space diversity

The table below relates dataset characteristics to CoRE’s H@1 and its improvement over the average H@1 of the hybrid retrieval baselines.

DatasetEntity typesRelation typesCandidate rulesCoRE H@1Average H@1 improvement over hybrid
AMAZON45650.71+1.34
MAG44863.37+9.89
PRIME10186549.34+18.41
  • CoRE outperforms the hybrid baseline average across all three settings, with its largest advantage on PRIME, which has the most diverse schema and largest rule set. Thus, its advantage is maintained when the dataset includes more diverse and complex schemas and rules. This might be because a richer rule space leads to more diverse reasoning patterns and therefore opens more diverse retrieval channels, while also making it harder to determine which structural constraints will work well. CoRE addresses this challenge by learning rule selection from actual retrieval feedback and adapting the retrievers to their assigned queries. This allows it to account for differences in the usefulness of available paths.

However, since these datasets also differ in domain and query characteristics, the above larger gains cannot be attributed solely to schema and rule-space diversity. To better isolate the effect of rule-space size, we further conduct a controlled rule-expansion experiment on PRIME.

2. Expanding PRIME from 65 to 130 rules

We use training-set retrieval failures to guide the proposal of additional rules:

  • Identify queries for exploration. We collect training queries for which no existing rule-conditioned retriever ranks a ground-truth answer as top-1. These queries identify potential limitations of the current rule set.
  • Propose candidate rules. We prompt an LLM with the graph schema and five sampled queries from this set, together with their ground-truth answer entity types, and ask it to propose one potentially useful retrieval rule.
  • Validate and expand. We check each proposal for validity on the graph and duplication with the existing rule set. We repeat this process until we obtain 65 new valid, non-duplicate rules.
  • Retrain CoRE. We combine the 65 original and 65 proposed rules and retrain CoRE with the expanded rule set.

Note that the above rule proposals are solely guided by training queries, without using validation/test queries and answers.

Rule-space settingRulesH@1MRR
Original6549.3461.27
Expanded13051.7963.34

CoRE improves both H@1 and MRR after enlarging the rule set, suggesting that it can exploit additional retrieval paths for queries less well served by the original rules. Together with the cross-dataset comparisons, this supports CoRE’s consistent effectiveness across datasets with diverse numbers and complexity of schemas and rules.

This expansion also demonstrates a practical approach to diverse retrieval needs, using training-query failures to propose schema-valid rules, then adapting routing and retrieval to the expanded rule set. Repeating this rule discovery process as queries or schemas evolve is a promising follow-up direction under the agentic self-evolution topic. We plan to systematically explore this topic as future work.

The rule-expansion procedure and results are included in Section 5.4, Figure 3(a), and the future direction discussion in the Conclusion of the revised paper.

[Q1] Necessity of alternating updates

We have added direct comparisons on AMAZON, MAG, and PRIME to examine whether simpler sequential training can achieve comparable performance.

  • Sequential baselines and budgets. We evaluate Retriever → Router and Router → Retriever, freezing the first component before training the second. Both keep the rest of the settings same as CoRE (Alternating). We test 150 epochs per component as it matches CoRE’s three alternating rounds of 50 epochs per component, totaling 300 epochs.
  • Alternating CoRE outperforms both sequential strategies on all three datasets. In sequential training, freezing the first component prevents it from responding to subsequent changes (feedbacks) in the other. Alternating co-adaptation allows both components to adjust, updated routing decisions reshape the retrievers’ query assignments, and improved retrievers provide updated feedback for routing.
SettingAMAZONMAGPRIME
Retriever → Router (2 stages × 150 epochs)50.1153.9945.15
Router → Retriever (2 stages × 150 epochs)48.6349.6340.14
Alternating (3 rounds × 50 epochs)50.7163.3749.34
  • The trajectories show that later rounds of adaptation improve upon the first round, with the largest gains occurring early and smaller refinements afterward. This is consistent with the intuition behind co-adaptation: the initial router update learns which rules work well with the available retrievers, substantially improving query assignments. Subsequent retriever updates then specialize retrieval to these revised assignments, providing updated feedback for the router. As the components become better aligned, later rounds require finer adjustments rather than large changes. Rapid convergence is therefore a practical advantage: CoRE obtains the benefits of mutual adaptation within a small number of rounds.
DatasetInitialR1 RetrieverR1 RouterR2 RetrieverR2 RouterR3 RetrieverR3 Router
PRIME2.542.8246.0246.8747.1150.3049.56
AMAZON27.0227.5649.7550.7649.7451.0949.86
MAG14.5621.0757.2464.5362.9764.7062.93

See Appendix F, Table 4 and Figure 11 in the revised paper for the sequential-training comparisons and optimization trajectories.

[Q2, W2] Routing objective

1. Routing objective and motivation

Our current implementation learns to select the rule–retriever pair that maximizes downstream retrieval reward. This differs from selecting the rule that best matches query semantics: a semantically appropriate rule may perform poorly if its retriever cannot reach or rank the correct evidence. We will replace “best explains” and related expressions with wording that explicitly describes this retrieval objective.

  • The existing case study illustrates this distinction. In Appendix C, Figure 7(b), GPT-5 selects institution → author → paper ← field of study for a query with institutional and topical constraints. Although semantically plausible, the field branch fails to reach the relevant papers. The broader institution → author → paper rule instead retrieves them at ranks 2 and 4, illustrating the situation described by the reviewer.
  • Lower performance from semantic routing supports our motivation for reward-driven routing. We compare CoRE against GPT-6-Luna selecting rules from query and rule descriptions, and BGE-large and BERT selecting rules by embedding similarity. All methods use the same fixed rule-conditioned retrievers.
Routing methodAMAZONMAGPRIME
Trained BERT router (CoRE)50.7163.3749.34
GPT-6-Luna semantic routing48.1759.4746.29
BGE-large embedding similarity46.4142.7541.78
BERT embedding similarity45.7649.7135.40

Reward-driven CoRE achieves higher H@1 across all three datasets. These results support learning from actual retrieval outcomes, since semantic compatibility alone does not capture how effectively a retriever executes a rule.

2. How often does semantic selection differ from the best retrieval choice?

Figure 7(a) illustrates this mismatch on MAG using GPT-5. We extend the analysis to all three datasets using GPT-6-Luna and BGE-large similarity.

For each query, we run every candidate rule through its retriever and compare the rank reward of the correct answer. We count a mismatch when the selected rule does not have the highest rank reward. The mismatch rate is the percentage of evaluated queries meeting this condition.

DatasetSemantic proxyQueries evaluatedMismatch rate (%)
AMAZONGPT-6-Luna1,71934.47
AMAZONBGE-large similarity1,71941.21
MAGGPT-6-Luna2,74827.61
MAGBGE-large similarity2,74836.63
PRIMEGPT-6-Luna2,89936.49
PRIMEBGE-large similarity2,89945.88

GPT-6-Luna selects a rule outside the retrieval-optimal set for approximately 28–36% of queries, while BGE-large similarity does so for 37–46%. Both proxies show the highest mismatch rate on PRIME and the lowest on MAG, and GPT-6-Luna exhibits fewer mismatches on every dataset.

These results show that mismatch occurs across domains. The measured rates can reflect both imperfect selection and differences between semantic plausibility and retrieval effectiveness.

We have revised the wording throughout the paper to clarify the retrieval-driven objective, particularly in Section 4.1, and added semantic-routing comparisons and mismatch analysis in Appendix G, Figure 12.

Reviewer_kF9ROpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

CoRE alternately trains a rule router and rule-conditioned retrievers using query-evidence pairs, without requiring externally annotated query-rule labels. Experiments on three STaRK datasets show strong average performance for CoRE-BERT-C, with additional analyses of component adaptation, router transfer, and optimization dynamics.

Critical strengths

CoRE learns rule selection from retrieval feedback without requiring externally annotated query-rule pairs. Its alternating design provides a practical way to align routing decisions with rule-conditioned retrieval. CoRE-BERT-C improves average H@1 over MoR from 48.93 to 54.47 and MRR from 58.77 to 65.97. The substantial gains on MAG and PRIME support the approach’s empirical value across different domains.

Critical weaknesses

Table 2 compares CoRE with unadapted routers or fixed BM25 scoring, so improvements conflate component training with alternating co-adaptation. Figure 5 provides useful optimization dynamics on MAG, but a matched-budget, train-then-freeze control would better establish the feedback loop’s specific benefit.

Questions for authors

With the same learned components and comparable training budgets, how does alternating training compare with training the retrievers once, freezing them, and then adapting the router? Can the round-by-round improvement shown on MAG in Figure 5 also be quantified on AMAZON and PRIME?

2 comments

We thank the reviewer for recognizing CoRE’s practical value and for the constructive suggestions on isolating the contribution of alternating co-adaptation. In response, we have added matched-budget sequential baselines and extended the round-by-round analysis to AMAZON and PRIME.

[Q1] Contribution of alternating co-adaptation

We compare alternating CoRE with two sequential training strategies on AMAZON, MAG, and PRIME.

  • Compared strategies.
    • Sequential Training Strategy 1: Retriever → Router: Train the retrievers for 150 epochs, freeze them, and then train the router for 150 epochs.
    • Sequential Training Strategy 2: Router → Retriever: Train the router for 150 epochs, freeze it, and then train the retrievers for 150 epochs.

CoRE performs three alternating rounds of retriever and router training, allocating 50 epochs per component in each round, resulting in 150 epochs per component (300 epochs total). To ensure a matched epoch budget, both sequential strategies allocate the same 150 epochs as mentioned above to each component but train them consecutively rather than alternatingly.

SettingAMAZONMAGPRIME
Retriever → Router (2 stages × 150 epochs)50.1153.9945.15
Router → Retriever (2 stages × 150 epochs)48.6349.6340.14
Ours CoRE (Alternating 3 rounds × 50 epochs)50.7163.3749.34
  • Alternating CoRE consistently outperforms both sequential training orders across all three datasets. Retriever → Router also performs better than the reverse order, suggesting that the router benefits more from feedback provided by adapted retrievers in one training round.
  • In sequential training, freezing the first component prevents it from responding to subsequent changes (feedback) in the other. Alternating co-adaptation allows both components to adjust; updated routing decisions reshape the retrievers’ query assignments, and improved retrievers provide updated feedback for routing.

Full details of this comparison are included in Appendix F.1, Table 4 of the revised paper.


[Q2] Round-by-round improvement on AMAZON and PRIME

We have added round-by-round H@1 results for AMAZON and PRIME alongside MAG. At each stage, we evaluate the corresponding router–retriever pair, starting from initialization and continuing after each retriever and router update.

  • All three datasets show a similar pattern: the first router update produces the largest improvement, followed by additional gains from retriever adaptation. Later router updates yield smaller gains. Each dataset reaches its highest observed H@1 after the third retriever update.
  • This pattern further justifies complementary roles for the two components: early router adaptation establishes effective query–rule assignments, and subsequent retriever updates improve retrieval under those assignments.
DatasetInitialR1 RetrieverR1 RouterR2 RetrieverR2 RouterR3 RetrieverR3 Router
PRIME2.542.8246.0246.8747.1150.3049.56
AMAZON27.0227.5649.7550.7649.7451.0949.86
MAG14.5621.0757.2464.5362.9764.7062.93

The complete optimization trajectories are shown in Appendix F.2, Figure 11 of the revised paper.

AuthorsOpenReview ↗

Looking Forward to Your Further Feedback

Dear Reviewer kF9R,

Thank you again for your positive assessment of our work and your constructive suggestions. We have addressed your questions in our responses and revised manuscript, and would appreciate any further thoughts you may have.

Should any questions remain, please let us know, we would be happy to provide further clarification. We sincerely appreciate your time and consideration.

Best regards,

The Authors

Comments on the paper

2 comments

We thank all three reviewers for their time, careful evaluation, and constructive suggestions. Your feedback has helped us strengthen the experimental evidence, clarify CoRE’s objective, and better characterize its capabilities. We have provided detailed responses to each concern and revised the manuscript accordingly. The main additions are:

  • Alternating co-adaptation. [kF9R Q1, Q2; svHx Q2] Matched-budget comparisons show that CoRE outperforms both sequential training orders across all three datasets. Additional round-by-round results clarify how early adaptation and subsequent refinements contribute to performance. Revised paper: Appendix F, Table 4 and Figure 11.
  • Rule-set sensitivity and refinement. [svHx W1; mJLT Q1] We evaluate rule quantity and quality, demonstrate that utility-based pruning largely preserves retrieval performance while reducing retrieval workload, and expand PRIME from 65 to 130 rules through retrieval-guided proposals, improving both H@1 and MRR. Revised paper: Section 5.4, Figure 3.
  • Routing objective and semantic alignment. [svHx Q2] We clarify that CoRE learns routing from retrieval rewards. Comparisons with semantic routing and quantitative mismatch analysis further support accounting for actual retriever behavior when selecting rules. Revised paper: Section 4.1; Appendix G, Figure 12.
  • Reward design. [mJLT Q2] Experiments on PRIME and MAG show that richer rewards can improve answer coverage and ranking quality, demonstrating how CoRE can accommodate different retrieval objectives. Revised paper: Appendix H, Table 5.
  • Failure analysis and correction. [mJLT Q3] We distinguish errors recoverable through better routing from failures that persist even with the best available rule, and demonstrate a lightweight routing correction that improves retrieval without invoking additional retrievers. Revised paper: Appendix E, Figure 10 and Table 3.

We hope these additions address the reviewers’ concerns and make the contributions and scope of CoRE clearer. Should any questions or concerns remain after reviewing our responses and revised manuscript, please let us know. We would be happy to provide further clarification and continue the discussion.

AuthorsOpenReview ↗

Summary of Rebuttal

Dear all,

During the rebuttal phase, we carefully addressed the reviewers' comments. One reviewer maintained their already positive score, while the other two acknowledged our responses and raised their scores.

We sincerely appreciate the reviewers' valuable suggestions and constructive feedback, which have helped us further strengthen the work.

Should there be any further questions, please do not hesitate to let us know. We would be happy to provide additional information to further polish the work.

Best,

Authors

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.