acceptodds
Under review as a conference paper at ICLR 2027

SPARK: Adaptive Parametric Knowledge Participation for Faithful Retrieval-Augmented Generation

Abstract

Retrieval-augmented generation (RAG) enhances the faithfulness of large language models (LLMs) by incorporating external knowledge into the generation process. However, conflicts between retrieved context and parametric knowledge can cause LLMs to rely on inconsistent internal knowledge, hindering their ability to faithfully leverage retrieved evidence. Existing approaches mitigate this issue by editing parametric knowledge to resolve such conflicts. However, locating and modifying knowledge encoded in model parameters is challenging and may inadvertently compromise the backbone LLM's capabilities. In this paper, we introduce SPARK, a novel framework that formulates parametric knowledge control as a latent participation policy learning problem, enabling LLMs to adaptively regulate the contribution of parametric knowledge during generation. Specifically, SPARK introduces a lightweight participation policy that dynamically modulates the contribution of FFN-mediated parametric knowledge while keeping the backbone LLM frozen. The participation policy is optimized with a faithfulness-oriented reward, enabling fine-grained control over parametric knowledge participation during generation. Extensive experiments on six datasets validate the effectiveness of SPARK, which introduces and optimizes only 0.01% additional trainable parameters while keeping the backbone LLM fully frozen, thereby preserving its general capabilities and achieving more faithful generation than competitive baselines. All datasets and code will be released via GitHub.

Primary area:
alignment, fairness, safety, privacy, and societal considerations
Keywords:
Knowledge Conflict, Reinforcement Learning, Retrieval-Augmented Generation

Reviews

Among:
Shape:
1234this paper, Top 4%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.33 over 3 reviews, above 96% of the others.
Reviewer_1BVcOpenReview ↗
Rating
4
Confidence
4
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

This work studies how to balance parametric and contextual knowledge in retrieval-augmented generation (RAG). The authors propose SPARK, a lightweight framework that adaptively regulates the contribution of each FFN layer through input-dependent participation gates. Instead of modifying the backbone parameters or suppressing predefined knowledge-related layers, SPARK learns these gates using GRPO with answer-level rewards while keeping the backbone frozen. The authors construct training examples that encourage the model to rely on parametric knowledge when the retrieved context is misleading and follow contextual evidence when its internal knowledge is incorrect. Experiments on MRQA, ConfiQA, and MMLU demonstrate improvements in question answering while largely preserving general capabilities.

Critical strengths
  1. The paper is clearly written, with a well-organized presentation of the motivation, methodology, and experimental results, making the main ideas easy to follow.

  2. SPARK is lightweight and parameter-efficient, as it freezes the backbone and only trains a small number of gating parameters. Learning the participation policy directly from generation-level feedback, without requiring explicit supervision for how parametric knowledge should participate, is an appealing design choice.

  3. The experiments cover multiple QA datasets, model sizes, and model families. The ablation studies also examine the contributions of the faithfulness reward, format reward, and regularization, providing useful evidence for the effectiveness of the proposed framework.

Critical weaknesses
  1. My main concern is about whether the reward function adequately captures contextual faithfulness. The method primarily relies on exact-match agreement with reference answers, which measures answer correctness but does not necessarily establish whether the model actually relies on the appropriate evidence. Evaluating evidence-grounded generation more directly would strengthen the paper's central claim.

  2. The construction of knowledge-conflict examples may not always reflect genuine conflicts. In particular, selecting a semantically related passage that does not contain the model's parametric answer does not guarantee that the passage contradicts that answer. This raises concerns about whether the training data reliably teaches the model to distinguish misleading evidence from incomplete context.

Questions for authors

see weakness

Reviewer_42AgOpenReview ↗
Rating
3(+1)
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

SPARK addresses conflicts between parametric knowledge and external context in RAG by applying input-dependent scalar gates to the output of each FFN. Each gate takes the current hidden state as input and attenuates or amplifies the corresponding FFN contribution. The gates are trained using GRPO with answer exact-match and output-format rewards, together with KL and parameter regularization.

Critical strengths

1. The architecture is simple and parameter-efficient, with clearly defined initial behavior.

SPARK adds an input-dependent scalar gate to each FFN layer, with additional trainable parameters amounting to less than 0.01% of the backbone’s parameter count. Allowing both attenuation and amplification of FFN contributions makes the approach more flexible than suppression-only methods. This is a practical design for adapting a frozen model without identifying individual knowledge-bearing neurons.

2. The method demonstrates meaningful performance improvements across multiple evaluation settings.

SPARK achieves the highest reported average MRQA F1 on both primary backbones and improves performance across all evaluated ConfiQA subsets. Additional experiments on Qwen and Llama models support its applicability beyond a single backbone.

Critical weaknesses

The contributions of reinforcement learning and input-dependent gating are not separately established.

Section 3.2 presents an objective for training the same gates with SFT, but the experiments do not compare this approach with GRPO. The full-parameter SFT baseline differs in both the trainable parameterization and the optimization objective, making it difficult to attribute the performance gains to GRPO. Furthermore, the paper does not compare against learned, input-independent FFN scaling factors for each layer under matched conditions.

Questions for authors

1. Controlled comparisons

Under identical training data and comparable hyperparameter search budgets, how does training the same gates with SFT compare with training them using GRPO? Additionally, how well do learned, input-independent FFN scaling factors perform under the same optimization objective?

2. Comparisons with the closest existing methods

How does SPARK compare with ParamMute and CoRect when the backbone, supplied contexts, and evaluation conditions are matched? Both are presented as closely related internal-intervention methods, but neither is included in the reported experimental comparisons.

Reviewer_rqdpOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

SPARK introduces input-dependent scalar gates that modulate FFN outputs during retrieval-augmented generation. The backbone remains frozen, while the gates are trained with GRPO using answer-matching and formatting rewards, together with KL and L2 regularization.

Critical strengths
  1. The method is simple and parameter-efficient. It supports both attenuation and amplification of FFN outputs, starts from an identity initialization, and requires no updates to the backbone weights. This is an appealing adaptation approach.

  2. The experimental coverage is a strength. The authors test several model sizes and families, and the improvements extend to the out-of-domain evaluation. The checks on general capabilities are also useful, given that the intervention affects computation throughout the network.

Critical weaknesses
  1. The construction in A.3 does not ensure that a "conflicting" passage actually contradicts the parametric answer. It only requires topical relevance and the absence of that answer. Such passages may instead be uninformative or concern a different entity. A.9 also uses this construction, so the TN/FP analyses also confound conflict type with context quality and answer availability.

  2. The ablations do not isolate the two central design choices. Training the same gates with SFT, and training a static gate for each layer with GRPO. The paper introduces the first option in Eq. 1, but the SFT baseline updates the backbone. Without these controls, it is hard to tell what is gained from RL or from making the gates input-dependent. The substantial improvement of the format-only variant also raises a practical question about how much of the gain comes from output formatting. The author should clarify whether answer extraction and format-failure handling are consistent across methods.

Questions for authors
  1. How often do the selected c−c^{-} passages explicitly contradict the correct answer, rather than merely omit it? Does the improvement hold on the explicitly contradictory subset?

  2. How do gate-only SFT and static-gate GRPO compare with SPARK under the same training and evaluation conditions? Can the authors also report format-validity rates?

  3. Are different instructions used for MRQA's context-resisting cases and ConFiQA? Please clarify what tells the model when to retain its parametric answer and when to follow the supplied context, since the intended behavior differs between these settings.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.