acceptodds
Under review as a conference paper at ICLR 2027

CiteProbe: From LLM Attention to Citations

Abstract

Sentence-level citations make Retrieval-Augmented Generation (RAG) outputs easy to verify by identifying the source sentences that support a generated answer. Yet existing methods face a trade-off between effectiveness and efficiency. Perturbation-based approaches require repeated LLM inference, while attention-based approaches typically rely on fixed aggregation rules that obscure informative variations across layers and heads. Can accurate citations instead be directly decoded from the attention weights without any additional pass? We show that they can, and that signals are concentrated in a few heads. To leverage this, we introduce CiteProbe, a lightweight approach that formulates sentence-level citation as a readout problem over the attention weights of a frozen LLM. For each candidate sentence, CiteProbe constructs an attention profile capturing the attention it receives from response tokens across individual layers and heads, thereby preserving the full layer–head structure. A lightweight, model-specific linear probe then learns to map these attention profiles to citation scores. This formulation exploits citation-relevant signals already present in the attention weights and scores all candidate sentences using a single generation pass, avoiding the repeated LLM inference required by perturbation-based methods. Experiments across six LLMs and four datasets show that CiteProbe consistently outperforms existing attribution methods while substantially reducing inference cost. These results demonstrate that sentence-level citation signals are directly decodable from structured LLM attention, offering an effective and efficient approach to citation attribution. We also show that CiteProbe can be applied to black-box LLMs under surrogacy mode. We release CiteProbe as open source at https://www.github.com/citeprobe/citeprobe.

Primary area:
interpretability and explainable AI
Keywords:
attention attribution, context attribution, sentence-level citation, retrieval-augmented generation, mechanistic interpretability, attention heads, linear probes, evidence attribution, LLM explainability

Reviews

Among:
Shape:
1234this paper, Top 15%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.00 over 3 reviews, above 85% of the others.
Reviewer_YwMeOpenReview ↗
Rating
3(+1)
Confidence
4
Informativeness
2
Presentation
2
Soundness
3(+1)
Read review
Paper summary

The paper introduces CiteProbe, a method for identifying source sentences that support an LLM-generated answer using attention patterns across individual attention heads. The authors first investigate which attention heads contain useful citation-related information and then train a lightweight linear probe on head-specific attention features to score and rank candidate source sentences based on their likelihood of supporting the generated answer.

Critical strengths
  • The problem addressed in this paper is relevant and important, particularly given the growing need for reliable citation attribution in LLM-generated responses.

  • The idea of using attention patterns from different layers and heads to identify supporting evidence is interesting. It also avoids the repeated inference required by several existing attribution methods.

  • CiteProbe has low additional inference cost compared with the evaluated baselines, making it a potentially practical approach for citation attribution in RAG systems.

Critical weaknesses
  • The main results focus on mAP and Top-1 accuracy, which measure how well CiteProbe ranks sources but do not show how much supporting evidence it actually identifies. Appendix L reports low recall in some cases, but this is not discussed in the main text.

  • Recall is particularly important because the paper evaluates CiteProbe on multi-hop datasets, where multiple pieces of evidence are needed to support an answer. Finding just one relevant sentence is not enough if other required evidence is missing, but the main results do not show how much of this evidence CiteProbe actually identifies.

  • Even with the current evaluation setup used in the main text, CiteProbe shows only small improvements over the Top-16-head baseline and sometimes performs worse. It is unclear how much the learned linear probe actually helps.

  • The claim that CiteProbe extends to proprietary models is overstated. The authors pass their answers through an open-source model and use its attention weights to find supporting evidence. They could do the same with ground-truth answers. This only shows that CiteProbe works on externally generated text.

Questions for authors
  1. Why does the main evaluation emphasize Top-1 accuracy without reporting recall, especially for questions that require multiple supporting sentences, when mAP already measures ranking performance?

  2. In the proprietary-model experiments, CiteProbe simply takes the generated answers and uses an open-source model to identify supporting evidence. The same could be done with ground-truth answers. What do these experiments demonstrate about CiteProbe's extension to proprietary models beyond its ability to process any given answer?

  3. How does CiteProbe turn its source predictions into actual citations? Are citations assigned to individual claims or the whole answer? Why is this comparison with citation generation methods only discussed in Appendix K and not in main text?

7 comments

Dear Reviewer YwMe,

Thank you for your detailed review. We are glad that you found the problem relevant and important. Below, we meticulously respond to each comment and outline the corresponding revisions.

W1. The main results focus on mAP and Top-1 accuracy, which measure how well CiteProbe ranks sources but do not show how much supporting evidence it actually identifies. Appendix L reports low recall in some cases, but this is not discussed in the main text.

Reply: Thank you for highlighting this concern. We agree that recall is important, and can directly inform multi-hop citation performance. The issue of low sporadic low recall occurs because of validation-selected operating points that maximizes F1. Given that an operating point can trade off recall for precision and vice versa, we added Area Under the Precision-Recall Curve (AUPRC) to Table 1 in the main text and precision-recall curves to Figure 15 in Appendix M.2. AUPRC evaluates the precision-recall trade-off across all thresholds, alleviating issues with single operating points selected on validation F1. CiteProbe achieves the highest mean AUPRC among all methods on five of six backbones. CiteProbe's stronger overall AUPRC is reflected in its favorable precision-recall trade-offs across most settings, with the ideal curve approaching the upper-right corner in Figure 15. We summarize the AUPRC figures as follows:

BackboneMethodHotpotQAMuSiQueTyDiQAWiCEMean
Phi-4-miniContextCite-32.498.439.559.349.461
ARC-JSD.615.536.575.378.526
AttnTrace.188.153.493.317.288
AT2.750.654.836.596.709
BM25.492.478.527.309.452
Top-16 heads.760.651.793.599.701
CiteProbe.762.690.810.618.720
Ministral-3-8BContextCite-32.539.499.573.425.509
ARC-JSD.613.584.553.438.547
AttnTrace.607.522.521.456.527
AT2.744.670.755.582.688
BM25.488.568.545.367.492
Top-16 heads.753.665.741.567.681
CiteProbe.750.724.751.587.703
Qwen3.5-9BContextCite-32.165.122.398.110.199
ARC-JSD.512.447.461.334.438
AttnTrace.580.514.759.512.591
AT2.426.369.703.423.480
BM25.474.497.582.390.486
Top-16 heads.756.704.783.598.710
CiteProbe.743.740.776.617.719
Gemma-4-12BContextCite-32.281.171.493.163.277
ARC-JSD.573.385.516.306.445
AttnTrace.079.066.667.076.222
AT2.708.663.871.586.707
BM25.535.511.572.376.499
Top-16 heads.784.687.824.584.719
CiteProbe.787.732.830.604.738
Claude Opus 5ContextCite-32 (M).562.548.442.292.461
ARC-JSD (G).657.619.507.327.527
AttnTrace (Q).701.695.628.447.618
AT2 (M).762.742.742.495.685
BM25.584.648.435.295.491
Top-16 heads (Q).779.736.690.474.669
CiteProbe (M).761.770.739.544.703
GPT-5.6 SolContextCite-32 (M).522.474.637.403.509
ARC-JSD (Q).698.631.744.512.646
AttnTrace (Q).677.649.813.603.685
AT2 (M).707.675.873.638.723
BM25.502.539.660.399.525
Top-16 heads (M).771.663.852.638.731
CiteProbe (G).744.707.835.628.728

M denotes Ministral, G denotes Gemma, and Q denotes Qwen surrogates.

W2. Recall is particularly important because the paper evaluates CiteProbe on multi-hop datasets, where multiple pieces of evidence are needed to support an answer. Finding just one relevant sentence is not enough if other required evidence is missing, but the main results do not show how much of this evidence CiteProbe actually identifies.
Q1. Why does the main evaluation emphasize Top-1 accuracy without reporting recall, especially for questions that require multiple supporting sentences, when mAP already measures ranking performance?

Reply: We agree that recall is important for identifying multiple supporting sentences, particularly for multi-hop questions. Top-1 measures the single-citation use case, and recall is shown through R/P/F1 and mAP. Since recall can be obtained by trading off precision, we address this concern directly by using reported data (Table 23, Appendix M.2) showing CiteProbe's Recall and Precision, alongside F1 and also the added AUPRC on multi-hop HotpotQA and MuSiQue, alongside the strongest competing baseline:

BackboneMethodHotpotQA RecallHotpotQA PrecisionHotpotQA F1HotpotQA AUPRCMuSiQue RecallMuSiQue PrecisionMuSiQue F1MuSiQue AUPRC
Phi-4-miniRunner-up69.1%†^{\dagger}72.7%†^{\dagger}70.9%†^{\dagger}.750$^{\dagger}$53.2%∗^{*}69.3%∗^{*}60.2%∗^{*}.536$^{*}$
CiteProbe76.4%66.6%71.2%.76262.6%69.0%65.6%.690
Ministral-3-8BRunner-up66.5%†^{\dagger}71.2%†^{\dagger}68.8%†^{\dagger}.744$^{\dagger}$60.5%∗^{*}64.7%∗^{*}62.5%∗^{*}.584$^{*}$
CiteProbe70.6%68.4%69.5%.75071.2%66.7%68.9%.724
Qwen3.5-9BRunner-up69.9%∗^{*}54.8%∗^{*}61.4%∗^{*}.512$^{*}$83.6%∗^{*}29.6%∗^{*}43.8%∗^{*}.447$^{*}$
CiteProbe74.6%65.7%69.9%.74374.3%65.8%69.8%.740
Gemma-4-12BRunner-up60.8%†^{\dagger}73.3%†^{\dagger}66.4%†^{\dagger}.708$^{\dagger}$53.6%∗^{*}45.9%∗^{*}49.4%∗^{*}.385$^{*}$
CiteProbe74.1%71.9%73.0%.78769.1%68.8%68.9%.732

$^{\dagger}$ denotes AT2, and ∗^{*} denotes ARC-JSD.

Overall, CiteProbe demonstrates consistently strong multi-hop evidence recovery, achieving higher recall in 7/8 settings, while maintaining higher F1 and AUPRC in all 8 settings against runner-up baselines. More specifically, on Qwen3.5-9B/MuSiQue, ARC-JSD reaches 83.6% recall by trading precision down to 29.6%. The resulting F1 of 43.8% and AUPRC of .447 show the cost of that trade-off.

Recall should therefore be interpreted together with precision and AUPRC. The evidence for multi-sentence recovery comes directly from the citation-set evaluation using Recall and F1, while mAP and Top-1 provide complementary information about ranking quality. AUPRC further evaluates performance across operating thresholds and avoids relying on a single threshold choice.

Recognizing your inputs, and as discussed in W1, we added AUPRC to comprehensively address recall concerns. On this standard metric, CiteProbe managed the highest mean AUPRC on five of six backbones.

W3. Even with the current evaluation setup used in the main text, CiteProbe shows only small improvements over the Top-16-head baseline and sometimes performs worse. It is unclear how much the learned linear probe actually helps.

Reply: Thank you for this observation. We agree that the gains over Top-16 are modest in some settings. Top-16 is a strong diagnostic baseline and essentially a restricted linear probe, derived from our preliminary analysis showing that a small subset of informative heads can already support effective citation attribution. We included Top-16 as a transparent diagnostic baseline, and CiteProbe generalizes this idea by learning head-specific weights jointly in a single linear fit, rather than relying on a separate head-ranking and selection procedure.

To quantify the difference more carefully, we report 95% paired bootstrap confidence intervals (CI) for CiteProbe − Top-16. Across the 48 settings, CiteProbe achieves 18 wins, 25 ties, and 5 losses, indicating modest gains over the strong Top-16 baseline.

CiteProbe vs Top-16

BackboneHotpotQAMuSiQueTyDiQAWiCE
Phi-4-mini (ΔmAP\Delta_{\text{mAP}})○ -.007 [-.016, .001]▲ .013 [.005, .021]○ .011 [-.001, .024]○ .002 [-.006, .010]
Phi-4-mini (ΔTop-1\Delta_{\text{Top-1}})○ .007 [-.013, .027]○ .013 [-.009, .035]○ .015 [-.005, .038]▲ .025 [.009, .042]
Ministral-3-8B (ΔmAP\Delta_{\text{mAP}})○ -.003 [-.014, .007]▲ .030 [.021, .039]○ .005 [-.021, .031]▲ .013 [.002, .024]
Ministral-3-8B (ΔTop-1\Delta_{\text{Top-1}})○ .021 [-.003, .044]▲ .047 [.022, .074]○ .007 [-.040, .055]▲ .054 [.030, .078]
Qwen3.5-9B (ΔmAP\Delta_{\text{mAP}})▼ -.014 [-.023, -.006]▲ .015 [.008, .021]○ -.012 [-.027, .003]○ .001 [-.008, .011]
Qwen3.5-9B (ΔTop-1\Delta_{\text{Top-1}})○ .002 [-.017, .021]▲ .032 [.013, .050]○ -.023 [-.050, .004]▲ .031 [.010, .051]
Gemma-4-12B (ΔmAP\Delta_{\text{mAP}})○ -.006 [-.013, .000]▲ .028 [.022, .034]○ .007 [-.005, .019]○ .006 [-.002, .013]
Gemma-4-12B (ΔTop-1\Delta_{\text{Top-1}})▼ -.018 [-.034, -.003]○ .012 [-.000, .025]▲ .023 [.003, .046]○ .015 [-.000, .031]
Claude Opus 5 (surr.) (ΔmAP\Delta_{\text{mAP}})▼ -.026 [-.035, -.016]▲ .014 [.005, .022]▲ .054 [.031, .078]▲ .050 [.037, .061]
Claude Opus 5 (surr.) (ΔTop-1\Delta_{\text{Top-1}})○ -.002 [-.022, .019]▲ .030 [.009, .051]▲ .102 [.058, .147]▲ .108 [.081, .135]
GPT-5.6 Sol (surr.) (ΔmAP\Delta_{\text{mAP}})▼ -.030 [-.039, -.020]▲ .022 [.014, .031]○ -.006 [-.021, .010]▼ -.016 [-.024, -.009]
GPT-5.6 Sol (surr.) (ΔTop-1\Delta_{\text{Top-1}})○ -.007 [-.023, .008]○ .012 [-.001, .025]○ .001 [-.026, .029]○ -.002 [-.017, .014]
Win (▲) / tie (○) / loss (▼)0 / 8 / 49 / 3 / 03 / 9 / 06 / 5 / 1

W4. The claim that CiteProbe extends to proprietary models is overstated. The authors pass their answers through an open-source model and use its attention weights to find supporting evidence. They could do the same with ground-truth answers. This only shows that CiteProbe works on externally generated text.
Q2. In the proprietary-model experiments, CiteProbe simply takes the generated answers and uses an open-source model to identify supporting evidence. The same could be done with ground-truth answers. What do these experiments demonstrate about CiteProbe's extension to proprietary models beyond its ability to process any given answer?

Reply: We thank the reviewer for raising this distinction. Indeed, the same approach could be applied to ground-truth or other externally generated answers, which highlights the broader applicability of CiteProbe. However, for proprietary LLMs, external text is precisely all that is available. The question is therefore whether CiteProbe transfers well. For instance, BM25 drops sharply on GPT-5.6 Sol, possibly because GPT-5.6 paraphrases more extensively and reduces lexical overlap. In contrast, CiteProbe continues to maintain strong performance. Therefore, our experiments establish three practical findings.

First, CiteProbe's performance transfers to closed-source models without retraining, despite differences in answer lengths, styles, and paraphrasing.

Second, the results are strong across different surrogates. CiteProbe achieves the highest mean performance on Claude Opus 5, and the highest mean Top-1 on GPT-5.6 Sol.

Third, this makes citation of proprietary-model outputs practical, since CiteProbe requires only a single forward pass.

Nonetheless, we agree that our original wording could be more precise and have revised "extending to proprietary models" to "providing citations for proprietary-model outputs through model surrogacy".

Reviewer_Vc4VOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes CiteProbe, a method for citation attribution, which trains a linear probe to predict citation relationship from a head-level attention map. The method is shown to be effective and extensive analyses are performed. However, important prior work is missing.

Critical strengths

The method is simple and addresses a practical problem. Experiments and analyses are generally complete and thorough, including held-out-dataset evaluation and surrogate experiments.

Critical weaknesses
  1. The closest prior method is missing. AT2 (Cohen-Wang et al., 2025) already uses attention aggregated over source and response tokens, learns a coefficient for each attention head, and applies the resulting readout to new examples. This closely overlaps with CiteProbe’s design, but AT2 is neither discussed nor evaluated.

  2. I was slightly confused at the flow of the paper. What's the purpose of Section 3.2? If it's saying that few heads do the job, the following main method goes back to using all heads (I'd expect at least a L1-regularized probe); if it's saying that head identity matters, then I think that's not very surprising.

  3. Section 3.3 describes a fused implementation compatible with FlashAttention, but Appendix C says that the reported experiments use eager attention. This is a bit conflicting.

Reference

  • Cohen-Wang, B., Chuang, Y.-S., and Mądry, A. (2025). Learning to Attribute with Attention. arXiv:2504.13752.
Questions for authors
  1. In Table 5, does the reported inference overhead includes attention extraction and any generation slowdown relative to ordinary optimized inference? These statistics, especially at long context, would be helpful.
7 comments

Dear Reviewer Vc4V,

Thank you for your thoughtful review. We are glad that you found CiteProbe simple and effective for a practical problem, and our experiments complete and thorough. Below, we respond to each comment and outline the corresponding revisions.

W1. The closest prior method is missing. AT2 (Cohen-Wang et al., 2025) already uses attention aggregated over source and response tokens, learns a coefficient for each attention head, and applies the resulting readout to new examples. This closely overlaps with CiteProbe’s design, but AT2 is neither discussed nor evaluated.

Reply: Thank you for highlighting AT2. We agree to include this arXiv preprint to Related Work and our experimental baselines. We summarize the main differences between AT2 and CiteProbe as follows:

CiteProbe's componentWhat AT2 does
Attention Weight Centering: Subtracts each head's mean attention across candidate sentences within an example (Equation 2).None. Uses each head's attention as is.
Scaling: Divides each head's attention scores by their standard deviation within an example (Equation 2).None. Uses each head's attention as is.
AdamW: Trains the linear probe using AdamW with decoupled weight decay, without normalizing the learned coefficients.Uses Adam and subsequently L1-normalizes the learned head coefficients.
Evidence labels: Trains the linear probe directly on annotated supporting-evidence labels.Trains using ablation-derived targets measuring the effects of masking context tokens.
Training loss: Binary Cross Entropy.Training loss: Negative Pearson Correlation.

We evaluate AT2 using the same protocol, and CiteProbe achieves higher mean mAP and Top-1 than AT2 on all six backbones:

BackboneAT2 mAPCiteProbe mAPAT2 Top-1CiteProbe Top-1
Phi-4-mini.818.831.830.856
Ministral-3-8B.807.822.804.835
Qwen3.5-9B.659.832.635.855
Gemma-4-12B.820.842.859.873
Claude Opus 5 (surr.).796.824.778.839
GPT-5.6 Sol (surr.).824.836.894.901

CiteProbe also avoids AT2's ablation-based training cost. Table 5 reports 242.3 minutes for CiteProbe versus 703.3 for AT2 on Gemma-4-12B. The advantage is therefore improved citation ranking with a simpler training procedure using evidence labels already available in the benchmarks.

The main revisions to the manuscript are:

  1. L70–72: Introduced AT2 as a baseline.
  2. L103–104: Added AT2 and clarified its use of ablation-derived supervision to learn attention-head weights.
  3. L283: Added AT2 as an attention-based attribution baseline.
  4. L324: Added AT2 results across all six LLMs and four datasets.
  5. L378–387, L420: Added AT2 to the lexical competition analysis and comparison.
  6. L439: Added AT2 to the citation failure analysis.

W2. I was slightly confused at the flow of the paper. What's the purpose of Section 3.2? If it's saying that few heads do the job, the following main method goes back to using all heads (I'd expect at least a L1-regularized probe); if it's saying that head identity matters, then I think that's not very surprising.

Reply: Thank you for this observation. We agree that head identity mattering is not itself surprising. Section 3.2 serves as a preliminary investigation, demonstrating that selecting informative heads achieves strong citation performance, but requires separate head ranking, sweeping k, and judging selection stability across datasets. This motivates CiteProbe to operationalize the process through a single linear fit.

Importantly, in Appendix D.2, we show that the 16 largest probe coefficients account for only 6.4–33.3% of the total absolute coefficient mass, indicating that sufficiency may not imply enforcing sparsity.

We also appreciate the suggestion of L1 regularization and have conducted an additional L1 and L2 comparison with CiteProbe to identify if there is an optimal linear setting. The results show that although L1 produces sparser readouts, citation performance between the probes does not vary meaningfully:

Paired differences, with 95% confidence intervals

ModelDifferenceΔmAPΔTop-1ΔAUPRC
Phi-4-miniCiteProbe − L1○ .002 [-.001, .006]○ .003 [-.004, .011]▲ .008 [.002, .013]
CiteProbe − L2○ -.001 [-.003, .002]○ .001 [-.003, .006]▼ -.003 [-.005, -.001]
L1 − L2○ -.003 [-.006, .000]○ -.002 [-.009, .006]▼ -.010 [-.014, -.006]
Ministral-3-8BCiteProbe − L1▲ .011 [.006, .015]○ -.004 [-.013, .005]▲ .016 [.012, .020]
CiteProbe − L2○ .000 [-.005, .005]○ .004 [-.007, .014]▲ .013 [.007, .019]
L1 − L2▼ -.010 [-.016, -.004]○ .007 [-.005, .020]○ -.003 [-.009, .003]
Qwen3.5-9BCiteProbe − L1○ -.001 [-.005, .002]▲ .008 [.001, .016]○ -.001 [-.005, .002]
CiteProbe − L2○ -.002 [-.005, .001]○ .004 [-.002, .010]▼ -.004 [-.006, -.001]
L1 − L2○ -.001 [-.003, .002]○ -.004 [-.010, .001]▼ -.003 [-.005, -.001]
Gemma-4-12BCiteProbe − L1○ .000 [-.003, .003]○ -.001 [-.007, .005]○ -.002 [-.005, .001]
CiteProbe − L2▲ .002 [.000, .004]○ .001 [-.003, .006]▲ .003 [.001, .004]
L1 − L2○ .002 [-.001, .005]○ .003 [-.003, .008]▲ .005 [.001, .008]

▲ marks an interval above zero, ▼ below zero, and ○ an interval containing zero.

L1 sparsity

ModelNon-zero coefficient/Total
Phi-4-mini88/768 (11%)
Ministral-3-8B662/1088 (61%)
Qwen3.5-9B85/128 (67%)
Gemma-4-12B158/768 (21%)

Taken together, these results reinforce our original motivation. Identifying informative heads is valuable, but explicitly enforcing sparsity is not necessary for strong citation performance. CiteProbe therefore retains a straightforward linear readout. We have revised the manuscript in Appendix D.2 (Table 12) to include this ablation.

W3. Section 3.3 describes a fused implementation compatible with FlashAttention, but Appendix C says that the reported experiments use eager attention. This is a bit conflicting.

Reply: Thank you for identifying this ambiguity. To clarify, all reported experiments use eager attention to maintain controlled experimental conditions. We have clarified this distinction in the updated manuscript at L240–245, Section 3.3 (Inference).

To elaborate, the reasons we develop the CiteProbe fused kernel are twofold. First, optimized attention implementations, such as FlashAttention, do not materialize the attention matrix, which means that CiteProbe has to accumulate attention differently by fusing with the optimized computations. Second, to claim that CiteProbe does not incur inference slowdown, it has to work outside of lab conditions, where compatibility with such attention optimizations becomes the default. Appendix C, which you have pointed to, is where we demonstrate that our implementation achieves numerical agreement up to floating-point rounding due to operations like softmax tiling.

Q1. In Table 5, does the reported inference overhead includes attention extraction and any generation slowdown relative to ordinary optimized inference? These statistics, especially at long context, would be helpful.

Reply: Thank you for the question. Table 5's reported inference overhead includes all slowdowns relative to eager inference. Relative to optimized inference, with the CiteProbe fused kernel, the overhead is not expected to differ from that of the eager setting. This is because the additional mathematical operations required by the fused kernel (Appendix C, Equations 9–12) to reproduce the results of Equations 1, 2, and 5 are trivial. To illustrate this in practice, we additionally timed generation with attention collection using Gemma-4-12B with the fused kernel (Triton port) on LongBench-Cite documents (4K-29K context length). We limit generation to 128 tokens, with three repeats after warmup, for 90 paired comparisons. The table reports the average time to process the prompt and generate the answer, confirming that the additional attention extraction time is negligible:

Prompt tokensCiteProbe offCiteProbe onExtra timeSlowdown
4,09612.94 s13.02 s+0.07 s0.6%
16,38419.22 s19.30 s+0.08 s0.4%
28,67229.84 s29.98 s+0.14 s0.5%

We have added this measurement to the updated manuscript in L1050–1064, Appendix C.3, Table 9.

Reviewer_piv3OpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

CiteProbe ranks candidate source sentences for an already generated answer. It sums answer-to-source attention separately for each layer and head, then standardizes each head’s scores across candidate sentences within the example. A logistic probe trained with class-weighted binary cross-entropy predicts supporting-evidence labels while the LLM remains frozen. The target is annotated support rather than causal dependence. The paper first selects Top-k heads by evidence correlation as a diagnostic, then compares the full linear readout, larger readouts, and pruning.

Open models can reuse generation attention. Closed-model answers require one additional teacher-forced pass through an open surrogate. An online-softmax sentence-sum algorithm avoids storing the full attention matrix. Evaluation uses four open backbones and two closed answer models on HotpotQA, MuSiQue, TyDiQA, and WiCE, training on three datasets and testing on the fourth. Each dataset supplies 500 training, 50 validation, and up to 1,000 test examples. Main ranking results retain only correctly answered examples and average three training seeds. Across 24 model-dataset combinations, mean mAP and Top-1 improve over ARC-JSD by approximately 3.9 and 7.0 percentage points. Further analyses cover surrogates, thresholded citations, incorrect answers, and lexical competition.

Critical strengths
  1. Preserving head-specific attention and fitting a small readout provides a support-ranking method that can reuse generation computation. Appendix C explains how to avoid materializing the full attention matrix and reports numerical agreement with eager calculation. Table 5 separates 242.3 minutes of one-time training from 12.1 minutes of additional inference, totaling 254.4 versus ARC-JSD’s 1,556.0 minutes. The implementation and cost accounting give the method practical value.

  2. The main study tests cross-dataset transfer and includes Top-16 heads, BM25, larger readouts, and pruning, rather than only expensive perturbation baselines. Table 1 supports the macro-average gains over ARC-JSD. The appendices report training-seed variation, thresholded citation results, and the sample counts retained after answer-correctness filtering. The paper also distinguishes support prediction from causal attribution and reports limited answer-correctness discrimination, keeping important scope boundaries explicit.

Critical weaknesses
  1. AT2, public since April 2025, also treats different heads’ attention as features, learns linear coefficients, and avoids repeated ablations on new examples. It is neither cited nor specifically discussed. CiteProbe instead uses supporting-evidence labels, within-example standardization, and a sentence-level implementation. These differences are useful, but describing earlier attention methods mainly as fixed aggregation overstates the novelty of learning a readout. The supervision and implementation increments need clearer positioning. I verified a public preprint and submission record, but not a peer-reviewed publication; omitting this preprint comparison alone would not justify rejection.

  2. The full probe does not consistently improve on simpler controls in Table 1. Qwen mean mAP is 0.832 versus Top-16’s 0.834; the GPT-surrogate values are 0.836 and 0.843. Qwen Top-1 is also slightly below BM25. Appendix D reports a maximum three-seed mAP standard deviation of 0.008, but this does not replace paired intervals for differences on the same test examples. The evidence supports inexpensive support ranking more strongly than a stable increment from full learned weighting over head selection or the abstract’s claim of consistently outperforming alternatives.

Questions for authors
  1. Relative to AT2, which existing results isolate the additions from evidence-label supervision, within-example standardization, and implementation changes? With attention features fixed, how do supervision targets change citation ranking and computational cost?

  2. What are the per-dataset paired intervals against Top-16 and BM25? Which examples account for the Qwen and GPT-surrogate mAP reductions, and do the main differences persist across training seeds?

  3. Were Table 1’s best surrogates chosen using validation or test results? What remains if each surrogate is fixed before testing? Citation thresholds and the linear learning rate already have validation-selection specifications; this question concerns surrogate choice only.

6 comments

Dear Reviewer piv3,

Thank you for your careful and constructive review. We appreciate your recognition of CiteProbe's practical value, comprehensive evaluation, and clearly defined scope. Your suggestions have helped us strengthen the manuscript through additional comparisons, statistical analyses, and methodological clarifications.

W1. AT2, public since April 2025, also treats different heads' attention as features, learns linear coefficients, and avoids repeated ablations on new examples. It is neither cited nor specifically discussed. CiteProbe instead uses supporting-evidence labels, within-example standardization, and a sentence-level implementation. These differences are useful, but describing earlier attention methods mainly as fixed aggregation overstates the novelty of learning a readout. The supervision and implementation increments need clearer positioning.

Q1. Relative to AT2, which existing results isolate the additions from evidence-label supervision, within-example standardization, and implementation changes? With attention features fixed, how do supervision targets change citation ranking and computational cost?

Reply: We thank the reviewer for highlighting the arXiv preprint AT2. We agree that including AT2 strengthens our work and have added it to Related Work and our experimental baselines. AT2 is competitive with ARC-JSD, but is outperformed by CiteProbe overall, as summarized below:

ModelARC-JSDAT2CiteProbe
mAPTop-1AUPRCmAPTop-1AUPRCmAPTop-1AUPRC
Phi-4-mini.796.811.526.818‾\underline{.818}.830‾\underline{.830}.709‾\underline{.709}.831.856.720
Ministral-3-8B.779.763.547.807‾\underline{.807}.804‾\underline{.804}.688‾\underline{.688}.822.835.703
Qwen3.5-9B.785‾\underline{.785}.744‾\underline{.744}.438.659.635.480‾\underline{.480}.832.855.719
Gemma-4-12B.782.786.445.820‾\underline{.820}.859‾\underline{.859}.707‾\underline{.707}.842.873.738
Claude Opus 5.774.755.527.796‾\underline{.796}.778‾\underline{.778}.685‾\underline{.685}.824.839.703
GPT-5.6 Sol.838.879.646.824.894‾\underline{.894}.723‾\underline{.723}.836‾\underline{.836}.901.728

On the point of describing earlier attention methods as "typically relying on fixed aggregation", this mainly reflects the broader published literature. In particular, the three published attention-based methods in our Related Work (Grad-SAM, Think-Twice, and AttnTrace) all use fixed and uniform aggregation across attention heads.

AT2 is a relevant exception, which we now explicitly acknowledge in Related Work. AT2 already learns a coefficient for each attention head. Here, we summarize the component differences between CiteProbe and AT2:

CiteProbe's componentWhat AT2 does
Attention Weight Centering: Subtracts each head's mean attention across candidate sentences within an example (Equation 2).None. Uses each head's attention as is.
Scaling: Divides each head's attention scores by their standard deviation within an example (Equation 2).None. Uses each head's attention as is.
AdamW: Trains the linear probe using AdamW with decoupled weight decay, without normalizing the learned coefficients.Uses Adam and subsequently L1-normalizes the learned head coefficients.
Evidence labels: Trains the linear probe directly on annotated supporting-evidence labels (Training loss: Binary Cross Entropy).Trains using ablation-derived targets measuring the effects of masking context tokens (Training loss: Negative Pearson Correlation).

(continued below)

(continued from above)

We ran ablations that start from AT2 and add/replace one component from CiteProbe. Additionally, to isolate the effect of AT2’s ablation labels, we ran the “CiteProbe + AT2 ablation labels” ablation which shows a drop in performance when evidence-labels are replaced. Training loss is coupled to the label types and not practical to ablate. The results are as follows:

AT2 vs CiteProbe Ablations (mAP / Top-1 / AUPRC)

AblationPhi-4-miniMinistral-3-8BQwen3.5-9BGemma-4-12B
AT2.818 / .830 / .709.807 / .804 / .688.659 / .635 / .480.820 / .859 / .707
AT2 + centering.818 / .827 / .707.807 / .804 / .689.682 / .693 / .509.823 / .860 / .714
AT2 + scaling.812 / .833 / .649.808 / .801 / .646.743 / .778 / .558.828 / .868 / .685
AT2 + AdamW.818 / .830 / .708.808 / .804 / .688.662 / .642 / .480.821 / .861 / .708
AT2 + evidence labels.778 / .808 / .647.761 / .769 / .611.752 / .779 / .602.751 / .783 / .589
CiteProbe + AT2 ablation labels.810 / .831 / .683.808 / .803 / .676.802 / .814 / .664.827 / .867 / .713
CiteProbe.831 / .856 / .720.822 / .835 / .703.832 / .855 / .719.842 / .873 / .738

The results show that centering and scaling improve mAP, most strongly on Qwen, although scaling alone lowers AUPRC on the other three backbones. Evidence-label supervision alone does not reproduce CiteProbe's gains either. Moreover, replacing CiteProbe's evidence-label supervision with AT2's ablation labels degraded CiteProbe's performance. This suggests that CiteProbe's improvements arise from the combination of components.

On computational cost, we added AT2 to 4.6 Efficiency Comparison (RQ5), which shows that AT2's training time on ablation labels is 2.9x of CiteProbe. If AT2 trains on evidence labels like CiteProbe, the overheads would be similar, except our earlier mentioned ablation of AT2 + evidence labels cannot match CiteProbe's performance.

The main revisions to the manuscript are:

  1. L70–72: Introduced AT2 as a baseline.
  2. L103–104: Added AT2 and clarified its use of ablation-derived supervision to learn attention-head weights.
  3. L283: Added AT2 as an attention-based attribution baseline.
  4. L324: Added AT2 results across all six LLMs and four datasets.
  5. L378–387, L420: Added AT2 to the lexical competition analysis and comparison.
  6. L439: Added AT2 to the citation failure analysis.

W2. The full probe does not consistently improve on simpler controls in Table 1. Qwen mean mAP is 0.832 versus Top-16’s 0.834; the GPT-surrogate values are 0.836 and 0.843. Qwen Top-1 is also slightly below BM25. Appendix D reports a maximum three-seed mAP standard deviation of 0.008, but this does not replace paired intervals for differences on the same test examples. The evidence supports inexpensive support ranking more strongly than a stable increment from full learned weighting over head selection or the abstract’s claim of consistently outperforming alternatives.
Q2. What are the per-dataset paired intervals against Top-16 and BM25? Which examples account for the Qwen and GPT-surrogate mAP reductions, and do the main differences persist across training seeds?

Reply: Thank you for this observation. Top-16 is our own preliminary head-selection method, showing that a small subset of informative heads already yields strong citation performance. It can be viewed as a restricted linear readout that assigns equal weights to selected heads and zero to the rest. CiteProbe's use of a linear layer operationalizes Top-16, giving a cleaner formulation that replaces the process of separate head ranking, selection, and head stability analysis, with a single linear fit. Nonetheless, we include Top-16 in full transparency as a diagnostic baseline.

As you suggested, we computed 95% paired CI against BM25 and Top-16, showing that CiteProbe improves significantly over BM25 (36 wins, 8 ties, 4 losses), with more modest gains over Top-16 (18 wins, 25 ties, 5 losses). The results are as follows:

CiteProbe vs BM25

BackboneHotpotQAMuSiQueTyDiQAWiCE
Phi-4-mini (ΔmAP\Delta_{\text{mAP}})▲ .052 [.040, .064]▲ .027 [.014, .041]▲ .021 [.003, .039]▲ .106 [.090, .123]
Phi-4-mini (ΔTop-1\Delta_{\text{Top-1}})▲ .026 [.001, .051]▲ .039 [.010, .070]▲ .042 [.009, .076]▲ .156 [.124, .188]
Ministral-3-8B (ΔmAP\Delta_{\text{mAP}})▲ .053 [.041, .066]▲ .025 [.013, .037]○ .014 [-.012, .039]▲ .076 [.060, .092]
Ministral-3-8B (ΔTop-1\Delta_{\text{Top-1}})▲ .045 [.020, .070]▲ .029 [.002, .058]○ .027 [-.019, .075]▲ .120 [.091, .149]
Qwen3.5-9B (ΔmAP\Delta_{\text{mAP}})▲ .020 [.009, .031]▼ -.012 [-.022, -.002]▼ -.024 [-.044, -.003]▲ .050 [.037, .065]
Qwen3.5-9B (ΔTop-1\Delta_{\text{Top-1}})○ -.009 [-.031, .014]▼ -.025 [-.047, -.003]▼ -.045 [-.083, -.006]▲ .070 [.044, .097]
Gemma-4-12B (ΔmAP\Delta_{\text{mAP}})▲ .051 [.040, .062]○ .002 [-.009, .013]○ .006 [-.009, .023]▲ .049 [.036, .063]
Gemma-4-12B (ΔTop-1\Delta_{\text{Top-1}})▲ .025 [.003, .046]○ .015 [-.009, .039]▲ .030 [.001, .061]▲ .049 [.021, .076]
Claude Opus 5 (surr.) (ΔmAP\Delta_{\text{mAP}})▲ .029 [.018, .040]▲ .012 [.002, .021]▲ .032 [.008, .057]▲ .092 [.077, .106]
Claude Opus 5 (surr.) (ΔTop-1\Delta_{\text{Top-1}})○ .018 [-.004, .040]○ .002 [-.020, .024]▲ .069 [.023, .117]▲ .161 [.130, .192]
GPT-5.6 Sol (surr.) (ΔmAP\Delta_{\text{mAP}})▲ .110 [.096, .124]▲ .062 [.051, .074]▲ .023 [.001, .045]▲ .092 [.077, .107]
GPT-5.6 Sol (surr.) (ΔTop-1\Delta_{\text{Top-1}})▲ .070 [.045, .096]▲ .040 [.019, .061]▲ .070 [.026, .116]▲ .106 [.078, .133]
Win (▲) / tie (○) / loss (▼)10 / 2 / 07 / 3 / 27 / 3 / 212 / 0 / 0

(continued below)

Comments on the paper

1 comment

Dear Area Chair and Reviewers,

We sincerely thank all reviewers for their constructive and positive feedback. The discussions have helped us further strengthen the paper through the following additions and clarifications.

1. Additional baseline comparison. We incorporated AT2 into our experimental baselines, providing a more comprehensive comparison with prior work. Although an arXiv preprint, AT2 is competitive with the strongest baseline (ARC-JSD). We show that CiteProbe nevertheless outperforms both.

2. Expanded citation quality evaluation. We added Area Under the Precision-Recall Curve (AUPRC) to the main results, alongside precision-recall curves and multi-hop evidence recovery analyses, to provide a more comprehensive assessment of citation quality beyond ranking metrics (mAP and Top-1).

3. Additional ablations and statistical analyses. We conducted component ablations, L1 and L2 regularization comparisons, and 95% paired bootstrap confidence intervals against our preliminary Top-16 method, and BM25. These additions provide further insight into CiteProbe's methodological choices and performance relative to these competitive baselines.

We believe these additions, guided by the reviewers' feedback, have further strengthened the evidence for CiteProbe's contributions. We sincerely appreciate the reviewers' valuable role in shaping and improving this work.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.