acceptodds
Under review as a conference paper at ICLR 2027

Phantom Merge: When Your Large Language Model Agents Pick One but Tell You About Another

Abstract

LLM agents increasingly operate over multiple entities, where reliable response generation requires not only retrieving relevant evidence but also preserving its association with the correct entity. We identify Phantom Merge (PM), an entity–evidence binding failure in which an agent attributes unsupported claims to its committed entity, potentially even when the task is completed correctly. We formalize PM at the claim level and characterize its prevalence and error subtypes across four agent domains and four backbone families. To address this problem, we propose a two-stage framework for claim-level risk estimation and correction. First, Anchor-Grounded Risk (AGR) combines a representational readout of the agent's pre-emission hidden state with Jacobian-based functional information, providing complementary slot-level and anchor-value risk scores. Second, Fixed-Anchor Correction (FAC) audits flagged claims and selectively revises them using observed anchor evidence or removes them when no supported replacement is available. On the held-out Shopping detection cohort, AGR-slot achieves an AUROC of 96.2% and an average precision of 97.3%, outperforming the evaluated detection baselines. In the end-to-end Shopping mitigation evaluation, FAC reduces trajectory-level PM from 51.4% to 6.2%, with an output-to-original claim ratio of 80.7%. These results demonstrate the prevalence of entity-specific grounding failures and the effectiveness of combining internal risk estimation with evidence-constrained claim correction.

Primary area:
interpretability and explainable AI
Keywords:
LLM agents, binding failure, claim attribution, representation probing, hallucination detection, agentic evaluation, activation analysis

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_MV96OpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

This paper studies entity–evidence binding failures in multi-entity LLM agents. It defines Phantom Merge (PM) as a failure in which an agent commits to one entity but attributes unsupported claims to it using evidence from another entity, user constraints, or conflicting fabricated values. The paper further separates PM into cross-entity misattribution, constraint-to-anchor projection, and anchor-slot fabrication.

For detection, the paper proposes Anchor-Grounded Risk (AGR), combining a supervised hidden-state probe with a Jacobian-based functional readout. For mitigation, Fixed-Anchor Correction (FAC) retains, rewrites, or removes flagged claims while preserving the selected entity. AGR-slot reaches 96.2% AUROC on the Shopping test set, and FAC reduces trajectory-level PM from 51.4% to 6.2%.

The problem is important, and entity-specific attribution is not fully captured by ordinary factuality or task-success metrics. However, the strongest detector does not directly use anchor values, mitigation is largely evaluated by the same AGR machinery used within the intervention pipeline, and closely overlapping recent work is missing. AGR-slot also generalizes poorly outside Shopping, while the linked code does not correspond to the current paper.

Critical strengths
  • A claim may be supported somewhere in the trajectory while still being false for the entity to which it is attributed.

  • Its post-hoc design could be integrated without retraining the original agent.

Critical weaknesses
  • AGR-slot combines a supervised representational score with Jacobian probability mass over a slot lexicon. It does not use the committed anchor’s identity or observed value. Its 96.2% AUROC may therefore reflect correlations between PM labels and claim or slot patterns rather than detection of incorrect entity binding. AGR-value uses actual anchor values for only 48 of 274 test claims and falls back to the representational branch for the remaining 226.

  • AGR-slot does not generalize well across domains. Its AUROC drops from 0.96 on Shopping to approximately 0.78, 0.69, and 0.60 on Telecom, Airline, and Retail, respectively. It is also worse than the representational readout on all three additional domains. The quote-template analysis does not rule out broader domain or slot-template leakage.

  • Post-intervention outputs are re-evaluated using the same AGR procedure involved in claim selection and functional auditing, with only manual spot checks. Residual PM should instead be measured through independent, blinded human annotation of all corrected outputs.

  • Deceptive Grounding [R1] studies claims supported by evidence belonging to the wrong entity and evaluates entity-attribution verification. ProvenanceGuard [R2] studies cross-source conflation in tool-using agents and performs source-aware verification and repair. Without a direct comparison, the claim that PM is a novel reliability problem is not adequately supported.

  • The Shopping annotation does not report the number of annotators, independent double annotation, pre-adjudication agreement, or κ\kappa. The other domains rely mainly on two GPT-5.6 Astra passes with limited human audits. “No disagreements after reconciliation” is not a reliability statistic.

  • Several methods use a common threshold of 0.05 despite having differently calibrated score scales, and external baselines are not uniformly retuned. Threshold-free AUROC results are more credible than the reported P/R/F1 comparisons.

  • Cross-backbone PM prevalence does not demonstrate cross-backbone AGR/FAC performance. The method’s reliance on internal hidden states and Jacobians makes such transfer nontrivial.

  • There are no latency, memory, Jacobian-computation, or correction-cost measurements. The method also requires model internals that are unavailable for many API-based agents.

[R1] Deceptive Grounding: Entity Attribution Failure in Clinical Retrieval-Augmented Generation

[R2] ProvenanceGuard: Source-Aware Factuality Verification for MCP-Based LLM Agents

Questions for authors

Refer to weaknesses

3 comments

Thank you for the review. All changes in the revised PDF are marked in blue.

W1. AGR-slot ignores the anchor's value; AGR-value covers 48 of 274 claims.

You are right about AGR-slot. We fixed slot assignment (annotated slot where available, otherwise parsed from the claim), which lowers AGR-slot to 0.866; extended the anchor-value target from 48 to all 274 test claims; and made the PM probe the main detector. With one slot rule, J-slot reaches 0.605, AGR-slot 0.866, AGR-value 0.895. The probe uses no slots: AUROC 0.888 [0.847, 0.924], AP 0.908, above every external baseline. It holds across slot types: 0.846 within slots containing both PM and correctly bound claims, 0.89 (CAP) and 0.88 (CEM) against correctly bound claims, and 0.883 on test claims whose quote template never appears in training.

Revision: Table 2 reports the PM probe and AGR-value. Appendix B reports J-slot and AGR-slot. §5.2 and Appendix B report the slot, subtype, and template results.

W2. AGR-slot generalizes poorly across domains; leakage.

Trained on Shopping only, the probe reaches AUROC 0.957 on Telecom, 0.816 on Airline, and 0.870 on Retail. The AGR-slot drop comes from its Shopping slot lexicon: Shopping claims are about price and rating, Telecom about plans and data allowances, Airline about fares and departure times, and the probe needs no lexicon. On leakage, training and test claims come from different trajectories (1,244 claims; 243 test trajectories), and the probe holds 0.883 on unseen templates (W1).

Revision: §5.2 explains the lexicon with examples; Fig. 2 is updated.

W3. Residual PM is measured with the same AGR procedure.

Residual PM uses no AGR score. A deleted claim leaves the output, an unchanged claim keeps its original gold label (including PM claims the gate missed), and a rewritten claim is checked by an expert annotator against the selected entity's evidence. A trajectory counts if any remaining claim is PM. Of the 57 rewrites, 1 still contained PM (1.8%).

Revision: §4 states this calculation; Appendix G reports the human audit.

W4. Comparison with R1 and R2.

R1 (Deceptive Grounding) studies entity attribution in clinical retrieval-augmented generation; we study agents that commit to an entity and then describe it. Our CAP type, where a user's requirement is stated as an attribute of the anchor, does not occur in R1 and is the most common PM type on Shopping. R2 (ProvenanceGuard) checks which tool a claim comes from, but in PM the mixed-up entities usually come from the same tool call (one search result listing several products), so the claim must be checked against the anchor's own evidence. We evaluated verifiers that receive the anchor's evidence, the step both rely on (anchor-only, source-agnostic, and cite-or-drop LLMs, DeBERTa-NLI); the best reaches 0.829 and the probe 0.888.

Revision: Related Work discusses R1 and R2. The contributions list PM in agent trajectories including CAP, its frequency across 4 domains and 4 backbones, and the two methods.

W5. Annotation reliability.

Shopping labels were annotated manually by an expert, and all 274 test claims received a second manual review. For τ³, on a stratified sample of 60 claims (20 per domain), LLM and independent human labels agree on 56 of 60 before adjudication (κ = 0.867, 95% CI [0.731, 0.967]). The annotator model is GPT-6 Astra (corrected typo). We did not retain the two original LLM passes separately, so their agreement before adjudication cannot be reported.

Revision: Appendix A describes the annotation process and this agreement.

W6. Common 0.05 threshold.

All methods now choose thresholds on the development split by maximum F1, and the test set is evaluated once. The probe has precision 0.758, recall 0.942, F1 0.840, and precision 0.779 at 90% recall.

Revision: Table 2 uses these thresholds for every method and names AUROC and AP as the main comparison.

W7. Cross-backbone coverage.

We replicated on Gemma-4-31B with the same protocol. The probe reaches AUROC 0.786 [0.739, 0.834] on 474 test claims (166 PM); a format-only control reaches 0.589. Six layers lie within 0.01 on development; test AUROC spans 0.786 (selected layer 38) to 0.833 (layer 41), mean 0.832. FAC's deletion branch lowers trajectory-level PM from 41.2% to 7.2% (160 to 28 of 388 trajectories, gold labels).

Revision: §5.2 (Fig. 2) and §5.3 (Fig. 5) add Gemma-4-31B; Appendix J gives details.

W8. Cost; model internals.

On one A100 (Qwen3-32B), the probe costs about 0.3 ms per claim (under 1% of generation time, no extra forward pass), AGR-value 10.7 ms, Jacobian estimation is a one-time offline 50 MB file, and FAC's rewrite takes 2.3 s per claim, against 1.9 s for one agent forward pass. The method applies to open-weight and self-hosted agents.

Revision: New Appendix I reports costs; Limitations states the need for hidden states. The repository matches the revised paper.

Thank you for the detailed response. I have read the rebuttal and the other reviewers’ comments. The additional results clarify several points, especially the evaluation of residual PM, threshold selection, computational cost, and the Gemma experiment.

I still have some reservations. The main detector does not directly use the selected entity or its evidence, the human evaluation covers the rewritten claims rather than all post-intervention outputs, and the comparisons with the closest related work remain indirect. The annotation study and the cross-backbone evaluation of the full FAC pipeline are also limited.

Overall, the rebuttal addresses some of my concerns, but not enough to change my assessment. I will keep my current score.

Thank you for the follow-up. We answer each remaining point.

1. The main detector and the selected entity or its evidence. Adding the anchor's value explicitly matches the probe: AGR-value, which targets the anchor's observed value, reaches AUROC 0.895 against 0.888 (DeLong p = 0.325), and verifiers that take the anchor's evidence as text reach 0.829 at best. The probe reads the hidden state at the first token of the claim, computed with the committed anchor and its evidence in context. FAC then uses the anchor's evidence explicitly: for each flagged claim it reads the anchor's own value and keeps, rewrites, or deletes the claim.

2. Human evaluation of post-intervention outputs. Every claim in every post-intervention output has a human label. Unchanged claims keep their expert label (all 2,000 Shopping trajectories were annotated manually), deleted claims leave the output, and an expert labeled all 57 rewritten claims against the anchor's evidence (1 still PM, 1.8%). Outputs of the deletion branch contain only original claims, so every output claim has a gold label.

3. Direct comparison with the closest related work. PM is a relation among three things: a claim, the entity the agent committed to, and that entity's evidence. Each related work checks a different relation, and neither releases code, so we implemented the comparison.

ProvenanceGuard checks the relation between a claim and its source, a tool output, by routing each claim to a source and comparing it with the claimed one. Its repository holds the paper and assets and no code, so we implemented every specified component from the paper (centroid routing, NLI support with token alignment, protected values, a calibrated random forest, stated-versus-routed comparison), choosing each unspecified detail in its favor, and we call the result our adaptation. On the 274 test claims AUROC is 0.739 with tool returns as sources and 0.736 with product records as sources and the anchor given as the claimed source (an oracle); the probe exceeds it by 0.152 [0.089, 0.213] (paired cluster bootstrap, 2,000 resamples). Two properties of PM explain this. In CEM, donor and anchor often share a source: for 12 of 26 CEM claims (46.2%) they sit in one search return, so the source comparison passes. In CAP, the value comes from the user's request, a source that supports it, and the error is attributing it to the product. Refining sources to product records moves the problem to routing: 83.3% of CEM claims are routed away from the anchor, but only 37.3% of correctly bound claims are routed to it, since a short attribute claim resembles many records and similarity cannot recover the agent's commitment.

Deceptive Grounding studies clinical RAG, where the queried entity is given and documents are retrieved for it. In PM the agent selects the entity among several observed ones, so the reference entity is the agent's own commitment, and correction must keep that entity fixed. Its code is not public, and its verification step corresponds to the anchor-evidence verifiers (best 0.829 against 0.888).

PM is therefore a problem that claim-level support checks, sampling-based uncertainty, and source-level provenance each leave open. It occurs in up to 55.6% of task-correct trajectories, includes CAP, is detected from the hidden state at the claim, and is corrected with the anchor fixed.

4. Annotation study and cross-backbone evaluation of the full FAC pipeline. Annotation. On τ³ we extended the human study to 120 stratified claims (40 per domain): human and LLM labels agree on 114 (95.0%), κ = 0.90, 95% CI [0.815, 0.967]. On Shopping, an independent second annotator labeled 100 test claims (50 PM, 50 correctly bound, drawn at random within each class) blind to the original labels and agreed on 88: 47 of 50 PM claims (94%) and 41 of 50 correctly bound claims (82%). Of the 12 disagreements, 9 are claims the second annotator read as PM, mostly CAP cases. Cross-backbone. FAC selects claims with the backbone's probe and then deletes or rewrites them with a fixed Qwen3-8B. On Gemma-4-31B we ran selection and deletion end to end on 388 test trajectories with gold labels: the probe reaches AUROC 0.786 [0.739, 0.834] (format-only control 0.589), and trajectory-level PM falls from 41.2% to 7.2% (160 to 28). Deletion is the main action: on Shopping it handles 125 of 187 flagged claims (67%), and on Telecom it alone reaches lower residual PM than rewriting. The rewrite branch is evaluated on Qwen3-32B, with all 57 rewrites human-checked.

Revision: Appendices A, C, D, and J report the annotation agreement, the paired differences, the ProvenanceGuard adaptation with its diagnostics, and the Gemma evaluation.

These revisions cover all four reservations. As the discussion deadline is approaching, we would appreciate your consideration of raising your score if they resolve your concerns. We welcome any remaining questions.

Reviewer_wKmCOpenReview ↗
Rating
2(+1)
Confidence
3
Informativeness
2
Presentation
2(+1)
Soundness
2(+1)
Read review
Paper summary

The paper focuses on the reliability of agentic pipelines in online shopping and similar web tasks. The authors consider a problem of Phantom Merge (PM) where the LLM confidently generates text and attributes it to some evidence, but the connection between the evidence, the query, and the response is wrong. To address it, the paper proposes a series of supervised detection approaches based on the LLM’s hidden representations. Additionally, the authors propose correction methods that act on the detected PMs and either remove the flagged claims or rewrite them. The authors also provide an experimental evaluation across several benchmarks.

Critical strengths
  1. The paper introduces a new reliability problem: the agent confidently generates text and attributes it to some evidence, but this connection is wrong.
  2. The authors propose a supervised method for detecting PM cases, and a correction method that reduces the number of PMs.
Critical weaknesses
  1. The paper is extremely hard to read and understand. The text should be completely rewritten and cleaned of AI-generated phrases and terms. I don’t understand what some terms mean in the context of the paper: “linear readout”, “slots”, “frozen computation”, “functional representation”, “probability mass”, “pre-emission representations”. I read every paragraph several times to understand something, but it was quite complicated.
  2. I don’t really understand the importance of the problem. It seems to be a quite narrow, rare case of LLM agents working only on online-shopping and similar tasks, and it currently sounds like a subset of hallucinations in general.
  3. The baseline selection could be improved. Specifically, the paper uses SelfCheckGPT as a sampling-based method, but there are numerous more relevant methods that outperform it, e.g., CoCoA [1].
  4. What are “PM-enriched trajectories”? How are they defined?
  5. It is unclear how these approaches were trained and tested: which training protocols are used and how the held-out sets are defined.
  6. “Corresponding hidden state at layer l∗l^*, selected on validation data”: how was this layer selected? Is hch_c the hidden state of a single token? Could you explain how the hidden states are selected for the claims?
  7. Table 2 and its description claim that integrating the representational probe with functional information improves performance. However, Figure 3 shows that this holds only for Shopping, while elsewhere Representational readout ≥\geq AGR-{any}. So the method is not better on average than a simple logistic regression. Moreover, OOD behavior is unclear: how will the method work if it is trained on τ\tau-bench and tested on Shopping, and vice versa?
  8. Minor issues:
    1. Figure 6 looks odd: both bars are percentages but on different scales, so the bar for 91% appears lower than the bar for 73%.
    2. Typo: “GPT-5.6 Astra”. I think there is no such model.

[1] Uncertainty Quantification for LLMs through Minimum Bayes Risk: Bridging Confidence and Consistency (Vashurin et al., NeurIPS 2025)

Questions for authors

See weaknesses 4-7

4 comments

Thank you for the review and for listing the terms that were unclear. All changes in the revised PDF are marked in blue.

W1. The paper is hard to read, and several terms are unclear.

We revised §3 and define each term where it first appears:

  • Slot: the attribute a claim is about. In "Product A costs $199", the slot is price and the value is $199.
  • Hidden state at the claim: the model's internal state at the first token of the claim, before the claim is written.
  • PM probe: a logistic-regression classifier on this hidden state that predicts whether the claim is PM.
  • Frozen computation: the layers of the agent's model above the probed layer, used without training.
  • AGR-value: a linear approximation of these layers (the Jacobian lens) that maps the hidden state to scores over next tokens, used for analysis.
  • Probability mass: the total predicted probability of a set of tokens.

Revision: §3 opens with three steps (read the hidden state, score it with the PM probe, correct flagged claims with the anchor's evidence) and marks AGR-value as analysis. Slot-based variants move to Appendix B.

W2. The importance of the problem is unclear; it seems narrow and a subset of hallucination.

PM is common, occurs in customer-service tasks as well as shopping, and existing hallucination detectors miss it.

  • Not only shopping: three of our four domains are customer-service tasks from τ³-Bench, and PM occurs in all 16 backbone–domain pairs.
  • Present when the task is solved: among task-correct trajectories, up to 55.6% contain PM (Qwen, Shopping) and 32.0% for Mixtral on Airline.
  • Different from general hallucination: the wrong value usually appears in the trajectory, for another entity or as the user's request. Detectors that check whether a claim is supported anywhere, or whether the model answers consistently, cannot see this. On our test set, SelfCheck-NLI with 5 samples reaches AUROC 0.693 and CoCoA 0.496, while the probe reaches 0.888.

Revision: The Introduction states these points; §5.1 and §5.2 give the numbers.

W3. Stronger baselines such as CoCoA.

We added CoCoA with the authors' code. For each claim we sampled the agent's final response 8 times from the same trajectory (temperature 0.7), matched the sampled claims to the original, and computed the CoCoA score. It reaches AUROC 0.496, against 0.888 for the probe. CoCoA measures disagreement between samples, but a PM claim can be produced the same way in every sample.

Revision: Table 2 adds CoCoA, and Appendix D gives its setup.

W4. What are PM-enriched trajectories?

They are 2,000 Shopping trajectories built so that the agent must compare products before recommending one: 250 standard ShoppingBench tasks plus 1,750 tasks asking for specific product attributes, where the recommendation tool requires viewing at least two products. In this set 46.75% of trajectories (935 of 2,000) contain PM. The τ³-Bench trajectories are sampled from each domain's normal task distribution without this selection.

Revision: Appendix A gives the selection rule and states that Shopping rates describe this set, not natural frequencies.

W5. How were the methods trained and tested?

All splits are by trajectory, each trajectory appears in one split only, and the test set is evaluated once. D_p (393 claims) trains the probe, D_c (289) calibrates scores, D_f (288) fits fusion weights and chooses thresholds, and the test set has 274 claims from 243 trajectories. All baselines use the same splits and choose thresholds on D_f.

Revision: Appendix B adds the split table.

W6. How was the layer selected? Is it a single token?

Yes, a single token: the hidden state at the first token of each claim, from layer 49 of 64 in Qwen3-32B, chosen by AUROC on the development split. Each claim has its own hidden state. Using the token just before the claim's value gives a similar test AUROC (0.913).

Revision: Appendix B states the selection rule and the position comparison.

W7. Fusion helps only on Shopping; out-of-domain behavior.

The main detector is now the probe. With one slot rule for all claims, fusion and the probe perform about the same on Shopping (AGR-value 0.895, probe 0.888), so we use the simpler probe. It is above DeBERTa-NLI with the anchor's evidence (0.829) and CoCoA (0.496). Trained on Shopping only, it reaches AUROC 0.957 on Telecom, 0.816 on Airline, and 0.870 on Retail. We train on Shopping because it has the largest expert-labeled set; for a new domain, a small labeled set can calibrate the probe.

Revision: Table 2 and Fig. 3 are updated, and §5.2 describes the cross-domain setting.

W8. Minor issues.

Fig. 5 now uses one 0–100% axis for both bars, and the annotator model name is corrected to GPT-6 Astra.

Thank you again. We would be glad to answer any further questions.

Thank you for the rebuttal. The presentation concerns are largely addressed. However, I still have the following concerns:

  1. The updated paper lacks detailed results on τ\tau-bench.
  2. Thank you for adding the cross-domain evaluation, but it lacks unsupervised baselines. Please add other methods to the cross-domain comparison, such as those used in Table 2.
  3. The rebuttal reports SelfCheck-NLI (5 samples) at 0.693 ROC-AUC, while Table 2 gives 81.5 for K=1. Could you explain why more samples degrade performance?
  4. Could you explain why CoCoA performs at a random level in terms of ROC-AUC? Is it an implementation issue or a task-specific feature?
  5. The PM probe’s CI overlaps with DeBERTa-NLI’s, so the real improvement over the baselines is unclear.

Thank you for the follow-up. We answer the five points in order.

1. Detailed τ-bench results. Shopping-trained probe, no retraining; τ³ labels from GPT-6 Astra (human–LLM agreement on 120 sampled claims: 95%, κ = 0.90, 95% CI [0.815, 0.967]). Cells give AUROC.

DomainPM / test claimsProbeDeBERTa-NLISelfCheck-NLI (K=1)Best LLM verifier
Telecom170 / 270 (63%)0.957 [0.93, 0.98]0.710.620.70
Airline41 / 203 (20%)0.816 [0.74, 0.89]0.630.830.83
Retail42 / 1,011 (4%)0.8700.760.85–

Telecom is the well-powered domain (170 PM claims) and the probe leads every method by at least 0.24. Airline and Retail have 41 and 42 PM claims: the probe's own Airline interval spans 0.15, so differences of 0.01–0.02 cannot be resolved (Retail's prevalence is 4%). Average precision still separates the methods (Airline: probe 0.59, SelfCheck and LLM verifier 0.16 to 0.19; Retail: baselines 0.02–0.03).

2. Unsupervised baselines across domains. The table adds DeBERTa-NLI, SelfCheck-NLI, and the LLM verifiers from Table 2, with one input rule for all domains.

3. Why more samples lower SelfCheck-NLI. SelfCheckGPT assumes that a model that knows a fact reproduces it across stochastic samples, while a hallucinated fact diverges; each sentence is scored by its contradiction with the other samples. PM breaks this assumption. The value is in the agent's context and the agent has committed to its anchor, so every sample repeats the same binding. The question PM asks is whether the value belongs to the committed entity, so our adaptation scores the claim against passages from the trajectory's observations. K=1 uses the anchor's own evidence. K=5 adds records of other products from the same rollout, which turns the check into a pooled-evidence check. The data follow this difference. Pooled passages raise the contradiction score of correctly bound claims from 0.167 to 0.445, since other products' attributes differ from the anchor's. They raise that of PM claims much less, from 0.545 to 0.642, since a CEM claim carries another product's value, which that product's record supports. The class gap falls from 0.378 to 0.197 and AUROC from 0.815 to 0.693. This is the defining property of PM, a value that is correct for some entity in the trajectory and wrong for the committed one: a check on the anchor's evidence detects it, and a check on pooled evidence accepts it. K=1 is the anchor-specific check used in Table 2.

4. Why CoCoA is at chance. CoCoA estimates the uncertainty of a generation as the product of a confidence term and the dissimilarity among sampled outputs (minimum Bayes risk), and is validated on QA, translation, and summarization against output correctness. A high score marks an answer the model is unsure of. A PM claim is a value copied from context by an agent committed to its anchor, so the model is confident and the samples agree: in a claim-level resampling diagnostic, 98.1% of PM claims were reproduced identically in at least 7 of 8 samples (83.9% of correctly bound claims), and 241 of 274 claims were identical in all 8, which gives them one score. Of the 33 claims that varied, 27 were correctly bound, paraphrases of long titles. A PM claim such as "Color: Blue" is stated identically every time, like a correct one. CoCoA measures epistemic uncertainty, and PM is a binding error made with confidence, so chance-level AUROC (0.496 [0.428, 0.561], authors' code, K = 8, T = 0.7) follows from the problem definition.

5. Overlapping intervals. Marginal intervals overlap even when one method is better on most resamples, so we pair them: a cluster bootstrap by trajectory (2,000 resamples). The probe exceeds SelfCheck-NLI (K=1) by 0.073 [0.006, 0.141] and the ProvenanceGuard adaptation by 0.152 [0.089, 0.213]; the marginal intervals of the three LLM verifiers (best 0.788) do not overlap the probe's [0.847, 0.924]. DeBERTa-NLI, the closest baseline, takes the anchor's evidence as its premise, while the probe reads only the hidden state at the claim; the probe is 0.059 higher on Shopping, and the gap is 0.25 on Telecom. The gain has the same direction in every comparison, and its size follows what each method sees: methods that miss the relation between claim and committed entity (sampling consistency, pooled-evidence support) fall furthest behind.

Revision: Appendix C adds the τ³ table and the paired differences; Appendix D states the SelfCheck K=1/K=5 construction, the CoCoA setup, and the diagnostic; Appendix B states that D_f fits the fusion weights and that every method chooses its threshold on the development split, correcting the threshold sentence in our first reply to W5.

These revisions cover all five points. As the discussion deadline is approaching, we would appreciate your consideration of raising your score if they resolve your concerns. We welcome any remaining questions.

Thank you for the detailed responses and additional experiments. The rebuttal addresses most of my concerns, and I have raised my overall rating as well as my presentation and soundness scores.

Reviewer_Ye1SOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2(+1)
Read review
Paper summary

The paper defines Phantom Merge: a claim in an agent’s final response is attributed to the entity it committed to but is not supported by that entity’s own observed evidence. For detection it proposes AGR, which fuses a linear probe on the pre-emission hidden state with a Jacobian-lens functional readout; for correction it proposes FAC, which gates, audits and rewrites or deletes flagged claims while keeping the anchor fixed. On a held-out Shopping cohort AGR-slot reaches 96.2% AUROC, and FAC reduces trajectory-level PM.

Critical strengths
  1. The failure mode is well motivated and cleanly separated from generic hallucination. Showing that it co-occurs with task success at high rates is a useful finding.
  2. The detection test on Shopping is well designed and shows care for statistical evidence.
Critical weaknesses
  1. The method does not transfer. Figure 3 shows AGR-slot falling (0.78/0.69/0.60) below the plain representational probe (0.94/0.83/0.87). On Telecom, FAC leaves 47.6% residual PM against 1.6% for simple deletion (tables F.1/F.2). The abstract and conclusion claim a general framework whilst the main text concedes AGR-slot only hold up on Shopping and FAC does not transfer on one of the three other domains.

  2. The mitigation evaluation is circular and the comparison hard to interpret. Post-intervention outputs are "re-evaluated using the same AGR procedure, with manual spot checks" (line 254), so the detector is evaluating itself. Anything it misses will be missed again. Under complete intervention, RARR deletes almost everything (tables G.1 and F.1), and CoVe does the same on the \tau^3 domains (table F.1), suggesting broken adaptations. Under the shared gate, FAC, FAD and RARR all reach the same 5.3% residual PM (table G.1), so FAC's advantage is only claim volume. Table G.2 shows 125 of 187 flagged claims deleted and only 57 rewritten, and the AGR heads retain just 62% of correctly bound claims (figure 6). FAC is mostly deletion with a modest rewrite branch, measured by the method under test.

Questions for authors
  1. Why does AGR-slot degrade so sharply out of domain?
  2. How is residual PM measured independently of AGR? Please report a confusion matrix with human annotators.
  3. What was inter-annotator agreement between the two LLM passes (and versus humans) before adjudication?
  4. How are claims segmented and slot-typed at test time, and is this independent of the annotation process?
3 comments

Thank you for the review. All changes in the revised PDF are marked in blue.

W1 / Q1. The method does not transfer. Why does AGR-slot degrade out of domain?

The drop comes from one Shopping-specific component, the slot lexicon of AGR-slot. The PM signal and the correction method transfer.

Each domain has its own attributes. Shopping claims are about price, rating, color, and size ("Product A has a rating of 4.8", slot: rating). Telecom claims are about plans, data allowances, and billing ("Your plan includes 10 GB of data", slot: data allowance). Airline claims are about fares, baggage, and departure times ("Your flight departs at 9:40", slot: departure time). AGR-slot measures probability on a lexicon of slot names built from Shopping attributes, so many Telecom, Airline, and Retail slots fall outside it and the lexicon term adds noise. Task-oriented dialogue systems likewise need a separate attribute ontology per domain.

The PM probe uses no lexicon. Trained on Shopping only, with the score direction fixed on Shopping training data, it reaches AUROC 0.957 on Telecom, 0.816 on Airline, and 0.870 on Retail.

Correction follows the same pattern. Without retuning, FAC lowers residual PM from 20.1% to 1.1% on Airline and from 3.1% to 0.4% on Retail. On Telecom the evidence is account records, and FAC's audit step was fit on product listings. FAC's deletion branch does not depend on this format and lowers residual PM on Telecom to 1.6% with the same gate.

Revision: The probe is the main detector in all domains. The Abstract and Conclusion state that the slot lexicon and audit step are set up per domain. §5.2 adds the examples above, Fig. 2 is updated, and Limitations states that the audit step should be fit per domain.

W2 / Q2. Is the correction evaluation circular, and is FAC mostly deletion? How is residual PM measured independently of AGR?

Residual PM comes from the original gold labels and a human review of every rewritten claim. No detector rescores the edited output. For each output, a deleted claim is absent and contributes no residual PM. An unchanged claim keeps its original gold label, including any PM claim the gate missed. A rewritten claim has new text, so an expert annotator checks it against the selected entity's evidence. A trajectory has residual PM if any remaining claim is labeled PM. Of the 57 rewritten claims, 1 still contained PM (1.8%).

Yes, most flagged Shopping claims are deleted, because the anchor's evidence lists no value to rewrite with. The 57 rewrites are the claims where the anchor's evidence settles the slot, and they separate FAC from deletion: at the same 5.3% residual PM, FAC keeps 59.3% of the original claims, against 35.8% for FAD.

RARR and CoVe run from their original code; we only adapted the input to supply the trajectory evidence. On Shopping they work (RARR keeps 37.0% of claims at 5.3% residual PM). On τ³, where the evidence is structured account and booking records, their verification step rejects almost every claim, so they reach low residual PM only by deleting almost all output. Generic verify-and-revise methods cannot tell PM claims from correctly bound claims in this setting.

Revision: §4 describes how each edit action enters residual PM. Appendix G adds the human check. Appendix E adds the gate's precision, recall, and F1 as the threshold varies on the development split, with 0.05 marked.

Q3. Inter-annotator agreement before adjudication.

On a stratified sample of 60 τ³ claims (20 per domain, half PM and half correctly bound), the LLM labels and independent human labels, collected before adjudication, agree on 56 of 60 claims (κ = 0.867, 95% CI [0.731, 0.967]). The human annotator did not see the LLM labels. We did not retain the two original LLM passes separately, so their mutual agreement before adjudication cannot be reported.

Revision: Appendix A describes the annotation process and reports this agreement in place of "no disagreements remained after reconciliation."

Q4. How are claims segmented and slot-typed at test time, independently of annotation?

Shopping uses the annotated claims, so no parser is used there. For τ³, a rule-based parser splits the final response into bullet points and reads each claim's slot from its attribute header, using no PM label. It finds 82.6% of the reference claims; for these, the slot matches in 93.3% of cases and the value in 86.7%.

Revision: New Appendix H describes the parser and these results.

Thank you again. We would be glad to answer any further questions.

Dear Reviewer Ye1S,

Thank you again for your detailed review. We addressed both weaknesses and all four questions through the following revisions and additions:

  • Cross-domain transfer (W1/Q1): The probe is now the main detector in every domain, and the slot lexicon and audit step are described as set up per domain. Trained on Shopping only, the probe reaches AUROC 0.957 on Telecom, 0.816 on Airline, and 0.870 on Retail, and 0.856 / 0.668 / 0.686 on Gemma-4-31B. §5.2 explains the AGR-slot drop with examples, and Fig. 2 and Appendix C (new τ³ table with DeBERTa-NLI, SelfCheck-NLI, and LLM-verifier baselines) report the results.
  • Correction without circularity (W2/Q2): Residual PM comes from gold labels, with deleted claims absent, unchanged claims keeping their labels, and all 57 rewritten claims checked by an expert (1 still PM). §4 states the calculation, Appendix G the human check, Appendix E the gate's precision, recall, and F1 across thresholds, and Appendix K the branch accounting (125 of 187 flagged Shopping claims are deleted, 57 are rewritten, and FAC keeps 59.3% of claims against 35.8% for FAD at the same 5.3% residual PM).
  • Inter-annotator agreement (Q3): On τ³, we extended the stratified human sample to 120 claims, where human and LLM labels agree on 114 (95.0%, κ = 0.90, 95% CI [0.815, 0.967]) before adjudication. On Shopping, a second annotator agrees with the original labels on 88 of 100 test claims. Appendix A reports the process and agreement.
  • Claim segmentation and slot typing (Q4): Shopping uses the annotated claims. For τ³, a rule-based parser reads each claim's slot from its attribute header without any PM label, finds 82.6% of the reference claims, and matches the slot in 93.3% and the value in 86.7% of them (Appendix H).

These revisions cover both weaknesses and all four questions in your review. As the discussion deadline is approaching, if you find that the revised manuscript resolves your concerns, we would appreciate your consideration of raising your score. We welcome any remaining questions.

Thank you for your detailed answer and the updated manuscript. Apologies for the late response.

The following concerns/questions are resolved/answered: Q3, Q4 and the circular evaluation.

Remaining concerns:

  • W1: The changes in the paper are quite substantial and some of the transfer claims (still present in the abstract) are not supported. I appreciate that you mention that the audit step should be fitted per domain but this is not done so we still do not know if it is a solution.
  • W1: The lead is also clear only on Telecom (table C.3): SC-NLI and the LLM verifier are level on Airline and on Retail the gap with SC-NLI is probably within noise. By the way, on this table, using bold for your value instead of the best value is misleading.

I have raised my soundness score. However, I have not changed my overall score because I still think the paper is slightly below the bar.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.