acceptodds
Under review as a conference paper at ICLR 2027

ROBUST-VERIFY: Adversarial Self-Play with Structured Reasoning for Robust Claim Verification under Corrupted Chain-of-Thought

23 views

Abstract

Structured chain-of-thought (CoT) reasoning has become standard in automated claim verification, improving interpretability by requiring a model to articulate intermediate steps before issuing a verdict. However, this approach exposes a previously unaddressed vulnerability that intermediate reasoning steps can be corrupted independently of the underlying claim or evidence, causing verdict errors even when the model possesses sufficient knowledge for a correct judgment. No existing framework provides a mechanism for detecting such corruptions. Zero-shot prompting across four model families (LLaMA-3-8B, Mistral-7B, Qwen2.5-7B, GPT-OSS-20B) achieves Corruption Detection Rate (CDR) near 0.000, confirming that detection requires explicit supervision regardless of model size. We introduce ROBUST-VERIFY, a three-phase training framework augmenting structured verification with an explicit Corruption Check element and training the model to detect and recover from corrupted reasoning through adversarial self-play. Phase 1 performs warm-up training on human-annotated chains with direct inconsistency-detection supervision. Phase 2 trains a Claim Verification (CV)-Polluter and CV-Agent through a policy-gradient method (GRPO) across a six-type corruption taxonomy at three severity levels. Phase 3 introduces Partial Answer Masking (PAM) to concentrate repair supervision on the post-corruption continuation. We present the first explicit evaluation of corruption detection in structured claim verification. Our model achieves consistently strong CDR across HoVer (2/3/4-hop) and FEVEROUS-S, with CDR-L1 ranging from 0.587 to 0.765, a 10x improvement over the best zero-shot baseline while also improving verdict recoverability under several corruption settings. Ablation confirms that PAM is necessary as its removal causes structural collapse (SP = 0.000). Our results show a detection-recovery trade-off between corruption severity levels, motivating CDR-aware reward design as future work.

Primary area:
foundation or frontier models, including LLMs
Keywords:
claim verification, chain-of-thought reasoning, adversarial self-play, corruption detection, large language models, fact checking, robustness, structured reasoning, reinforcement learning, GRPO

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_YhogOpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
1
Soundness
1
Read review
Paper summary

This paper introduces ROBUST-VERIFY, a three-phase training framework augmenting structured verification with an explicit Corruption Check element and trains the model to detect and recover from corrupted reasoning through adversarial self-play. Results show a detection-recovery trade-off between corruption severity levels, motivating CDR-aware reward design as future work.

In my view, this manuscript violates the journal’s policy on the use of AI tools. Although the authors declare that they employed AI tools for writing and editing, the content—from the methodology to the experiments—lacks human readability and closely resembles an AI-generated technical report. I therefore recommend rejecting this manuscript.

Critical strengths
  1. I think employing adversarial training in claim verification is an excellent idea.

  2. The way it is formulated in this paper is a strong point.

Critical weaknesses
  1. The details in the paper lack readability; the use of CA (Classification Accuracy) in place of macro-F1 is highly unusual. Furthermore, the formulas and content in Figure 1 are extremely difficult to decipher. In addition, the paper features a significant amount of writing characteristic of technical reports—content that I strongly suspect was generated by AI (SP = 0.000).

  2. Setting aside the concerns regarding AI-generated content, the experimental setup is poorly organized. First, the paper fails to include any state-of-the-art closed-source models as baselines. Second, it is not specified whether the datasets used for claim verification operate under an "open-book" or "closed-book" setting.

Questions for authors
  1. What is the performance of your method compared to RL from scratch

  2. The Open-book setting is also essential for claim verification. Can you cover that one?

  3. Please provide a clearer explanation of what these metrics specifically mean, as well as how they are calculated.

2 comments

Q1. "The paper features a significant amount of writing characteristic of technical reports --- content that I strongly suspect was generated by AI (SP = 0.000)."

Disagree; this is a metric misreading. SP (Structure Preservation) is a reported experimental result, not a writing quality score. It measures the fraction of our seven verification-format elements that are well-formed in a model's output chain. The reviewer appears to have read Table 4 rather than Table 1. The two relevant rows are:

ModelTableSettingSP
Zero-shot LLaMA-3.1-8BTable 1, line 396–458oracle open-book0.484-0.588
GPT-OSS-20BTable 4, line 654scale comparison0.000

GPT-OSS-20B outputs in a native format incompatible with our seven-element structure (stated at line 669), which is why its SP is 0.000. The Zero-shot LLaMA SP values are not 0.000.

Regarding AI-assisted writing: we declared the use of AI tools for writing and editing assistance in the submission, consistent with ICLR 2027 policy. The methodology, experimental design, all code, and all numerical results are entirely the authors' work. We welcome any specific technical question about the content.


Q2. "The paper fails to include any state-of-the-art closed-source models as baselines. It is not specified whether the datasets used for claim verification operate under an open-book or closed-book setting."

Clarify - the setting is stated; closed-source baseline already included. The setting is oracle-evidence open-book: evidence is drawn from dataset annotations, identical to STRIVE (Gong et al., 2025). We will add this as the first sentence of Section 3.1.

Regarding closed-source baselines: Table 4 already includes a GPT-class model (GPT-OSS-20B, zero-shot) as a scale comparison. The paper's contribution is a training framework evaluated on open-weight models where fine-tuning is possible. Closed-source API models cannot be trained with GRPO or PAM, so they evaluate a different question (zero-shot prompting scale) rather than the framework itself. That said, adding a GPT-4o zero-shot oracle-evidence row as an additional reference point is feasible through the API and we will include it, while making clear that it is a scale reference, not a comparable training baseline.


Q3. "What is the performance of your method compared to RL from scratch?"

Clarify; RL from scratch is not viable here. Applying GRPO directly to the base model without Phase 1 warmup produced CDR = 0.000 throughout the entire training run in our preliminary experiments. The base model never generates the INCONSISTENCY DETECTED sequence, so the detection reward is permanently zero and the agent has no gradient signal (sparse-reward collapse). This is consistent with GASP (Li et al., 2026) and ISC (Han et al., 2024), both of which demonstrate that supervised initialization is a prerequisite for RL fine-tuning on structured generation tasks with sparse rewards. Phase 1 is not optional scaffolding; it bootstraps the reward signal into a non-degenerate regime. We will add a sentence reporting this result explicitly in Section 3.2.


Q4. "The use of CA (Classification Accuracy) in place of macro-F1 is highly unusual. Please provide a clearer explanation of what these metrics specifically mean, as well as how they are calculated."

Already stated in the paper- CA is macro-F1. CA = macro-F1 computed on uncorrupted input chains. This is stated in the Metrics paragraph of Section 4 and in the Table 1 column header ("CA (macro-F1)"). The reviewer's question is answered by reading the paper. We use macro-F1 rather than raw accuracy because HoVer and FEVEROUS-S evaluation splits have label imbalance, making raw accuracy misleading, a standard choice in claim verification. We will add a one-sentence clarification to the metrics paragraph and provide explicit per-class computation formulas for all five metrics (CA, RC, CDR, RQ, FC) in the revised appendix to prevent any further confusion.


Thanks for your response.

However, I am sorry that you do not understand my comments and most of my concerns are still there. Table 1 reports CA, RC-L1/L2/L3, CDR-L1/L2, RQ-L1/L2, and SP for all four datasets across the three model configurations (Zero-shot, SFT-only, and ROBUST-VERIFY full). This kind of presentation is not readable for human readers.

I will keep my rating.

Reviewer_w2m8OpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This work tackles an important problem of detecting corruption in intermediate reasoning steps of chain-of-thought in LLMs with respect to the task of claim verification. The authors propose a three-phase framework named ROBUST-VERIFY to detect and recover from corrupted reasoning chains. In the first stage, the authors warm-start the model to produce a structured reasoning chain (with 7 steps/elements, as the authors state in Section 3.1) to verify a claim conditioned on evidence. Then, in the second phase, they induce an adversarial self-play where a polluter corrupts the reasoning chain based on three severity levels and six corruption types, and an agent has to recover the continuation given the corrupted prefix. Both the polluter and agent are trained using binary correctness rewards and another reward component that checks if the chain conforms to the expected structure with GRPO. The authors evaluate on HoVer and FEVEROUS and Robust-verify appears to have a superior Corruption detection rate, with some trade-off in verification performance.

Critical strengths
  1. The paper tackles an important problem in detecting corruption in reasoning chains for claim verification and also proposes a recovery mechanism.

  2. I think the use of self-play here is very interesting, but it is not completely self-play (detailed below in weaknesses), as the setup assumes access to ground-truth labels and source claim data from FEVEROUS and HoVEr.

  3. I appreciate the ablations and additional qualitative examples in Appendix.

Critical weaknesses
  1. Some experimental details are missing in the paper. Since HoVER and FEVEROUS have a corpus of evidence, what retrieval and re-ranking approach was employed to fetch relevant evidence for verifying the test claims? I could not find it in Section 4 or in the Appendix either. Or instead, do you assume access to golden / oracle evidence? If so, I do not think that is a realistic setup. This is a major concern, as many of the corruption strategies in CV-Polluter rely on evidence.

  2. Writing could be a bit more structured. For instance, Section 3.1 should start with the problem formulation more clearly and also describe the general fact-checking pipeline adopted for this work. As mentioned in the previous comment, it is not clear at all how evidence is retrieved for each test claim. Do you use claim decomposition as it is complex verification and it is also a form of structured reasoning? This is not clear, and Section 3.1 is very dense and short, probably due to AI-assisted polishing. i would recommend the authors expand on it a bit and make the setup clearer.

  3. While HoVer was one of the first multi-hop FC (fact-checking) benchmarks, it is quite saturated and is also created synthetically from Wikipedia snippets. It also does not deal with structural challenges (where evidence is split across hybrid sources: structured and unstructured data) associated with real-world complex claims. The claims fact-checked in HoVer are not realistic in the sense that these are usually not the claims fact-checkers or journalists encounter on a daily basis. More real-world claim-based complex benchmarks like Averitec [3], ClaimDecomp [4], I feel current evaluation is limited and heavily relies on HoVer. I would like to see if the approach generalizes well to real-world complex claims such as the benchmarks listed in the earlier sentence. This would also demonstrate immense practical utility of the proposed ROBUST-vERIFY framework for real-world fact-verification pipelines.

  4. I think some additional works need to be discussed in the related work section. Currently, there are only 2 subsections that focus on structured CoT-based reasoning for LLMs and self-play RL for LLMs. I recommend. third subsection that discusses common attacks/corruption studies of COT in LLMs and related mitigation/detection strategies like [1,2]. This list is not exhaustive and current related work discussion is very thin.

  5. Self-play would assume continuous interaction for generating data from the environment and learning a joint verifier like works absolute zero, R1, or relying on environment signals for rewards. The current reward design still requires manual human-annotated verdicts. I would request the authors to perhaps rename it to an adversarial training setup or draw an analogy to an adversarial game rather than pure self-play or at least not overstate the self-play part.

[1] Chain-of-Defensive-Thought: Structured Reasoning Elicits Robustness in Large Language Models against Reference Corruption Wenxiao Wang, Parsa Hosseini, Soheil Feizi

[2] Thought Purity: A Defense Framework For Chain-of-Thought Attack Zihao Xue, Zhen Bi, Long Ma, Zhenlin Hu, Yan Wang, Xueshu Chen, Zhenfang Liu, Kang Zhao, Jie Xiao, Jungang Lou

[3] AVeriTeC: A Dataset for Real-world Claim Verification with Evidence from the Web Michael Schlichtkrull, Zhijiang Guo, Andreas Vlachos

[4] Generating Literal and Implied Subquestions to Fact-check Complex Claims Jifan Chen, Aniruddh Sriram, Eunsol Choi, Greg Durrett

Questions for authors
  1. Line 158 specifies that each sample is tagged with the name of the dataset. If a dataset has a large number of inconsistencies, wouldn't the model overfit to this superficial pattern from metadata rather than actually learning the pattern causing the inconsistency?

  2. Could the tradeoff in performance be due to the polluter dominating the CV-Agent. Since you induce an adversarial play, couldn't na equilibrium objective be introduced between CV-Polluter and CV-Agent and also as proxy for covergence. Perhaps this will ensure both these policies are optimal

  3. Does the PAM phase only apply to training instances where the model was unable to recover corrupted reasoning? MINOR COMMENTS

  4. What is the rationale behind using 10,000 resamplings for the bootstrap?

  5. The expansion for PAM is not provided in the abstract and is quite confusing on what it refers to until its introduced later in introduction. Please revise this.

  6. Same for CDR, though it is easy to map to. Please ensure the abstract is self-contained. Introduce the short forms before their usage.

  7. The text in Figure 1 is quite hard to read. Please increase the font size within the blocks.

  8. it would be nice to have a motivating example in the introduction showing an instance of corruption and how the proposed method mitigates it.

3 comments

Q1. "What retrieval and re-ranking approach was employed to fetch relevant evidence for verifying the test claims? Or do you assume access to golden / oracle evidence? If so, that is not a realistic setup."

Clarify — this is the standard controlled setup in the field. ROBUST-VERIFY operates in the oracle-evidence open-book setting: evidence sentences e\mathbf{e} are drawn directly from dataset annotations. This is not a limitation unique to our work — it is the established controlled protocol used by every structured claim verification system we cite, including STRIVE (Gong et al., 2025), FEVEROUS-S (Aly et al., 2021), and HoVer (Jiang et al., 2020). Isolating reasoning quality from retrieval noise is the deliberate scope of this paper. Retrieved-evidence evaluation is an important extension and we will include it in future work, but the oracle-evidence setting is not an unrealistic choice — it is the field standard for the reasoning component we study. We will state this explicitly as the first sentence of Section 3.1 to avoid any ambiguity.


Q2. "Line 158 specifies that each sample is tagged with the name of the dataset. If a dataset has a large number of inconsistencies, wouldn't the model overfit to this superficial pattern from metadata rather than actually learning the pattern causing the inconsistency?"

Disagree, with two mitigations. First, the dataset tag appears on both clean and corrupted chains during training (Section 3.1), so it cannot reliably predict the Corruption Check step. Second, FPR = 0.000 (see JotP Q1) provides direct evidence against this hypothesis: if the model were using the dataset tag as a shortcut, it would generate false positives on clean chains carrying the same tags. It does not.


Q3. "Self-play would assume continuous interaction for generating data from the environment and learning a joint verifier like Absolute Zero, R1. The current reward design still requires manual human-annotated verdicts. I would request the authors to rename it to an adversarial training setup."

Agree; we accept this correction. The CV-Agent reward uses ground-truth verdict labels yiy_i (Eq. 3), so the setup is not fully self-contained self-play. We will revise the framing to "adversarial training with self-generated corruptions" throughout (title, abstract, section headings, body). The core mechanism is unchanged; only the framing is corrected.


Q4. "I think some additional works need to be discussed in the related work section. I recommend a third subsection that discusses common attacks/corruption studies of CoT in LLMs and related mitigation/detection strategies like Chain-of-Defensive-Thought [1] and Thought Purity [2]."

Agree; we will add it. We will add Related Work subsection 2.3 covering Chain-of-Defensive-Thought (Wang et al., 2025) and Thought Purity (Xue et al., 2025). These are the closest defense-side works to ours and their absence is a gap we acknowledge.


Q5. "HoVer is quite saturated. I would like to see if the approach generalizes well to real-world complex claims such as AVeriTeC [3] and ClaimDecomp [4]."

Disagree that HoVer is saturated for this task; agree on future benchmarks. HoVer is saturated for claim verification accuracy (CA). It is not saturated for corruption detection. Every baseline in Table 1 — including 7B+ models — achieves CDR = 0.000 on HoVer. A benchmark where state-of-the-art models score zero on the primary metric of a paper is not saturated; it is unsolved for that task. The reviewer conflates CA saturation with CDR saturation, which are orthogonal. Within the oracle-evidence scope, HoVer and FEVEROUS-S represent different evidence formats (multi-hop Wikipedia snippets vs. hybrid structured/unstructured), providing coverage across the two main evidence-format families. AVeriTeC (Schlichtkrull et al., 2023) and ClaimDecomp (Chen et al., 2022) both require retrieved evidence; we will name them as next-step benchmarks in the conclusion.


Q6 (Minor). "PAM and CDR are not expanded in the abstract. Figure 1 text is hard to read. A motivating example in the introduction would be helpful. Does the PAM phase only apply to training instances where the model was unable to recover?"

Presentation changes: agree. PAM scope: already stated in the paper.

We will (1) expand PAM and CDR on first use in the abstract, (2) increase Figure 1 font size, and (3) add a motivating example showing a corrupted entity resolution step and its Corruption Check flag.

On PAM scope: this is already stated in Section 3.3. LPAML_{\mathrm{PAM}} (Eq. 5) applies exclusively to failed recovery episodes, i.e., those where v^(r^i)≠yi\hat{v}(\hat{r}_i) \neq y_i. On successful recovery, no repair snippet is generated and LPAML_{\mathrm{PAM}} contributes zero to the agent loss. We will add a forward pointer so this cannot be missed.

Q7. "Could the tradeoff in performance be due to the polluter dominating the CV-Agent? Couldn't an equilibrium objective be introduced as a proxy for convergence?"

Agree this is a real risk; we have two mitigations. The zero-sum reward structure (RP=1−RAR_P = 1 - R_A, Eq. 4) means the game has no equilibrium incentive for the polluter to ease off once the agent collapses. We address this with: (1) checkpoint selection on best validation RC rather than final-step reward, selecting the iterate before collapse; (2) PAM providing an imitation-learning anchor that prevents the agent's output distribution from drifting under aggressive polluter updates. A formal equilibrium regularizer (e.g., a Nash-inspired penalty on reward variance within a batch) is an interesting extension we will note as future work.


Reviewer_w2m8OpenReview ↗

Response to authors

Thank you for your response. However, I emphasise that retrieval is one of the hardest tasks in fact-checking, and noisy evidence or hard distractors can lead to their own flavor of corruption in reasoning chains, which would not be covered in your current work. Also, in response to Q5 regarding HoVer being saturated for claim verification and not CDR, it still doesn't explain how CDR could be high if claim verification is saturated. Because there might be fewer naturally occurring corruptions if the models are already achieving the highest accuracy. Granted, the baselines might perform poorly on your artificially generated corruptions. Also I would request you to use less generative AI when crafting responses. For instance, "The reviewer conflates CA saturation with CDR saturation, which are orthogonal." is not entirely correct. Sure, a correct verdict does not always mean an uncorrupted chain, but a heavily corrupted chain has very less chance of leading to right verdict. So they are not really othogonal. Since ICLR allows an upload of revised pdf your response would be more strong if you had made some of the revisions mentioned in rebuttal and uplaoded a revised pdf.

Reviewer_JotPOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper studies corruption of intermediate reasoning chains in structured claim verification while keeping the underlying claim and evidence unchanged. ROBUST-VERIFY adds an explicit Corruption Check element and combines supervised warm-up with GRPO-based adversarial self-play and partial answer masking (PAM) repair guidance. The reported CDR-L1 increases from 0 for the evaluated LLaMA-3-8B zero-shot baseline to 0.587–0.765 across four datasets. However, evaluation confounds and inconsistencies limit the interpretation of these gains.

Critical strengths
  1. Targeting corruption of intermediate reasoning, rather than changes to the underlying evidence, is a relevant robustness setting. The explicit corruption taxonomy and structured verification protocol provide a concrete framework for studying this problem.
  2. Separately assessing detection, verdict recovery, joint performance, and structural compliance is conceptually useful. The qualitative examples also illustrate why a correct verdict need not imply successful detection or repair.
Critical weaknesses
  1. The warmup set is 65% Refuted and all Refuted examples contain the detection token; without false-positive rates on clean inputs, high CDR may reflect Refuted bias plus surface string matching rather than genuine detection.
  2. Training and evaluation share the same corruption taxonomy, with no held-out-type or human-written corruption evaluation. The demonstrated robustness is consequently limited to the tested corruption distribution.
  3. Recovery and clean accuracy drop below zero-shot baselines (e.g., RC-L1 0.292 vs 0.393 on FEVEROUS-S); gains concentrate on a metric the paper itself introduces.
  4. The Corruption Check ablation is missing, there is no cross-seed variance, and self-consistency or self-correction baselines are absent.
Questions for authors

See the weaknesses.

1 comment

Q1. "The warmup set is 65% Refuted and all Refuted examples contain the detection token; without false-positive rates on clean inputs, high CDR may reflect Refuted bias plus surface string matching rather than genuine detection."

Clarify + new result. We ran a full FPR evaluation after submission. Starting from 982 Supported-labeled chains, we applied three filters: (1) removed 643 chains that already contained INCONSISTENCY DETECTED or Answer: Refuted (mislabeled, not flagged by our model), (2) removed 92 blank chains, (3) removed 44 truncated chains. This left 203 verified clean chains. ROBUST-VERIFY flags zero:

Clean chainsFPR95% bootstrap CI (N=10,000N = 10{,}000)
2030.000[0.000,,0.018][0.000,\\,0.018]

CDR Precision =1.000= 1.000 (bootstrap CI [0.982,,1.000][0.982,\\,1.000]). A string-matching model would flag a substantial fraction of clean chains; FPR = 0.000 with an upper bound of 0.018 rules out that explanation. We will add this as a dedicated paragraph in Section 4 and include the full bootstrap table in the appendix.


Q2. "Recovery and clean accuracy drop below zero-shot baselines (e.g., RC-L1 0.292 vs. 0.393 on FEVEROUS-S); gains concentrate on a metric the paper itself introduces."

Agree in part; disagree on interpretation. The RC-L1 drop on FEVEROUS-S is real and we do not dismiss it. It is caused by the detection-recovery trade-off documented in Finding 2 (Section 5.2): self-play increases CDR at the cost of recovery precision at L1, the mildest severity where the zero-shot model's higher RC-L1 carries CDR = 0.000. A model that recovers without detecting is not useful in any pipeline that must know when to trust a chain. The joint metric RQ-L1 makes this precise: ROBUST-VERIFY scores 0.309 vs. 0.000 for zero-shot on FEVEROUS-S, a result that cannot come from shortcut recovery. RC at L2 and L3 on FEVEROUS-S both improve over zero-shot (−0.035-0.035 and +0.039+0.039), confirming the effect is severity-specific, not a systematic regression. We will make the RQ comparison more prominent in the discussion.


Q3. "The Corruption Check ablation is missing."

Agree; partial ablation available. Table 3 (ROBUST-VERIFY w/o PAM) already shows what removing the repair anchor does: CDR collapses to 0.000 and SP collapses to 0.000. A direct ablation removing the Corruption Check element entirely was not completed: approximately 57% of warm-up chains reverted to degenerate format without it (Section 6), and subsequent self-play produced repeated GRPO backward-pass failures. The w/o PAM run on HoVer-2 alone consumed >300 GPU-hours near the PBS walltime limit, making a full repeat on a modified format infeasible within the submission window. We will add this explicitly to the limitations paragraph.


Q4. "There is no cross-seed variance and no self-consistency or self-correction baselines."

Clarify + new results.

Cross-seed: A second seed run (seed = 123) is in progress. We will include the CDR-L1 results in Appendix Table 6 of the camera-ready revision. Preliminary output from the first 500 self-play steps shows no qualitative deviation from the main-table behavior.

Self-consistency: Majority voting over N=7N = 7 completions at T=0.7T = 0.7 (Wang et al., 2023) recovers +0.10+0.10 CA (macro-F1) on HoVer-3/4, the CA drop on those datasets is a sampling-bias artifact, not a structural failure. However, CDR = 0.000 for self-consistency: majority voting over verdicts cannot substitute for explicit detection supervision because it generates no Corruption Check flags. We will add this paragraph to Section 5 with the citation.


Q5. "Training and evaluation share the same corruption taxonomy, with no held-out-type or human-written corruption evaluation."

Agree; this is stated in the paper. Section 6 ("Evaluation alignment") acknowledges that CDR reflects in-distribution detection. Evaluating on held-out generator types and human-written adversarial chains is the most important open problem. We will strengthen this statement and frame it as the primary future direction.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.