- Rating
- 1
- Confidence
- 3
- Informativeness
- 1
- Presentation
- 1
- Soundness
- 1
Read review
Paper summary
This paper introduces ROBUST-VERIFY, a three-phase training framework augmenting structured verification with an explicit Corruption Check element and trains the model to detect and recover from corrupted reasoning through adversarial self-play. Results show a detection-recovery trade-off between corruption severity levels, motivating CDR-aware reward design as future work.
In my view, this manuscript violates the journal’s policy on the use of AI tools. Although the authors declare that they employed AI tools for writing and editing, the content—from the methodology to the experiments—lacks human readability and closely resembles an AI-generated technical report. I therefore recommend rejecting this manuscript.
Critical strengths
-
I think employing adversarial training in claim verification is an excellent idea.
-
The way it is formulated in this paper is a strong point.
Critical weaknesses
-
The details in the paper lack readability; the use of CA (Classification Accuracy) in place of macro-F1 is highly unusual. Furthermore, the formulas and content in Figure 1 are extremely difficult to decipher. In addition, the paper features a significant amount of writing characteristic of technical reports—content that I strongly suspect was generated by AI (SP = 0.000).
-
Setting aside the concerns regarding AI-generated content, the experimental setup is poorly organized. First, the paper fails to include any state-of-the-art closed-source models as baselines. Second, it is not specified whether the datasets used for claim verification operate under an "open-book" or "closed-book" setting.
Questions for authors
-
What is the performance of your method compared to RL from scratch
-
The Open-book setting is also essential for claim verification. Can you cover that one?
-
Please provide a clearer explanation of what these metrics specifically mean, as well as how they are calculated.
2 comments
Thanks for your response.
However, I am sorry that you do not understand my comments and most of my concerns are still there. Table 1 reports CA, RC-L1/L2/L3, CDR-L1/L2, RQ-L1/L2, and SP for all four datasets across the three model configurations (Zero-shot, SFT-only, and ROBUST-VERIFY full). This kind of presentation is not readable for human readers.
I will keep my rating.
Q1. "The paper features a significant amount of writing characteristic of technical reports --- content that I strongly suspect was generated by AI (SP = 0.000)."
Disagree; this is a metric misreading. SP (Structure Preservation) is a reported experimental result, not a writing quality score. It measures the fraction of our seven verification-format elements that are well-formed in a model's output chain. The reviewer appears to have read Table 4 rather than Table 1. The two relevant rows are:
GPT-OSS-20B outputs in a native format incompatible with our seven-element structure (stated at line 669), which is why its SP is 0.000. The Zero-shot LLaMA SP values are not 0.000.
Regarding AI-assisted writing: we declared the use of AI tools for writing and editing assistance in the submission, consistent with ICLR 2027 policy. The methodology, experimental design, all code, and all numerical results are entirely the authors' work. We welcome any specific technical question about the content.
Q2. "The paper fails to include any state-of-the-art closed-source models as baselines. It is not specified whether the datasets used for claim verification operate under an open-book or closed-book setting."
Clarify - the setting is stated; closed-source baseline already included. The setting is oracle-evidence open-book: evidence is drawn from dataset annotations, identical to STRIVE (Gong et al., 2025). We will add this as the first sentence of Section 3.1.
Regarding closed-source baselines: Table 4 already includes a GPT-class model (GPT-OSS-20B, zero-shot) as a scale comparison. The paper's contribution is a training framework evaluated on open-weight models where fine-tuning is possible. Closed-source API models cannot be trained with GRPO or PAM, so they evaluate a different question (zero-shot prompting scale) rather than the framework itself. That said, adding a GPT-4o zero-shot oracle-evidence row as an additional reference point is feasible through the API and we will include it, while making clear that it is a scale reference, not a comparable training baseline.
Q3. "What is the performance of your method compared to RL from scratch?"
Clarify; RL from scratch is not viable here. Applying GRPO directly to the base model without Phase 1 warmup produced CDR = 0.000 throughout the entire training run in our preliminary experiments. The base model never generates the
INCONSISTENCY DETECTEDsequence, so the detection reward is permanently zero and the agent has no gradient signal (sparse-reward collapse). This is consistent with GASP (Li et al., 2026) and ISC (Han et al., 2024), both of which demonstrate that supervised initialization is a prerequisite for RL fine-tuning on structured generation tasks with sparse rewards. Phase 1 is not optional scaffolding; it bootstraps the reward signal into a non-degenerate regime. We will add a sentence reporting this result explicitly in Section 3.2.Q4. "The use of CA (Classification Accuracy) in place of macro-F1 is highly unusual. Please provide a clearer explanation of what these metrics specifically mean, as well as how they are calculated."
Already stated in the paper- CA is macro-F1. CA = macro-F1 computed on uncorrupted input chains. This is stated in the Metrics paragraph of Section 4 and in the Table 1 column header ("CA (macro-F1)"). The reviewer's question is answered by reading the paper. We use macro-F1 rather than raw accuracy because HoVer and FEVEROUS-S evaluation splits have label imbalance, making raw accuracy misleading, a standard choice in claim verification. We will add a one-sentence clarification to the metrics paragraph and provide explicit per-class computation formulas for all five metrics (CA, RC, CDR, RQ, FC) in the revised appendix to prevent any further confusion.