- Rating
- 3(+1)
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper studies shared errors in self-evolving search agents, where a proposer generates questions and pseudo-labels and a solver supplies feedback for subsequent curricula. It diagnoses co-cheating through a post-hoc, source-grounded audit of labels and solver responses. The proposed CrossFit method partitions source documents and uses auxiliary solvers trained on complementary partitions to score proposals, while retaining all admitted tasks for main-solver training. Multi-sample verification provides a separate intervention on label quality. Experiments cover three evolution rounds, two Qwen3.5 model sizes, seven downstream benchmarks, and fixed-bank evaluator comparisons. The authors report reduced false agreement and improved downstream question answering.
Critical strengths
-
CrossFit targets a specific feedback dependency rather than treating every pseudo-label error as interchangeable. Section 3.2 and Figure 4 exclude the evaluated source from the auxiliary solver's training while preserving the main solver's access to all admitted tasks. This provides a concrete way to limit direct recycling of source-derived mistakes without requiring an external truth oracle during training.
-
The downstream results provide evidence beyond the audit metric. Table 1 reports gains of 8.8 and 8.4 percentage points over Dr. Zero on the same 1,325-question evaluation suite, with improvements on all seven benchmarks at both scales. Table 5 shows that the advantage grows between rounds 2 and 3. These results support the practical value of changing proposer feedback for the learned main solver.
Critical weaknesses
-
The central co-cheating measurements lack independent validation. Section 2 and Appendix B use one model to construct references and judge labels and responses, but report no human-validation results or adjudication prompts that would establish the reliability of false-agreement judgments. Table 4 reports approximately 85–86% audit coverage, without specifying how unresolved cases enter each statistic, and similar coverage across treatments does not establish comparable judgment error rates. These omissions matter because the claimed reductions depend on distinguishing shared incorrect answers from valid alternatives and unsupported cases. A small blinded validation across treatments would materially strengthen this diagnosis.
-
The fixed-bank experiment is insufficiently specified to verify its causal interpretation. Sections 5.2 and 5.3 do not identify the bank's originating run and round, its overlap with each evaluator's training questions, or the scoring rule for replay accuracy. For the 4B coupled control, Section 5.2 reports 68.7% solver truth and 88.1% replay accuracy without explaining how these quantities differ. The same-source, full-data, random-row, and half-budget controls are relevant, but the missing protocol details prevent checking whether the comparison isolates source ancestry rather than exact-question exposure or differences in evaluation. This limits the mechanism claim, independently of the downstream gains.
Questions for authors
-
Could you provide a small blinded human validation of the audit's false-agreement judgments across treatments, with the observed disagreements and an explicit account of how unresolved cases enter the reported rates?
-
Could you specify the fixed-bank construction and each evaluator's initialization and training overlap, and define replay accuracy relative to solver truth, so that the source-exclusion comparison in Figure 7 can be reconstructed?
9 comments
Response to Reviewer C2e7 (2/7): W1 (continued)
Continuation of our response to Reviewer C2e7.
W1 (continued). Can audit errors or unresolved cases explain the reduction in false agreement?
Blinded human-review protocol. Three author annotators participated: R1 and R2 independently reviewed every sampled question, and R3 adjudicated disagreements. R1 and R2 each received the same anonymized question, source/evidence packet, adopted label, and five saved responses. Method, model scale, training round, reward, audit verdicts, and the other annotator's verdicts were hidden. Each annotator reconstructed the evidence-backed acceptable-answer set and classified the label and each response as correct, incorrect, or unresolved; decisions were locked before comparison. A complete decision consists of the label verdict and the five response verdicts. If any of the six is unresolved, the decision is treated as unresolved, while the case record retains the partial verdicts. When R1 and R2 agree exactly, that decision is final; otherwise, R3 independently reviews the same evidence packet and makes the final decision. R3's decision may remain unresolved. Two unresolved initial decisions count as exact agreement but remain unresolved.
| Model | Method | R1–R2 exact agreement / 30 | R1 and R2 both unresolved | Sent to R3 / 30 | Final human decision unresolved / 30 |
|---|---|---|---|---|---|
| 4B | Dr. Zero | 19 / 30 (63.33%) | 0 | 11 / 30 (36.67%) | 0 / 30 (0.00%) |
| 4B | MSV | 21 / 30 (70.00%) | 0 | 9 / 30 (30.00%) | 2 / 30 (6.67%) |
| 4B | CrossFit | 22 / 30 (73.33%) | 0 | 8 / 30 (26.67%) | 0 / 30 (0.00%) |
| 4B | MSV + CrossFit | 21 / 30 (70.00%) | 1 | 9 / 30 (30.00%) | 1 / 30 (3.33%) |
| 9B | Dr. Zero | 20 / 30 (66.67%) | 1 | 10 / 30 (33.33%) | 2 / 30 (6.67%) |
| 9B | MSV | 20 / 30 (66.67%) | 0 | 10 / 30 (33.33%) | 1 / 30 (3.33%) |
| 9B | CrossFit | 19 / 30 (63.33%) | 0 | 11 / 30 (36.67%) | 2 / 30 (6.67%) |
| 9B | MSV + CrossFit | 24 / 30 (80.00%) | 0 | 6 / 30 (20.00%) | 0 / 30 (0.00%) |
Across all 240 questions, R1–R2 exact agreement is 166/240 (69.17%), including two both-unresolved cases; R3 adjudicates 74/240 (30.83%), and the final human decision remains unresolved for 8/240 (3.33%). Two final unresolved cases retain the initial both-unresolved decision; six remain unresolved after R3 adjudication. Among the 202 questions that both initial annotators resolve completely, agreement is 164/202 (81.19%). Final human-decision coverage is 232/240 (96.67%). The audit and the final human decisions jointly resolve 194 questions, or 970 pairs.
Audit instruction and outputs. The automated auditor and human annotators use the following evidence-based decision rule.
Given the question, source/evidence packet, adopted label, five saved responses and frozen label-match flags, establish the set of acceptable answers supported by the question and evidence. Record supporting passages and valid alternative answers. Judge the label and each response as correct, incorrect or unresolved, with an evidence-based reason. Supported aliases and valid alternatives count as correct; disagreement with a preferred answer string alone does not establish error. Retain unresolved when evidence is insufficient, contradictory or unable to disambiguate the answer. Preserve partial verdicts without guessing. Mark shared wrong only when the frozen matcher is positive and the label and response express the same incorrect answer. Mark lost credit only when the label is incorrect, the response is correct, and the matcher rejects it. Leave a predicate unresolved if its evidence is undecidable. Do not alter saved answers, match flags, reward, admission or updates; report matcher discrepancies explicitly. Return acceptable answers, supporting citations, six verdicts, five shared-wrong/lost-credit decisions and unresolved reasons.
The output record contains question_id, reader_id, acceptable_answers, evidence_citations, adopted_label, and responses. Each verdict contains status, reason, and evidence_ids; the five response entries also contain slot, frozen_match, shared_wrong, and lost_credit. status is correct, incorrect, or unresolved. Slots must be exactly 1–5. The aggregation applies the decision rules above.
Response to Reviewer C2e7 (3/7): W2
Continuation of our response to Reviewer C2e7.
W2. Does the fixed-bank comparison isolate source ancestry from exact-question exposure?
We use two complementary controls. First, fixed-bank replay keeps the evaluated questions, adopted labels, and matcher unchanged while varying the evaluator's training history. Figure 7 compares coupled and source-ID feedback on the same questions; the supplementary replay compares same-source and source-ID OOF auxiliary solvers on the same 3,000-question bank.
The table focuses on the 829 questions with wrong adopted labels. Five answers per question give 4,145 responses per evaluator. Wrong-label match counts responses repeating the incorrect label; reference-correct answer counts responses judged correct against the reference. Lower values in the first column and higher values in the second indicate improvement.
| Model | Auxiliary evaluator | Wrong-label match | Reference-correct answer |
|---|---|---|---|
| 4B | Same-source | 946 / 4,145 (22.82%) | 2,158 / 4,145 (52.06%) |
| 4B | Source-ID OOF | 65 / 4,145 (1.57%) | 3,072 / 4,145 (74.11%) |
| 9B | Same-source | 1,302 / 4,145 (31.41%) | 1,996 / 4,145 (48.15%) |
| 9B | Source-ID OOF | 20 / 4,145 (0.48%) | 3,540 / 4,145 (85.40%) |
Source-ID OOF reduces repetition of wrong labels while increasing correct answers on the same questions. Wrong-label matching falls from 22.82% to 1.57% at 4B and from 31.41% to 0.48% at 9B, while reference correctness rises from 52.06% to 74.11% and from 48.15% to 85.40%. Thus, the improvement includes recovering correct answers on questions whose adopted labels are wrong.
In the supplementary replay, exact-question-ID overlap is 0% for both evaluators; routed source-ID overlap is 100% for same-source and 0% for source-ID OOF. The following F rates use all 15,000 responses per evaluator.
| Model | Evaluator | Exact-question overlap (%) | Routed source overlap (%) | F, all responses |
|---|---|---|---|---|
| 4B | Same-source auxiliary | 0.00 | 100.00 | 946 / 15,000 (6.31%) |
| 4B | Source-ID OOF | 0.00 | 0.00 | 65 / 15,000 (0.43%) |
| 9B | Same-source auxiliary | 0.00 | 100.00 | 1,302 / 15,000 (8.68%) |
| 9B | Source-ID OOF | 0.00 | 0.00 | 20 / 15,000 (0.13%) |
Second, the reward-label control fixes the training curriculum and changes the labels used to compute the main solver's training rewards. Each cell below reports adopted-label rewards / reference-label rewards. Evaluation still measures F against the original adopted labels and correctness against the reference, so changing the training reward labels does not change the evaluation definitions.
| Model | Dr. Zero main-solver reward labels | F (%) | Solver truth (%) |
|---|---|---|---|
| 4B | Adopted / reference | 9.05 / 4.05 | 75.46 / 80.94 |
| 9B | Adopted / reference | 7.27 / 3.02 | 79.56 / 84.80 |
Reference-label rewards lower F and improve reference correctness at both scales on the fixed curriculum. Together, these controls show that both evaluator training history and reward-label choice affect shared-error reproduction. The exact-question exposure details needed to interpret source ancestry separately are specified in Q2.
Response to Reviewer C2e7 (1/7): W1
Thank you for your time and for looking closely at how we computed the audit and replay scores. Your questions prompted a human re-review of the audit and a comparison of replay outcomes on the same questions.
Response. We checked whether the original model audit's judgments agree with human judgments. We re-reviewed 30 questions per treatment and scale, comparing the adopted label and five saved solver answers. A label–response pair is counted as false agreement when both share the same incorrect answer. We compare only pairs whose status is resolved by both the audit and the human review, using the same sample weights and denominator for both F estimates.
With human judgments as the reference, precision measures how many audit-flagged false agreements humans confirm; recall measures how many human-identified false agreements the audit detects. These metrics assess judgment reliability on the jointly resolved validation sample.
Human judgments reproduce CrossFit's lower false-agreement rate: 6.8% versus 2.4% for Dr. Zero and CrossFit at 4B, and 9.2% versus 2.7% at 9B. Weighted precision is 95.3–100%, and recall is 100% on the jointly resolved pairs. This supports the audit's reliability and the CrossFit comparison on the human-reviewed sample.
These validation-sample rates are distinct from Table 4's averages over 43 training steps. Audit coverage is reported as J/E (Appendix B); unresolved cases are addressed in Q1 below.