acceptodds
Under review as a conference paper at ICLR 2027

Byzantine-Robust Federated RAG via Aligned Calibration and Fixed-Membership Conformal Prediction

Abstract

Language models answer questions more accurately when they can consult relevant documents, an approach called retrieval-augmented generation (RAG). Many valuable collections, such as medical records or company files, cannot be pooled in one place because of privacy rules or ownership. Federated RAG leaves each collection with its owner: each owner, or node, searches its own documents and scores candidate answers, and a central hub combines the scores. Because the hub cannot inspect the nodes, some of them, called Byzantine, may be compromised, faulty, or misled by malicious instructions hidden in documents, and report arbitrary scores. We want the hub to return a set of candidate answers that contains the correct one with a chosen probability, the guarantee offered by conformal prediction. Conformal prediction keeps every answer whose score passes a cutoff, and the hub sets that cutoff in a calibration step, using questions with known answers. An unknown group of nodes, no larger than a declared bound, may misreport both in this step and when new questions are answered. Existing methods assume every node is honest or protect only the calibration step. Our method rests on a simple observation: the honest nodes are the same in both steps. The hub therefore has all nodes score the same calibration questions. It keeps a candidate answer only if some plausible group of honest nodes, using its own scores in both steps, would keep it. We prove three guarantees. The resulting sets contain the correct answer with the chosen probability in finite samples, whatever the Byzantine nodes report. No method using the same information can return smaller sets without risking the loss of an answer the honest nodes support. If nodes fail at random, the guarantee weakens only by the probability that more nodes fail than declared. We tested the method in simulations and on real question-answering tasks, including medical exams. We also used a panel of language models as nodes, some of them hijacked by injected instructions. Our answer sets reached the target probability whenever no more nodes misbehaved than declared. They were also clearly smaller than those of simpler methods offering the same protection, most of all when the declared bound was generous. Simply averaging the nodes' scores could miss the target. In practice, an operator can therefore declare a cautious bound on the number of bad nodes without paying much for it in the size of the answer sets.

Primary area:
probabilistic methods (Bayesian methods, variational inference, sampling, UQ, etc.)
Keywords:
conformal prediction, distribution-free uncertainty quantification, federated learning, Byzantine robustness, retrieval-augmented generation, prediction sets, robust aggregation, coverage guarantees

Reviews

Among:
Shape:
1234this paper, Top 15%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.00 over 3 reviews, above 85% of the others.
Reviewer_tHFzOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
3
Read review
Paper summary

The paper proposes Byzantine-robust conformal prediction for federated RAG with frozen scoring pipelines and a fixed, unknown set of malicious nodes. All nodes score shared calibration questions. The hub considers every possible honest group consistent with a declared corruption budget and retains a candidate if the same group supports it during both calibration and prediction. Validation includes synthetic experiments, four multiple-choice benchmarks using GPT-2, and an OpenBookQA experiment using prompt-injected Llama-3.2-3B-Instruct nodes.

Critical strengths
  1. This paper explores a realistic hospital scenario involving federated RAG and sensitive patient information.
  2. The experiments vary attacks, corruption budgets, dependence, and quantization, and include deliberate assumption violations.
Critical weaknesses
  1. The four benchmark experiments use GPT-2 (Tab. 16), whose accuracy is approximately 24–28% against 25% chance, with prediction sets retaining roughly 3.6 of four options. The stronger-model evaluation is limited to Llama-3.2-3B-Instruct on OpenBookQA. Without stronger LLM/RAG baselines under comparable evidence access and coverage targets, it remains unclear whether the method produces useful predictions beyond reducing overhead in weak scoring systems.
  2. Worst case inflation can produce full answer sets, while exact enumeration scales combinatorially. The paper establishes a focused theoretical improvement, but does not sufficiently demonstrate how consequential that improvement is in realistic federated deployments.
  3. The experimental presentation is too fragmented, key results are relegated to extensive appendices, requiring repeated navigation between sections. A consolidated main-text table would make the practical significance of the improvements easier to assess.
Questions for authors
  1. Do authors have results with stronger models on the medical benchmarks, or comparisons against stronger LLM/RAG systems at matched coverage and comparable evidence access?
  2. Why must the method keep every answer that a possible honest group would accept, instead of simply meeting the target coverage?
  3. In practice, how would an operator estimate the maximum number of malicious nodes and ensure that the honest nodes remain the same during calibration and prediction?
8 comments
AuthorsOpenReview ↗

Response to Reviewer tHFz (1/7)

Dear Reviewer tHFz,

Thank you for the careful and specific review, and for the comparisons you proposed: stronger models on the medical benchmarks, baselines at matched coverage and evidence, including retrieval from the whole pooled corpus, and one consolidated table. We ran or added each of them, and took the presentation score to heart.

In brief.

  • Useful predictions with strong scorers (W1, Q1): with Qwen2.5-14B, most OpenBookQA and MMLU-Med questions get a single answer at a 90% target. Robustness costs little when nobody attacks, and the proposed rules hold the target where averaging and the median fall below it.
  • Realistic deployments (W2): with capable scorers, fixed-set comes close to the clean oracle and is smaller than deletion. At a 90% target or lower (finer reports for the Qwen scorers), nearly full sets arise only under one switching attack and cause no error; exact enumeration takes at most 17.3 ms per query for up to 32 nodes failing independently at up to 5% each.
  • One consolidated table (W3): Table 1, on the tenth page, gathers every scorer and benchmark.

Terms.

  • Clean oracle: the conformal set the honest nodes alone would give.
  • Integrity: never dropping an answer that some possible honest group would accept.
  • Low–high, high–low: the corrupt nodes send the lowest score at calibration and the highest at the query, or the reverse.

New text is in blue in the revised PDF.

AuthorsOpenReview ↗

Response to Reviewer tHFz (2/7): W1

W1. Weak scorers, and stronger baselines with comparable evidence and coverage

Thank you for pressing on this. With a strong scorer, the method gives useful predictions, not just lower overhead. Its sets are small where the scorer is accurate, often a single answer, and they stay valid under attack, where averaging and the median do not. We re-ran the four benchmarks with three stronger scorers (the 7B on three) and compared fixed-set, at the same 90% target, with pipelines built on the same corpus (no retrieval, a single node, the unguarded 16-node mean, and retrieval from the whole pooled corpus) and with seven aggregation rules at the hub. Single-pipeline methods such as TRAQ, C-RAG and RobustRAG give no coverage guarantee when some of the hub's sources may send arbitrary reports; each can run inside a node, so they complement this layer.

  • Useful predictions. With Qwen2.5-14B at every node and two of 16 nodes corrupt, fixed-set returns a single answer for 83% of OpenBookQA and 63% of MMLU-Med questions (1.18 and 1.49 answers on average; GPT-2 kept 3.60–3.66), at coverage 0.91–0.92 (Table 1).
  • Matched baselines at the same target, with nobody attacking (Table 24). Mean answers kept at 90%:
Benchmark, scorerNo retrievalWhole corpus16-node meanFixed-set (A=2A=2)
OpenBookQA, Llama-3.2-3B1.7521.8031.7191.849
OpenBookQA, Qwen2.5-7B1.2821.5761.4121.530
OpenBookQA, Qwen2.5-14B1.1011.1591.1131.175
MMLU-Med, Llama-3.2-3B1.8621.9591.9442.098
MMLU-Med, Qwen2.5-7B1.5361.7881.7381.854
MMLU-Med, Qwen2.5-14B1.3921.4141.3901.476
MedQA-US, Llama-3.2-3B2.6022.6692.6332.786
MedQA-US, Qwen2.5-7B2.4572.5662.5502.662
MedQA-US, Qwen2.5-14B2.1982.2492.1762.291
MedMCQA, Llama-3.2-3B2.6482.7892.7332.882
MedMCQA, Qwen2.5-14B2.5882.6122.6622.766

On these pairs, the Byzantine guarantee costs fixed-set only 0.06–0.15 answers per question over the unguarded 16-node mean (at most 0.23 on any row of Table 24, the literature tasks included), and 0.07–0.32 over the model without retrieval. A single node, which sees only its own shard, is 0.9–4.6 points less accurate than the 16-node mean on every exam pair, and with the Qwen scorers it keeps more answers than fixed-set on all seven pairs.

  • Under attack, the averaging rules and the median fall below the target; the proposed rules stay at 0.90–0.91 (Table 25). Coverage at 90% under low–high reporting, over the eleven benchmark–scorer pairs:
Aggregation rule at the hubCoverage
Plain mean0.65–0.86
Winsorized mean0.82–0.86
Trimmed mean0.82–0.87
Median of means0.82–0.86
Coordinate-wise median0.86–0.88
Krum0.89–0.91
Local-marginal rule (no alignment across nodes)0.91–0.96
Fixed-set, joint-threshold, deletion (A=2A=2)0.90–0.91

Krum, which follows one selected node, stays near the target here but has no guarantee: when the two nodes send the maximum at calibration only, it falls to 0.879 (7B, MedQA-US). The local-marginal rule, which ignores which calibration questions the nodes share, has no guarantee either, and its sets are larger than fixed-set's on ten of the eleven pairs. The two other rules that are valid by construction keep larger sets: under this attack, the pp-value merger keeps 2.051–4.000 answers with the Qwen scorers, and the guarded symmetric rule (a trimmed mean plus a worst-case guard) all or nearly all four on six of the seven Qwen pairs.

In the revision: Table 1 (p. 10) and Section 5 (lines 451–458 and 512); Tables 24–25 (Appendix T.2, pp. 62–63); Section 1 and Appendix A for TRAQ, C-RAG and RobustRAG.

AuthorsOpenReview ↗

Response to Reviewer tHFz (3/7): W1 (cont.) and Q1

W1 (continued). Two tasks where retrieval matters, and what the exam benchmarks show

On two literature tasks, the federation adds accuracy as well as validity under attack. We added the two tasks of the same medical suite that need the literature: BioASQ yes/no (518 held-out questions, 2 options) and PubMedQA* (400 questions, 3 options). Each of 16 nodes retrieves four passages from its own shard of PubMed (Table 41):

Task, scorerAccuracy: no retrieval / 16-node mean / whole corpusAnswers kept at 90%: no retrieval / fixed-set (2 corrupt)Low–high coverage: fixed-set / plain mean
BioASQ yes/no, Llama-3.2-3B53.3 / 71.0 / 83.8%1.750 / 1.5120.901 / 0.825
BioASQ yes/no, Qwen2.5-14B51.4 / 73.9 / 87.3%1.735 / 1.4570.901 / 0.823
PubMedQA*, Llama-3.2-3B36.0 / 52.5 / 76.0%2.236 / 2.3030.906 / 0.796
PubMedQA*, Qwen2.5-14B9.5 / 39.2 / 78.5%2.837 / 2.6350.911 / 0.633
  • Smaller sets than the model alone, at the guarantee. With two corrupt nodes, fixed-set keeps fewer answers than the model without retrieval on three of the four pairs, at coverage 0.93–0.95. Under low–high it covers 0.90–0.91, where the plain mean falls to 0.63–0.83.
  • Retrieval helps when it finds the evidence. A node whose four passages include an article of the question's gold evidence answers 75.7–91.1% of the questions correctly; the other nodes answer 36.7–68.4% correctly.
  • Replicating articles across nodes helps further. The one exception above, PubMedQA* with Llama-3.2-3B (2.303 answers against 2.236 without retrieval), arises because each article sits with only one node of 16, and averaging dilutes it. When each article is held by 2, 4 or 8 nodes instead, the mean is right on 58.8, 63.2 and 71.0% of the questions (52.5% with one), and fixed-set keeps 2.135–2.192 answers, fewer than without retrieval. On BioASQ, accuracy rises from 71.0% to 81.1%, and fixed-set's sets shrink from 1.510 to 1.278 answers.
  • Pooling the whole corpus is more accurate still (76.0–87.3%), but that is exactly what a federation cannot do; replication closes part of the gap.
  • What the four exam benchmarks test. On OpenBookQA and the three medical exam benchmarks, the 16-node mean is at most 0.7 points more accurate than the model without retrieval, and its sets are at most 0.034 answers smaller (Table 24). These benchmarks therefore test the aggregation rule, the paper's contribution; the two literature tasks test what federated retrieval adds.

In the revision: Table 1 (p. 10) and Section 5 (lines 457–458 and 512); Table 41 (Appendix T.10, p. 81).

Q1. Stronger models on the medical benchmarks; stronger systems at matched coverage

Yes to both. With Qwen2.5-14B, fixed-set keeps 1.49 answers on MMLU-Med at 90%, a single answer for 63% of the questions, and 2.31 and 2.77 on MedQA-US and MedMCQA, where that scorer is right on 64% and 54% of the questions (Table 1, which also has Llama-3.2-3B and Qwen2.5-7B). Our response to W1 gives the comparison at matched coverage and evidence: with nobody corrupt, the guarantee costs fixed-set 0.06–0.23 answers per question over the unguarded 16-node mean, and when the corrupt nodes report low scores at calibration and high ones at the query, the averaging rules and the median fall below the target on the exam benchmarks while the proposed rules stay at 0.90–0.91 (Tables 24–25).

In the revision: Table 1 (p. 10); Tables 24–25 (Appendix T.2, pp. 62–63).

Reviewer_ihTBOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper studies Byzantine-robust prediction in federated retrieval-augmented generation (RAG), where some nodes may report arbitrary scores during both calibration and query phases, potentially breaking the coverage guarantees of conformal prediction. The authors propose a fixed-set inference method based on aligned calibration and the assumption that the set of honest nodes remains unchanged across both phases. The method considers possible honest subsets and retains a candidate answer if at least one subset supports it in both phases. The authors provide theoretical guarantees for finite-sample coverage, prediction set minimality under specific conditions, and set size. Experiments on synthetic data, multiple-choice QA benchmarks, and prompt-injected language models show that the method maintains the target coverage while producing smaller prediction sets than some robust baselines.

Critical strengths
  1. The paper addresses a practically relevant problem by considering Byzantine attacks during both calibration and query phases. While some existing federated conformal prediction methods mainly focus on protecting calibration, this work considers malicious nodes that may behave differently across the two phases, potentially breaking the coverage guarantees of existing approaches.

  2. The key idea is clear and intuitive: the same set of honest nodes should be used in both calibration and query phases. Rather than allowing different possible honest subsets in each phase, the proposed method requires a consistent subset across both, providing a clear motivation for its design.

  3. The paper provides a relatively thorough theoretical analysis. Beyond finite-sample coverage guarantees, the authors establish inclusion relationships among different prediction rules and prove the minimality of the fixed-set method under a specific integrity requirement. The assumptions and scope of these guarantees are also clearly stated.

Critical weaknesses
  1. Limited practical value demonstrated by the real-world QA experiments. Although the paper evaluates several real-world QA datasets, the main experiments use GPT-2 as the answer scorer, achieving only 24.10%–28.12% accuracy, close to the 25% random-guessing level for four-choice questions. Even the honest-mean oracle retains about 3.5 out of four candidate answers on average, while the proposed fixed-set method typically retains around 3.6–3.7. Therefore, although the method reduces prediction set size while maintaining coverage, the improvement is relatively small in absolute terms, and the resulting sets remain less informative for practical answer selection. The authors also evaluate a stronger Llama model, but these experiments are limited to OpenBookQA, leaving its effectiveness on medical QA tasks with stronger models unclear.

  2. Limited applicability to open-ended RAG tasks. Although the paper focuses on federated RAG, the proposed method assumes a known and finite set of candidate answers, and all real-world QA experiments use four-choice questions. However, practical RAG systems often involve open-ended answer generation, where the correct answer may not be included in a predefined candidate set. Therefore, the current theoretical guarantees and experimental results do not directly extend to open-ended RAG tasks, limiting the practical scope of the proposed method.

  3. The fixed honest-node assumption may limit robustness against dynamic attacks. A key assumption behind the proposed method and its theoretical guarantees is that the set of honest nodes remains unchanged across calibration and query phases. However, in practical RAG systems, prompt injection attacks often depend on the retrieved documents and specific questions. The same node may be misled by malicious instructions for some queries but behave normally for others. As a result, the identities of compromised nodes may change across questions or over time, which raises concerns about the applicability of the proposed method in such dynamic settings.

Questions for authors
  1. Could the authors evaluate the method with stronger language models on real medical QA benchmarks? In particular, when the base model has better answer prediction accuracy, can the proposed method still maintain the target coverage while achieving meaningful reductions in prediction set size?

  2. Could the authors further discuss how the proposed method could be extended to open-ended RAG generation? For example, if the correct answer is not included in a predefined candidate set, or if candidate answers are generated by a language model, would the current coverage guarantees still apply?

  3. Could the authors further discuss or experimentally evaluate how the method performs when the identities of compromised nodes change across questions or over time? How would this affect the coverage and prediction set size, even if the number of compromised nodes per query remains within the declared budget?

7 comments
AuthorsOpenReview ↗

Response to Reviewer ihTB (1/6): W1

Dear Reviewer ihTB,

Thank you for a precise review. Your account of what the guarantees assume, a known finite candidate list and a fixed honest group, is exactly right, and your questions ask what happens with stronger models and when each assumption is relaxed. The revision answers each with new experiments, and the last two also with new theory.

In brief.

  • Stronger models on medical QA (W1, Q1): coverage holds and the sets shrink as accuracy grows; with Qwen2.5-14B, most MMLU-Med and OpenBookQA questions get a single answer.
  • Open-ended answers (W2, Q2): lists written by a language model at the hub, with a "none of these" entry for a missing answer, keep every guarantee (new Proposition S.6), measured with a missing option and with model-written lists.
  • Changing identities (W3, Q3): deletion stays valid when the compromised nodes change from question to question (new Theorem S.1), measured on eight benchmark–scorer pairs; a document planted before calibration, if left unchanged and not chosen with knowledge of the calibration questions, needs no budget.

Terms.

  • Clean oracle: the conformal set the honest nodes alone would give.
  • Low–high, high–low: the corrupt nodes send the lowest score at calibration and the highest at the query, or the reverse.

New text is in blue in the revised PDF.

W1. Practical value beyond GPT-2

We agree that GPT-2's sets are not informative; with stronger scorers, the same method gives small sets at the same guarantee. GPT-2 keeps 3.6–3.7 of 4 answers, as Section 5 notes, so the revision adds three stronger scorers on the four benchmarks, three of which are medical (the 7B is not run on MedMCQA).

  • Three stronger scorers (Table 1; Section 1, lines 135–137). With Qwen2.5-14B, fixed-set keeps 1.18 answers on OpenBookQA and 1.49 on MMLU-Med (GPT-2: 3.65 and 3.60), with coverage of at least 0.91 in every row of the table, and it still saves 0.10 and 0.26 answers over deletion.
  • Single-answer sets, the quantity that matters for answer selection: share of questions on which fixed-set returns one answer, at targets of 90% / 80% (Table 23):
Llama-3.2-3BQwen2.5-7BQwen2.5-14B
MMLU-Med36.6% / 57.3%45.4% / 74.2%63.4% / 89.0%
MedQA-US17.4% / 32.4%20.2% / 34.3%29.8% / 53.5%
MedMCQA14.2% / 23.1%not run20.2% / 32.0%
OpenBookQA43.7% / 72.0%57.9% / 84.1%83.1% / 93.6%
  • Set size follows the scorer's accuracy. On MedQA-US and MedMCQA, where the 14B is right on 64.1% and 54.2% of the questions, fixed-set keeps 2.309 and 2.774 answers. Even the clean oracle keeps 2.179 and 2.665 there, and the model without retrieval 2.198 and 2.588 (Table 24). A 90% set cannot be small when the scorer is right on only about half of the questions.
  • Two literature tasks where retrieval matters (BioASQ yes/no and PubMedQA*; Table 41). The 16-node mean is right on 71.0–73.9% and 39.2–52.5% of the questions, against 51.4–53.3% and 9.5–36.0% without retrieval. Fixed-set keeps 1.46–1.51 of 2 and 2.30–2.64 of 3 answers at 90%. Under low–high reporting, it covers 0.90–0.91 where the plain mean falls to 0.63–0.83.

In the revision: Section 1 (lines 135–137); Table 1 (p. 10) and Section 5 (lines 451–458 and 512); Tables 23–24 (Appendix T.2, pp. 61–62); Table 41 (Appendix T.10, p. 81).

AuthorsOpenReview ↗

Response to Reviewer ihTB (2/6): Q1

Q1. Stronger models on real medical QA

Yes to both parts: coverage is maintained, and the sets shrink as the scorer's accuracy grows.

  • Coverage is maintained. With two corrupt nodes of 16 sending the largest score, fixed-set covers 0.91–0.93 at a 90% target on the medical benchmarks, with every scorer. Over all seven constructed threats within the budget, all eleven benchmark–scorer pairs and all three proposed rules, the lowest coverage is 0.898 at 90%, exactly the clean oracle's own coverage in that setting (the 14B on MedMCQA under low–high), and 0.798 at 80%.
  • The sets shrink with accuracy. Answers of 4 kept by fixed-set at 90%, with the accuracy of the 16-node mean in parentheses:
Llama-3.2-3BQwen2.5-7BQwen2.5-14B
MMLU-Med2.114 (66.9%)1.865 (73.3%)1.486 (78.7%)
MedQA-US2.802 (55.3%)2.673 (55.6%)2.309 (64.1%)
MedMCQA2.905 (47.5%)not run2.774 (54.2%)

GPT-2, at 24–28% accuracy, kept 3.6–3.7. Compared with deletion, fixed-set removes 0.18–0.21 answers with the 3B, 0.26–0.29 with the 7B and 0.26–0.32 with the 14B (0.057–0.077 with GPT-2).

  • Report resolution. For the sharply peaked Qwen scorers, Table 1 uses 24-bit reports, which change no guarantee. At the default 8 bits, their cutoff on MedQA-US and MedMCQA falls within one rounding step of the top of the scale, and every proposed rule then keeps all four answers when two nodes send the largest score. Finer reports cost a few bytes each, never enlarge fixed-set or deletion, and bring both toward the oracle (Tables 23 and 26).

In the revision: Table 1 (p. 10); Table 23 (Appendix T.2, p. 61); Table 26 (Appendix T.3, p. 64), with 6 to 24 bits.

AuthorsOpenReview ↗

Response to Reviewer ihTB (3/6): W2

W2. Open-ended RAG tasks

The revision extends the guarantees beyond predefined lists, and tests the extension. New Proposition S.6 covers candidate lists written for each question by a language model, with a "none of these" entry for a correct answer the list lacks. Fully free-form answers, with no finite list at all, remain outside the guarantees, as Section 6 now states (lines 533–534).

  • The list need not be predefined or shared across questions. The theorems index candidates by their position in each question's own list; multiple-choice options already differ from question to question. The proofs use only four things: the list is finite, fixed before the nodes score, the same for every node and in both phases, and the calibration labels say which listed candidate is correct.
  • Generated lists, and an answer that may be missing (Proposition S.6). Let a generator, frozen before calibration and run by the hub, write each question's list with "none of these" as its last entry, and let a rule fixed before calibration map the reference answer to the first matching candidate, or to "none of these". If the (question, list, label) triples are exchangeable, Theorems 4.1, 4.3, 4.4 and Corollary 4.2 hold as stated: each proposed set contains the matching candidate, or "none of these" when none matches, with probability at least 1−α1-\alpha.
  • The hub writes the list. A list proposed by the nodes is not covered, since a Byzantine node could leave the answer out (Remark S.7).
  • Experiment 1: the answer may be missing (Table 37). One option of every OpenBookQA question is replaced by "None of the other answers" (the correct option on 250 of the 1,000 questions); every node retrieves again and Llama-3.2-3B rescores. Coverage (mean size), nobody corrupt:
OracleFixed-setJoint-thresholdDeletion
Answer replaced ("none" is correct)0.8777 (2.678)0.9080 (2.835)0.9148 (2.867)0.9367 (3.012)
Answer present0.9116 (2.272)0.9286 (2.421)0.9315 (2.450)0.9425 (2.583)

No proposed rule drops an answer the oracle keeps. (The guarantee is marginal, not per group: the oracle's 0.878 on the roughly 167 such questions of each split is about one binomial standard error below 0.90.)

  • Experiment 2: lists written by a language model (Table 42). Qwen2.5-14B reads the question alone and writes three candidates, "None of these" is added, and 16 nodes score the four options; a judge model fixed in advance labels each list by the candidate that matches the reference answer. With two corrupt nodes, fixed-set covers 0.92–1.00 at 90% and keeps "None of these" on 91–100% of the questions where it is right.

In the revision: Section 6 (lines 533–534); Proposition S.6 and Remark S.7 (Appendix S.3, pp. 58–59); Table 37 (Appendix T.8, p. 77); Table 42 (Appendix T.11, p. 83).

Reviewer_mhjwOpenReview ↗
Rating
4(+1)
Confidence
2
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

This paper studies conformal prediction for federated RAG when a fixed, unknown subset of nodes can arbitrarily manipulate both calibration and query reports. It proposes aligned calibration, in which all nodes score the same calibration examples, and a fixed-set rule that retains a candidate whenever some feasible honest subset supports it using that subset’s own calibration threshold and query mean. The paper establishes finite-sample coverage, pointwise minimality under an entrywise interval model with clean-score integrity, and a coverage bound under random node failures. It also analyzes two relaxations, joint-threshold and deletion. Experiments use synthetic scores, four QA benchmarks scored by GPT-2, and an OpenBookQA panel scored by Llama-3.2-3B-Instruct. The results demonstrate reduced conservatism relative to the relaxations and coverage failures of unprotected aggregation under certain attacks.

Critical strengths
  • A useful structural insight with a precise inference rule. The paper identifies a specific source of unnecessary conservatism: allowing the hypothetical honest group to change across calibration examples or between calibration and prediction. Requiring the same group throughout yields a clear construction and a provable hierarchy of prediction sets. This provides a concrete explanation for how fixed-set can remove additional candidates while preserving the honest-mean oracle, and constitutes a useful contribution to robust uncertainty quantification.

  • Careful theoretical treatment supported by an executable implementation. The analysis handles arbitrary reports in both phases, directional quantization errors, and random failures, while distinguishing marginal coverage from oracle preservation and set-size efficiency. The minimality result is explicitly qualified by its information model. The supplementary implementation is consistent with the stated rules, and spot checks of the downstream Llama computations reproduce the archived statistics. Together with the simulations and the stronger-model panel, this provides credible support for the main construction.

Critical weaknesses
  • The practical value of clean-score integrity needs stronger justification. The paper clearly distinguishes clean-score integrity from marginal coverage, and Appendix R demonstrates that pruning can violate oracle containment. However, these results do not fully establish when preserving every candidate admitted by a compatible honest-mean oracle is preferable in practice to alternative valid constructions. The p-value merger, for example, provides valid coverage under its stated assumptions and returns smaller sets under some attacks (Table 6). A concrete application-level justification for accepting additional conservatism to preserve the honest-mean oracle would strengthen the significance of the minimality result in Theorem 4.3.
  • The practical gains with capable RAG scorers are validated in a narrow setting. On the four GPT-2 benchmarks, point accuracy is near chance, the oracle already retains approximately 3.5 of four candidates, and the absolute reduction over deletion under stable-maximum reporting is only 0.057–0.077 candidates per query (Table 13). The Llama evaluation provides more convincing evidence of useful set-size reductions, but it is limited to OpenBookQA and a single instruction-induced failure mode (Appendix R). Thus, the evidence for practical gains across QA tasks and more varied retrieval-based attacks remains limited, even though the theoretical coverage guarantee is broader.
Questions for authors
  • Application-level justification: Beyond showing that alternative methods can violate oracle containment, could you provide a concrete use case or analysis explaining when such a violation is consequential despite valid marginal coverage? In particular, what downstream requirement makes preserving the entire honest-mean oracle set worth the potentially larger prediction sets?

  • Focused additional validation: Could you provide one additional evaluation with a capable scorer beyond the current OpenBookQA setting, for example on a medical QA benchmark, reporting coverage and absolute set sizes for fixed-set, deletion, the p-value merger, and the oracle? This would help assess whether the practical gains extend beyond the current strong-model example. Please also clarify the scope of the prompt-injection claims by distinguishing the controlled instruction changes in the current experiment from attacks delivered through retrieved passages.

8 comments
AuthorsOpenReview ↗

Response to Reviewer mhjw (1/5): W1

Dear Reviewer mhjw,

Thank you for a careful review, and for spot-checking the Llama computations. Your account of what clean-score integrity adds over marginal coverage is exactly the distinction the paper draws, and you asked for the two things that test it: a concrete case for integrity, and a capable scorer beyond OpenBookQA. The revision adds both.

In brief.

  • Integrity (W1, Q1): within the budget, fixed-set misses only where the clean oracle misses (new Proposition S.8). When two corrupt nodes target one medical subject, an unguarded mean keeps its average coverage while 7–15% of that subject's questions lose an answer the clean oracle keeps (Llama-3.2-3B); fixed-set loses none.
  • Capable scorers (W2, Q2): three scorers on the four benchmarks (the 7B on three), two of them also on two literature tasks, with the coverage and set sizes you asked for on all three medical benchmarks.
  • Injection through retrieved passages (Q3): the submitted attack is now named an injected instruction; text planted in retrieved passages is analyzed (new Corollary S.4) and measured.

Terms.

  • Clean oracle: the conformal set the honest nodes alone would give.
  • Low–high, high–low, stable maximum: the corrupt nodes send the lowest score at calibration and the highest at the query; the reverse; or the largest score in both phases.
  • Peaked scorer: one that puts nearly all its probability on one answer.

New text is in blue in the revised PDF.

W1. The practical value of clean-score integrity

Integrity controls where errors fall; valid coverage controls only how many. A rule with valid coverage may miss up to 10% of the time, and an attacker may decide which questions those misses land on. Integrity takes that choice away. In application terms: when single-answer sets are acted on and larger sets go to a person, corrupt nodes within the budget can cause extra escalations, but can never hide an answer the honest evidence supports (Q1 develops this case). And in most of our settings integrity costs nothing relative to the pp-value merger, whose sets are larger.

  • What integrity guarantees (new Proposition S.8; Section 4, lines 355–357). Within the budget, every miss of fixed-set is a miss of the clean oracle C∘\mathcal C^{\circ}. So for any group of questions EE, including one an attacker selects, Pr⁡{Y∈Cfs(X)∣E}≥Pr⁡{Y∈C∘(X)∣E}\Pr\lbrace Y\in\mathcal C^{\rm fs}(X)\mid E\rbrace\ge\Pr\lbrace Y\in\mathcal C^{\circ}(X)\mid E\rbrace, whatever the corrupt nodes report. They can enlarge a set, but they cannot create an error or move one.
  • Measured on the submission's own failure mode (Table 36). Here Llama-3.2-3B judges on OpenBookQA are misled by an injected instruction in both phases, with a budget of 2; we use the 464 of 500 calibration splits on which at most two judges, drawn at random, are misled (the oracle keeps about 974 question–answer pairs per split):
RuleCoverageMean sizeSplits dropping an oracle answerOracle pairs dropped per split
Fixed-set0.9241.86200
Joint-threshold0.9331.95100
Deletion0.9522.15800
pp-value merger0.9812.816420 (90.5%)2.32
All-node mean (no guarantee)0.9011.654–59.0

The all-node mean sits at its nominal coverage, so a coverage monitor sees nothing, yet it drops 6.1% of the oracle's pairs (0.9% when nobody is misled). The merger is 51% larger than fixed-set and still not a superset of the oracle.

  • With peaked scorers at 8 and 16 bits, the merger keeps nearly every answer. The merger drops a candidate only if 2A+12A+1 nodes reject it at level α(A+1)/(K−A)\alpha(A+1)/(K-A), here 2.1%; at these resolutions a peaked scorer gives the correct answer the top possible report on more than 2.1% of a node's calibration questions, and that node's pp-values then never get small enough to reject anything. So with Qwen2.5-7B and 14B the merger keeps 3.97–4.00 of 4 answers in all 98 settings within the budget at 8 and 16 bits, and 2.442–4.000 at Table 1's 24 bits, more than fixed-set on every pair (Table 27 shows why).
  • Where the merger is smaller. Under high–low reporting, with Llama on OpenBookQA it keeps 2.857 answers against fixed-set's 3.990, and with the Qwen scorers at 24 bits it is smaller on four of seven pairs. Elsewhere fixed-set is smaller, for example in 48 of the 56 settings of the submission's GPT-2 study (Table 15). The merger remains a good choice for a small federation with a wide report range and a scorer that is not peaked, when high–low attacks are expected and the honest-mean reading is not needed.

In the revision: Section 4 (lines 355–357 and 386–389); Proposition S.8 (Appendix S.4, p. 59); Tables 26–27 (Appendix T.3, pp. 64–65); Table 36 (Appendix T.7, p. 76).

AuthorsOpenReview ↗

Response to Reviewer mhjw (2/5): Q1

Q1. When does a violation matter despite valid marginal coverage?

When an attacker can choose which questions bear the errors, and someone acts on the answers. Valid marginal coverage holds on average over all questions, so it does not stop corrupt nodes from concentrating the misses on one group. Our new experiment does this to one medical specialty: the unguarded mean keeps its average coverage, yet on the targeted subject it covers up to 2.23 points less than the clean oracle, while fixed-set creates no new miss.

  • The downstream requirement. Take a system that acts on single-answer sets and sends larger sets to a person. With integrity, corrupt nodes within the budget can cause more escalations, but can never hide an alternative the honest evidence supports: if fixed-set returns the single answer yy, the clean oracle returns {y}\lbrace y\rbrace or nothing (Proposition S.8, part 2), so every alternative the honest evidence cannot rule out reaches the person. The price is extra escalations, which Theorem 4.4 confines to answers whose clean score lies within a known width of the clean cutoff. The same holds whenever participants have conflicting interests: one that wants an answer suppressed gains nothing.
  • A targeted medical specialty (new; three scorers; Table 34). Two corrupt nodes of 16 send the largest score on the questions of one of MMLU-Med's six subjects, in both phases, each subject in turn (budget 2, 90%). On the targeted subject the unguarded mean covers up to 2.23 points less than the clean oracle with Llama-3.2-3B (0.65 and 0.83 points with the 7B and 14B): 7–15% of the subject's questions lose an answer the clean oracle keeps (2–5% with the Qwen scorers), and up to 2.2% lose the correct answer although the clean oracle has it. Fixed-set stays above the oracle on every subject with every scorer and creates no new miss (16-bit reports for the Qwen scorers).
  • Errors can also be pushed away. If the two nodes instead reverse their ranking on one subject, the unguarded mean loses up to 3.2 points on the other subjects.
  • A random tenth of the questions (Table 35; Llama, OpenBookQA, three corrupt nodes, budget 3). When the nodes send the largest score on those questions only, the unguarded mean covers 0.884 there (0.902 without attack), and 12.3% of them lose an answer the clean oracle keeps (3.1% without attack). The proposed rules lose none.

In short, an unguarded average can keep its coverage while moving its errors onto, or away from, a group of questions an attacker chooses. This is not an argument against the pp-value merger, which is valid by construction and shows no such shift (0.9837 on the targeted tenth, 0.9830 on the rest); the case against the merger is its size, which with peaked scorers at 8 and 16 bits means nearly every answer (W1).

In the revision: Section 4 (lines 355–357); Proposition S.8 (Appendix S.4, p. 59); Tables 34–35 (Appendix T.7, p. 74); Table 36 (Appendix T.7, p. 76).

AuthorsOpenReview ↗

Response to Reviewer mhjw (3/5): W2

W2. Gains beyond GPT-2 and one Llama setting

Correct for the submission; the revision widens the evidence on both fronts you name. It adds three capable scorers on the four benchmarks (the 7B on three) and a much broader range of attacks. Across all of them, fixed-set is never larger than deletion, and its sets shrink as the scorer improves.

  • Set sizes (Table 1; two corrupt nodes sending the largest score, 90% target). Fixed-set keeps 1.9–2.9 answers of 4 with Llama-3.2-3B, 1.5–2.7 with Qwen2.5-7B and 1.2–2.8 with Qwen2.5-14B, at coverage 0.91–0.93; GPT-2 kept 3.6–3.7.
  • Savings over deletion (Table 23). Fixed-set removes 0.18–0.24 answers with the 3B, against the 0.057–0.077 you cite for GPT-2, and 0.099–0.355 with the Qwen scorers: about half to three quarters of deletion's excess over the oracle, on every pair.
  • Single-answer sets (Table 23). Share of questions on which fixed-set returns one answer at 90% / all four at 90% / one answer at 80%:
Llama-3.2-3BQwen2.5-7BQwen2.5-14B
MMLU-Med36.6 / 11.4 / 57.3%45.4 / 7.0 / 74.2%63.4 / 1.8 / 89.0%
MedQA-US17.4 / 36.5 / 32.4%20.2 / 31.7 / 34.3%29.8 / 19.5 / 53.5%
MedMCQA14.2 / 38.3 / 23.1%not run20.2 / 39.2 / 32.0%
OpenBookQA43.7 / 6.4 / 72.0%57.9 / 1.1 / 84.1%83.1 / 0.0 / 93.6%
  • A broader range of attacks. Every pair is analyzed under the seven constructed threats of the GPT-2 study. The lowest coverage of the three proposed rules is 0.898 at 90%, exactly the clean oracle's own coverage in that setting (the 14B on MedMCQA under low–high), and 0.798 at 80%. Attacks now also come through retrieved passages (planted false claims; Q3), through candidate lists that may lack the answer, and through identities that change from question to question.
  • Where retrieval matters. On BioASQ yes/no and PubMedQA*, the 16-node mean is right on 71.0–73.9% and 39.2–52.5% of the questions (51.4–53.3% and 9.5–36.0% without retrieval), and fixed-set keeps 1.46–1.51 of 2 and 2.30–2.64 of 3 answers at coverage 0.93–0.95 (Table 41).

In the revision: Table 1 (p. 10) and Section 5 (lines 451–458 and 512); Tables 23–25 (Appendix T.2, pp. 61–63); Tables 29–30 (Appendix T.5, p. 68); Table 41 (Appendix T.10, p. 81).

Comments on the paper

3 comments
AuthorsOpenReview ↗

Response to all reviewers (1/3)

Dear Area Chair and Reviewers,

We thank the reviewers for careful and constructive reports. Reviewer mhjw finds that the paper clearly distinguishes clean-score integrity from marginal coverage, and their spot checks of the Llama computations reproduced the archived statistics; Reviewer ihTB notes that the method reduces set size while maintaining coverage, and asks for reductions that matter in practice; Reviewer tHFz credits a focused theoretical improvement, and asks how consequential it is in realistic deployments. The three rules, the theorems of Section 4 and their proofs are unchanged. The revision adds three stronger scorers, which answer those two questions, along with attacks through retrieved passages, compromised nodes that change between questions and generated candidate lists, with new theory where needed; the table below answers each question raised. New text is in blue in the revised PDF.

Terms.

  • Fixed-set, joint-threshold, deletion: the paper's three rules. Budget AA: the declared maximum number of corrupt nodes.
  • Clean oracle: the conformal set the honest nodes alone would give; the hub cannot compute it, since it does not know who is honest.
  • Integrity: never dropping an answer that some possible honest group would accept; within the budget, fixed-set then misses only where the clean oracle misses.
  • Low–high, high–low, stable maximum: the corrupt nodes send the lowest score at calibration and the highest at the query; the reverse; or the largest score in both phases.
Question (reviewers)What the revision showsWhere
Are the sets useful with a capable scorer, also on medical QA? (all)Yes. With Qwen2.5-14B, and two of 16 nodes corrupt and sending the largest score, fixed-set gives a single answer for 83% of OpenBookQA and 63% of MMLU-Med questions at a 90% target (GPT-2 kept 3.60–3.66 of 4 answers). Three scorers on four benchmarks (the 7B on three), two of them also on two literature tasksTable 1 (p. 10), Sec. 5; Tables 23, 26, 41
How does it compare with stronger baselines at matched coverage? (tHFz)With nobody attacking, robustness costs 0.06–0.23 answers per question over the unguarded 16-node mean. On two literature tasks, the 16-node mean is 16.5–29.7 points more accurate than the model without retrievalTables 24, 41
Do the rules hold where standard aggregation fails? (mhjw, tHFz)Under low–high on the exam benchmarks, the plain mean covers as little as 0.65 with the stronger scorers (0.55 with GPT-2), and robust averages and the median 0.82–0.88; the three proposed rules cover 0.90–0.91Sec. 5; Tables 1, 25
Why keep every answer an honest group would accept? (mhjw, tHFz)Coverage alone lets attackers choose where errors fall. When two corrupt nodes target one MMLU-Med subject, the plain mean keeps its average coverage, yet 7–15% of that subject's questions lose an answer the clean oracle keeps (Llama-3.2-3B); fixed-set loses none (Prop. S.8). In every row of Table 1, the pp-value merger keeps more answers than fixed-setSec. 4; App. S.4; Tables 26–27, 34–36
Attacks through retrieved passages, and nodes that change? (mhjw, ihTB)Planted false text makes a poisoned node pick the planted option on 52.4–99.5% of questions, yet within the budget fixed-set covers at least 0.90. Text planted before calibration, left unchanged and not chosen with knowledge of the calibration questions, needs no budget (Cor. S.4), and deletion stays valid when the corrupt nodes change between questions (Thm. S.1)Secs. 2, 3, 5; App. S.1–S.2; Tables 29–32
What if the correct answer is not among the candidates? (ihTB)Lists generated at the hub, with a "none of these" entry, keep every guarantee (Prop. S.6); measured with a missing option and with lists written by an LLMSec. 6; App. S.3; Tables 37, 42
How bad is the worst case, and what does exactness cost? (tHFz)At a 90% target or lower (finer reports for the Qwen scorers), nearly full sets need one switching attack and cause no error. Exact enumeration takes at most 17.3 ms per query up to 32 nodes failing independently at up to 5% each; two exact shortcut checks (Prop. S.9) settle most decisions, and deletion needs noneSecs. 4, 6; App. S.5; Tables 28, 33, 38–40
One consolidated main-text table? (tHFz)Added, and reproduced in full in our third commentTable 1 (p. 10)
AuthorsOpenReview ↗

Response to all reviewers (2/3)

A. What is new

Theory, with proofs in Appendix S:

ResultWhere
Deletion stays valid when the compromised nodes change from question to question; beyond the budget it loses at most the probability of exceeding itTheorem S.1, Remark S.2
Fixed-set and joint-threshold can lose coverage when the compromised nodes change, by a two-node exampleProposition S.3
Text planted before calibration, left unchanged and not chosen with knowledge of the calibration questions, needs no budget; the statement fails otherwiseCorollary S.4, Remark S.5
Candidate lists generated at the hub, with a "none of these" entry, keep every guarantee; lists proposed by the nodes do notProposition S.6, Remark S.7
Within the budget, fixed-set misses only where the clean oracle misses, on any group of questionsProposition S.8
Two exact shortcut checks decide most candidates without enumerationProposition S.9

Experiments (Appendix T), mostly in 16-node federations with real retrieval:

  • three stronger scorers on the four benchmarks (the 7B on three), at 90% and 80% targets and every report resolution (Tables 23, 26 and 27);
  • two literature tasks where retrieval matters, BioASQ yes/no and PubMedQA* (Table 41);
  • matched baselines, and every aggregation rule under low–high reporting (Tables 24–25);
  • attacks through retrieved passages (Tables 29–30), and compromised nodes that change between questions (Tables 31–32);
  • attacks aimed at one medical subject or at a random tenth of the questions, and which oracle answers each rule drops (Tables 34–36);
  • a missing correct answer, and candidate lists written by a language model (Tables 37 and 42);
  • the worst case along the target, federation size, report range, enumeration cost and budgets (Tables 28, 33 and 38–40).

Reproducibility. The supplement holds the code of every new study, every score file it uses and each study's frozen record; the rebuttal's paper_tables.py rebuilds every table of the revision from those records, and the studies need only numpy and pytest.

B. What we clarified

Four points of scope are now stated in the paper, where the reviewers asked:

  1. The submitted injection experiment replaced the instruction a judge receives. Section 5 and the captions of Figures 3–4 now call it an injected instruction, and text planted in retrieved passages is treated separately (Corollary S.4, Appendix T.5). (mhjw)
  2. The scope of the guarantees. They hold for any finite candidate list fixed before the nodes score, including lists generated at the hub with a "none of these" entry (Proposition S.6); Section 6 now says so, and that free-form answers with no list are outside them. (ihTB)
  3. Changing identities. Deletion keeps its guarantee when the compromised nodes change from question to question (Theorem S.1); Section 3 now names it as the rule to deploy then, since fixed-set and joint-threshold assume a fixed honest group. (ihTB)
  4. The worst case and the cost of exactness. Section 6 now states both: under high–low reporting every rule returns nearly every answer while the target is well above the scorer's accuracy, and the proposed rules' sets shrink fast once it is not; two exact shortcut checks settle 94.3–100% of fixed-set's decisions without enumeration, and deletion needs none. (tHFz)
AuthorsOpenReview ↗

Response to all reviewers (3/3)

The consolidated table (Table 1 of the revision, p. 10). In every row, 16 nodes score the candidates, two of them corrupt and sending the largest score in both phases; the declared budget is A=2A=2 and the target 90%. Sizes are the mean number of answers kept, out of 4 (2 for BioASQ, 3 for PubMedQA*). Cov. and Single are fixed-set's coverage and its share of single-answer sets; the last column is the coverage of fixed-set and of the plain mean under low–high reporting.

ScorerBenchmarkAcc.OracleFixed-setCov.DeletionMergerSingleLow–high
GPT-2OpenBookQA–3.5253.6540.92763.7214.0002.4%0.9149 / 0.5459
GPT-2MedQA-US–3.5573.6590.92283.7363.9290.1%0.9101 / 0.5865
GPT-2MedMCQA–3.5403.6470.92423.7213.9640.8%0.9122 / 0.5989
GPT-2MMLU-Med–3.5193.5960.91563.6533.8950.9%0.9053 / 0.6094
Llama-3.2-3BOpenBookQA73.0%1.7211.8650.91992.1042.85743.7%0.9041 / 0.8208
Llama-3.2-3BMedQA-US55.3%2.6252.8020.92632.9793.48317.4%0.9109 / 0.7009
Llama-3.2-3BMedMCQA47.5%2.7342.9050.92393.1143.58114.2%0.9066 / 0.6530
Llama-3.2-3BMMLU-Med66.9%1.9482.1140.92282.3222.86636.6%0.9034 / 0.7776
Qwen2.5-7BOpenBookQA78.1%1.4191.5480.92041.9032.97157.9%0.9009 / 0.8364
Qwen2.5-7BMedQA-US55.6%2.5492.6730.91122.9303.59220.2%0.8980 / 0.7052
Qwen2.5-7BMMLU-Med73.3%1.7421.8650.91442.1522.82745.4%0.8996 / 0.8284
Qwen2.5-14BOpenBookQA86.1%1.1151.1830.91941.2822.44283.1%0.9016 / 0.8640
Qwen2.5-14BMedQA-US64.1%2.1792.3090.91292.6284.00029.8%0.8993 / 0.7693
Qwen2.5-14BMedMCQA54.2%2.6652.7740.91253.0804.00020.2%0.8978 / 0.6954
Qwen2.5-14BMMLU-Med78.7%1.3941.4860.91451.7453.63463.4%0.8992 / 0.8505
Llama-3.2-3BBioASQ yes/no71.0%1.4171.5120.92841.6582.00048.8%0.9014 / 0.8254
Llama-3.2-3BPubMedQA*52.5%2.0282.3030.92672.4943.0003.7%0.9057 / 0.7958
Qwen2.5-14BBioASQ yes/no73.9%1.3661.4570.93021.7282.00054.3%0.9012 / 0.8230
Qwen2.5-14BPubMedQA*39.2%2.3732.6350.95292.9703.0004.3%0.9107 / 0.6331

(Acc. is the top-answer accuracy of the 16-node mean. Llama and GPT-2 reports have 8 bits, the Qwen reports 24; Table 26 gives the other resolutions. Means over 100 calibration splits, 500 on OpenBookQA except for GPT-2, with standard errors of at most 0.4 points of coverage; fixed-set's low–high means below 0.90 are at most 1.6 standard errors below it.)

C. How the paper changed

Every new or changed passage is in blue, and the main text uses the tenth page for Table 1. Nothing was removed and no submitted number changed; submitted tables move up one number (submitted Table 13 is Table 14), and theorem, figure and appendix numbers are unchanged.

  • Section 1 (lines 135–137): small sets with a stronger scorer.
  • Section 2 (lines 175–179): planted text and changing identities.
  • Section 3 (lines 280–282): which rule to deploy.
  • Section 4 (lines 325–326, 355–357 and 386–389): the budget for independent failures; integrity; the merger.
  • Section 5 (lines 399, 451–458, 512 and 518–524) and the captions of Figures 3–4 (p. 9): the submitted attack named an injected instruction; stronger scorers, planted text and changing identities.
  • Section 6 (lines 533–537): the scope of the guarantees, the worst case and the exact shortcut checks.
  • Appendix S (pp. 55–59): a new theorem, a corollary and four propositions, with proofs. Appendix T (pp. 60–83): every new experiment, Tables 23–42.
  • Supplement: the code, score files and frozen record of every new study.

We are grateful for the reviewers' time and look forward to the discussion.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.