acceptodds
Under review as a conference paper at ICLR 2027

Beyond Answer Preservation: Scientific Evidence Minimization for Private LLM Reasoning

Abstract

Large language models can support scientific reasoning, but using external models often requires disclosing private scientific context. Reducing this disclosure while retaining useful answers is not sufficient for grounded reasoning: response preservation and downstream utility do not by themselves guarantee that the released context contains sufficient evidence to support the answer. We formulate **Scientific Evidence Minimization**, which seeks a policy-compliant, low-disclosure context that remains sufficient for grounded scientific reasoning. We introduce **CEDAR**, a verification-guided framework that iteratively evaluates an external answer, diagnoses missing evidence, and selectively expands the released context within the disclosure policy. We evaluate CEDAR across multiple scientific domains, reasoning tasks, and disclosure policies, explicitly separating answer correctness from evidence sufficiency to characterize how much scientific information is actually necessary for grounded reasoning.

Primary area:
alignment, fairness, safety, privacy, and societal considerations
Keywords:
Data Minimization, Scientific Evidence Sufficiency, Selective Disclosure, Grounded Reasoning, Privacy-Preserving LLMs

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_izWaOpenReview ↗
Rating
3
Confidence
2
Informativeness
3
Presentation
3
Soundness
2
Read review
Paper summary

This paper studies Scientific Evidence Minimization: limiting private scientific context shared with external LLMs while preserving sufficient evidence for grounded reasoning. It proposes CEDAR, a verification-guided framework that iteratively refines disclosed context and falls back to local execution when needed. Experiments across four scientific domains show improved grounded task success under strict disclosure constraints, although verifier reliability remains a limitation on open-ended tasks.

Critical strengths
  1. Well-motivated problem formulation backed by clear empirical evidence: The distinction between answer preservation and evidence sufficiency is compelling and well-substantiated by the controlled study, which shows that a sparse support-complete subset (21.6% of relations) achieves nearly the same GTS as the full context (0.750 vs. 0.764), whereas a budget-matched random subset drops to 0.333 GTS despite having similar raw answer accuracy (0.875 vs. 0.889).

  2. Rigorous evaluation and thorough system auditing: The paper separates the operational search-time verifier from the frozen post-hoc evaluator, and transparently audits both intermediate trajectory exposure (search-wide union) and the respective roles of verification gating versus iterative refinement.

Critical weaknesses
  1. Search-time verifier errors on open-ended reasoning tasks (Q1/Q3): In Table 2, CEDAR achieves lower GTS on open-ended tasks Q1 (0.425) and Q3 (0.259) compared to the single-pass One-Shot baseline (0.569 and 0.363, respectively). As shown in the Appendix G.1 audit (Table 16), the search-time verifier exhibits high false-acceptance rates on Q1 and Q3 (57.6% and 74.1% GTS disagreement with the frozen post-hoc evaluator, respectively), causing the search to terminate prematurely before sufficient evidence is disclosed.

  2. Assumption of pre-constructed structured graphs and single model-pair evaluation: CEDAR assumes that the private scientific workflow is already represented as a structured graph x=(S,R) with pre-defined exact/abstract representations and relation edges, leaving the impact of automated graph construction from raw unstructured scientific text unexplored. Furthermore, the experiments only evaluate a single local/external model configuration (openai/gpt-oss-120b locally and gpt-5.4-mini-2026-03-17 externally), leaving cross-model generalization unverified.

Questions for authors
  1. Clarification on improving Q1 and Q3 performance: Regarding CEDAR's lower GTS relative to One-Shot on Q1 and Q3 in Table 2, is the primary bottleneck the search-time verifier being overly permissive on open-ended answers (as indicated in Table 16), or the initial evidence contract selecting overly sparse seed nodes? Could tightening the verifier prompt directly mitigate this issue?

  2. Clarification on the local and external model pairing: The experiments deploy a 120B local model (openai/gpt-oss-120b) in the trusted environment while using gpt-5.4-mini-2026-03-17 as the external model. Could the authors briefly clarify the primary rationale behind this pairing (e.g., controlling API costs across iterative external calls)

  3. Future work on extending to raw scientific text: Since the state nodes, abstract representations, and evidence relations x=(S,R) are pre-constructed in the benchmark, which module of CEDAR (e.g., contract generation or failure diagnosis) do the authors expect to be most sensitive to extraction noise when applied to unannotated raw experimental records?

Reviewer_zDZtOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
1
Soundness
2
Read review
Paper summary

This paper studies how to minimize scientific information disclosed to an external LLM while preserving sufficient evidence for grounded answers. Motivated by the gap between answer preservation and evidential support, it formulates Scientific Evidence Minimization over scientific state and evidence relations. CEDAR combines query-conditioned selection, verifier-guided disclosure expansion, and trusted-local fallback. Evaluation covers 160 workflows across four domains, five question types, and two simulated disclosure policies, plus a separate controlled study. CEDAR achieves 0.578 Grounded Task Success (GTS), versus 0.428 for One-Shot and 0.308 for adapted Data Minimization. Its advantage over One-Shot occurs under the strict policy; their permissive-policy results are statistically indistinguishable.

Critical strengths

[Controlled Evidence for the Importance of Grounding]

  • The paper clearly distinguishes answer correctness from evidential support. Support-Complete and Random-Matched contexts achieve similar answer accuracy, 0.889 versus 0.875, but substantially different GTS, 0.750 versus 0.333, at the same relation budget (Table 1). Shuffled-binding and matched-deletion controls further support the importance of retaining the right evidence.

[Transparent Evaluation and Disclosure Audits]

  • Search-time verification is separated from post-hoc evaluation, gold annotations are excluded from inference, and workflow-level bootstrap resampling accounts for related questions. Appendix G additionally reports routing, verifier disagreement, and disclosure accumulated across intermediate candidates. These audits substantially improve the interpretability of the results.
Critical weaknesses

[Uneven Gains and Unclear Refinement Benefit]

  • CEDAR significantly underperforms Policy-Max overall (0.578 vs. 0.648), shows no significant improvement over One-Shot under the permissive policy, and is significantly worse than One-Shot on Q1 and Q3 (Tables 2, 12). Its strict-policy gains also involve local recovery unavailable to the main baselines. Gate/LOCAL achieves 0.706 GTS versus CEDAR’s 0.694 at (B=4) on the diagnostic subset (Table 3). Although CEDAR reduces disclosure and increases external completion, matched recovery and cost comparisons are needed to establish refinement’s incremental benefit.

[Large Disagreement in Grounding Verification]

  • Among verifier-accepted cases, the post-hoc evaluator rejects GTS for 57.6% of Q1 and 74.1% of Q3 cases (Table 16), revealing substantial disagreement over the stopping criterion. Because the evaluator is conservative relative to humans, these are not confirmed grounding-failure rates. Targeted human adjudication is needed to establish the reliability of accepted releases.
Questions for authors
  1. Are questions explicitly declassified or sanitized before external execution? Appendix A.5 labels [110] as protected, but the Table 8 Q5 question contains it verbatim. Does exposure accounting include the transmitted question? This determines the scope of “zero protected-exact exposure.”
  2. Can human-validation results be reported by method, alongside adjudication of verifier-accepted but evaluator-rejected CEDAR cases? The evaluator’s GTS recall is 0.720 against humans (Table 10); this analysis would clarify verifier reliability and the robustness of method rankings.
  3. What independent checks validate required support sets, abstraction text, and protection assignments? Can the complete annotations, component prompts, and specificity weights be provided to support reproduction?
Reviewer_JTq8OpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper formulates Scientific Evidence Minimization, aiming to minimize scientific information disclosed to external LLMs while retaining sufficient evidence for grounded reasoning. The authors propose CEDAR, a verification-guided framework that iteratively expands policy-compliant disclosures and falls back to trusted-local execution when necessary. Experiments cover 160 scientific workflows, four domains, five question types, and two disclosure policies. CEDAR achieves 0.578 overall Grounded Task Success (GTS), outperforming One-Shot (0.428) and Data Minimization (0.308). A controlled study shows that evidential relations and their correct bindings can matter beyond answer correctness alone.

Critical strengths
  1. The paper distinguishes correct answers from answers supported by the disclosed context. With matched relation budgets, support-complete and random subsets achieve similar answer accuracy (0.889 vs. 0.875) but substantially different GTS (0.750 vs. 0.333).
  2. CEDAR respects disclosure constraints, supports trusted-local fallback, and accounts for cumulative disclosure across the search trajectory rather than only the final release.
Critical weaknesses
  1. The benefit of iterative refinement is confounded by local fallback. The baselines lack CEDAR's trusted-local fallback, while CEDAR's end-to-end GTS includes local execution using the full private context. A full-benchmark comparison with an equivalently equipped One-Shot baseline is necessary to isolate the contribution of iterative refinement.
  2. The search-time verifier is unreliable on key reasoning tasks. Appendix G.1 reports GTS disagreement rates of 57.6% for Q1 and 74.1% for Q3 among verifier-accepted external cases. This weakens confidence in the verification mechanism central to CEDAR. Human validation also reports only 78.3% evaluator–human agreement on GTS, leaving uncertainty about the robustness of some conclusions.
  3. Does the use of a local agent or other intelligent gating mechanism to control the information disclosed to the external model compromise its task-solving performance? The authors should quantify the utility trade-off between selective disclosure and providing the external model with the full scientific context.
  4. Could progressively revealing small amounts of information lead the external model to generate hallucinated or premature answers based on incomplete evidence? Furthermore, how does this iterative process affect task completion time, token consumption, and overall computational cost compared with directly providing the full context in a single request?
  5. How is the reliability of the local agent responsible for assessing evidence sufficiency and controlling disclosure ensured? Given that this component determines whether additional information should be released or computation should remain local, errors in its judgments could compromise both answer quality and the effectiveness of the disclosure policy. The authors should provide stronger evidence of its reliability, particularly for complex scientific reasoning tasks.
Questions for authors

see weaknesses

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.