- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper addresses disconnects among patient evidence, clinical decisions, executed actions, and final documentation in long-horizon EHR tasks. The authors propose CLINCH, a clinical agent harness that requires no fine-tuning of the main model. A Clinical Workspace maintains task, patient, and decision state. Clinically-Aware Context Management, Decision-Anchored Knowledge Integration, and Evidence-Grounded Task Verification support evidence recovery, patient-specific knowledge application, and closed-loop checks against runtime-recorded actions and results.
The experiments cover three base models, the full PhysicianBench task set, and a fixed 60-task subset of MedAgentBench. Mean single-run task success increases by 4.7–21.0 and 11.1–19.4 percentage points, respectively. The authors provide three runs per main-experiment configuration, task-level confidence intervals, a token-budget control, grouped ablations, and detailed case studies.
Critical strengths
S1. The research question is closely aligned with practical EHR workflows, and the system design is intuitive.
Distinguishing unretrieved information from negative findings and intended actions from completed operations, while checking documentation against execution, addresses important clinical-agent failure modes. The Clinical Workspace and runtime log incorporate these distinctions into ongoing execution and have a clear practical motivation.
S2. The full system improves performance across multiple models and two environments, supported by substantial analysis.
The main experiments use three runs. The authors report task-level confidence intervals, checkpoint failure categories, token usage, and a budget control. Case studies illustrate how evidence recovery, decisions, and execution connect. The selection of the MedAgentBench subset and the mechanisms enabled in that environment are also described explicitly.
Critical weaknesses
W1. The incremental value of CLINCH's specific design remains unclear. Comparisons focus on the original agents, extended execution budgets, and mechanism removal. Direct comparisons with related agents that support memory, history recovery, or review are missing, making it difficult to distinguish the benefits of standard workflow support from those of CLINCH's specific design.
W2. Statistical and mechanistic evidence for the components remains limited. Each ablation is run once with DeepSeek, while the full system uses a three-run mean; FDR and closed-loop checks are removed together. Appendix B reports ablation token usage, but finer attribution and cross-model ablations are missing. Appendix D provides cases and text excerpts, but does not show a complete input–output comparison for each mechanism.
Questions for authors
Q1. How much does CLINCH improve over other memory or review augmented agents? Please add at least one related agent baseline under the same base model, tool access, information sources, and comparable budgets, and compare task success, checkpoint outcomes, and token usage.
Q2. Are the component contributions stable, and do they vary across base models? Please repeat each ablation three times and report means, variability, and confidence intervals for paired gains. Corresponding ablations on the other models could use the same predefined subset covering the main task types. The full framework has already been tested on three models; this experiment would assess the components' cross-model stability.
Q3. Do the performance gains of each mechanism justify its additional token usage? Please present the token usage already reported in Appendix B alongside Table 4, separate FDR-call costs from subsequent revision costs, and refine the grouped ablations. Repeated runs and budget controls would help establish the marginal benefits and resource efficiency of each mechanism group.
Q4. Could the existing cases in Appendix D show more directly how each mechanism changes the information available to the model and its actions? Please provide comparisons of original tool outputs and their compressed versions, patient facts and decision records, and FDR inputs, feedback, and before-and-after text and action changes.
2 comments
We have added the related-agent comparison requested in Q1. Our RetroSum reproduction reaches 30.0% Pass@1 and 78.4% CP with DeepSeek-V4-Pro, and 26.0% and 78.4% with GLM-5.2. CLINCH reaches 34.3% and 80.6%, and 45.7% and 84.6%, respectively. Token usage and the one-run versus three-run reporting convention are included in the General Response update.
We thank the reviewer for the precise and actionable questions.
Q1: Related agent baselines. We compared CLINCH with two generic agents that use the same base model, tools, and a similar token budget (details in General Response 1). Generic continuation reaches 22.0% Pass@1 and 66.9% CP, and a generic critic/reflection loop reaches 29.0% and 70.7%, compared with 34.3% and 80.6% for CLINCH. CLINCH's checkpoint pass rate exceeds the loop's by 9.9 points (95% CI [4.8, 15.2]).
Q2: Component contributions. Each ablation group targets one failure mode, and removing it raises the corresponding failure category in Table 4. Without the outline and clinical compression, retrieval failures rise from 7.7 to 14.0 per 100 tasks. Without decision-anchored retrieval, reasoning failures rise from 64.0 to 73.0. Without FDR, documentation failures rise from 37.3 to 42.0. The new no-knowledge control adds a further comparison at the component level: with knowledge cards, decision-time retrieval, and medical review all disabled, CLINCH still reaches 33.0% Pass@1 and 79.1% CP (details in General Response 2).
Q3: Token usage per mechanism. We will add token usage to Table 4 for each configuration: 2,299K for the full system, 1,679K without the outline and compression, 2,207K without decision-anchored retrieval, and 1,142K without FDR. FDR calls themselves account for 1.3% of tokens. The difference between the full system and the FDR ablation comes from the retrieval, execution, and revision work that FDR's feedback triggers.
Q4: Mechanism-level examples. We will add a section to Appendix D that traces each mechanism with verbatim excerpts from the two cases. For context management, it shows a 212K-character laboratory result stored in full, with a 20K-character visible window and an explicit marker that more content remains. For decision-anchored knowledge, it shows the query (patient context, pending decision, candidate plan), the retrieved evidence and its stated limitations, the resulting decision record with interpretation and change conditions, and the status change from planned to executed after the medication hold was confirmed by read-back. For FDR, it shows the draft's claim that the patient was counseled, FDR's finding that the operation ledger contained no matching communication, the communication the agent then created, the revised note citing that record, and the second review accepting it.