acceptodds
Under review as a conference paper at ICLR 2027

Guidelines as Control Logic for Multimodal Sequential Clinical Decision-Making

Abstract

Real-world clinical decision-making is dynamic, requiring clinicians to continuously acquire patient information and make sequential decisions according to Clinical Practice Guidelines (CPGs). Existing approaches typically incorporate CPG knowledge in two forms: external or parametric. External approaches commonly rely on retrieval-augmented generation (RAG), which injects large amounts of retrieved CPGs into the model context, following a “Guideline as Context” paradigm. Parametric approaches encode guideline knowledge into model parameters, yet still fail to transform guidelines into explicit, traceable, and easily updatable decision control logic. To address these limitations, we propose CPGAgent, a guideline-grounded framework that introduces a “Guidelines as Control Logic” paradigm for multimodal sequential clinical decision-making. Our main idea is to encode CPGs as latent guideline memory slots and dynamically activate relevant slots according to evolving patient states, enabling guideline-guided trajectory planning. For evaluating dynamic decision trajectories, we introduce CPGTrajBench, a multimodal executable benchmark in which evidence is progressively revealed through agent actions. We further develop a guideline-grounded evaluation framework with trajectory-planning rubrics, assessing guideline-consistent planning across the entire clinical trajectory. Experiments show that CPGAgent achieves the best trajectory performance across three cancer types, matching the strongest baseline on one cohort and outperforming it by 9.6 and 9.2 points on the other two. Anonymized code is provided in the reproducibility statement.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
Medical Agents, Clinical Practice Guidelines, Agentic Benchmarking

Reviews

Among:
Shape:
1234this paper, Top 32%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.50 over 4 reviews, above 68% of the others.
Reviewer_sZSUOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper presents a clinical AI system that uses clinical guidelines to guide decisions as patient information changes over time. It stores guideline knowledge in memory and selects the most relevant parts for each step of the decision process. The paper also introduces a benchmark for testing whether clinical AI systems make decisions that follow guidelines throughout a patient care process. The method performs better than most baselines on the cancer datasets they evaluate.

Critical strengths

1- The paper introduces a clear conceptual shift by treating clinical guidelines as state dependent control logic for sequential decision making. 2- The paper presents a novel evaluation framework by progressively revealing multimodal evidence and assessing complete clinical trajectories.

Critical weaknesses

1- The paper addresses a timely problem, and the different components of the proposed framework are combined in a coherent and technically sound way. However, the overall method feels more like a combination of existing techniques applied to a new problem than a fundamentally novel methodological contribution.

2- The evaluated cohorts are relatively small. For datasets of this size, reporting only point estimates is insufficient, and I would expect mean and standard deviation values across repeated runs or resampling.

3- A major concern is the reliance on LLMs both to construct the ground truth trajectories and to evaluate model performance. This creates a potential source of evaluator bias, especially in cases where the same or closely related models are involved in both prediction and evaluation.

4- The train, validation, and test separation is not described clearly enough. Given the use of learned guideline representations and benchmark construction procedures, the paper should make the data separation and potential sources of leakage explicit.

Questions for authors

1- Given the relatively small cohort sizes, could you report variability across runs or resampling, such as mean and standard deviation, rather than only single point estimates?

2- How were the ground truth trajectories generated and validated, and how much of this process relies on LLM generated outputs?

3- Could you clarify whether the same or closely related LLMs are used both for generating predictions and evaluating them? If so, how do you mitigate potential evaluator bias or self preference?

4- Could you provide a clearer description of the train, validation, and test splits, including whether any patient, guideline, or trajectory information can overlap across these partitions?

5 comments
AuthorsOpenReview ↗

Response to Reviewer sZSU: 1) Novelty

1) Novelty

Critical Weaknesses:

1- The paper addresses a timely problem, and the different components of the proposed framework are combined in a coherent and technically sound way. However, the overall method feels more like a combination of existing techniques applied to a new problem than a fundamentally novel methodological contribution.

Response:

Thank you for this suggestion. While our framework builds upon established techniques, its novelty lies in their integration for state-aware guideline control, particularly through Dynamic Anchor Attention (DAA). Unlike Q-Former, which uses fixed learned query embeddings, our DAA dynamically selects the most decision-relevant guideline memory as the anchor based on the current patient state. Then, we enable State-aware knowledge integration through two stages. First, the patient state guides the extraction of decision-relevant information from the anchor guideline memory, effectively querying what matters for the current patient state. The resulting state-aware representation then attends to other selected guideline memories to identify complementary information from additional sections. This allows the Planner to focus on key clinical decisions while incorporating relevant cross-section knowledge.

AuthorsOpenReview ↗

Response to Reviewer sZSU: 2) Cohort Size and Statistical Variability

2) Cohort Size and Statistical Variability

Critical Weaknesses and Question:

2- The evaluated cohorts are relatively small. For datasets of this size, reporting only point estimates is insufficient, and I would expect mean and standard deviation values across repeated runs or resampling.

1- Given the relatively small cohort sizes, could you report variability across runs or resampling, such as mean and standard deviation, rather than only single point estimates?

Response:

We thank the reviewer for this suggestion. We first note that CPGTrajBench is comparable in scale to representative case-based multimodal clinical benchmarks, as summarized below.

BenchmarkCasesClinical ModalitiesInteraction
AgentClinic-NEJM [1]120Text, EHR, PathologyDialogue/Simulation
MTBBench [2]66 (26 multimodal + 40 longitudinal)Text, Pathology, MolecularExecutable Environment
CPGTrajBench (Ours)109Text, EHR, Radiology, Pathology, MolecularExecutable Environment

Unlike conventional static QA benchmarks, CPGTrajBench integrates five clinical modalities across three cancer types, requiring progressive evidence acquisition and multi-step clinical planning. Each reference trajectory is further validated by clinical experts. The construction of this multimodal, interactive, and guideline-grounded benchmark is itself a core contribution of our work.

Nevertheless, we acknowledge that 109 cases remain relatively limited for statistical estimation and generalizability. To address this concern, we have updated Table 2 to report mean ± bootstrap SD from 10,000 within-cohort patient-level resamples (seed 42).

Specifically, CPGAgent achieves 47.957 ± 1.808 on LUAD (30 cases), 49.663 ± 0.788 on UCEC (40 cases), and 55.519 ± 1.622 on NPC (39 cases), with an overall score of 51.289 ± 0.813. The same bootstrap procedure is applied to all baselines.

References

[1] Schmidgall, Samuel, et al. “AgentClinic: A Multimodal Benchmark for Tool-Using Clinical AI Agents.” npj Digital Medicine, vol. 9, 2026, article no. 499, doi:10.1038/s41746-026-02674-7.

[2] Vasilev, Kiril, et al. “MTBBench: A Multimodal Sequential Clinical Decision-Making Benchmark in Oncology.” Advances in Neural Information Processing Systems 38, Datasets and Benchmarks Track, 2025, pp. 12217–12246, doi:10.52202/085713-0368.

AuthorsOpenReview ↗

Response to Reviewer sZSU: 3) Trajectories Generation

3) Trajectories Generation

Questions For Authors:

2- How were the ground truth trajectories generated and validated, and how much of this process relies on LLM generated outputs?

Response:

As detailed in Section 4.2 and Appendices G–H, reference trajectories are constructed through iterative LLM generation, step-level verification, and clinical expert review.

First, LLM-based construction and verification. At each step, a DeepSeek-V4-Pro Generator proposes guideline-grounded actions and state transitions based on the current patient state. A separately invoked DeepSeek-V4-Pro Verifier checks these proposals against the complete patient record, identifies inconsistencies, and supports state correction before the next step.

Second, clinical expert validation. As detailed in the newly added Appendix H, clinicians review complete trajectories for clinical appropriateness, guideline consistency, and cross-step coherence. Localized errors are corrected, while trajectories with major unresolved errors are rejected. Of 135 trajectories reviewed, 109 were accepted and 26 failed clinical expert review (Table 10).

Thus, while trajectory generation and initial verification rely on LLMs, the final reference trajectories undergo independent clinical expert validation instead of relying solely on LLM outputs.

Reviewer_Gc2DOpenReview ↗
Rating
3(+1)
Confidence
4
Informativeness
1
Presentation
2(+1)
Soundness
2
Read review
Paper summary

This paper proposes CPGAgent, a guideline-grounded framework for multimodal sequential clinical decision-making. It introduces a "Guidelines as Control Logic" paradigm that dynamically activates latent guideline memories based on evolving patient states. The framework combines a Guideline Planner with state-conditioned retrieval, Soft Gate, and Dynamic Anchor Attention, alongside a Medical Executor for multimodal evidence acquisition. The authors also introduce CPGTrajBench, a benchmark that evaluates guideline adherence across clinical decision trajectories. Experiments on three cancer cohorts (109 cases) show that CPGAgent outperforms existing baselines, with ablation studies supporting the effectiveness of its memory selection and fusion mechanisms.

Critical strengths

This paper tackles a clinically important problem: keeping sequential decisions guideline-consistent as the patient state evolves. Its modular design separates guideline planning from multimodal execution, linked by a structured, auditable Patient State Machine. CPGTrajBench is a valuable contribution. It reveals multimodal evidence progressively through agent actions and scores trajectories with required, acceptable, conditional, and unsafe action sets, which accommodate multiple valid clinical paths. CPGAgent ranks first on all three cohorts, and its advantage grows with case complexity.

Critical weaknesses
  1. Does updating the clinical guidelines require retraining the Memory Encoder and Planner Decoder? The paper provides no experiments on guideline updates to support the claim that the framework is "easily updatable."

  2. There may be potential data leakage between training and evaluation. If a patient's ground-truth trajectory in CPGTrajBench was used to train the Planner Decoder in Stage 2, the model may have already learned the correct actions, even when patient information is revealed incrementally during testing. Could the authors clarify the data splitting strategy?

  3. Why does Executor Only perform worse than Codex GPT-5.6-sol despite using the same underlying model? Its All Cohorts score is 34.511 versus 39.044, and its NPC score (28.693) is the lowest in the table.

  4. Since Rubric Trajectories are generated by an LLM, how does the framework handle errors in the initial trajectory? Is there a mechanism for correction, or do errors propagate through subsequent planning steps?

Questions for authors

I suggest that the authors improve the overall logical flow and coherence of the manuscript, as some sections lack sufficient clarity and explanation.

4 comments
AuthorsOpenReview ↗

Response to Reviewer Gc2D: 1) Updatability

1) Updatability

Critical Weaknesses:

  1. Does updating the clinical guidelines require retraining the Memory Encoder and Planner Decoder? The paper provides no experiments on guideline updates to support the claim that the framework is "easily updatable."

response:

We thank the reviewer for raising this important question. To evaluate guideline updatability, we conducted a cross-version guideline transfer experiment. The original model was trained using NSCLC 2010, SCLC 2010, CSCO Endometrial Cancer 2023, and CSCO Nasopharyngeal Carcinoma 2022. We then directly re-encoded four updated guidelines using the frozen Memory Encoder, without additional annotation or retraining of any model components.

Guideline UpdateTrajectory Score
NSCLC, SCLC 2010 → 201547.957 → 49.006
CSCO UCEC 2023 → 202549.663 → 44.821
CSCO NPC 2022 → 202555.519 → 55.496

The results demonstrate that CPGAgent can incorporate newer guideline versions without retraining the Memory Encoder, Soft Gate, or Planner Decoder, while maintaining competitive trajectory performance. The corresponding experimental results have been incorporated into Section 5.2 (Ablation Experiments and Trajectory Evaluation) of the revised manuscript.

Importantly, our Memory Encoder and Planner are not specialized to particular guideline versions, but are trained to generalize across clinical guidelines. Therefore, CPGAgent can incorporate updated guidelines simply by re-encoding the revised text into latent memory slots, without additional annotation or model retraining.

AuthorsOpenReview ↗

Response to Reviewer Gc2D: 2) Data Leakage

2) Data Leakage

Critical Weaknesses:

  1. There may be potential data leakage between training and evaluation. If a patient's ground-truth trajectory in CPGTrajBench was used to train the Planner Decoder in Stage 2, the model may have already learned the correct actions, even when patient information is revealed incrementally during testing. Could the authors clarify the data splitting strategy?

Response:

We thank the reviewer for raising this concern. We clarify that Planner training and CPGTrajBench evaluation follow two entirely independent data construction pipelines, with no patient-level overlap.

As detailed in the newly added Appendix A (Guideline-Derived Planner Training Data Construction), Planner training data are constructed exclusively from clinical guideline documents. Specifically, AE, RETRIEVE, and CONTINUE employ rule-based supervision derived from guideline passages, while PLAN uses GPT-5.6-sol to generate synthetic patient states and corresponding guideline-grounded clinical plans. No real patient records, benchmark cases, or reference trajectories are used in Planner training.

In contrast, CPGTrajBench is constructed independently from 109 real clinical cases, using progressively revealed multimodal evidence and guideline-grounded trajectory generation and verification, as described in Section 4.2. Therefore, training–evaluation separation is ensured at the data-source level.

AuthorsOpenReview ↗

Response to Reviewer Gc2D: 3) Agent Harness Differences

3) Agent Harness Differences

Critical Weaknesses:

  1. Why does Executor Only perform worse than Codex GPT-5.6-sol despite using the same underlying model? Its All Cohorts score is 34.511 versus 39.044, and its NPC score (28.693) is the lowest in the table.

response:

Although Executor Only and Codex GPT-5.6-sol use the same underlying LLM, they operate under substantially different agent harnesses. Executor Only is our lightweight clinical execution agent, primarily limited to predefined medical skills, such as WSI slicing, CT slicing, and clinical report reading, without an explicit planning mechanism. In contrast, Codex is a mature, commercially developed general-purpose agent system, equipped with a sophisticated agent harness that supports autonomous file exploration, code execution, and flexible tool orchestration. Therefore, Codex's stronger performance is understandable despite using the same backbone model.

AuthorsOpenReview ↗

Response to Reviewer Gc2D: 4) Trajectory Error Handling

4) Trajectory Error Handling

Critical Weaknesses:

Since Rubric Trajectories are generated by an LLM, how does the framework handle errors in the initial trajectory? Is there a mechanism for correction, or do errors propagate through subsequent planning steps?

Response:

Our framework incorporates two complementary mechanisms to mitigate error propagation: step-level LLM verification and case-level clinical expert review.

First, step-level verification enables iterative correction. At each step, a separately invoked Verifier checks generated actions and state transitions against the relevant guidelines and complete patient record, identifies inconsistencies, and supports patient-state correction before subsequent trajectory generation (Section 4.2; Appendix G).

Second, clinical expert review provides additional oversight. As detailed in the newly added Appendix H, clinicians assess the clinical appropriateness, guideline consistency, and cross-step coherence of complete trajectories. Of the 135 trajectories reviewed, 109 were accepted and 26 failed clinical expert review (Table 10).

Reviewer_SVraOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes CPGAgent, which encodes clinical practice guidelines (CPGs) as latent memory and facilitates sequential planning through patient-state-conditioned retrieval, a "soft gate" mechanism, and dynamic anchor attention. The system comprises a guideline planner, a medical executor, and a shared patient state. Additionally, the authors introduce CPGTrajBench—a benchmark that reveals multimodal evidence in a step-by-step manner—to evaluate decision-making based on guideline-aligned trajectory scoring. Experiments across three types of cancer demonstrate that the system outperforms the baselines.

Critical strengths
  1. The problem addressed is of practical significance. Shifting from static medical Q&A to a framework involving evidence acquisition and sequential decision-making under conditions of incomplete information aligns more closely with clinical workflows. The focus on action preconditions, unresolved information, and adherence to clinical guidelines holds substantial research value.
  2. The system design is well-structured. It features a clear division of labor between the planner and the executor, utilizing a structured representation of the patient's state to integrate guideline retrieval, tool invocation, and evidence updates. This architecture facilitates the management of multimodal information and enables the tracking of state transitions throughout the decision-making process.
  3. The experiments provide promising preliminary evidence. The system demonstrates significant performance improvements on UCEC and NPC datasets, as well as in overall scoring. Furthermore, ablation studies and complexity analysis substantiate the efficacy of guideline-augmented planning, with the results largely consistent with the study's initial motivation.
Critical weaknesses
  1. While the main experiments support the effectiveness of the complete system—with module ablation studies providing preliminary evidence of individual contributions—the absence of a standard RAG planner that matches the training and inference budgets limits our ability to assess the distinct advantages of the proposed memory architecture.
  2. Certain attributes of the "control logic" have yet to be fully verified. While neural memory mechanisms can facilitate a generalized form of control, current experiments primarily demonstrate the efficacy of guideline-conditioned planning. They do not directly substantiate stronger claims—such as the logic being "explicit, traceable, and easily updateable." It is recommended to either conduct additional experiments involving the modification or deletion of guidelines, or to adjust the relevant claims to be more measured and precise.
  3. The description regarding data construction and the separation of training data is insufficient. While the text explicitly states that the LUAD data originates from TCGA, it fails to adequately specify the sources for UCEC and NPC, nor does it explain how the clinical cases were converted into benchmark tasks. Furthermore, details concerning the scale, partitioning, and deduplication of the training datasets are required to assess whether they are truly independent of the cases used for evaluation.
Questions for authors
  1. Controlled RAG comparison: Could you compare CPGAgent with a conventional text-RAG planner using the same planner backbone, guideline corpus, patient-state inputs, executor, and interaction budget? This would help isolate the contribution of the proposed latent-memory architecture.
  2. Control-logic claims: What evidence supports the claim that latent guideline memories provide explicit, traceable, and easily updateable decision logic? Could guideline-editing experiments demonstrate targeted behavioral changes, stability of unrelated decisions, and the cost of updating the system?
  3. Benchmark construction: Could you clarify the data sources and case-selection criteria for UCEC and NPC, and explain how clinical records are converted into initial patient states and progressively revealed multimodal evidence?
3 comments
AuthorsOpenReview ↗

Response to Reviewer SVra: 1) Controlled RAG comparison

1) Controlled RAG comparison

Critical Weaknesses and Questions:

  1. While the main experiments support the effectiveness of the complete system—with module ablation studies providing preliminary evidence of individual contributions—the absence of a standard RAG planner that matches the training and inference budgets limits our ability to assess the distinct advantages of the proposed memory architecture.
  2. Controlled RAG comparison: Could you compare CPGAgent with a conventional text-RAG planner using the same planner backbone, guideline corpus, patient-state inputs, executor, and interaction budget? This would help isolate the contribution of the proposed latent-memory architecture.

response:

Thank you for this suggestion. We conducted two additional controlled experiments: GPT-5.6-sol + Text-RAG and Qwen3.5-9B Planner + Text-RAG, using the same dual-agent architecture, guideline corpus, patient-state inputs, Medical Executor, and interaction budget. Both Text-RAG baselines reuse CPGAgent's memory ranking mechanism, replacing the selected latent memories with their original guideline text. This ensures identical retrieval and selection, isolating the contribution of latent memory representation. The corresponding experimental results have been incorporated into Section 5.1 (Main Results) of the revised manuscript.

MethodLUADUCECNPCOverall
GPT-5.6-sol + Text-RAG45.03543.85350.61246.597
Qwen Planner + Text-RAG47.56045.54851.92648.384
CPGAgent47.95749.66355.51951.289

CPGAgent outperforms GPT + Text-RAG by 4.692 points and the same-backbone Qwen Planner + Text-RAG by 2.905 points overall, supporting the contribution of our guideline memory architecture beyond the Planner backbone. We attribute these gains to two key designs:

(1) Compact guideline representation. Retrieved guideline text alone averages 12,013 characters (15,002 tokens), introducing substantial contextual overhead. Our Memory Encoder compresses guidelines into compact latent slots, reducing the need to repeatedly process lengthy text.

(2) State-aware knowledge integration. We enable State-aware knowledge integration through two stages. First, the patient state guides the extraction of decision-relevant information from the anchor guideline memory, effectively querying what matters for the current patient. The resulting state-aware representation then attends to other selected guideline memories to identify complementary information from additional sections. This allows the Planner to focus on key clinical decisions while incorporating relevant cross-section knowledge.

AuthorsOpenReview ↗

Response to Reviewer SVra: 2) Guideline Update Experiments

2) Guideline Update Experiments

Critical Weaknesses and Questions:

  1. Certain attributes of the "control logic" have yet to be fully verified. While neural memory mechanisms can facilitate a generalized form of control, current experiments primarily demonstrate the efficacy of guideline-conditioned planning. They do not directly substantiate stronger claims—such as the logic being "explicit, traceable, and easily updateable." It is recommended to either conduct additional experiments involving the modification or deletion of guidelines, or to adjust the relevant claims to be more measured and precise.
  2. Control-logic claims: What evidence supports the claim that latent guideline memories provide explicit, traceable, and easily updateable decision logic? Could guideline-editing experiments demonstrate targeted behavioral changes, stability of unrelated decisions, and the cost of updating the system?

response:

To evaluate guideline updatability, we conducted a cross-version guideline transfer experiment. The original model was trained using NSCLC 2010, SCLC 2010, CSCO Endometrial Cancer 2023, and CSCO Nasopharyngeal Carcinoma 2022. We then directly re-encoded four updated guidelines using the frozen Memory Encoder, without additional annotation or retraining of any model components.

Guideline UpdateTrajectory Score
NSCLC, SCLC 2010 → 201547.957 → 49.006
CSCO UCEC 2023 → 202549.663 → 44.821
CSCO NPC 2022 → 202555.519 → 55.496

The results demonstrate that CPGAgent can incorporate newer guideline versions without retraining the Memory Encoder, Soft Gate, or Planner Decoder, while maintaining competitive trajectory performance. The corresponding experimental results have been incorporated into Section 5.2 (Ablation Experiments and Trajectory Evaluation) of the revised manuscript.

Importantly, our Memory Encoder and Planner are not specialized to particular guideline versions, but are trained to generalize across clinical guidelines. Therefore, CPGAgent can incorporate updated guidelines simply by re-encoding the revised text into latent memory slots, without additional annotation or model retraining.

AuthorsOpenReview ↗

Response to Reviewer SVra: 3) Benchmark Construction

3) Benchmark Construction

Critical Weaknesses and Question:

The description regarding data construction and the separation of training data is insufficient. While the text explicitly states that the LUAD data originates from TCGA, it fails to adequately specify the sources for UCEC and NPC, nor does it explain how the clinical cases were converted into benchmark tasks. Furthermore, details concerning the scale, partitioning, and deduplication of the training datasets are required to assess whether they are truly independent of the cases used for evaluation.

Benchmark construction: Could you clarify the data sources and case-selection criteria for UCEC and NPC, and explain how clinical records are converted into initial patient states and progressively revealed multimodal evidence?

Response:

We thank the reviewer for highlighting the need to clarify benchmark provenance and construction. We have added Appendix H (Clinical Case Selection and Expert Review) to detail the data sources, case-selection criteria, and quality-control process.

First, regarding data sources and case selection, CPGTrajBench includes LUAD cases from TCGA and UCEC and NPC cases from two independent clinical centers. To preserve anonymity during double-blind review, we withhold the specific institutional names and will disclose them in the final manuscript upon acceptance. We apply a two-stage screening process: initial screening for disease-specific eligibility, clinical record consistency, and required multimodal data, followed by clinical expert review of reference trajectories. As reported in Appendix H (Table 10), 1,091 initial candidates were screened, 135 passed the initial screening, and 109 were retained after expert review (30 LUAD, 40 UCEC, and 39 NPC).

Second, regarding benchmark construction, we have clarified the process in Section 4.2 and Appendices G–H. As detailed in Appendix H, each cohort has a predefined initial decision point ($ T_0 $): LUAD and UCEC begin after surgery and pathological confirmation but before adjuvant therapy, while NPC begins after diagnostic biopsy and preliminary staging but before anticancer treatment. During offline construction, a guideline-grounded Generator proposes actions and state transitions, which are verified against complete patient records before clinical expert review (Section 4.2; Appendix G). During online evaluation, agents start with partial patient information and progressively acquire multimodal evidence through clinical tools.

Reviewer_34VEOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

This paper tries to address how LLM agents can follow clinical practice guidelines in sequential clinical decision-making, where patient evidence is revealed step by step. The authors propose CPGAgent, in which a trained Qwen3.5-9B planner and a GPT-5.6-sol executor communicate through a structured patient state. Guidelines are encoded offline into latent memory slots, and at each step a learned gate selects a few of them and an anchor-based attention fuses them into the planner's input. The authors also introduce CPGTrajBench, 109 oncology cases across three cancer types, with LLM-generated and expert-reviewed trajectory rubrics and an alignment-based trajectory score. CPGAgent ranks first on all three cohorts, on par with the strongest baseline on LUAD and about 9 points ahead on UCEC and NPC.

Critical strengths

I think the paper asks the right question. Whether an agent stays consistent with guidelines over a whole trajectory, as evidence becomes available, is much closer to clinical practice than single-question accuracy, and an executable multimodal benchmark in this direction would be valuable to the community. The decomposition of the system is also sensible: guideline-grounded planning is kept apart from evidence acquisition, and the patient state gives an auditable record of what has been established and what is still missing. As a method, the approach is plausible and could be built upon.

Critical weaknesses

My main concern is that the experiments do not isolate what the paper claims to contribute. The paper presents guidelines as explicit, traceable and easily updatable control logic, but in the method they become latent vectors from a trained encoder, selected by a learned gate and fused into a 9B decoder, which is closer to a learned form of memory-augmented retrieval. The most natural alternative, a strong model such as GPT-5.6-sol or Claude-Opus-5 acting as the planner over retrieved guideline text, is not evaluated, and the paper does not explain why a trained 9B planner is needed. This omission is notable because the benchmark's reference trajectories are themselves produced by retrieving guidelines and letting an LLM propose the next actions (p. 6). The two RAG baselines are generic single-agent systems and do not answer this question. The ablation points in the same direction: with random memory selection the system still scores above every baseline (Tables 2-3), which suggests that much of the gain comes from the trained planner rather than from guideline control. The updatability claim is likewise not supported. No update procedure is described or tested, and since the planner and the gate are trained on trajectories and provenance tied to the current guidelines, incorporating an important new therapy may require additional annotation or retraining, which has not been investigated and would be the very limitation the paper attributes to parametric approaches.

I am also not yet convinced that the evaluation can carry the comparison. The reference rubrics are generated and verified by the same LLM, and trajectories are scored by GPT-5.6-sol, which also serves as the backbone of the executor and of most baselines. Clinician review is mentioned only briefly, agreement between the score and clinical judgment is not reported, and the paper describes differently what the rubric verifier can see (p. 6 vs. pp. 18-19). The source of the planner's training trajectories, and their separation from the 109 evaluation cases, is not described. With 109 cases, a single run and only the authors' own benchmark, the evidence is thin, particularly on LUAD, where the margin is 0.021.

Beyond these two points, the technical components (compression of text into latent slots, retrieve-then-rerank, Q-Former-style fusion, staged supervised and preference training) are largely established, and the paper does not discuss related work on context compression. The NSCLC guideline used is the 2010 version, which differs substantially from current practice, without an explanation of how versions are matched to cases. A number of details needed for reproduction, including the scoring weights, the baseline configurations and the later training stages, are not given.

Questions for authors

How does a strong model acting as the planner over retrieved guideline text, in the same two-agent setup and budget, compare with CPGAgent? And the same trained planner reading guideline text instead of latent memories? Where do the planner's training trajectories and provenance annotations come from, and are they separated from the evaluation cases and from the rubric-generation pipeline? What would it take for CPGAgent to follow a new guideline version that introduces an important new therapy: re-encoding only, or new annotation and retraining?

5 comments
AuthorsOpenReview ↗

Response to Reviewer 34VE: 1) Latent Memory Effectiveness

1) Latent Memory Effectiveness

Critical Weaknesses and Questions:

My main concern is that the experiments do not isolate what the paper claims to contribute. The paper presents guidelines as explicit, traceable and easily updatable control logic, but in the method they become latent vectors from a trained encoder, selected by a learned gate and fused into a 9B decoder, which is closer to a learned form of memory-augmented retrieval. The most natural alternative, a strong model such as GPT-5.6-sol or Claude-Opus-5 acting as the planner over retrieved guideline text, is not evaluated, and the paper does not explain why a trained 9B planner is needed. This omission is notable because the benchmark's reference trajectories are themselves produced by retrieving guidelines and letting an LLM propose the next actions (p. 6). The two RAG baselines are generic single-agent systems and do not answer this question. The ablation points in the same direction: with random memory selection the system still scores above every baseline (Tables 2-3), which suggests that much of the gain comes from the trained planner rather than from guideline control.

How does a strong model acting as the planner over retrieved guideline text, in the same two-agent setup and budget, compare with CPGAgent? And the same trained planner reading guideline text instead of latent memories?

response:

Thank you for this suggestion. We conducted two additional controlled experiments: GPT-5.6-sol + Text-RAG and Qwen3.5-9B Planner + Text-RAG, using the same dual-agent architecture, guideline corpus, patient-state inputs, Medical Executor, and interaction budget. Both Text-RAG baselines reuse CPGAgent's memory ranking mechanism, replacing the selected latent memories with their original guideline text. This ensures identical retrieval and selection, isolating the contribution of latent memory representation. The corresponding experimental results have been incorporated into Section 5.1 (Main Results) of the revised manuscript.

MethodLUADUCECNPCOverall
GPT-5.6-sol + Text-RAG45.03543.85350.61246.597
Qwen Planner + Text-RAG47.56045.54851.92648.384
CPGAgent47.95749.66355.51951.289

CPGAgent outperforms GPT + Text-RAG by 4.692 points and the same-backbone Qwen Planner + Text-RAG by 2.905 points overall, supporting the contribution of our guideline memory architecture beyond the Planner backbone. We attribute these gains to two key designs:

(1) Compact guideline representation. Retrieved guideline text alone averages 12,013 characters (15,002 tokens), introducing substantial contextual overhead. Our Memory Encoder compresses guidelines into compact latent slots, reducing the need to repeatedly process lengthy text.

(2) State-aware knowledge integration. We enable State-aware knowledge integration through two stages. First, the patient state guides the extraction of decision-relevant information from the anchor guideline memory, effectively querying what matters for the current patient. The resulting state-aware representation then attends to other selected guideline memories to identify complementary information from additional sections. This allows the Planner to focus on key clinical decisions while incorporating relevant cross-section knowledge.

We agree that Planner training improves its clinical planning capabilities, as evidenced by the higher performance of Qwen Planner + Text-RAG (48.384) over GPT-5.6-sol + Text-RAG (46.597). However, replacing retrieved guideline text with latent memories further improves performance to 51.289, demonstrating that the gains arise from both Planner training and our proposed guideline memory mechanism.

AuthorsOpenReview ↗

Response to Reviewer 34VE: 2) Updatability

2) Updatability

Critical Weaknesses and Questions :

The updatability claim is likewise not supported. No update procedure is described or tested, and since the planner and the gate are trained on trajectories and provenance tied to the current guidelines, incorporating an important new therapy may require additional annotation or retraining, which has not been investigated and would be the very limitation the paper attributes to parametric approaches.

What would it take for CPGAgent to follow a new guideline version that introduces an important new therapy: re-encoding only, or new annotation and retraining?

response:

We thank the reviewer for raising this important question. To evaluate guideline updatability, we conducted a cross-version guideline transfer experiment. The original model was trained using NSCLC 2010, SCLC 2010, CSCO Endometrial Cancer 2023, and CSCO Nasopharyngeal Carcinoma 2022. We then directly re-encoded four updated guidelines using the frozen Memory Encoder, without additional annotation or retraining of any model components.

Guideline UpdateTrajectory Score
NSCLC, SCLC 2010 → 201547.957 → 49.006
CSCO UCEC 2023 → 202549.663 → 44.821
CSCO NPC 2022 → 202555.519 → 55.496

The results demonstrate that CPGAgent can incorporate newer guideline versions without retraining the Memory Encoder, Soft Gate, or Planner Decoder, while maintaining competitive trajectory performance. The corresponding experimental results have been incorporated into Section 5.2 (Ablation Experiments and Trajectory Evaluation) of the revised manuscript.

Importantly, our Memory Encoder and Planner are not specialized to particular guideline versions, but are trained to generalize across clinical guidelines. Therefore, CPGAgent can incorporate updated guidelines simply by re-encoding the revised text into latent memory slots, without additional annotation or model retraining.

AuthorsOpenReview ↗

Response to Reviewer 34VE: 3) Evaluation

3) Evaluation

Critical Weaknesses and Questions:

I am also not yet convinced that the evaluation can carry the comparison. The reference rubrics are generated and verified by the same LLM, and trajectories are scored by GPT-5.6-sol, which also serves as the backbone of the executor and of most baselines. Clinician review is mentioned only briefly, agreement between the score and clinical judgment is not reported, and the paper describes differently what the rubric verifier can see (p. 6 vs. pp. 18-19). The source of the planner's training trajectories, and their separation from the 109 evaluation cases, is not described. With 109 cases, a single run and only the authors' own benchmark, the evidence is thin, particularly on LUAD, where the margin is 0.021.

Where do the planner's training trajectories and provenance annotations come from, and are they separated from the evaluation cases and from the rubric-generation pipeline?

response:

We thank the reviewer for raising these important concerns. We have expanded the descriptions of training–evaluation separation, clinical expert review, evaluator bias mitigation, and statistical uncertainty in the revised manuscript.

Training data and data leakage. As clarified in the newly added Appendix A, Planner training and CPGTrajBench evaluation follow separate data construction pipelines. Planner training data are derived exclusively from clinical guidelines. AE, RETRIEVE, and CONTINUE use rule-based supervision, while PLAN uses GPT-5.6-sol to generate synthetic patient states and guideline-grounded clinical plans. The synthetic PLAN data are additionally reviewed by two medical experts (Appendix I). No evaluation patient records, reference trajectories, or rubrics are used for Planner training. The pipelines share guideline knowledge but have no patient-level or trajectory-level data overlap.

Reference trajectory reliability and verifier access. Reference trajectories are generated and verified using DeepSeek-V4-Pro through independent calls without shared conversation history. The Generator observes only the current patient state and relevant guidelines, whereas the Verifier accesses the complete patient record exclusively for verification and controlled evidence disclosure. We have clarified this distinction consistently in Section 4.2 and Appendix G. Furthermore, reference trajectories undergo clinical expert review for guideline consistency and clinical appropriateness. As detailed in Appendix H (Table 10), 26 of the 135 cases passing initial screening were rejected during expert review, leaving 109 clinically accepted cases.

Evaluator bias mitigation. Although GPT-5.6-sol serves as the semantic Judge and as the Executor backbone, it does not directly assign trajectory scores. As detailed in Appendix E (Tables 7–8), the Judge provides structured semantic labels at predefined decision points, while trajectory alignment, action coverage, penalties, and final scores are computed by a deterministic algorithm with fixed coefficients. All methods are evaluated independently against the same reference rubrics, reducing opportunities for model-specific scoring preferences.

Statistical uncertainty. We have updated Table 2 to report mean performance with bootstrap uncertainty from 10,000 within-cohort patient-level resamples. We are also conducting multi-seed experiments to assess run-to-run variability; however, due to time and computational resource constraints, these experiments cannot be completed within the rebuttal period. As an alternative, we report bootstrap estimates to quantify case-sampling uncertainty, while acknowledging that they do not capture variability across independent runs. CPGAgent achieves the highest overall mean score (51.289), exceeding the strongest overall baseline by 2.905 points.

Comments on the paper

1 comment
AuthorsOpenReview ↗

Rebuttal Summary for the Area Chair

Dear Area Chair,

We sincerely thank you and all four reviewers for your constructive feedback. We have carefully addressed the reviewers' concerns through substantial additional experiments, detailed clarifications, and revisions to the manuscript. We would like to briefly highlight the key updates:

  1. Controlled RAG comparisons. We added two controlled Text-RAG baselines using GPT-5.6-sol and the same trained Qwen3.5-9B Planner. Under matched architectures and interaction budgets, CPGAgent achieves an overall score of 51.289, outperforming the two baselines (46.597 and 48.384). These results provide direct evidence for the contribution of our guideline memory mechanism beyond Planner training.
  2. Guideline updatability. We conducted cross-version guideline transfer experiments across all three cancer cohorts. Updated guidelines can be incorporated by re-encoding them with the frozen Memory Encoder, without additional annotation or model retraining. The corresponding experimental results have been updated into the revised manuscript.
  3. Benchmark reliability and data independence. We clarified that Planner training uses exclusively guideline-derived synthetic data, with no patient-level or trajectory-level overlap with CPGTrajBench. We further detailed the clinical expert validation process, in which 109 of 135 reviewed cases were accepted, and clarified the separation between LLM-based semantic judgments and deterministic trajectory scoring.
  4. Statistical reliability and reproducibility. We added uncertainty estimates from 10,000 patient-level bootstrap resamples and expanded the manuscript of training procedures, baseline configurations, scoring coefficients, and benchmark construction.

We believe these additions directly address the major concerns raised in the reviews and substantially strengthen the empirical evidence and methodological transparency of our work.

As the discussion period is approaching its end, we would greatly appreciate it if you could kindly encourage the reviewers to revisit our detailed responses and provide further feedback on whether their concerns have been adequately addressed. We remain fully committed to constructive discussion and would be happy to promptly clarify any remaining issues.

Thank you very much for your time and consideration.

Best regards, The Authors

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.