- Rating
- 2
- Confidence
- 4
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper formulates colon cancer treatment planning as an interactive RL problem under partial observability and partial verifiability. It constructs a patient-specific environment where agents sequentially acquire clinical information before recommending treatment, with rewards evaluating information gathering, treatment selection, and reasoning alignment. Experiments show that GRPO and GDPO improve clinical decision-making performance, with the GDPO-trained Llama-3.1-8B outperforming GPT-5.2 under the proposed evaluation framework.
Critical strengths
Interesting problem formulation. The paper addresses an important challenge in clinical decision-making, where multiple treatment plans and reasoning trajectories can be clinically valid rather than having a single correct answer. This provides a meaningful setting for exploring reinforcement learning under partial verifiability.
Critical weaknesses
-
Limited novelty and insufficient discussion of related work. Active information acquisition through RL is not specific to colon cancer treatment planning and has been explored in prior work such as ProMed (ACL 2026). The paper does not sufficiently distinguish its approach from existing methods or demonstrate the advantages of its reward design.
-
Questionable validity of partially verifiable rewards. Although the paper argues that multiple treatment plans can be clinically valid, its reward is largely based on a single clinician's historical decision. It remains unclear whether the LLM judge can reliably recognize reasonable alternatives that differ from the ground truth, raising concerns about the validity of the reward and the reported improvements.
Questions for authors
-
How does the proposed framework differ from existing RL-based active clinical information acquisition methods, such as ProMed (ACL 2026)?
-
Why is a trajectory-level LLM-judged reward preferred over explicit information-gain-based rewards? Could the authors provide comparisons to demonstrate its advantages?
-
How does the LLM judge distinguish clinically reasonable treatment alternatives from incorrect predictions when they differ from the clinician's ground truth? Has this capability been validated through independent clinical expert evaluation?
3 comments
(continue)
Q3.
An independent clinician assessment shows that the regimen scores from the GPT-5.2 judge agree with the clinician's grades of the treatment plans (Spearman's rank correlation coefficient 0.80). We draw 30 patients from the held-out test set. Patients the clinician had already seen in the earlier 34-case study were excluded. The clinician reviews the treatment plans from GDPO-Llama and GPT-5.2 for those patients in blinded fashion along with the full patient representation and the original treatment plan from the physician as the comparison point. Each treatment plan is graded with one of the following choices:
- Same regimen,
- Clinically equivalent,
- Reasonable alternative,
- Not appropriate, or
- Cannot judge.
The clinician judges the choice of drugs and ignores dose, schedule and number of cycles. We then compare each grade with the regimen score that the GPT-5.2 judge gave the same rollout.
| Model | Same regimen | Clinically equivalent | Reasonable alternative | Not appropriate | Cannot judge | Acceptable (95% CI) |
|---|---|---|---|---|---|---|
| GDPO-Llama | 14 | 4 | 4 | 7 | 1 | 22 of 29, 76% (58-88%) |
| GPT-5.2 | 8 | 4 | 10 | 3 | 5 | 22 of 25, 88% (70-96%) |
Table 2. Clinician grades of the plans from GDPO-Llama and GPT-5.2 for 30 held-out test patients, rated blind against the treating oncologist's plan. Acceptable plans include same regimen, clinically equivalent or reasonable alternative, excluding the plans that cannot be judged. We report the 95% Wilson intervals.
The clinician rated 76% of GDPO-Llama's plans and 88% of GPT-5.2's plans acceptable, i.e., same regimen, clinically equivalent or reasonable alternative (95% intervals 58-88% and 70-96%). GDPO-Llama more often chose the physician's regimen (14 against 8 plans), while GPT-5.2 more often gave a reasonable alternative (10 against 4).
| Clinician grade | Number of plans | Mean judge score |
|---|---|---|
| Same regimen | 22 | 4.68 |
| Clinically equivalent | 8 | 3.50 |
| Reasonable alternative | 14 | 2.64 |
| Not appropriate | 10 | 2.20 |
| Cannot judge | 6 | 3.17 |
| Spearman's rank correlation coefficient between clinician and GPT-5.2 judge | - | 0.80 |
| Agreement on same or equivalent, % (Cohen's ) | - | 76% (0.48) |
Table 3. The agreement between the clinician assessment and the GPT-5.2 judge's scoring on the regimen accuracy on plans from GDPO-Llama and GPT-5.2 for 30 patients. The rubric gives 5 to an exact match with the regimen from the original treatment plan, 3-4 to a clinically equivalent regimen, 2 to a regimen of the right class that is not equivalent, and 0-1 to a wrong class. We report the Spearman's rank correlation coefficient between the judge's scores and the clinician's grades. For the agreement, a plan counts as same or equivalent for the judge when it scores 3-5, and for the clinician when it is graded same regimen or clinically equivalent. Plans the clinician cannot judge are excluded from the rank correlation and the agreement.
The judge's regimen score is concordant with the clinician's grade; the mean judge score is 4.68 for plans graded as the same regimen, 3.50 for clinically equivalent, 2.64 for reasonable alternative and 2.20 for not appropriate. Of the six plans the clinician could not judge, 4 are GPT-5.2 plans that list several conditional options without choosing a single one, and 2 are for one patient whose original plan was unclear.
For the plans that are graded not appropriate by the clinician, five of GDPO-Llama's 7 plans are rated not appropriate because they leave out radiation or surgery from a multimodal curative plan for rectal cancer or anal squamous cell carcinoma although the plans suggest the right chemotherapy. The missing modality is a limit of the agent and the rubric, which focuses on the accuracy of the regimen, since our evaluation targets the chemotherapy plan.
Thank you for your responses, I decide to maintain my original score.
Thank you for your review. We appreciate your recognition of the problem formulation and your questions on the reward design.
Q1.
We agree that active information acquisition through RL is not specific to oncology and has been explored in ProMed (Ding et al., ACL 2026). The difference between ProMed and our work is in the clinical decision problem and in how the reward is constructed.
ProMed studies medical question answering, where a ground-truth answer is available, and rewards each question by its Shapley information gain toward predicting the correct answer. Treatment planning, on the other hand, can have multiple acceptable treatment options for the same patient. Since the medical records contain only the realized treatment plan, we cannot observe the other acceptable plans or the decision processes that could lead to them, which makes both the treatment decision and the decision process partially verifiable. We therefore grade the whole trajectory against the realized plan and the documented decision process instead of each question. We do not claim novelty in the RL algorithm or in active information acquisition itself, but in the interactive environment and the patient-specific reward for treatment planning under partial verifiability. We will add ProMed and this distinction to the related work.
Q2.
We conduct the ablation study to compare the trajectory-level LLM-judged reward with rule-based rewards for information acquisition and regimen selection. The rule-based information acquisition reward is the F1 score between the factors the agent asks about and those in the physician's documented decision process, matched by keyword over 26 factors such as stage, biomarkers and labs. While the rule-based information acquisition reward is not directly comparable with the Shapley information gain proposed in ProMed, it measures how much of the documented decision process the agent covers.
The ablation study is set up as follows. The judge is Qwen3.8-27B-FP8 with a LoRA adapter fine-tuned on the past judging completions from GPT-5.2, and the mediator is Qwen3-1.7B with a LoRA adapter. We train Llama-3.1-8B-Instruct with GDPO for 300 steps under six reward settings with group size 8, 5 turns, the same patient order and seed 42. We evaluate the checkpoint at step 300 on the validation split with 143 patients with one trajectory per patient with greedy decoding.
Table 1. Reward ablation on the validation split (143 patients, one greedy trajectory per patient at step 300, scored by the fine-tuned judge). In the LLM-judged setting, the policy is rewarded by the fine-tuned judge; in the rule-based setting, the reward is computed by deterministic rules. Decision-factor recall is the share of the physician's decision factors the agent asked about. The full table can be found at our comment above.
In the rule-based reward setting for information acquisition, the agent exploits the reward by asking more questions. With the rule-based decision-factor reward alone, the agent asks about more of the physician’s decision factors (recall 0.547), but it asks 41.4 questions per case and completes only 80.4% of the cases. Imposing the LLM-judged reward along with the rule-based reward still leads to 38.5 questions per case and 85.3% completed cases. On the other hand, the LLM-judged information and reasoning rewards raise the decision-factor recall from 0.369 to 0.459 at 25.4 questions per case, compared to 17.8 without them, while the agent still completes 95.8% of the cases. In principle, an information-gain reward can also be measured against the documented decision process, and the rule-based decision-factor reward is our closest version of it. We could not run ProMed's Shapley information gain within the rebuttal period, since it estimates the gain of each question with repeated model calls, and we will add this comparison to the discussion.