acceptodds
Under review as a conference paper at ICLR 2027

COTCAgent: Preventive Consultation via Probabilistic Chain-of-Thought Completion

Abstract

Sequential diagnosis requires ranking diseases from a sparse intake under a public budget of follow-up questions, which is diagnostic triage with missing findings rather than screening of people who have no symptoms. Because laboratory series arrive irregularly and histories remain incomplete, the ranking must be updated as new facts arrive, yet most pipelines are limited by unnamed numeric trends, unverifiable free-form chain of thought, and a negative bias from treating unasked findings as absent. To solve these, we propose COTCAgent, a sparse-intake consultation agent built on Chain-of-Thought Completion (COTC), which couples three modules on one shared ternary evidence log. The Temporal-Statistics Adapter (TSA) turns time series into typed predicates with named statistics, while the COTC module scores diseases with a learnable knowledge-base energy in which unknown findings add zero energy, after which bounded probabilistic completion selects unknown findings under the question budget and writes each answer back as present, absent, or unknown. Edge strengths and an absence scale are trained with listwise ranking under a sparse curriculum, so that the same energy chooses the next finding by a surrogate entropy reduction. Experiments on DDXPlus and MIMIC-IV show that these modules improve ranking under a matched question budget, with information-gain completion above asking nothing or at random and above protocol-matched askers and a local LLM, thereby providing a foundation for future calibrated scoring and prospective preventive consultation.

Primary area:
neurosymbolic & hybrid AI systems (physics-informed, logic & formal reasoning, etc.)
Keywords:
sequential diagnosis, preventive consultation, chain-of-thought completion, ternary evidence, knowledge-base scoring, information gain, sparse clinical intake, medical decision support

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_F9pCOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
1
Soundness
2
Read review
Paper summary

This paper introduces the COTCAgent framework for sequential diagnosis scenarios. The framework proposes a Temporal-Statistics Adapter (TSA) to convert the temporal data into typed evidence with verifiable named routines. This work uses a ternary representation of evidence, present, absent, and unknown, to better distinguish different states of the evidence. It treats the disease–evidence relationships as learnable weights to help identify more informative questions during multi-round consultations. This work also represents the chain of thought as an evidence scoring log, making each round of diagnosis traceable to entries in the knowledge base.

Critical strengths
  1. This work addresses practical problems and has a clear motivation. It identifies the need to ask useful follow-up questions when initial patient information is sparse. It also highlights limitations of some existing methods, including treating unasked-about findings as absent and describing numerical trends without specifying the underlying estimators, making slope and change-point claims difficult to verify.

  2. The experiments are relatively thorough, and the appendix provides details about the experimental setup and parameter settings.

Critical weaknesses
  1. The presentation of the method could be clearer. Although the authors provide many details about the proposed method, I still find it difficult to follow the descriptions of some modules and understand how they fit into the overall pipeline. For example, Section 3.2 introduces several technical terms, such as “rule-parsed intent,” “named estimator family,” “executable routine,” “planner,” “trend queries,” and “abrupt-change queries.” Concrete examples illustrating these concepts and their relationships would help readers better understand how the method works.
  2. Section 3.2 describes a finite schema covering slope, change-point score, and cohort z-score. However, the rationale for selecting these statistics and their adequacy for extracting relevant temporal evidence could be better explained. It would be helpful to discuss whether additional statistics, such as within-patient variability, could provide useful evidence beyond the current representation. I suggest organizing the explanation of each statistical output in a separate paragraph and clarifying what changes to the TSA schema and downstream knowledge base would be required to support additional statistics.
Questions for authors

Please address the questions and clarification requests raised in the Weaknesses section above.

Reviewer_bLPoOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper presents COTCAgent, a structured diagnostic completion framework that converts longitudinal records into typed evidence, ranks diseases with a learned knowledge-base energy, and selects follow-up questions using posterior mass or surrogate information gain. COTC-IG achieves 88.1% Hit@1 on DDXPlus and improves MIMIC-IV Hit@1 from 72.7% to 79.4%, although it is nearly tied with the simpler mass policy.

Critical strengths
  1. The paper addresses efficient disease diagnosis under a limited question budget, which is a practically important problem.
  2. The methodology is described in sufficient detail to make the computational steps transparent and auditable.
  3. The proposed method achieves strong diagnostic performance with only a limited number of follow-up questions.
Critical weaknesses
  1. The experiments do not isolate the paper’s central claimed contributions. In particular, there is no direct comparison between the proposed present/absent/unknown representation and a binary or unknown-as-absent alternative. There is also no downstream TSA-versus-no-TSA ablation on MIMIC-IV. It is therefore unclear how much of the improvement comes from these design choices rather than from the learned ranker or the overall system combination.

  2. The broader comparative evidence is not fully controlled. The internal same-ranker comparison between acquisition policies is informative, but the external baselines use different ranking models and may not receive equivalent access to the disease-evidence knowledge base. Moreover, DDXPlus uses synthetic cases and an exact-answer oracle, while the MIMIC-IV task uses coarse phenotype labels and shows almost no difference between IG and the simpler mass policy. These limitations weaken the broader claims about clinical applicability and superiority over existing systems.

Questions for authors
  1. For each external baseline, could the authors clarify its training data, ranking model, access to the disease-evidence knowledge base, and hyperparameter budget? Could the methods also be compared using the same ranker or equivalent prior knowledge?

  2. The individual components appear closely related to existing work on temporal feature extraction, knowledge-based disease scoring, and active feature acquisition. Could the authors clarify the main technical novelty beyond integrating these components?

Reviewer_45qZOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

This paper proposes COTCAgent, a framework for sequential diagnosis under incomplete clinical information. It integrates a Temporal-Statistics Adapter (TSA), learnable knowledge-base scoring with ternary evidence, and information-gain-based question selection. Experiments on DDXPlus and MIMIC-IV demonstrate improved diagnostic ranking under limited questioning budgets, achieving 88.1% Hit@1 on DDXPlus with an average of 3.61 questions.

Critical strengths
  1. The framework integrates ternary evidence representation with learnable knowledge-base scoring, enabling interpretable disease ranking while distinguishing missing from negative findings.

  2. COTC-IG improves diagnostic ranking on DDXPlus from 37.9% to 88.1% Hit@1 with only 3.61 questions on average, demonstrating efficient evidence acquisition under limited questioning budgets.

Critical weaknesses
  1. The authors propose the TSA to handle irregular temporal variations in numerical clinical measurements. However, many clinical conditions lack sufficiently frequent longitudinal observations. For example, a patient with diabetes may visit the hospital only twice a year, resulting in just two laboratory measurements. How does TSA handle such sparse and irregularly sampled clinical data?

  2. The manuscript does not sufficiently isolate the incremental contribution of the Temporal-Statistics Adapter. A controlled ablation comparing raw laboratory values, abnormality indicators, and TSA-derived longitudinal features under identical downstream ranking and acquisition settings would help clarify its effectiveness.

  3. The proposed Information Gain (IG) approximates the probability of a positive finding as pj=∑i:Mij=1P~(di)p_j=\sum_{i:M_{ij}=1}\tilde P(d_i), rather than explicitly modeling P(ej=1∣di)P(e_j=1\mid d_i). Since P~(di)\tilde P(d_i) is not necessarily calibrated, this approximation may introduce bias into IG estimation and question selection. A comparison with likelihood-based Bayesian IG would strengthen the methodological justification.

  4. The model architecture appears closely aligned with the synthetic data generation process, raising concerns about generalizability to real-world clinical settings. Indeed, the performance differences among IG-based, probability-mass-based, and random selection become marginal on MIMIC. A simple Naive Bayes baseline that estimates P(ej∣di)P(e_j\mid d_i) directly from training data and performs Bayesian information gain maximization should be included. Although the authors argue that discriminative scores cannot provide calibrated mutual information, this limitation arises from their modeling choices. Given the large training dataset available in DDXPlus, estimating these conditional likelihoods should be feasible. If such a simple baseline achieves comparable performance, the necessity of the proposed COTC framework would be questionable.

  5. All methods are primarily compared under a fixed budget of five rounds with two questions per round (5 × 2). However, the appendix reports that a 3 × 5 configuration can achieve 94.4% performance. It remains unclear whether the relative rankings of the methods would remain consistent under different interaction budgets. A systematic evaluation across varying numbers of rounds and questions per round is recommended.

Questions for authors

Q1 The original DDXPlus paper reports 99.21% GTPA@1 for AARLC and 97.15% for BASD, while your reimplementations reach 28.0% and 31.1%. What explains this gap? Do the baselines have access to M, age, and sex as COTC does?

Q2 DDXPlus assumes evidences are conditionally independent given the disease, which Eq. 5 approximates. How does COTC-IG compare with a naive-Bayes model fit on the same data with exact information gain?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.