- Rating
- 1
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 1
Read review
Paper summary
This article studies the problem of discovering multi hop biomedical causal mechanisms from sparse and asynchronous clinical observations, proposes the TRACE framework, and combines time aware graph construction, literature verification, MCTS reverse causality search, macro skill reuse, and structured conformal prediction. The core viewpoint of the author is to narrow down the candidate mechanism space through time and knowledge constraints, then use literature evidence to support active search, and use conformal predictive control to control the uncertainty of output. The paper reported good candidate ranking performance on MIMIC, ADE Time, and CoMAGC, and reported near target 95% coverage on MIMIC and FB15k Subset, while demonstrating the inference acceleration brought by macro skill reuse. The research question has practical value, and combining mechanism search with uncertainty control is also attractive. However, the relationship between "candidate mechanism credibility", "true causal structure", and "statistical coverage guarantee" has not been fully established in current papers. There are still some key issues regarding method definition, theoretical basis, and experimental verifiability.
Critical strengths
The paper reported good candidate ranking performance on MIMIC, ADE Time, and CoMAGC, and reported near target 95% coverage on MIMIC and FB15k Subset, while demonstrating the inference acceleration brought by macro skill reuse. The research problem has practical value, and combining mechanism search with uncertainty control also has certain appeal
Critical weaknesses
-
The coverage objects and prediction sets of structured conformal prediction are not fully defined, and existing algorithms are insufficient to support the core guarantee of the paper. Equations (10) to (12) and the pseudocode did not use yi, only calculating the validation score of the graph itself. The triple probability product of equation (10) can be used as a heuristic inconsistency score, but without a dependency structure explanation, it cannot be directly explained as a joint probability that the entire path holds.
-
The construction of a time perception graph has not yet established an effective identification process from clinical events to causal structures. The time similarity in equation (3) is symmetrical and can express the degree of event proximity, but it cannot determine the causal direction itself. Although formula (4) introduces Bayesian structural scoring and acyclic constraints, it does not define how clinical events are composed of random variables and repeated observation samples, nor does it explain the specific form of likelihood, missing value handling, and causal identification assumptions.
-
Insufficient information on dataset construction and baseline adaptation makes it difficult to determine the reliability and fairness of the main results. Section 4.1 only briefly lists four data sources, without providing the task sample size, input-output instances, annotation process of real root causes or paths, candidate space size, and complete partitioning of training, validation, calibration, and testing sets.
The implementation of active open domain search and continuous updates is not yet sufficient to support its method proposition. Section 3.3 allows the language model to propose new upstream reasons, but equation (7) relies on the Rotate embedding of candidate entities. The paper does not explain how new entities outside the knowledge graph are represented, nor does it distinguish between "discovering new relationships between known entities" and "discovering unincorporated entities and mechanisms".
-
- The existing experiments can provide preliminary signals of performance improvement, but are not sufficient to validate the core mechanisms and statistical conclusions. In Table 2, the absolute MRR improvement of TRACE relative to the best baseline of each dataset is 0.090, 0.080, and 0.069, respectively, which cannot directly correspond to the highest 10.4% state-of-the-art baseline improvement mentioned in the abstract. The author should clarify and unify the comparison criteria. The coverage rates of 95.2% and 95.4% in Table 3 do not have matching test sample sizes, confidence intervals, or duplicate partitioning results, so it cannot be claimed to "strictly maintain" target coverage based solely on point estimates; The results of 94.8% and 94.9% in Table 4 cannot be explained as module removal leading to guarantee failure in the absence of statistical analysis. If the MRR decreases simultaneously after removing the conformal module in Table 4, it is necessary to explain whether conformal gating has changed the sorting, evaluation samples, or training process. The 54.3% acceleration brought by macro skills has potential value, but should be included in the cost of building and maintaining the skill library, reporting hit rate, cold start performance, and accuracy under the same time budget, and compared with regular path caching.
Questions for authors
-
Can provide clear prediction objectives, set construction, coverage events, and theorem proofs, and report the overall coverage rate, rejection rate, and error rate in the accepted samples separately?
-
Clarify the semantics of nodes and edges, implement global constraints on the merged graph, and report the accuracy of edge direction, structural distance, and performance under mixed conditions on simulated data or expert annotated data with known structures?
-
Can compare greedy, bundle search, MCTS, and caching strategies under a fixed validator, compare different conformal scores under a fixed candidate set, and report coverage performance divided by path length, disease category, and time?
My native language is Chinese. After reading this manuscript, I provided my opinion in Chinese and had it translated into English by AI. The editor seems to believe that this is an AI generated opinion, and they are requesting me to make further comments. But I think their viewpoint is obviously inappropriate.