- Rating
- 1
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 1
Read review
Paper summary
This paper proposes PRISM for TCM prescription recommendation. It builds symptom–symptom, symptom–herb and herb–herb graphs from training data and derives weak “main” vs. “auxiliary” herb roles from a statistical specificity score. Main herbs are predicted first and auxiliary herbs are then conditioned on them, with three latent states presented as an interpretable reasoning trajectory. On TCM-PD, PRISM outperforms six baselines.
Critical strengths
- The main-then-auxiliary structure is intuitive. Role supervision is obtained from training prescriptions without annotation, with a sample-specific number of main herbs set by a gap rule.
- Gains over six baselines are consistent on all nine metrics and sizable (e.g., Recall@20 0.476 vs. 0.419 for TCMRGCL). Graphs and role statistics are built from the training split only.
- The paper includes component ablations, sensitivity analyses over four hyperparameters (Tables 8–11), and a code link.
- The comparison against random role assignment (Table 4) is a reasonable attempt to test role validity, although it is limited to a consistency measure.
Critical weaknesses
- The paper offers limited methodological novelty, primarily combining existing techniques, and lacks sufficient experimental validation to demonstrate the unique contributions and necessity of the proposed framework.
- The references contain duplicate entries for Jin et al. (2020a, 2020b) and Yang et al. (2022a, 2022b), indicating inconsistencies in reference management and citation accuracy.
- Core claims are not isolated by the ablations. No variant removes the main–auxiliary mechanism. The relevant rows of Table 2 only drop losses, so the branches presumably still feed the final score. The “w/o Main-Auxiliary” model in Table 3 is never defined or evaluated quantitatively. The “w/o Main-herb BCE” collapse (F1@5 0.2346 → 0.0657; the text says 0.0651) is hard to read as evidence that core herbs are crucial. That loss has weight 0.1, the prescription loss still supervises every herb, and precision rises from K=10 to K=20 (0.064 → 0.112). This looks like a degenerate ranking (e.g., an unsupervised main branch whose logits dominate the summed score). the authors should explain it rather than present it as evidence for role modeling. Fig. 3(b) is largely expected by construction and confounded by sample difficulty.
- Component gains are small relative to unreported variance, and the tuning protocol is unclear. Everything uses one seed. Merely varying c in Table 11 moves F1@5 by 0.003 and F1@10 by 0.004, which is larger than the graph-ablation gaps (≤0.002). In Fig. 3(a), removing SH or HH gives a higher F1@10 than the full model (0.2601/0.2591 vs. 0.2581), and all gaps are at most about 0.002; the text cites only F1@5/F1@20. In Fig. 3(c), removing the latent module costs about 0.013 in F1@5/F1@10, but at F1@10/F1@20 most of that gain comes from z0 alone and z1/z2 add at most about 0.002 (F1@20 even dips from z0 to z0+z1), so it is unclear that the progressive steps, rather than an extra head, drive the gain. The default configuration scores 0.2358/0.2604/0.2368 (F1@5/10/20) in Tables 8–11 but 0.2346/0.2581/0.2369 in Tables 1–2, with no explanation. There is no validation split (Sec. 4.1), yet early stopping is used (Table 12) and the adopted settings are best or near-best in Tables 8–11. It is therefore unclear whether test data drove early stopping and hyperparameter selection (a common protocol on TCM-PD, but it should be stated).
- The margin over baselines is large and well beyond noise;the concern is what produces it. Without what appears to be the ranking loss (“w/o FARL”, never defined), PRISM falls below TCMRGCL and SCEIKG in F1 at every cutoff. Baselines use their original settings (App. A.3), so the added ranking and multi-task supervision is an uncontrolled advantage. The final score also sums four heads, including a global head without role-specific supervision. Head weights are unreported and no ablation removes this head, so it is not shown that the progressive path drives the predictions. By the paper’s own descriptions (Sec. 4.2, App. A.3), SMGCN/SMRGAT already use SS/SH/HH graphs, TCMRGCL is hierarchical, and PresRecST is staged. The difference behind Challenges 1 and 3 is not analyzed. Evaluation uses a single dataset (the abstract says “datasets”; Table 2 is labeled “Dataset1”).
- Interpretability and role validity are unsupported. Table 6 reports cosine similarities for three cases. The high z1–z2 value (0.926) is what residual updates produce, not evidence of reasoning, and there is no faithfulness test or TCM-expert assessment. The “consistency” metrics (Tables 4–5) are undefined and may rely on the same statistics that define the roles. “Role overlap = 0” holds by construction (random roles also score 0), and no downstream test with random roles is given. Roles are never compared with the monarch–minister–assistant–envoy roles that motivate the work. In both case studies (full coverage 8/8 and 7/8 vs. an average Recall@20 of 0.48), the main set is small (1/8 gold herbs) and nearly all of the gain appears in the auxiliary stage, but no intervention (e.g., perturbing the predicted main herbs) shows that main-herb guidance causes it. How the cases were chosen is not stated.
Questions for authors
- With multiple seeds, what do the following achieve: a flat model without main–auxiliary decomposition, random roles in the full model, no global head, and no structural-affinity terms in the logits?
- What exactly are FARL and the latent consistency loss (the latter is absent from the objective in Sec. 3.6), how are the z0-only and z0+z1 variants decoded, and what are the learned head weights?
- What data drove early stopping and the choices in Tables 8–11, and why do the default-setting results differ between Tables 1–2 and Tables 8–11?
- How does the strongest baseline perform with the same ranking/multi-task supervision, and how does PRISM perform on another dataset?
- How are the consistency metrics defined? What is the distribution of main-herb set sizes under the largest-gap rule? Can roles be checked against annotated formulas or physicians, and how were the case studies selected? Several herbs also recur across unrelated cases in Tables 3 and 7; how do the predictions compare with a training-frequency baseline?