acceptodds
Under review as a conference paper at ICLR 2027

TumorCast: A Cancer World Model for Longitudinal Tumor Forecasting

Abstract

Anticipating how a tumor will grow and spread could help clinicians adapt therapy in time, yet forecasting its evolution is difficult: tumors exhibit substantial heterogeneity, and their trajectories depend on anatomical location, current disease state, local tissue context, and the patient’s clinical history. Existing approaches often predict clinical outcomes directly from observed CT scans or records, or generate future CT scans, leaving the evolution of CT latent states insufficiently modeled. To address this gap, we propose TumorCast, a cancer world model for longitudinal tumor forecasting that predicts CT latent states conditioned on clinical history, interval treatment, and transition interval. Built on the joint-embedding predictive architecture (JEPA), TumorCast learns reusable predicted CT latent states through a Representation–Dynamics–Readout curriculum, without reconstructing future CT scans. First, longitudinal CT scan pairs adapt a CT encoder and latent predictor through future-state supervision, while clinical text provides conditioning inputs for state transitions. Next, chained training learns successive state transitions through recursive rollout, with future-state supervision propagating backward through the trajectory so that later prediction errors refine earlier state transitions and predicted intermediate states. Finally, a shared clinical reader (Qwen3.5-9B/27B) combines predicted CT latent states with clinical context to forecast lesion diameter, lesion size change, and liver metastasis status. To support evaluation, we curate a multimodal tumor forecasting benchmark comprising 1,344 patients, 3,606 CT scans with corresponding radiology reports, and 11,728 structured clinical entries covering clinical history and interval treatment, yielding 2,400 forecasting examples across liver and pancreas. Experiments show that TumorCast outperforms rendered-slice vision-language models (VLMs) and text-only baselines, highlighting the success of our multimodal JEPA design and potential clinical effectiveness.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
Cancer world model, Longitudinal Forecasting, precision medicine

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 2 reviews, above 27% of the others.
Reviewer_wNBhOpenReview ↗
Rating
2
Confidence
4
Informativeness
1
Presentation
2
Soundness
2
Read review
Paper summary

This paper introduces TumorCast, a model to predict longitudinal CT scans in latent space. The paper adopts the JEPA framework and incorporates clinical information to guide the future state generation. The authors validates the effectiveness of generated latent states with an LLM as the backbone on lesion diameter, size, and metastasis status prediction tasks on a private longitudinal CT dataset.

Critical strengths

The topic of predicting longitudinal CT in latent space is novel and of interest to the research community.

Critical weaknesses
  • The technical contributions are limited because the authors follow the standard JEPA framework and do not extend beyond this technique.
  • The downstream evaluation is problematic with an LLM as the backbone. LLMs are powerful and might easily neglect the impact of input latent states.
Questions for authors
  1. The technical contributions are limited. This work used the standard JEPA framework consisting of a teacher-student architecture and a predictor. Cross attention between text and vision and AdaLN are incorporated. The training paradigm separates the update of the encoder and predictor. Autoregressive training is simplified. Details such as KV cache, mask attention, and impact of supervising the endpoints only need to be added. The authors should consider extending beyond these techniques to strengthen this work scientific contributions.

  2. The downstream evaluation is problematic. On downstream tasks, the authors finetuned an LLM (Qwen3.5) to process latent states and output predictions including lesion diameter, lesion size change, and liver metastasis status. However, an LLM is powerful and can easily demolish or ignore the impact of the latent state. Hence, this setting cannot determine the contributions of the latent states. To verify their effectiveness, the authors should design a setting where the latent states are the dominant factor, such as linear probing.

  3. In addition, the downstream tasks do not expand the scope of conventional methods, and the discovery is not novel. Predicting diameter, size, and metastasis status are regression or classification tasks that can be done by supervised learning conditioned on the current CT scan. The authors should configure downstream tasks to improve the impact of this study, such as counterfactual response, intermediate state interpretation, etc.

  4. The interpretability of the proposed model is weak. Since the major claim of this paper is that the progression process is important, the authors should consider provide interpretation of the learned intermediate states to demonstrate its relevance to final prediction.

  5. The claim in the introduction section that JEPA-type models discard the predictor does not hold. In model-based planning, the JEPA’s predictors were trained and applied.

  6. The use of terminology lacks rigidity, such as “chained training” -> “autoregressive training”. The word “curriculum” is often used to describe training with increasing levels of difficulty, but this work uses it to describe different stages of model development.

  7. The clarity of equations needs to be improved, such as equations (4) and (7). Adding description of the symbols could help.

In sum, the technical contributions and downstream application need to be improved.

3 comments
AuthorsOpenReview ↗

Author Response to Reviewer wNBh (Part 1)

(Q1-1) Technical Contributions and Relationship to the JEPA Framework

We appreciate the opportunity to clarify how TumorCast advances beyond a direct application of standard JEPA. Standard JEPA relies on dense, regular observations—an assumption that breaks down in clinical oncology, where CT scans are sparse, irregularly timed, and driven by multimodal interventions (EHR and treatments). To overcome these barriers, we introduce three non-trivial, clinically grounded adaptations:

  • Spatially Preserved State Space: Rather than compressing 3D scans into global vectors that lose spatial detail, we define 3D spatially structured CT representations as evolving latent states, enabling fine-grained evaluation of tumor progression and local metastasis.
  • Discrete Multimodal Transition Dynamics: Clinical trajectories comprise asynchronous, discrete multimodal events, as sparse 3D CT snapshots punctuated by irregular clinical records. Using Cross-Attention and AdaLN, we fuse unstructured clinical history, interval treatments, and dynamic time spans into the Predictor to model cross-modal state transitions under varying therapies.
  • Sparse Chained Supervision & LLM Decoding: For sparse follow-ups, we introduce Chained Latent Supervision to curb multi-step rollout drift, paired with a cross-modal LLM Reader to translate predicted latent states into clinical outcomes.

(Q1-2) Clarification on the comment: “training separates the update of the encoder and predictor”

Rather than updating the encoder and Predictor in isolation, we adopt a progressive strategy to balance domain adaptation with training stability:

1. Joint Adaptation: Initially, the 3D CT encoder and Predictor are updated jointly to adapt the generic static visual encoder to a domain-specific representation space tailored for longitudinal lesion prediction. 2. Frozen-Encoder Predictor Training: Once adapted, we freeze the encoder to prevent the follow-up supervision targets (St+1S_{t+1}) from drifting during optimization, allowing the Predictor to stably learn treatment- and interval-driven disease dynamics within a fixed latent space.

(Q1-3) Why Name It “Chained Latent Supervision” Instead of “Autoregressive”?

We thank the reviewer for noting this connection. While multi-step rollout shares a recursive form with autoregressive inference, we adopt Chained Latent Supervision to accurately define our training objective and avoid terminological ambiguity.

  • Sparse, Irregular Follow-Ups: Standard autoregressive modeling typically assumes dense, regularly spaced sequences to capture long-range history. Because clinical follow-ups are highly sparse and irregularly timed, we instead supervise local, treatment- and Δt\Delta t-conditioned transition chains across consecutive visits.
  • Cross-Step Gradient Flow: To curb rollout drift beyond single-step ground-truth training, we feed each predicted state into the next step with retained gradients, allowing downstream losses to backpropagate through intermediate states.

In the revision, we will clarify its conceptual link to autoregressive rollout while consistently using Recursive Latent Rollout for multi-step state prediction at inference.

AuthorsOpenReview ↗

Author Response to Reviewer wNBh (Part 2)

(Q2-1) Can You Design an Evaluation Where Latent States Are the Primary Factor?

We thank the reviewer for this suggestion. We added two evaluations that use the predicted latent states without the bridge or LLM reader.

1. Latent trajectory alignment. With the encoder and predictor frozen, we measured the cosine similarity between predicted and observed latent changes, ΔS=Sfuture−Scurrent\Delta S=S_{\text{future}}-S_{\text{current}}, on the 171 size test cases.

Pairing configurationMean cosine similarity [95% CI]
Another patient’s predicted change vs. true change0.0112 [−0.0058, 0.0361]
Training-set mean organ change vs. true change0.0357 [−0.0023, 0.0768]
Patient’s own predicted change vs. true change0.3282 [0.2789, 0.3814]

The patient-matched prediction achieved substantially higher alignment than the two near-zero baselines, supporting that the predictor captures patient-specific latent trajectories.

2. Linear probing. With the encoder and predictor frozen, we trained organ-specific ridge probes on the full predicted latent states to estimate future lesion diameter, without the bridge or LLM reader. Regularisation was selected using only training patients. On the 171 size test cases, linear probing achieved 43.12% BAcc, outperforming the best fine-tuned open-weight VLM baseline (36.25%) by 6.87%.

Together, these reader-free evaluations show that the predicted latent states capture patient-specific changes and contain task-relevant information recoverable through linear probing.

(Q2-2) Does the LLM Ignore the Latent State?

We thank the reviewer for this question. We performed a fixed-reader latent-replacement test on TumorCast-27B, replacing each case’s latent with predicted latents from three same-organ patients while keeping the question and history unchanged. Results are averaged over the three replacements on 171 size samples and 71 liver-metastasis samples.

MetricCorrectly matched latentRandom latent replacement (mean of 3)
Size BAcc ↑48.66%43.68%
Liver-metastasis AUPRC ↑71.43%65.39%
Liver-metastasis BAcc ↑78.68%74.84%

Random replacement reduced the three metrics by 4.98%, 6.04%, and 3.84%, showing that the LLM uses patient-specific latent information.

(Q3-1) Can you conduct other downstream task like treatment-conditioned response analysis?

We thank the reviewer for this suggestion. Because individual counterfactual outcomes are unobservable, we conducted a counterfactual sensitivity analysis on the 127 size-forecasting cases with recorded treatment plans. We replaced the original treatment with “No anti-tumour treatment” in both the latent-dynamics module and the text-based question-answering stage, while keeping all other inputs unchanged.

MetricResult
Changed growth status56 / 127 (44.09%)
Worsened growth status49 / 127 (38.58%)
Unchanged / Regressed growth status71 / 127 (55.91%) / 7 / 127 (5.51%)
Mean tumor diameter increase (No Treatment vs. Original)+4.38 mm

These results confirm that our model follows clinical logic, predicting +4.38 mm larger tumor growth and 38.58% disease progression when treatment is removed.

(Q3-2 & Q4) Are the Learned Intermediate States Interpretable?

  • Clinical interpretability (Q2-1). Trajectory alignment and linear probing show that the predicted latent states preserve patient-specific evolution and contain lesion-size-related signals recoverable by a linear probe.

  • Relevance to final predictions (Q2-2). Replacing the patient-matched latent states reduced model performance, indicating that the Reader uses patient-specific latent information when generating its predictions.

These results show that the learned intermediate states encode patient-specific disease evolution and clinical attributes and contribute to the final predictions, rather than being opaque features ignored by the Reader.

AuthorsOpenReview ↗

Author Response to Reviewer wNBh (Part 3)

(Q5 & Q6 & Q7) Terminology Standardization and Equation Clarity

We thank the reviewer for these helpful suggestions. In the revised manuscript, we will standardize our terminology and clarify our mathematical formulations as follows:

  • Predictor retention in JEPA. We acknowledge that V-JEPA 2-AC retains its predictor for action-conditioned planning. We will correct our statement and instead emphasise TumorCast’s predictor design for discrete treatment events and variable time intervals.
  • Chained Latent Supervision (Q1-3): We clarify this term to denote multi-step latent rollout under sparse, irregular CT sampling, distinguishing it from standard dense autoregressive training.
  • Staged Training: We agree that "Curriculum Learning" could be misleading since samples are not ordered by difficulty, and we will replace it with Staged Training throughout the manuscript.
  • Equations and Symbols: We will add complete symbol definitions and detailed explanations for all equations to improve readability.
Reviewer_ZqPpOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary
  • This paper introduces TumorCast, a model for forecasting tumor outcomes through future CT latent-state prediction

  • Training proceeds through representation adaptation, dynamics learning with paired and chained prediction losses, and clinical readout adaptation. Evaluation uses a curated longitudinal cohort across patients and forecasting examples

Critical strengths
  • Predicting follow-up CT representations provides a learning signal beyond the final clinical labels. Retaining the predictor for recursive forecasting and separating dynamics learning from clinical readout form a coherent approach to this problem.

  • In Table 2, removing the chained loss reduces lesion-size-change balanced accuracy so the chained-training ablation provides encouraging evidence.

Critical weaknesses
  • the paper does not provide the test counts by task, class, horizon, and rollout subset needed to interpret the results. Table 1 mentions sample counts in its caption, yet does not display them. The sensitivity and specificity values also suggest potentially different evaluation denominators between baselines and TumorCast

  • Table 1 compares native 3D processing with rendered-slice baselines, so those gains cannot be attributed solely to future-state prediction.

Questions for authors
  • Were all methods in Table 1 evaluated on exactly the same examples?

  • For “Bypass latent predictor,” was the reader retrained to forecast from observed CT states using the same clinical information, horizon inputs, and comparable training budget?

  • How were continuous scores obtained for metastasis AUPRC, given the binary JSON output in Appendix D.4?

2 comments
AuthorsOpenReview ↗

Author Response to Reviewer ZqPp (Part 1)

(W1 & Q1) Evaluation Sample Details

We thank the reviewer for attention for a detailed breakdown of the test samples. We have carefully reviewed the evaluation records and will provide counts by task, class, prediction horizon subset in the final version. As discussed in our response to Reviewer 1edU, we found that the baseline liver-metastasis evaluation omitted two required cleaning and filtering criteria. We will correct this and report results for all models before and after filtering to ensure fair comparisons.

1. Test Sample Counts

- Test set composition by task, organ, and Class distribution.

We enforced patient-level separation and used the same fixed training/test partition for all methods. The main single-step test set contains 242 queries, including 171 size queries and 71 liver-metastasis queries. The full-rollout subset contains 45 two-step endpoints.

Evaluation setQueriesPatientsClass distribution
Liver size (single-step)10557Shrink 36 / Stable 28 / Growth 41
Pancreas size (single-step)6634Shrink 26 / Stable 22 / Growth 18
Liver metastasis7154Negative 51 / Positive 20
Size full rollout (two-step endpoints)4527Shrink 18 / Stable 13 / Growth 14

- Test set composition by prediction horizon.

Prediction horizon (days)Size: Shrink / Stable / GrowthLiver metastasis: Negative / Positive
0–9072 (25 / 22 / 25)28 (22 / 6)
91–18057 (21 / 17 / 19)17 (13 / 4)
181–36528 (11 / 9 / 8)15 (12 / 3)
366–73014 (5 / 2 / 7)11 (4 / 7)
Total171 (62 / 50 / 59)71 (51 / 20)

2. Correction to the Liver-Metastasis Evaluation

The designed liver-metastasis evaluation set contains 71 samples (20 positive, 51 negative). When evaluating the baselines, we inadvertently omitted the two required cleaning and filtering criteria and included 9 additional cases, testing on a raw 80-sample set. TumorCast-9B and TumorCast-27B were evaluated on the designed 71-case set. The 9 additional cases are listed below.

  • The 730-day filter. Because only one size-forecasting test case exceeded 730 days, which is insufficient for meaningful duration-stratified analysis. We therefore standardised the evaluation horizon across all tasks to ≤\le 730 days. The baseline liver-metastasis evaluation omitted this filter and included 6 additional cases.

  • The follow-up CT preprocessing filter. TumorCast uses follow-up CTs for training supervision, so only successfully preprocessed CT pairs were retained in the training and test cohorts. The baseline evaluation overlooked this filter because the baselines use only the current CT, inadvertently including 3 additional cases.

Patient IndexPrevious CT dateFollow-up CT dateInterval (days)Liver-metastasis labelOriginal Exclusion Reason
12013-04-212017-04-261466NegativeTime filter
22011-04-272013-07-16811PositiveTime filter
32013-05-252019-07-212248NegativeTime filter
42016-04-072022-08-172323PositiveTime filter
52020-04-242022-09-03862NegativeTime filter
62017-05-252024-01-282439NegativeTime filter
72014-09-232015-04-05194PositiveFuture CT file corrupted; skipped
82016-03-272016-05-1751NegativeFuture CT file corrupted; skipped
92015-12-312016-07-15197NegativeFollow-up CT liver segmentation failed; skipped

Our response to Reviewer 1edU (W1 & Q1) presents the detailed results for both test sets. TumorCast-9B and TumorCast-27B consistently outperform the strongest baseline in two forcasting tasks. On the unified 71-case set, they achieve 77.16% and 78.68%, exceeding the strongest baseline (75.93%) by 1.23% and 2.75%, respectively. On the raw 80-case set, they achieve 79.94% and 81.24%, exceeding the strongest baseline (78.18%) by 1.76% and 3.06%, respectively.

In the final version, we will correct this oversight and consistently report results on the 71-sample in the main paper, ensuring a fair comparison across methods.

AuthorsOpenReview ↗

Author Response to Reviewer ZqPp (Part 2)

(W2 & Q2) Predictor Bypass With and Without Reader Retraining

We thank the reviewer for this suggestion. In the original predictor-bypass setting, the reader was not retrained, while the predictor was skipped only during inference. We therefore add Direct-3D-encoder-27B, which trains the bridge and reader to predict future outcomes directly from the current CT, without the predictor. Direct-3D and TumorCast use the same M3D encoder, training samples, and optimisation settings. All three models are evaluated on the same 171 size samples and 71 liver-metastasis samples (20 positive, 51 negative).

MetricTumorCast-27BPredictor bypass at inferenceDirect-3D with reader retraining
Size BAcc ↑ (171 samples)48.6643.8644.04
Diameter MAE (mm) ↓10.1911.3110.53
Liver-metastasis BAcc ↑ (71 samples)78.6877.1671.27

With identical 3D inputs and training budgets, TumorCast-27B improves size-forecasting BAcc and liver-metastasis BAcc by 4.62% and 7.41%, respectively, over the specifically trained Direct-3D baseline. These results provide direct evidence that explicitly modelling latent tumour-state evolution improves future-outcome prediction.

(Q3) How are continuous scores derived for computing liver-metastasis AUPRC?

We thank the reviewer for requesting clarification on the AUPRC calculation. Table 4 in Appendix D.4 uses simplified JSON outputs (pseudocode) to illustrate the task and class definitions rather than the complete inference output. The actual evaluation saves both the binary prediction and a continuous positive-class score. AUPRC is computed from the continuous scores and ground-truth labels.

Continuous score calculation. For each case, TumorCast evaluates two predefined candidate outputs: {"future_liver_metastasis": 0} and {"future_liver_metastasis": 1}. Using the reader’s token probabilities, we sum the conditional log-probabilities of all tokens in each complete candidate output, including the fixed end token, to obtain ℓ0\ell_0 and ℓ1\ell_1. These values are normalised into a continuous positive-class score:

{\exp(\ell_0)+\exp(\ell_1)}.$$ Both candidates are evaluated for every case, and the ground-truth label is not used to calculate the score. An example of the saved inference output is: ```json { "class_prediction": 1, "probability": 0.994089, "candidate_log_likelihoods": [-5.130993, -0.005957] } ``` Here, `probability` is the continuous score $q$. For commercial models, the continuous score is obtained from the `p_future_positive` field in the returned JSON. AUPRC is then calculated using the same AP implementation for all models.

Comments on the paper

2 comments
AuthorsOpenReview ↗

Author Response to Reviewer 1edU (Part 1)

(W1 & Q1) Detailed Clarification of Liver Metastasis Test Samples

We thank the reviewer for highlighting this discrepancy. To ensure a strictly fair comparison on an identical test set, we audited our evaluation pipeline. Our designed evaluation set for liver metastasis contains 71 samples (20 positive, 51 negative; patient-level split with no train–test overlap). However, When evaluating the baselines, we inadvertently omitted the necessary data-cleaning and filtering criteria, resulting in an unpolished 80-sample set (23 positive, 57 negative).

Below, we provide a thorough clarification of the two necessary filtering rules, followed by fair comparisons under both settings.

1. Explanation of Sample Differences:

  • Time-window filtering (≤\le 730 days, 6 samples excluded). Because the size-forecasting test set contained only 1 sample exceeding 730 days, lacking statistical power for duration-stratified analysis. So we standardized the evaluation horizon across all tasks to ≤\le 730 days. Omitting this filter, baseline evaluate the 6 cases below. All dates in the table have been de-identified:
Patient IndexPrior CT DateFollow-up CT DateInterval (Days)Label
12013-04-212017-04-261466Negative
22011-04-272013-07-16811Positive
32013-05-252019-07-212248Negative
42016-04-072022-08-172323Positive
52020-04-242022-09-03862Negative
62017-05-252024-01-282439Negative
  • Follow-up CT preprocessing failure (3 samples excluded). Because TumorCast uses follow-up CTs for training supervision, so we retained only successfully preprocessed CT pairs in both training and test sets. During baseline evaluation, we overlooked this filter because the baselines use only the current CT, inadvertently including 3 samples below.
Patient IndexPrior CT DateFollow-up CT DateInterval (Days)LabelExclusion Reason
72014-09-232015-04-05194PositiveFuture CT file corrupted; skipped
82016-03-272016-05-1751NegativeFuture CT file corrupted; skipped
92015-12-312016-07-15197NegativeFollow-up CT liver segmentation failed; skipped

2. Supplementary Comparisons on Identical Test Sets.

To provide a strictly fair comparison, we re-scored all baselines by excluding the aforementioned 9 samples from their previous predictions. For the supplementary comparison, we evaluated TumorCast on the complete set of 80 samples using the original model weights and classification thresholds.

  • Unified 71-Sample Test Set (20 positive, 51 negative). TumorCast-9B and TumorCast-27B achieve BAcc scores of 77.16% and 78.68%, respectively, exceeding the top-performing baseline (Qwen3.5-27B VLM SFT, 75.93%) by 1.23% and 2.75%.
ModelAUPRC ↑Sen. ↑Spec.↑BAcc ↑
GPT-5.6 Sol53.1210.00100.0055.00
GPT-6 Astra56.885.00100.0052.50
Claude Opus 555.5015.0096.0855.54
Gemini 3.8 Flash53.4125.0098.0461.52
MedGemma 1.5-4B ZS37.3930.0068.6349.31
Qwen3.5-9B VLM ZS50.8325.0098.0461.52
Qwen3.5-27B VLM ZS56.0435.0084.3159.66
MedGemma 1.5-4B SFT38.9965.0056.8660.93
Qwen3.5-9B VLM SFT60.1760.0084.3172.16
Qwen3.5-27B VLM SFT71.2695.0056.8675.93
Clin-JEPA64.2060.0084.3172.16
Qwen3.5-9B text SFT61.4795.0054.9074.95
Qwen3.5-27B text SFT68.5770.0080.3975.20
TumorCast-9B63.1070.0084.3177.16
TumorCast-27B71.4375.0082.3578.68
  • Raw 80-Sample Test Set (23 positive, 57 negative). The table below presents the supplementary results for TumorCast alongside selected SFT and commercial baselines from the original manuscript. TumorCast-9B and -27B achieve BAcc scores of 79.94% and 81.24%, respectively, outperforming the best baseline (Qwen3.5-27B text SFT, 78.18%) by 1.76% and 3.06%.
ModelAUPRC ↑Sen. ↑Spec. ↑BAcc ↑
GPT-5.6 Sol ZS45.758.70100.0054.35
GPT-6 Astra ZS51.254.35100.0052.17
Claude Opus 5 ZS52.0513.0496.4954.77
Gemini 3.8 Flash ZS49.4121.7498.2559.99
MedGemma 1.5-4B SFT38.7265.2254.3959.80
Qwen3.5-27B VLM SFT70.6595.6554.3975.02
Qwen3.5-27B text SFT71.1773.9182.4678.18
TumorCast-9B71.3573.9185.9679.94
TumorCast-27B72.9478.2684.2181.24

These results demonstrate that TumorCast maintains advantage for liver metastasis forecasting on both test sets. In the final version, we will consistently report results on the unified 71-sample set.

AuthorsOpenReview ↗

Author Response to Reviewer 1edU (Part 2)

(W2 & Q2) Clarification of interval-Treatment inputs.

We thank the reviewer for the attention to the timing of treatment inputs. We use interval action to refer to the treatments a patient received between two consecutive CT scans, recording only factual details such as treatment names, timing, and duration.

1. Reasons for defining interval actions

Our study focuses on how a patient’s multimodal state evolves from one observation to the next for tumour forecasting. The ground truth for key tumour attributes were extracted from 3D CT images or reports, while the available CT and EHR data do not provide a synchronous CT for every treatment event. Therefore, we use sparse CT examinations as state-observation endpoints and aggregate treatments between scans into interval actions.

2. Construction details

  • History and interval treatments. Treatments before the first CT are included in clinical history. Treatments recorded between two CT scans form the interval action and are paired with the observed follow-up CT to construct a supervised sample. Using only treatments known at the first CT would omit later treatments, leaving an incomplete treatment trajectory for predicting the follow-up CT.

  • Input content. The interval action records only the treatments delivered, timing, and course details, without response assessments, treatment reasons, or other clues about disease status. This minimises the influence of extra clinical information in the treatment description.

(W3 & Q3) Direct-Prediction Baseline with the Same 3D Input

We thank the reviewer for suggesting this important ablation. To isolate the contribution of the dynamics predictor, we add a Direct-3D-encoder-27B baseline and compare it with the full TumorCast and the original predictor-bypass setting. All three models use the same trained M3D encoder and 27B reader architectures. Direct-3D predicts future outcomes directly from the current CT features without using the predictor.

  • Training setting. TumorCast and Direct-3D initialise their bridge and 27B reader from the same pretrained weights and train them under identical optimisation settings.

  • Test setting. All three models are evaluated on the same 171 tumour-size samples and 71 liver-metastasis samples (20 positive and 51 negative). This matches the evaluation cohort specified in our response to W1 & Q1.

MetricTumorCast-27BTumorCast with Predictor BypassedDirect-3D-encoder-27B
Size BAcc ↑48.6643.8644.04
Diameter MAE (mm) ↓10.1911.3110.53
Liver-metastasis BAcc ↑ (71 samples)78.6877.1671.27

In this experiment, TumorCast-27B has higher size-forecasting BAcc and liver-metastasis BAcc than the specifically trained Direct-3D baseline by 4.62% and 7.41%, respectively. The results support the contribution of explicitly modelling latent tumour-state evolution.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.