acceptodds
Under review as a conference paper at ICLR 2027

NeuroBelief: Observer-Centric World Modeling of Information Acquisition in Multisequence MRI

Abstract

World models typically predict how a system changes after an action. In information acquisition, however, the underlying system may remain unchanged while the observer's task state changes as new evidence becomes available. This raises a different prediction problem: before acquiring a candidate observation, can we anticipate how it would update the current task state? We study this problem in multisequence brain MRI and introduce NeuroBelief. The currently observed sequences are encoded into a spatial Pathology Belief. Given this belief and the identity of a candidate sequence, NeuroBelief predicts the Pathology Belief that would be formed after acquisition, without access to the candidate image. Once the sequence is acquired, the same encoder constructs the corresponding target state, allowing the predicted transition to be evaluated directly. On BraTS 2023 and BraTS Africa, the predicted transitions depend on both the current patient state and the candidate sequence, and generalize to observation and acquisition combinations excluded from transition training. Using these predicted states for sequential acquisition, NeuroBelief reduces the number of acquired sequences by 48.4% internally and 44.7% externally relative to acquiring all four sequences, while mean Dice remains 0.68 and 1.41 percentage points below the corresponding four sequence references. These results show that world modeling can also be used to predict how new observations change an observer's task state, providing a directly testable basis for adaptive information acquisition. Source code is provided in the supplementary material.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
world models, information acquisition, predictive state modeling, multisequence MRI

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_A69KOpenReview ↗
Rating
2
Confidence
3
Informativeness
3
Presentation
3
Soundness
2
Read review
Paper summary

This paper introduces NeuroBelief, an observer-centric framework for predicting how acquiring an additional MRI sequence would change a task-specific spatial representation. A Belief Encoder constructs a deterministic “Pathology Belief” from available sequences, and an action-conditioned transition model predicts the representation after acquisition without accessing the candidate image. A Value Head estimates segmentation performance to guide sequence selection and stopping. Experiments on BraTS 2023 and BraTS Africa evaluate transition prediction, generalization to withheld transition combinations, and sequential acquisition.

Critical strengths
  • Novel and clearly-articulated problem setting. Moving from "world models predict how a system changes" to "predicting how an observer's task state updates as evidence arrives," with a testable predict-then-verify loop (Fig. 1), is more principled than treating missing-modality MRI as a simple re-sampling problem.
  • Careful control design. The paper includes meaningful predictive baselines (Identity, Spatial Population, Direct Logit Delta, Global, Seen Spatial Population), and reports paired 95% CIs and matched-budget sensitivity analyses (Table 7), indicating an honest approach to fair comparison.
Critical weaknesses
  • Unclear necessity of spatial state prediction. Acquisition ultimately depends on predicted Dice, yet a matched direct Dice/gain regression baseline is missing. Direct Logit Delta nearly matches spatial accuracy, leaving the benefit of predicting a full latent state insufficiently isolated.
  • Deterministic predictions without outcome uncertainty. Pathology Belief is explicitly a deterministic representation rather than a probabilistic posterior. The model therefore does not characterize alternative possible acquisition outcomes or provide uncertainty-based stopping guarantees.
  • One-step greedy acquisition. The policy evaluates immediate predicted gains rather than performing multistep planning. It may overlook complementary sequences whose value emerges primarily when acquired together.
  • Limited acquisition advantage. Population Fixed achieves slightly higher internal Dice with fewer sequences. At the primary threshold, budget-matched differences are negative internally and small externally. Paired confidence intervals are needed to establish the reliability of the policy-level differences.
  • Incomplete evaluation of inference-time decisions. Table 1’s task-consequence metrics use ground-truth Dice of decoded predictions rather than Value Head estimates. Although valid retrospectively, these results do not establish accurate inference-time candidate ranking, gain estimation, or stopping calibration.
Questions for authors
  • Please provide the missing architectural and preprocessing details, λ_mse, λ_mean, ε_rank, temperature T, baseline training protocols, and code location, and reconcile the inconsistencies in Figure 5 and the abstract.
  • Are results available for directly predicting candidate Dice or gains using the same Belief Encoder and comparable supervision? If not, what existing evidence distinguishes the benefit of state prediction from that of the encoder and value-learning objectives?
  • How would this policy actually be deployed in a real clinical scan? Given that the data already contain all four sequences, does "reducing 48.4%" imply a genuine decrease in scan time / cost? Can you provide a proxy model for acquisition time or cost?
  • Why start from T1c rather than the more common clinical starting set T1c+T2? How do reduction and Dice vary across different starting points?
5 comments

We thank the reviewer for their careful review. Our point-by-point responses are provided below:

Reply to Q1: Model Architecture. The Belief Encoder uses a 3D CNN with three levels and 32, 64, and 128 channels, respectively. Four MRI sequences are provided in a fixed order, with missing channels set to zero. A sequence availability mask is embedded and added to the spatial features to obtain the current state, BtB_t. Spatial Dynamics maps the candidate sequence identity to a vector of 128 dimensions, concatenates it with BtB_t at each spatial location, and predicts a residual ΔB\Delta B, yielding B^t+1=Bt+ΔB\hat B_{t+1}=B_t+\Delta B.

Segmentation Training and Evaluation. We train the segmentation model using MRI patches of 128×128×128128\times128\times128 voxels. At testing, a sliding window of the same size covers each patient’s entire MRI volume. Window predictions are combined into a complete segmentation map, which is compared with manual annotations to calculate DSC for whole tumor (WT), tumor core (TC), and enhancing tumor (ET).

Hyperparameters, Baselines, and Code. We use λmse=0.1\lambda_{\mathrm{mse}}=0.1, λmean=0.5\lambda_{\mathrm{mean}}=0.5, εrank=0.01\varepsilon_{\mathrm{rank}}=0.01, and temperature T=0.05T=0.05. RFNet, mmFormer, M3AE, and UniME in Table 3A follow their respective official training protocols and retain their training settings. All methods use the same patient splits and available sequence combinations. M3AE and UniME retain both pretraining and segmentation fine tuning stages.

The model is described in the supplementary material. We will provide a public repository containing the model implementation, training scripts, and evaluation code.

Figure 5 and the Abstract. Figures 5a and 5b show stopping counts, totaling 258/500 acquisitions on BraTS 2023 and 210/380 on BraTS Africa, corresponding to the abstract’s reductions of 48.4% and 44.7%. Their colors indicate whether final DSC is within 0.01 below each patient’s reference using all four sequences. Figure 5c shows the cumulative proportion of patients who have stopped and met this criterion. The abstract reports the mean DSC decreases across all patients, 0.68 and 1.41 percentage points, respectively. The 0.01 tolerance is used for retrospective evaluation, while τ=0.03\tau=0.03 determines stopping.

Reply to Q2: We compared Direct Q with NeuroBelief using the same frozen Belief Encoder, regional DSC supervision, and value loss. Given the current patient state and a candidate sequence, Direct Q predicts the WT, TC, and ET DSC values after acquiring that sequence. NeuroBelief first predicts the resulting spatial state and then estimates the regional DSC values.

Test CohortMethodBest Candidate Hit Rate ↑Final DSC ↑Cases with Remaining DSC Gain >0.03 ↓Acquisition Reduction (%) ↑
BraTS 2023 (125 cases)NeuroBelief83.60%0.847 ± 0.1294/125 (3.2%)48.4%
Direct Q82.96%0.844 ± 0.11915/125 (12.0%)52.2%
BraTS Africa (95 cases)NeuroBelief76.74%0.819 ± 0.12817/95 (17.9%)44.7%
Direct Q77.37%0.814 ± 0.12925/95 (26.3%)50.8%

Best candidate hit rate measures how often the model selects the sequence with the greatest actual DSC gain from the same observed state. The remaining metrics come from complete closed loop evaluations starting from T1c, with acquisition reductions calculated relative to all four sequences. Stopping thresholds were selected using the same development set criteria, yielding 0.03 for NeuroBelief and 0.05 for Direct Q. After stopping, we checked whether any remaining sequence could increase the actual mean DSC by more than 0.03.

Both methods achieved similar candidate hit rates. Direct Q used fewer sequences, while NeuroBelief achieved a slightly higher mean final DSC and stopped less frequently when a remaining sequence could provide a gain above 0.03.

After acquisition, the spatial prediction can be compared at each location and channel with the state produced by the same frozen encoder from the updated observations. In Table 2, for sequence transitions unseen during dynamics training, NeuroBelief achieved Latent L1 errors of 0.111 on BraTS 2023 and 0.118 on BraTS Africa. Direct Q outputs only regional DSC values, so this spatial metric does not apply to it.

Reply to Q3: The 48.4% figure represents the average reduction in the number of acquired sequences compared to the full four-sequence protocol within the BraTS 2023 internal test cohort. During retrospective evaluation, the model is restricted to using only the MRI data from the sequences already selected at each step. In a clinical workflow, after T1c and each subsequent acquisition, the model would update the patient state and recommend another sequence or stopping, subject to clinical protocol requirements.

We use the acquisition duration of each sequence as a proxy for cost. According to the routine 3T MRI protocol for gliomas reported by De Luca et al. [1], the acquisition times are 3 minutes and 50 seconds for both T1n and T1c, 2 minutes and 40 seconds for T2w, and 4 minutes and 08 seconds for FLAIR. We calculated the total duration for the selected sequences on a case-by-case basis and then determined the mean for the entire cohort. Both approaches include T1c, and its imaging time is factored into the estimates; however, the time required for contrast agent injection and the subsequent waiting period for enhancement is excluded.

Test CohortDuration for Full Four-Sequence ProtocolMean Estimated Duration (NeuroBelief)Reduction in Estimated Duration
BraTS 2023 (125 cases)14 min 28 sec8 min 07 sec43.9%
BraTS Africa (95 cases)14 min 28 sec8 min 30 sec41.3%

[1] De Luca F, et al. Diagnostic Performance of Fast Brain MRI Compared with Routine Clinical MRI in Patients with Glioma Grades 3 and 4: A Pilot Study. AJNR, 2025.

Reply to Q4: We start with a single sequence so that the strategy can decide whether to add T2w based on each patient’s available images. On the development set of 125 patients, the mean DSC across WT, TC, and ET was 0.814 for T1c, 0.712 for T2w, 0.703 for FLAIR, and 0.648 for T1n. We therefore selected T1c as the starting sequence.

Using the same frozen model, we compared four individual starting sequences and the T1c + T2w starting combination. For each configuration, the stopping threshold was selected using the same development set criterion. Acquisition reductions are relative to all four sequences, with the starting sequences included in the total. Table 1. Acquisition reduction and final DSC for different starting configurations

Starting ConfigurationBraTS 2023 Acquisition Reduction (%) ↑BraTS 2023 Final DSC ↑BraTS Africa Acquisition Reduction (%) ↑BraTS Africa Final DSC ↑
T1c48.40 ± 6.920.847 ± 0.12944.74 ± 10.250.819 ± 0.128
T1c + T2w49.40 ± 3.840.842 ± 0.12349.74 ± 2.560.815 ± 0.128
T1n25.40 ± 3.150.847 ± 0.13228.68 ± 9.630.816 ± 0.138
T2w49.60 ± 4.470.841 ± 0.12649.74 ± 2.560.815 ± 0.128
FLAIR49.40 ± 3.840.849 ± 0.12047.37 ± 7.710.814 ± 0.131

Compared with starting from T1c + T2w, starting from T1c yields a smaller acquisition reduction and a slightly higher mean final DSC in both cohorts.

We would like to thank the reviewer once again for their careful review; we will incorporate the necessary revisions and additions.

Dear Reviewer,

I hope this message finds you well. As the discussion period is nearing its end with less than eight hours remaining, I wanted to ensure we have addressed all your concerns satisfactorily. If there are any additional points or feedback you'd like us to consider, please let us know. Your insights are invaluable to us, and we're eager to address any remaining issues to improve our work.

Thank you for your time and effort in reviewing our paper.

Reviewer_4TuEOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper treats MRI sequence acquisition as "observer-centric world modeling", the pathology is fixed, and acquiring a sequence changes only the observer's belief. A belief encoder maps the available sequences to a spatial belief, supervised with tumour segmentation. A residual dynamics model predicts the belief after acquiring a candidate sequence without seeing it, and a value head predicts Dice and drives greedy selection with a stopping threshold. On BraTS 2023, with external testing on BraTS-Africa, the paper reports that predicted belief changes beat several controls, that the model generalises to withheld transitions, that the encoder beats missing-modality segmenters, and that closed-loop acquisition from T1c cuts sequences by 48.4% (internal) and 44.7% (external) with Dice close to fixed protocols.

Critical strengths
  1. The formulation is clean and directly checkable. Predicted and realised beliefs are in the same space, so transitions can be verified directly, and the withheld-transition protocol is a reasonable test of compositional generalisation. This makes the paper's claims testable in a way many acquisition papers are not.
  2. The transition-model controls and external test are careful. The paper compares against a population-mean update, a spatially pooled control, a direct logit-change predictor and patient/candidate permutation tests, and evaluates externally on BraTS-Africa without retraining, reporting the drop honestly. The limitations are stated clearly.
Critical weaknesses
  1. The paper assumes the most useful next sequence differs between patients, but in this setting it mostly doesn't. After T1c, the policy picks FLAIR and stops for 91% of internal and 77% of external cases (Table 8). It doesn't beat the Population Fixed policy (Table 3B), and even a per-patient oracle gains under 0.01 Dice over the best fixed path (Table 6). So the 48% reduction really comes from using two sequences instead of four, which a fixed protocol also achieves. The setup also removes what would make adaptivity matter in practice. Every run starts from post-contrast T1c, so T1n is never acquired. All sequences cost the same. "Acquiring" a sequence just means unmasking a volume that is already registered and skull-stripped. The only goal is segmentation Dice, not a diagnostic decision. The paper shows that a world model can be trained on this benchmark, but not that adaptive acquisition helps. To support the motivation, it needs a setting where the value of a sequence actually varies, e.g. different starting sequences, unequal costs, or a diagnostic endpoint.

  2. The model is a deterministic, one-step regressor of frozen encoder features, used greedily over at most three candidates, with no rollouts, planning or uncertainty. Predicting how an observation changes a belief is already the standard setup in belief-state POMDPs, PSRs and active feature acquisition, all of which the paper cites. What's missing is evidence that predicting the belief is needed at all. The natural comparisons are a simpler model that predicts each sequence's value directly from the current belief, and an AFA method like EDDI run in closed loop. Without these, the framing and the title promise more than the method delivers.

Questions for authors
  1. In what clinical or task setting would per-patient sequence choice beat a fixed protocol, and can you show it (e.g., free or T1n-first start, per-sequence costs, or an endpoint other than Dice)?
  2. Is Population Fixed effectively T1c → FLAIR → stop chosen on development data, and is there any statistically significant closed-loop difference between it and NeuroBelief at matched budget?
  3. What does predicting the post-acquisition belief enable that a direct value predictor Q(B_t, a) or an AFA baseline cannot, and how do those alternatives perform in closed loop?
  4. Is the stopping rule driven by signal? The predicted-gain error (MAE 0.054 internal, 0.083 external) is larger than the stopping threshold τ = 0.03.
4 comments

We thank the reviewer for their careful review. Our point-by-point responses are provided below: Reply to Q1: We consider a single MRI examination in which the images already acquired guide which sequence to acquire next and when to stop. A fixed strategy applies a path determined on the development set to all patients; NeuroBelief reassesses the benefit of further acquisitions for each patient after every sequence. In Table 3B, the fixed strategy stops after T1c and FLAIR for all patients in BraTS Africa, whereas NeuroBelief acquires a third sequence for 20 patients.

NeuroBelief and the fixed strategy achieve mean final DSC values of 0.819 and 0.809, with acquisition reductions of 44.7% and 50.0%, respectively.

The comparison at a matched average acquisition budget is provided in our response to Question 2.

We selected T1c because it achieved the highest mean DSC when used alone on the development set: 0.814 for T1c, 0.712 for T2w, 0.703 for FLAIR, and 0.648 for T1n. Following your suggestion, we also evaluated the same model starting from T1n, allowing it to select subsequent sequences and stop autonomously. Using the same development set criteria, we selected T1n + T1c + FLAIR as the fixed protocol, corresponding to a 25% acquisition reduction. All reductions are relative to acquiring four sequences, with the starting sequence included in the acquisition count.

Test CohortNeuroBelief Acquisition Reduction (%) ↑Fixed Strategy Acquisition Reduction (%) ↑NeuroBelief Final DSC ↑Fixed Strategy Final DSC ↑
BraTS 2023 (125 cases)25.40 ± 3.1525.00 ± 0.000.847 ± 0.1320.852 ± 0.118
BraTS Africa (95 cases)28.68 ± 9.6325.00 ± 0.000.816 ± 0.1380.825 ± 0.129

Starting from T1n, NeuroBelief uses fewer sequences, while the fixed strategy achieves a higher mean DSC.

Our current endpoint is the mean DSC across WT, TC, and ET, with acquisition decisions based on predicted DSC gains. We agree that other clinical endpoints merit investigation, such as whether the selected sequences support accurate identification of tumor progression, and could guide future acquisition strategies.

Reply to Q2: Yes. At the primary threshold of τ = 0.03, Population Fixed uses the average DSC gains on the development set to determine the path T1c → FLAIR → Stop. It acquires these two sequences for all patients in both test cohorts, reducing acquisitions by 50% relative to the complete protocol of four sequences.

The table reports results at matched average acquisition budgets. Population Fixed values are interpolated references, and paired differences are calculated as NeuroBelief minus the reference.

Test CohortMatched Acquisition ReductionNeuroBelief DSC ↑Population Fixed DSC ↑Paired Difference and 95% CI
BraTS 2023 (125 cases)48.40%0.8470.848−0.001 [−0.008, 0.004]
BraTS Africa (95 cases)44.74%0.8190.812+0.007 [−0.000, 0.015]

At matched average acquisition budgets, the mean DSC differences were not statistically significant in either cohort.

However, Population Fixed reflects average acquisition patterns in a limited development cohort, whereas NeuroBelief flexibly adjusts the acquisition path based on each patient’s available images. After each acquisition, it reassesses whether to continue and which sequence to select.

Reply to Q3: NeuroBelief predicts the spatial state after acquiring a candidate sequence, which can be decoded by the segmentation head to identify where segmentation labels may change. For example, it can separately predict the states resulting from adding FLAIR or T2w. Once FLAIR is acquired, the same encoder produces the updated state, allowing comparison with the prediction at each spatial location. Direct Q predicts each candidate’s regional DSC, while our MRI adaptation of EDDI ranks candidates by expected information gain [1]. Neither directly outputs this verifiable spatial state. In Table 2, for sequence transitions unseen during training, NeuroBelief achieves L1 errors of 0.111 and 0.118 on BraTS 2023 and BraTS Africa, compared with 0.159 and 0.164 for population average updates.

In the closed loop evaluation, all three strategies start from T1c and reassess the remaining candidates after each acquisition to select another sequence or stop. Their stopping thresholds are selected on the same development set using the same criteria. The same frozen segmenter evaluates each patient’s final DSC.

StrategyBraTS 2023 Acquisition Reduction (%) ↑BraTS 2023 Final DSC ↑BraTS Africa Acquisition Reduction (%) ↑BraTS Africa Final DSC ↑
NeuroBelief48.40 ± 6.920.847 ± 0.12944.74 ± 10.250.819 ± 0.128
Direct Q52.20 ± 8.410.844 ± 0.11950.79 ± 5.710.814 ± 0.129
EDDI49.00 ± 5.850.848 ± 0.11947.63 ± 7.360.810 ± 0.136

Acquisition reductions are relative to all four sequences and include the initial T1c acquisition. Direct Q achieves the greatest reduction in both cohorts. EDDI achieves a slightly higher mean final DSC on BraTS 2023, while NeuroBelief achieves the highest on BraTS Africa.

We also checked whether any remaining sequence could improve the actual mean DSC by more than 0.03 after stopping. Using this common retrospective criterion, the proportions for NeuroBelief, Direct Q, and EDDI were 4/125 (3.2%), 15/125 (12.0%), and 9/125 (7.2%) on BraTS 2023, and 17/95 (17.9%), 25/95 (26.3%), and 22/95 (23.2%) on BraTS Africa, respectively. NeuroBelief therefore stopped less frequently while a remaining sequence could still provide this gain.

[1] Ma C, Tschiatschek S, Palla K, et al. EDDI: Efficient Dynamic Discovery of High-Value Information with Partial VAE. ICML, 2019.

Reply to Q4: At each step, the Value Head estimates the DSC gain from each candidate sequence relative to the current state. If the maximum predicted gain exceeds τ\tau, the strategy acquires that sequence and updates the state using the acquired image; otherwise, it stops.

The gain errors of 0.054 and 0.083 in Table 1 evaluate decoded spatial predictions against actual task consequences using ground truth annotations. They do not directly measure the error of the Value Head scores used for stopping.

We selected τ\tau on the development set of 125 cases. Using the mean DSC of 0.860 with all four sequences as the reference, we chose the threshold that maximized acquisition reduction while keeping the decrease in mean DSC within 0.01.

Development Set ConfigurationAcquisition Reduction (%) ↑Mean DSC ↑DSC Decrease
All four sequences0.0%0.860—
τ=0.00\tau=0.0038.4%0.8580.002
τ=0.01\tau=0.0144.6%0.8580.002
τ=0.02\tau=0.0246.8%0.8570.003
τ=0.03\tau=0.0347.6%0.8560.004
τ=0.05\tau=0.0556.4%0.8490.011

Among the evaluated thresholds, τ=0.03\tau=0.03 maximized acquisition reduction within the specified DSC tolerance and was fixed for both test cohorts.

We would like to thank the reviewer once again for their careful review; we will incorporate the necessary revisions and additions.

Thank you for the detailed rebuttal and running all the requested experiments. My main concern is still that NeuroBelief and Population Fixed are not significantly different in either cohort. Considering I am already giving a score of 3, I personally cannot raise my score further unless the authors can find a setting where sequence value truly varies across patients to support the central claim that belief-predictive per-patient acquisition improves on simpler alternatives.

We thank the reviewer for their response and for carefully evaluating our additional experiments. While we understand your decision to maintain the original rating, we would like to offer further patient-level evidence in the hope that you might reconsider our work.

The core issue you raised concerns whether the value of a sequence truly varies across patients and whether such variation can be effectively incorporated into acquisition decisions. To this end, we calculated, for each method, the proportion of patients for whom at least one unacquired sequence could still have improved the actual mean DSC by more than 0.03 after acquisition had stopped:

MethodInternal cohort: Avg. number of sequencesInternal cohort: Remaining DSC gain >0.03External cohort: Avg. number of sequencesExternal cohort: Remaining DSC gain >0.03
Population Fixed2.0008/125 (6.40%)2.00026/95 (27.37%)
Direct Q1.91215/125 (12.00%)1.96825/95 (26.32%)
EDDI2.0409/125 (7.20%)2.09522/95 (23.16%)
NeuroBelief2.0644/125 (3.20%)2.21117/95 (17.89%)

These results indicate that, while a T1c+FLAIR protocol is sufficient for most patients, a subset of patients can still obtain substantial segmentation gains from additional sequences. Specifically, after the fixed protocol ended, 8 patients in the internal cohort and 26 patients in the external cohort could have gained more than 0.03 in mean DSC from an additional acquisition. Consequently, a streamlined protocol that is effective at the population level does not imply that the value of additional acquisitions is uniform across patients.

At each method’s actual stopping point, NeuroBelief leaves 4 such cases in the internal cohort and 17 in the external cohort, fewer than any of the compared methods in both cohorts. The table also reports the average number of acquired sequences, presenting acquisition effort alongside the remaining gains at stopping for each strategy. This provides a complementary perspective to average DSC: even when methods yield similar average segmentation performance, they may differ in the proportion of patients with substantial unrealized gains at the point of stopping.

Together with the transition prediction and patient-swapping experiments presented in the manuscript, these results highlight the capabilities explored by NeuroBelief: maintaining streamlined acquisition paths suitable for the majority of patients while using patient-specific outcome predictions to guide decisions on whether to continue or stop acquisition. We hope these supplementary results will encourage you to reconsider our work’s overall contribution in terms of patient-level acquisition needs, state prediction, and stopping decisions.

Reviewer_yXnsOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

This paper formulates information acquisition as predicting changes in an observer’s task state rather than changes in the underlying physical system. Its main contribution is NeuroBelief, which predicts the spatial task representation that would result from acquiring a candidate MRI sequence, using only the current representation and the candidate’s identity. The predicted state can be compared with the state encoded after acquisition and is also used to guide sequence selection and stopping. Experiments on BraTS 2023 and BraTS Africa assess prediction accuracy, dependence on patient state and candidate identity, and generalization to transition combinations withheld from dynamics training. In sequential acquisition, the method reduces the number of sequences both internally and externally, with mean Dice decreases relative to the four-sequence reference.

Critical strengths
  1. The paper makes a useful distinction between predicting the consequence of an acquisition and directly choosing an acquisition. The frozen encoder provides a consistent target for evaluating the predicted state, and the experiments go beyond downstream segmentation accuracy. In particular, the conditioning controls and withheld-transition evaluation help establish that the model learns patient- and candidate-dependent updates rather than simply reproducing an average modality effect.

  2. The evaluation includes an external cohort without adaptation and gives a fairly measured account of the acquisition results. The authors report the remaining gap to the four-sequence reference, acknowledge the limited internal advantage over population policies, and provide matched-budget comparisons. This makes the practical findings easier to assess than a sequence-reduction figure alone would.

Critical weaknesses
  1. The stopping threshold of 0.03 is selected on the development set, but the selection criterion is not specified. Although Table 7 reports results at alternative thresholds, the paper should explain what performance–acquisition trade-off motivated the primary setting.

  2. Figure 5 reports that 40 of 95 external cases fall outside the 0.01 Dice tolerance, but not the magnitude of those deviations. Reporting the distribution of patient-level Dice differences relative to the four-sequence reference would clarify whether the small mean loss conceals substantially larger drops in some patients.

  3. The closed-loop results report mean Dice across WT, TC, and ET. Reporting the performance change for each region relative to the four-sequence reference would make it easier to assess whether the small average loss masks a larger decrease in a particular region.

Questions for authors
  1. Could you briefly describe the Belief Encoder backbone and how the candidate embedding conditions Spatial Dynamics? A brief architectural description would help, as Figure 2 and Appendix B mainly describe the workflow and training procedure.
  2. How were the segmentation baselines in Table 3A trained? Were their original method-specific training schedules retained, or was a common training schedule used? This information would help interpret the performance differences.
5 comments

We thank the reviewer for their careful review. Our point-by-point responses are provided below:

Reply to Q1: The Belief Encoder takes four sequences as input, arranged in a fixed order, and feeds them into a three-stage 3D CNN; missing channels are zero-padded. The three stages consist of 32, 64, and 128 channels, respectively, with each stage containing two 3×3×3 convolutional layers, followed by InstanceNorm and ReLU activations. Two downsampling operations with a stride of 2 reduce each spatial dimension of the output to one-quarter of the input size. A sequence availability mask is projected via an MLP into a 128-dimensional space, then broadcast and added to the spatial features to yield the current state, BB.

The Spatial Dynamics module processes the 4D one-hot encodings of the candidate sequences through an MLP (with dimensions 4→256→256) and subsequently projects them into a 128-dimensional space. This vector is broadcast across all spatial locations and concatenated with BB along the channel dimension. Three 3×3×3 convolutional layers then predict a residual, ΔB\Delta B, resulting in a prediction of the next state prior to data acquisition: B^t+1=Bt+ΔB\hat{B}_{t+1} = B_t + \Delta B.

Reply to Q2: RFNet, mmFormer, M3AE, and UniME were all trained according to their respective official protocols, with their original training configurations preserved. For M3AE and UniME, both the pre-training and segmentation fine-tuning stages were maintained.

All methods used the same 1,001 training and 125 development cases from BraTS 2023. The input data were constructed based on the specific combinations of available sequences defined in this study. Table 3A compares the performance of these methods using the same patients and the same subsets of sequences, covering 125 internal test cases and 95 external test cases.

Some notes on Critical Weaknesses:

Reply to W1: On the 125 development cases, we selected the threshold that maximized acquisition reduction while keeping the mean DSC decrease within 0.01 relative to all four sequences. At τ = 0.03, acquisition reduction was 47.6%, with a DSC decrease of 0.004. At τ = 0.05, reduction increased to 56.4%, but the DSC decrease of 0.011 exceeded the specified tolerance.

Reply to W2: Among the 40 external cases whose DSC decreased by more than 0.01 relative to their individual references using all four sequences, the median decrease was 0.029, with an interquartile range of 0.018 to 0.043. Specifically, 33 cases showed a decrease of no more than 0.05, 5 cases showed a decrease greater than 0.05 but no more than 0.10, and 2 cases showed a decrease exceeding 0.10; the maximum observed decrease was 0.156.

Reply to W3: At τ = 0.03, we compared regional DSC at stopping with each patient’s reference using all four sequences. Mean paired differences are final DSC minus the reference, calculated before rounding.

Test CohortTumor RegionDSC With All Four Sequences ↑NeuroBelief Final DSC ↑Mean Paired Difference
BraTS 2023 (125 cases)WT (Whole Tumor)0.914 ± 0.0600.899 ± 0.095−0.015
TC (Tumor Core)0.862 ± 0.1720.862 ± 0.173+0.000
ET (Enhancing Tumor)0.786 ± 0.1970.780 ± 0.204−0.006
BraTS Africa (95 cases)WT (Whole Tumor)0.915 ± 0.0380.906 ± 0.055−0.009
TC (Tumor Core)0.816 ± 0.2080.805 ± 0.209−0.011
ET (Enhancing Tumor)0.770 ± 0.1650.747 ± 0.171−0.023

The largest mean decrease was in WT on BraTS 2023 and ET on BraTS Africa.

We would like to thank the reviewer once again for their careful review; we will incorporate the necessary revisions and additions.

Thank you for reporting the patient-level and regional Dice results. My main remaining question is about the intended clinical use. At the primary stopping threshold, every acquisition path in Table 8 starts with postcontrast T1 and omits precontrast T1. Could you give a concrete example of a patient group and clinical indication for which this shorter protocol would be worthwhile? In that setting, what evidence would show that omitting the sequence preserves the information radiologists need for diagnosis or management? A small change in segmentation Dice alone does not tell me whether the shorter exam would still serve its clinical purpose.

Thank you for raising this clinical question. We share the aspiration that this research will eventually be applied in clinical practice. Achieving this goal requires the collective effort of our team and other researchers. If, in the future, clinical workflows were to be managed by fully autonomous medical agents, these models would be responsible for interpreting pathological information and determining what additional diagnostic examinations are required. Our research specifically explores how a model can decide, based on the information already acquired, whether further data collection is necessary.

Regarding the points you raised:

Specific clinical applications. One potential application we are considering involves patients with a confirmed diagnosis of glioma who have recently undergone a comprehensive diagnostic MRI. In such cases, quantitative assessment of the tumor’s extent is often required for treatment planning. If a reduced set of MRI sequences can provide sufficient spatial information about the lesion, it could help shorten acquisition times and reduce the burden on patients. Similar practices already exist in clinical settings; for instance, the glioma radiotherapy planning protocol at OHSU does not include a pre-contrast T1-weighted (T1n) sequence [1].

Rationale for not selecting T1n. T1n was not selected in Table 8 because the model predicted that this sequence would offer little additional benefit for the current segmentation task. NeuroBelief prioritizes sequences predicted to yield the greatest DSC gain and stops when the maximum predicted gain among all remaining candidates does not exceed the specified threshold. The final segmentation results are then used to evaluate the effectiveness of these selections. Therefore, the decision not to select T1n is based on its predicted marginal utility under the current observational context, rather than an a priori assumption that the sequence is inherently unimportant.

Supporting evidence. There is significant overlap in the information provided by different MRI sequences for segmentation tasks. De Sutter et al. demonstrated that using only T1c and FLAIR sequences can achieve glioma segmentation performance comparable to that obtained using a full four-sequence protocol [2]. This finding aligns with our observation: when the existing sequences already provide the core information necessary for segmentation, the marginal benefit of adding a T1n sequence diminishes.

This evidence provides a basis for reducing the number of sequences acquired for lesion delineation and highlights the potential for this approach in treatment planning. We will add a section on limitations to discuss how, beyond segmentation performance, one might evaluate whether the retained information is sufficient to meet specific clinical diagnostic and therapeutic requirements.

References

[1] OHSU. MR Brain W only—Limited Radiation Therapy Protocols.

[2] De Sutter S, et al. Modality redundancy for MRI-based glioblastoma segmentation. International Journal of Computer Assisted Radiology and Surgery. 2024;19:2101–2109.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.