- Rating
- 1
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 1
Read review
Paper summary
This paper proposes CoSeg-MARL, a communicative multi-agent reinforcement learning framework for iterative brain tumor segmentation from multimodal MRI. Three agents specialize in whole tumor (WT), tumor core (TC), and enhancing tumor (ET), exchanging information through shared representations and explicit message passing. Each agent predicts global segmentation controls and a coarse spatial refinement grid, which a deterministic applicator converts into updated predictions. Training combines supervised Dice and binary cross-entropy losses with an actor–critic objective and hierarchy-aware rewards. Evaluation uses 2D slices from a 200-patient subset of BraTS 2020 with patient-level partitioning. The paper reports mean Dice scores of 0.821 for cold-start refinement and 0.916 for refinement initialized by a pretrained U-Net, compared with 0.826 for the backbone alone.
Critical strengths
-
The agent formulation reflects the structure of the segmentation task. Assigning agents to WT, TC, and ET provides a coherent way to model region-specific refinement while accounting for their nested anatomical relationships. Combining global controls with spatial refinement coefficients also offers a flexible action representation.
-
The reported warm-start improvements are substantial across multiple segmentation measures. Table 1 reports improvements over the pretrained backbone in Dice and ASSD for all three regions. Patient-level partitioning is also an appropriate precaution against leakage between slices from the same patient. If verified, these results would support the potential value of the complete refinement pipeline.
Critical weaknesses
-
Inconsistent Dice and IoU values. Table 1 reports warm-start WT Dice of 0.927 and IoU of 0.725. For the same binary masks and consistent averaging, these values are incompatible: mean Dice of 0.927 implies mean IoU of at least approximately 0.864. Similar inconsistencies appear in other entries. This raises a substantial concern about metric computation or reporting.
-
The contribution of reinforcement learning is not established. The actor also receives direct Dice and BCE supervision, but no supervised-only refinement baseline is provided. Consequently, the reported gains cannot be attributed specifically to reinforcement learning.
-
The benefit of multi-agent cooperation is not isolated. There is no comparison against a single policy jointly refining all three regions or against agents without explicit communication. Since all agents already access shared masks, logits, and representations, the additional value of message passing remains unclear.
-
Hierarchy preservation is motivated but not directly evaluated. The paper does not report hierarchy-violation rates or isolate the effects of the hierarchy reward and hierarchy-control action. Improved Dice alone does not establish more anatomically consistent predictions.
-
The deterministic applicator is insufficiently specified. The paper describes threshold adjustment, temperature scaling, morphology, and hierarchy enforcement without giving the exact update equations, parameter mappings, or operation order. These choices determine the transition dynamics and are essential for reproduction.
-
The optimization procedure is incomplete and potentially inconsistent with the stated algorithm. The hybrid loss weights and gradient paths are unspecified. Moreover, the critics are described as receiving each agent’s own action rather than the joint action, leaving the relationship to standard MADDPG unclear.
-
The fairness of the baseline comparison is uncertain. Table 2 does not clearly explain whether all methods were retrained on the same subset and evaluated using identical preprocessing, patient splits, and slice-level metrics. References associated with some entries concern different segmentation settings, making the provenance of the reported numbers particularly important.
-
The evaluation has limited coverage. Experiments use a selected 200-patient subset of one dataset and report slice-level averages. The criteria for selecting patients and informative slices are insufficiently defined, and volumetric performance is not evaluated. This limits conclusions about full-volume brain tumor segmentation.
-
The test-time refinement procedure is unclear. Training termination can depend on ground-truth Dice, but the inference stopping rule and maximum number of refinement steps are not specified. These details affect both reproducibility and the validity of the reported evaluation.
10 Computational cost and training stability are not assessed. The framework adds iterative actor evaluations and multiple critics during training, but reports no inference latency, memory usage, parameter counts, or variability across independent runs. The practical cost of the reported improvements therefore remains unknown.
- Several reporting discrepancies require correction. Table 2 gives a Dice standard deviation of 4.80, whereas the accompanying text states 2.64%. Distance units are unclear for the 2D evaluation, and ASSD is compared with ABD without establishing equivalence. These discrepancies compound concerns about the reliability of the results.
Questions for authors
-
Were Dice and IoU calculated from identical binary masks, samples, and averaging weights? How do you explain the incompatible values in Table 1, particularly warm-start WT Dice of 0.927 versus IoU of 0.725?
-
How are slices with absent WT, TC, or ET handled when computing Dice, IoU, HD95, ASSD, and RVD? Are the same slices included in every metric?
-
What precisely defines an “informative” slice, and how were the 200 patients selected? Were these criteria fixed before training, and do they use tumor annotations?
-
Are the final table results calculated exclusively on the 30 test patients? Which split is used for the Dice curves in Figure 4, checkpoint selection, and hyperparameter tuning?
-
What stopping criterion is used when ground-truth masks are unavailable? Please specify the maximum refinement horizon and confirm that test labels are not used to select the best intermediate prediction.
-
What are the exact equations and operation order for converting the four global controls and spatial grid into updated logits and masks? How are conflicting hierarchy controls from different agents resolved?
-
What is the complete actor objective, including all loss weights? Through which outputs and operations are the supervised gradients propagated, particularly when thresholding and morphology are involved?
-
Do the critics receive the joint actions of all agents or only their respective agent’s action? If only the latter, how are the effects of other agents’ simultaneous updates accounted for?
-
Which Table 2 results were obtained through your own experiments, and which were taken from previous publications? Were all entries evaluated on the same BraTS version, patient split, tumor targets, and metric definitions?
10 Does the full method outperform the same refinement actor trained using only supervised losses under an identical training and inference budget?
-
What happens when explicit communication is removed, when one policy predicts all three regions, and when hierarchy penalties are disabled? Please report both segmentation accuracy and hierarchy-violation rates.
-
How consistent are the gains across independent training seeds and individual test patients? What inference time and number of refinement steps are required relative to the pretrained U-Net?
Since I have not found the authors' responses to my concerns, I will keep my rating.