acceptodds
Under review as a conference paper at ICLR 2027

SymmetryBreak: Resolving Bilateral Ambiguity in Training-Free Brain MRI Anomaly Localization

Abstract

Localizing brain abnormalities from an unlabeled MRI without task-specific train- ing, lesion annotations, or external healthy-brain references remains challenging. Although the brain’s bilateral organization provides an intrinsic within-subject ref- erence, conventional contralateral comparison can identify where the hemispheres differ without determining which side is abnormal, often producing mirrored re- sponses. We introduce SymmetryBreak, a training-free framework that converts bilateral latent asymmetry into directional abnormality localization. Given a sin- gle MRI slice, a frozen vision encoder produces spatially organized latent rep- resentations, which are divided into corresponding left and right hemispheres through deterministic horizontal mirroring. SymmetryBreak first computes a bilat- eral cosine discrepancy map to identify regions of symmetry breaking. It then con- structs an image-internal reference from low-discrepancy correspondences and measures side-specific latent deviations from this reference. A shared-scale direc- tional comparison determines which hemisphere exhibits stronger abnormality, and a global hemisphere decision suppresses mirrored responses before construct- ing a localization map. SymmetryBreak requires no task-specific training or exter- nal healthy-brain atlases. Across BraTS 2019, BraTS 2021, and BraTS-Africa, SymmetryBreak achieves an average Dice Similarity Coefficient (DSC) of 67.4% and an AUPRC of 78.4%, outperforming the evaluated baselines. These results demonstrate that intrinsic anatomical symmetry, combined with image-internal directional inference, can provide a simple training-free paradigm for localized brain abnormality detection without an external normal reference.

Primary area:
unsupervised, self-supervised, semi-supervised, and supervised representation learning
Keywords:
Unsupervised, Zero Shot detection, MRI, Medical Imaging

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_Ub3cOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper proposes a training-free pipeline for 2D (slice-wise) Brain MRI tumor localization from pretrained vision models, named SymmetryBreak. The method consists of the following steps: the contralateral pairs of two hemisphere is partitioned by mid-point slice (at location W/2, given W the width of the feature map(s)). Then, latent feature statistics statistics are calculated through low-discrepancy locations. Then standard deviations from this latent feature statistics statistics provides a hint of the abnormal hemisphere. Finally, a thresholding is applied to obtain the localization mask for brain tumors. The paper evaluates the approach on three public datasets.

Critical strengths

The proposed method is easy to follow and understand. The method itself is easy to apply with good details of implementation. The motivation of the design is clear and in most cases the proposed method works well. In some cases it outperforms existing learning-based methods on Dice score and AUPRC.

The idea of utilizing frozen vision models (e.g., MedSAM) is interesting for such localization tasks. I also appreciate the objective discussion raised by the authors.

Critical weaknesses
  1. The proposed method includes quite a few assumptions that is appearing too empirical. And the inference process is mainly based these empirical assumptions with a large-pretrained vision segmentation model. For example, from sec 3.2, the pipeline starts from a partition that devides the image (feature) along the width-dimension evenly. The bilateral latent symmetry breaking, despite its name, it essentially a pixel-wise cosine distance map from left and right hemisphere. However, I would like to point out that this relies on both well affine registration to an atlas or good bilateral references, of which has been done at the stage of dataset preparation in BraTs.

The following abnormality score, is measured by standard deviation to the image latent feature statistics statistics such as mean. However, I found the methodology seems to be heavily relying on the hyperparameter choice of these rules, e.g., the region used for calculating the ‘healthy reference’ and segmentation threshold. This leaves the work rather empirical and insufficiently established.

  1. I also found the proposed methodology needs more careful validation beyond the given metrics and the failure cases in appendix do not seem to be easily resolved given the nature of the method of hard-choice of hemispheres and thresholding-based segmentation from feature maps. Please see questions for a detailed comment.
Questions for authors
  1. In the main tables, the authors uses DSC and AUPRC for evaluating the accuracy. I would like to ask whether DSC is a proper and fair metric? Since the proposed method can be tuned with multiple hyperparameters such as the threshold of segmentation, would this be also beneficial if the same search is applied to other baseline methods?

  2. The use of vision foundation models, especially, segmentation foundation models is reasonable. However, does the data leakage happen: could the author investigate this like whether MedSAM has been trained on this kind of task at least brain segmentation?

  3. However, from Fig.3, it seems that the SB, due to the hard-threshold on a certain hemisphere, fails to segment the correct tumor. I would like to ask whether the assumption of locating the tumor in one hemisphere is too strict as some abnormality can happen across the midpoint of brain? Overall, does the proposed method has the ability to deal with non-linear or even non-midpoint cases?

  4. From the failure cases, it seems that the proposed method struggles a lot on the small tumor part: I would like to ask whether this could be fixed without retraining anything?

5 comments
AuthorsOpenReview ↗

Response to Reviewer Ub3c (1/3)

We thank the reviewer for the careful review and constructive feedback. We address the comments below.

1. Reliance on alignment and registration (W1)

We agree that BraTS is co-registered, which simplifies midline alignment (as noted in Appendix E). SymmetryBreak relies on approximate within-subject left-right correspondence and does not require an external atlas. However, its sensitivity to misalignment and the potential benefit of alignment preprocessing had not been evaluated.

To address this, we designed a controlled experiment on BraTS-Africa T1 in which we first introduced random rigid misalignment to each slice through rotations of up to {2°, 5°, 10°} and horizontal translations of up to {2%, 5%, 10% } of image width. We then applied a rigid registration procedure to estimate and correct these transformations, restoring approximate midline alignment before evaluating SymmetryBreak. This experiment allows us to distinguish the performance degradation caused by misalignment from the effectiveness of alignment preprocessing.

We evaluate performance under three conditions: (1) original images, (2) randomly misaligned images, and (3) misaligned images with rigid registration. This comparison quantifies both the impact of misalignment and the extent to which alignment preprocessing restores SymmetryBreak's performance.

Experimental conditionDSC (%)AUPRC (%)
Original images60.569.6
Randomly misaligned images55.464.2
Misaligned + rigid alignment59.968.8

This evaluation is relevant because rigid alignment to an estimated mid-sagittal plane is a conventional preprocessing operation in neuroimaging pipelines. Our objective is not to claim that SymmetryBreak is inherently invariant to arbitrary geometric transformations, but rather to establish whether its reliance on approximate left-right correspondence can be addressed through a practical preprocessing step.

SymmetryBreak assumes approximate left-right anatomical correspondence. In settings with uncontrolled head positioning or substantial acquisition-related misalignment, an alignment preprocessing step may be necessary.

2. Dependence on hyperparameters and empirical rules (W2)

The sensitivity plots in Figs. 4–7 exhibit broad performance plateaus, suggesting that the proposed method does not rely on narrowly tuned hyperparameter values. To further investigate its sensitivity to dataset-specific tuning, we conducted cross-dataset hyperparameter selection experiments. Specifically, hyperparameters were selected using one dataset and evaluated on a different, out-of-sample dataset without further adjustment. This protocol assesses whether configurations selected on one dataset generalize to another.

The results are reported in the table below. The performance differences remain small, with maximum decreases of 0.7 percentage points in DSC and 0.9 percentage points in AUPRC on BraTS-Africa T2, and 0.3 percentage points in both metrics on BraTS-2019 T1ce.

For reference, we also report oracle configurations, where hyperparameters are selected and evaluated on the same target dataset. These configurations provide an indication of the potential benefit of dataset-specific optimization. Their improvements over the original paper configuration are modest (at most 0.7 percentage points in DSC and 0.4 percentage points in AUPRC).

Overall, these results suggest that the proposed method is relatively insensitive to the choice of hyperparameters and that configurations selected on one dataset transfer effectively to other datasets, reducing the need for dataset-specific empirical tuning.

Selected OnOptimal Values (α\alpha, β\beta, γ\gamma, η\eta)Tested OnDSC (%)AUPRC (%)Split TypeGap vs. Paper (DSC / AUPRC)
BraTS-2021 T1ce (Paper)(0.40, 4.0, 0.75, 0.95)BraTS-Africa T257.666.6Out-of-sample0.0 / 0.0
BraTS-2019 Flair(0.45, 5.0, 0.70, 0.85)BraTS-Africa T256.965.7Out-of-sample-0.7 / -0.9
BraTS-Africa T2 (Oracle)(0.40, 4.0, 0.80, 0.85)BraTS-Africa T258.167.0In-sample+0.5 / +0.4
BraTS-2021 T1(0.40, 4.0, 0.70, 0.85)BraTS-2019 T1ce67.881.2Out-of-sample-0.3 / -0.3
BraTS-2019 T1ce (Oracle)(0.55, 4.0, 0.75, 0.90)BraTS-2019 T1ce68.881.7In-sample+0.7 / +0.2
BraTS-Africa Flair(0.35, 5.0, 0.80, 0.95)BraTS-2019 T1ce68.081.3Out-of-sample-0.1 / -0.2
AuthorsOpenReview ↗

Response to Reviewer Ub3c (2/3)

3. Is DSC fair, and would the same search help the baselines? (Q1)

We consistently used a fixed segmentation threshold of (τ=0.5)\tau=0.5), without performing method-specific threshold optimization. To investigate whether threshold tuning could substantially affect the reported results, we conducted an additional experiment on BraTS-2021 T1 dataset, searching for the optimal threshold for each method over (τ\tau ∈\in {0.1, 0.2, …\ldots, 0.9}), using the same tuning split and selection criterion.

As shown in the table, threshold optimization results in only marginal improvements in DSC across all methods (at most 0.5 percentage points). In particular, SymmetryBreak achieves a DSC of 69.6% at the default threshold of 0.5, compared with 69.8% using the optimized threshold. These results indicate that the fixed threshold of 0.5 already provides near-optimal segmentation performance for our method.

MethodDSC, default (0.5)DSC, tuned τ\tauAUPRC
f-AnoGAN28.828.934.1
DenoisingAE39.439.947.5
AnoDDPM61.862.173.4
AutoDDPM67.467.479.3
SymmetryBreak69.669.881.3

We therefore retain the fixed threshold of 0.5 in the main experiments to ensure a consistent evaluation protocol.

4. Foundation-model leakage (Q2)

MedSAM's training collection includes BraTS, so we cannot rule out overlap between its pretraining data and some of our evaluation cases. Rather than asserting that overlap is unlikely, we ran a controlled experiment to test whether our results depend on it.

Leakage control. We replaced the MedSAM encoder with backbones that differ in pretraining exposure, keeping the pipeline, hyperparameters, and evaluation protocol fixed. Experiments use BraTS-Africa T2:

BackboneMedical pretrainingBraTS masks seenDSCAUPRC
MedSAM ViT-BYes (segmentation)Yes57.666.6
BiomedCLIPYes (image-text only)No55.365.1
SAM ViT-BNoNo54.563.7
DINOv2 ViT-BNoNo46.954.4

Findings.

  1. Memorization does not explain the results. SAM ViT-B has never seen medical images or segmentation masks, yet it reaches 54.5 DSC and 63.7 AUPRC, within 3.1 DSC and 2.9 AUPRC of MedSAM. The method's core behavior therefore does not require exposure to the test distribution.

  2. Pretraining exposure plausibly helps, modestly. Backbones with medical pretraining score higher, and we do not dismiss this. The MedSAM-SAM difference of about 3 DSC is an upper bound on what leakage or domain pretraining could contribute in our setting, and it does not alter our conclusions relative to the baselines. BiomedCLIP, which saw no segmentation masks, falls between the two, suggesting that domain-relevant features help generally rather than through memorized labels.

  3. Backbone quality matters. DINOv2 degrades substantially (46.9 DSC), indicating that the method benefits from features that preserve fine spatial structure and that not every general-purpose encoder is suitable. We do not claim backbone-agnosticism; we claim the approach is not contingent on medical-domain exposure.

Task novelty. MedSAM was trained for supervised, prompt-based segmentation. Our setting is unsupervised anomaly detection via left-right asymmetry, with no labels or prompts at inference, so the model was never optimized for this objective on any data it saw.

We will (i) document the pretraining exposure of each backbone, and (ii) discuss residual leakage risk as a limitation in our camera ready manuscript.

AuthorsOpenReview ↗

Response to Reviewer Ub3c (3/3)

5. Small tumors and the hard hemisphere choice (Q3, Q4)

We acknowledge that hard hemisphere gating may impose a restrictive assumption when abnormalities extend across the brain midline or when the hemisphere-specific evidence is ambiguous. In the original implementation, the hard gating mechanism selects one hemisphere based on the estimated evidence, which may suppress relevant information from the opposite hemisphere.

To investigate this limitation, we considered a soft-gating strategy that replaces the hard argmax operation with temperature-controlled softmax weighting:

S(p)=D(p)⋅softmax⁡(ELT,ERT),S(p) = D(p) \cdot \operatorname{softmax}\left(\frac{E_L}{T}, \frac{E_R}{T}\right),

where TT controls the smoothness of the hemisphere selection. Unlike hard gating, this formulation allows contributions from both hemispheres, particularly when their corresponding evidence scores are similar. We use T=1.5T = 1.5

This modification does not require retraining the network and provides a more flexible mechanism for handling ambiguous hemisphere assignments.

To investigate whether the small-tumor segmentation performance could be improved without retraining, we evaluated soft hemisphere gating as a post-processing modification.

Specifically, we compared the original hard-gating strategy against soft gating on the BraTS-2021 FLAIR dataset. For this evaluation, tumors were categorized according to their relative volume: small (less than 10%), medium (10–30%), and large (greater than 30%) of the total brain image size.

The quantitative results in terms of Dice Similarity Coefficient (DSC, %) are presented below:

VariantSmallMediumLarge
Hard gating39.478.582.1
Soft gating52.275.378.9

The results demonstrate that soft gating substantially improves small-tumor segmentation, increasing DSC from 39.4% to 52.2% (an absolute improvement of 12.8 percentage points).

Reviewer_QZyWOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes SymmetryBreak, a training-free method for finding brain tumors in MRI images. The main idea is to compare the left and right sides of the brain and determine which side is more likely to be abnormal. The method uses a frozen image encoder and does not need training on tumor labels or a separate healthy reference dataset. The authors test it on three BraTS datasets and report good results compared to the selected baselines.

Critical strengths
  1. The idea is simple and interesting. Comparing the two sides of the brain can show where they differ, but not which side is abnormal. The paper provides a useful way to address this problem without additional training.

  2. The authors test the method on three datasets and different MRI modalities. The ablation studies are also useful and show that the main components of the method improve the results.

Critical weaknesses
  1. I have a concern about how the hyperparameters were selected. The paper says that ground-truth tumor masks were not used for parameter tuning. However, Appendix A shows that four parameters were selected based on DSC and AUPRC results on BraTS-2021 T1ce. These metrics require ground-truth masks. It is not clear whether the same data were used for both parameter selection and final evaluation. This could affect the reported results.

  2. The method selects only one hemisphere as abnormal. This may work for tumors on one side of the brain, but it can fail for tumors near the center or on both sides. The authors also show that choosing the wrong hemisphere leads to a large drop in performance. I think this limitation needs more analysis.

Questions for authors
  1. Can the authors clarify how the four hyperparameters were selected? Were they fixed before the experiments in Appendix A? If the BraTS-2021 ground-truth masks were used to select them, was a separate test set used for the final evaluation?

  2. How does the method perform on tumors near the midline or affecting both hemispheres? Can the authors report the results separately for these cases?

3 comments
AuthorsOpenReview ↗

Response to Reviewer QZyW (1/2)

We thank the reviewer for the careful review and constructive feedback. We address the comments below.

1. Hyperparameter Selection & Evaluation Protocol (W1 / Q1)

We agree that the distinction between model parameter optimization and inference hyperparameter selection was not sufficiently clear in the original manuscript, particularly in Section 4.1. While our method involves zero learnable model parameters, we recognize that our original phrasing was ambiguous and overstated. We directly address your three sub-questions below:

  • (i) How were the four hyperparameters (α\alpha, β\beta, γ\gamma, η\eta) selected?
    The four inference hyperparameters were selected empirically via manual search over predefined ranges using DSC and AUPRC on BraTS-2021 T1ce (which required ground-truth masks for evaluation). Once determined on BraTS-2021 T1ce, these exact values were fixed and applied unchanged across all other datasets and MRI modalities without any further tuning.

  • (ii) Were they fixed before the Appendix A experiments?
    No. They were selected from the Appendix A sensitivity experiments, meaning the BraTS-2021 T1ce figures in the main paper represent tuning-set results rather than an independent evaluation.

  • (iii) Was a separate test set used for final evaluation?
    Not for BraTS-2021 T1ce in the original submission. However, because those parameters were locked afterward, all other datasets and modalities other than BraTS-2021 T1ce reported in our main tables serve as strict out of sample test sets.

Cross-Dataset Hyperparameter Transfer

To further verify that our chosen configuration does not overfit to BraTS-2021 T1ce, we conducted a cross-dataset transfer study. We applied the exact same manual selection procedure independently to several source datasets (solely for this analysis) and evaluated the resulting parameters unchanged on target datasets:

Selected OnOptimal Values (α\alpha, β\beta, γ\gamma, η\eta)Tested OnDSC (%)AUPRC (%)Split TypeGap vs. Paper (DSC / AUPRC)
BraTS-2021 T1ce (Paper)(0.40, 4.0, 0.75, 0.95)BraTS-Africa T257.666.6Out-of-sample0.0 / 0.0
BraTS-2019 Flair(0.45, 5.0, 0.70, 0.85)BraTS-Africa T256.965.7Out-of-sample-0.7 / -0.9
BraTS-Africa T2 (Oracle)(0.40, 4.0, 0.80, 0.85)BraTS-Africa T258.167.0In-sample+0.5 / +0.4
BraTS-2021 T1(0.40, 4.0, 0.70, 0.85)BraTS-2019 T1ce67.881.2Out-of-sample-0.3 / -0.3
BraTS-2019 T1ce (Oracle)(0.55, 4.0, 0.75, 0.90)BraTS-2019 T1ce68.881.7In-sample+0.7 / +0.2
BraTS-Africa Flair(0.35, 5.0, 0.80, 0.95)BraTS-2019 T1ce68.081.3Out-of-sample-0.1 / -0.2

(Note: The In-sample rows represent dataset-specific oracle upper bounds where tuning and testing data coincide, rather than reported generalization results.)

Across all out-of-sample transfer experiments, performance drops by at most 0.7% DSC and 0.9% AUPRC relative to our paper's fixed configuration. Most notably, our paper's BraTS-2021 T1ce configuration achieves 57.6% DSC and 66.6% AUPRC on the out-of-sample BraTS-Africa T2 dataset, falling within just 0.5% and 0.4% of the dataset-specific oracle. This confirms that the selected hyperparameters generalize robustly across different cohorts and imaging modalities without overfitting.

We revised Section 4.1 and Appendix A to state that the four inference hyperparameters were selected using DSC/AUPRC on BraTS-2021 T1ce, and we will label BraTS-2021 T1ce results as tuning-set results in all tables. The configuration was never re-tuned for any other dataset or modality in the main results. The per-dataset selections above were used only for this analysis.

AuthorsOpenReview ↗

Response to Reviewer QZyW (2/2)

2. Sensitivity to Bilateral and Midline-Crossing Tumors (W2/Q2)

We agree that SymmetryBreak's hemisphere-selection mechanism is primarily designed for unilateral abnormalities and may be less reliable when tumors cross the midline or involve both hemispheres. In particular, selecting a single dominant hemisphere can suppress abnormal regions in the contralateral hemisphere.

Nevertheless, SymmetryBreak is designed as a lightweight, training-free localization approach, particularly relevant to clinical settings where annotated or normal data are limited and task-specific training or fine-tuning may be impractical. Its applicability to predominantly unilateral cases therefore represents an important practical use case, although we acknowledge that bilateral and midline-crossing tumors remain challenging.

To better characterize this limitation, we conducted two complementary analyses. First, in table below, we quantified the distribution of unilateral, midline-crossing, and bilateral tumors across all evaluated benchmarks. This provides dataset-specific evidence of the prevalence of each tumor pattern without relying on external estimates. We use complete dataset labels to quantify this distribution.

DatasetsUnilateral Tumor (%)Bilateral + Midline Tumor (%)
BraTS-201988.511.5
BraTS-202187.612.4
BraTS-Africa85.914.1

Second, we group the cases for BraTS-2019 T2 evaluation set. Let ff be the fraction of tumor pixels outside the dominant hemisphere, and mm the fraction inside a central band of ±10% of the image width. We call a case unilateral if f<5% and m<10%, bilateral if f≥30%, and midline-crossing otherwise. For each group we report the number of cases and the results for SymmetryBreak and for AutoDDPM, the strongest baseline.

GroupNSB DSC (%)AutoDDPM DSC (%)
Unilateral29575.766.7
Midline-crossing1460.159.5
Bilateral2653.652.6

These additional analyses quantify the prevalence of the tumor configurations targeted by our method and characterize its performance when the underlying unilateral assumption is violated. We acknowledge that SymmetryBreak is not intended to replace methods designed explicitly for bilateral tumor localization, but offers a computationally lightweight, training-free alternative for settings where data and computational resources are constrained.

AuthorsOpenReview ↗

Gentle reminder

Dear Reviewer QZyW,

Thank you again for your time and thoughtful review of our paper.

We wanted to briefly follow up on our responses addressing your concerns. The key results relevant to your comments are summarized below:

  • Hyperparameter selection: We clarified that BraTS-2021 T1ce was used for hyperparameter selection and explicitly identify the corresponding results as tuning-set results. We added cross-dataset transfer experiments, where performance changes only modestly compared with the original configuration.
  • Midline and bilateral tumors: We report a breakdown of unilateral, midline-crossing, and bilateral cases.

We hope these clarifications and additional analyses address the concerns you raised. We would appreciate your feedback on whether our responses resolve the main issues.

Best regards,

The Authors

Reviewer_8rEUOpenReview ↗
Rating
2
Confidence
3
Informativeness
1
Presentation
3
Soundness
1
Read review
Paper summary

This paper proposes SymmetryBreak, a training-free method for brain MRI anomaly localization using a frozen vision encoder. It uses bilateral feature differences and a patient-specific reference to identify abnormal regions and determine which hemisphere is affected. Experiments on three BraTS datasets show promising results across different MRI modalities.

Critical strengths
  1. Novel formulation of unseen-sample familiarity through influence functions.

  2. Empirical validation of influence approximation in the sample-incremental setting.

Critical weaknesses

Major weaknesses

  1. The method assumes that deterministic horizontal mirroring induces anatomically meaningful correspondences in the frozen latent space. However, correspondence at the spatial index level does not necessarily guarantee anatomical or semantic feature alignment, particularly for ViT-based encoders that lack explicit reflection equivariance. Moreover, as illustrated in Figure 2, even the MRI images themselves exhibit noticeable anatomical asymmetry, further calling into question the validity of this assumption.

  2. In the 'Patient-Specific Reference Region' part, the proposed patient-specific reference is constructed from low bilateral discrepancy locations, yet low discrepancy does not necessarily imply normal anatomy. Furthermore, the reference statistics are computed by aggregating features from potentially heterogeneous anatomical tissues, such as gray matter, white matter, and cerebrospinal fluid, which may have substantially different feature distributions. Using a single mean and standard deviation across these regions may cause normal anatomical variations to be incorrectly identified as abnormalities. Additionally, the reference statistics are independently estimated for the two hemispheres, potentially affecting the comparability of their anomaly scores. The authors should clarify whether these reference statistics provide reliable and anatomically meaningful evidence for directional abnormality attribution.

  3. Inconsistency Between Reference Selection and Feature Statistics. The reference region is selected using cosine discrepancy between normalized features, while the reference mean and standard deviation are computed from the original, unnormalized features. Since cosine similarity ignores feature magnitude, locations with similar feature directions but substantially different magnitudes may still be included in the reference region. The authors should clarify whether this inconsistency affects the reliability of the patient-specific reference statistics.

  4. The method compares anomaly scores from the left and right hemispheres to determine which side is abnormal. However, these scores are calculated using different means and standard deviations for each hemisphere, so a higher score does not necessarily mean that side is more abnormal. The shared normalization step does not fully solve this problem. In addition, the method always selects one hemisphere, even when both sides have similar anomaly scores or the lesion affects both hemispheres. The authors should better explain why these scores can be directly compared and provide experiments to validate the accuracy of hemisphere selection.

  5. Lack of Robustness Evaluation to Spatial Misalignment.

  6. The paper lacks baselines using the same MedSAM backbone, making it unclear whether the improvements come from the proposed method or the pretrained features.

Minor weaknesses

Inconsistency in the Interpretation of Hyperparameter α.

There is a clear inconsistency between the mathematical definition of the patient-specific reference region and the sensitivity analysis in Appendix A.1. According to Eq. (4), the reference region is defined as N=p(p)≤Qα(D)\mathcal{N}={p(p)\leq Q_\alpha(D)}. Therefore, increasing α should monotonically expand the reference region by including more locations with higher bilateral discrepancy. However, Appendix A.1 states that setting α excessively high (α > 0.6) "overly restricts feature selection," which directly contradicts the mathematical formulation. The authors should clarify this inconsistency and provide a more accurate explanation of how α affects reference region selection and the resulting localization performance.

Questions for authors
  1. Unclear Hyperparameter Selection Protocol. The paper states that ground-truth annotations are used only for evaluation and not for parameter optimization. However, Appendix A selects several hyperparameters based on DSC and AUPRC results on BraTS 2021 T1ce. It is unclear whether these experiments were performed on a separate validation set or the same data used for final evaluation. The authors should clarify how the hyperparameters were selected to rule out potential test-set tuning.
6 comments
AuthorsOpenReview ↗

Response to Reviewer 8rEU (1/4)

We thank the reviewer for the careful review and constructive feedback. We also have one request. The Critical Strengths praise a novel formulation of unseen-sample familiarity through influence functions and an empirical validation of influence approximation in the sample-incremental setting. SymmetryBreak contains neither influence functions nor a sample-incremental setting. We would be grateful if the reviewer re-checked that section. We address the comments below.

1. Does mirroring give meaningful correspondence? (W1)

(a) Alignment. BraTS scans are co-registered to a common anatomical template, and skull-stripped by the organizers, so the mid-sagittal plane lies near the image center. This is why a deterministic flip works in our setting. We agree that co-registration does not guarantee a perfectly centered midline, and that arbitrary clinical scans may be worse. We therefore measured the sensitivity directly. On BraTS-Africa T1, we applied random rigid misalignment to each slice (rotations up to 2°/5°/10°, horizontal shifts up to ±2%/5%/10% of image width). We then applied rigid registration re-alignment before running SymmetryBreak:

Experimental conditionDSC (%)AUPRC (%)
Original images60.569.6
Randomly misaligned images55.464.2
Misaligned + rigid alignment59.968.8
  • Misalignment costs about 5 points of DSC. It hurts, but doesn't break the method.
  • Even misaligned, SB (55.4 / 64.2) stays above AutoDDPM on the same setting (53.1 / 61.8, Table 3).
  • Rigid alignment recovers 4.5 of the 5.1 lost DSC points. Correspondence is a real dependency, and a standard preprocessing step handles most of it.

(b) Are mirrored features comparable? If they weren't, healthy brains would show high discrepancy. They don't: mean bilateral discrepancy is 0.1025 ± 0.015 on 200 IXI healthy images versus 0.146 ± 0.035 on 200 BraTS tumor images (Fig. 1b), an effect size of d ≈ 1.6. The distributions do overlap, so we never use D alone as the score. Removing the bilateral term also drops DSC from 73.6 to 23.2 (BraTS-19 T2), so the flipped comparison carries real signal.

(c) Flipping features vs. encoding a flipped image. Agreed, ViTs aren't reflection-equivariant. Across encoder with the pipeline fixed on BraTS-Africa T2, DSC is 57.6 (MedSAM), 55.3 (BiomedCLIP), 54.5 (SAM) and 46.9 (DINOv2). All beat raw-pixel asymmetry (18.7) by a wide margin, so the signal isn't an artifact of one encoder. We don't claim backbone-agnosticism, since DINOv2 is clearly weaker.

2. Reference region, tissue heterogeneity, and per-hemisphere statistics (W2)

We agree N is “low-discrepancy,” not “normal,” and mentioned in Sec. 3.2, Three clarification we want to stated:

  • Common-mode errors cancel. If a tissue class is under-represented in N, its Z-score is inflated at both corresponding locations, because the anatomy is approximately symmetric. That bias cancels in Δ = Ã_L − Ã_R. The final score S = D·W also requires high bilateral discrepancy, so a symmetric tissue effect is suppressed twice.

  • Paired statistics. N is defined on the same index set for both hemispheres. μ_L/μ_R and σ_L/σ_R therefore differ only through real hemispheric differences, such as a left-right bias-field gradient, which is exactly what we want to normalize out. Without separate standardization, such a gradient would cause false directional attribution.

  • Empirical evidence. The reference module is the least critical component. Removing it costs −3.8 / −1.9 / −5.7 DSC on BraTS-19 T2 / BraTS-21 FLAIR / BraTS-Africa (derived from Table 4), versus −21.9 to −23.9 for hemisphere selection and −38.8 to −41.1 for directional attribution. The method is not fragile with respect to this module.

AuthorsOpenReview ↗

Response to Reviewer 8rEU (2/4)

3. Cosine selection vs. raw-feature statistics (W3)

You're right that the paper doesn't justify this, We tested whether the mismatch matters.

We conduct Experiment on BraTS-21 FLAIR, all else fixed:

VariantDSC (%)AUPRC (%)
Default (select on cosine, stats on raw)77.387.0
Stats on L2-normalized features77.987.4
Select N by Euclidean distance (magnitude-aware)76.485.1
Select N by cosine, drop top 10% magnitude outliers77.086.8
Median/MAD instead of mean/std75.486.0

What we conclude.

  • The mismatch is not a source of fragility. Every variant lands within 2 DSC points of the default. Trimming magnitude outliers from N changes DSC by only 0.3, which suggests σ inflation from magnitude outliers is small in practice.
  • Your suggested fix is slightly better. Computing statistics on normalized features gives +0.6 DSC and +0.4 AUPRC. We had argued that raw features keep a useful magnitude cue, and these results don't support that. Magnitude information is not needed here. We will adopt the normalized version as the default, so selection and statistics share one feature space, and will revise it in main manuscript.
  • Magnitude-aware selection does worse. Selecting N by Euclidean distance costs 0.9 DSC and 1.9 AUPRC, so cosine is the better selection rule.
  • Robust statistics did not help. Median/MAD loses 1.9 DSC, so we keep mean/std.

These gains are small, and we ran this on one dataset-modality pair. We will rerun the normalized-statistics variant and report paired bootstrap confidence intervals, so readers can see whether the +0.6 is real. We will also report the σ ratio between N and the trimmed N.

AuthorsOpenReview ↗

Response to Reviewer 8rEU (3/4)

4. Are left and right scores comparable, and why always one side? (W4)

You raise two separate issues: whether A_L and A_R can be compared, and whether forcing one side is valid. We concede part of the first, address the second with new data, and add a direct accuracy measurement.

1. Are the scores comparable?

The pooled median/std in the "shared scale" step is one affine transform applied to both sides. It does not undo the per-hemisphere standardization.

The narrower claim we make is this. Each side is expressed in units of deviation from its own low-discrepancy reference, and Δ compares those units at mirrored locations. Two design details support this:

  • N is defined on one index set, so μ_L/σ_L and μ_R/σ_R come from anatomically paired locations. Differences between them reflect real left-right differences, such as a bias-field gradient. Standardizing each side separately removes those offsets instead of mistaking them for pathology.
  • High-discrepancy locations, including most of the lesion, are excluded from N on both sides. A one-sided tumor therefore does not contaminate its own reference.

2. Hemisphere-selection accuracy.

Table 6 reports forced-correct and forced-wrong results but not accuracy. Treating overall DSC as a mixture of the two gives a rough estimate of how often the correct side is chosen:

SettingOverallCorrect sideWrong sideImplied accuracy
BraTS-19 T273.674.02.9≈ 99%
BraTS-21 FLAIR77.379.83.6≈ 97%
BraTS-Africa T1ce55.358.97.2≈ 93%

This estimate is likely optimistic, because hard cases tend to be both mis-routed and low-DSC. We measure the accuracy directly against ground-truth laterality, broken down by tumor size and distance to the midline.

3. Bilateral and midline-crossing tumors.

You're right that argmax has no abstention option, and that is a limitation. We quantified how often it matters. Unilateral tumors make up 88.5 / 87.6 / 85.9% of BraTS-2019 / 2021 / Africa. On the BraTS-2019 T2 evaluation set, split by tumor configuration: we group the cases for evaluation set. Let ff be the fraction of tumor pixels outside the dominant hemisphere, and mm the fraction inside a central band of ±10% of the image width. We call a case unilateral if f<5% and m<10%, bilateral if f≥30%, and midline-crossing otherwise. For each group we report the number of cases and the results for SymmetryBreak and for AutoDDPM, the strongest baseline.

GroupN (Total Number)SB DSC (%)AutoDDPM DSC (%)
Unilateral29575.766.7
Midline-crossing1460.159.5
Bilateral2653.652.6

Performance drops on bilateral and midline-crossing cases, as expected. It does not collapse, but with N = 14 and 26 we only claim it is comparable to the strongest baseline there, not better.

We also evaluated replacing the hard argmax with temperature-controlled soft gating (T=1.5T = 1.5) over both sides so that both contribute when hemispheric evidence is comparable:

S(p)=D(p)⋅softmax⁡(ELT,ERT),S(p) = D(p) \cdot \operatorname{softmax}\left(\frac{E_L}{T}, \frac{E_R}{T}\right),

so that both contribute when evidence is similar. On BraTS-21 FLAIR, DSC by tumor volume (small / medium / large):

VariantSmallMediumLarge
Hard gating39.478.582.1
Soft gating52.275.378.9

Soft gating improves small tumors by 12.8 points, with a trade-off on medium and large tumors (-3.2 points).

5. Same-backbone baselines (W6)

To isolate the contribution of our proposed method from the pretrained feature representations, we conducted additional experiments using the same MedSAM-ViT-B feature backbone for all methods. We compared SymmetryBreak against two baselines: raw bilateral cosine discrepancy with thresholding and within-image Mahalanobis/kNN scoring without symmetry modeling.

The results are summarized below:

Method (MedSAM-ViT-B features)BraTS-2019 T2 (DSC / AUPRC)BraTS-2021 FLAIR (DSC / AUPRC)BraTS-Africa T2 (DSC / AUPRC)
Raw bilateral cosine discrepancy D, thresholded66.5/74.868.8/75.146.4/56.7
Within-image Mahalanobis / kNN, no symmetry68.5/79.471.3/82.949.5/59.2
SymmetryBreak73.6 / 87.777.3 / 87.057.6 / 66.6

SymmetryBreak consistently outperforms both baselines across all three datasets, achieving DSC improvements of 5.1, 6.0, and 8.1 percentage. These results demonstrate that the improvements are not solely due to MedSAM's pretrained features but also reflect the contribution of our proposed symmetry-based discrepancy modeling.

Comments on the paper

1 comment

Dear Reviewers,

Thank you again for your feedback. Our general response is below.

SymmetryBreak is a training-free localization method for data-limited settings: no tumor annotations for training and no healthy-brain cohort. Because the reference comes from the same image, it does not depend on a training distribution, which matters for out-of-distribution cohorts. It targets predominantly unilateral tumors (85.9-88.5% of total BraTS dataset) and needs only approximate left-right alignment, a standard 2D preprocessing step.

Key evidence (in our individual responses).

  • Misalignment (BraTS-Africa T1): random rigid misalignment lowers DSC from 60.5 to 55.4. Rigid re-alignment recovers 59.9.
  • Hyperparameters: settings selected on other datasets lose at most 0.7 DSC and 0.9 AUPRC. Target-specific tuning gains at most 0.7 DSC.
  • Thresholds: tuning the threshold for every method changes DSC by at most 0.5.
  • Backbones (BraTS-Africa T2): MedSAM 57.6, BiomedCLIP 55.3, SAM 54.5, DINOv2 46.9 DSC. SAM has no medical pretraining yet is within 3.1 DSC of MedSAM.
  • Ablations (BraTS-2019 T2): removing bilateral discrepancy, directional attribution, or hemisphere selection drops DSC from 73.6 to 23.2, 34.8, and 51.7.
  • Tumor configuration (BraTS-2019 T2): DSC is 75.7 on unilateral cases, 60.1 on midline-crossing, 53.6 on bilateral.

Limitations we acknowledge.

  • Hard hemisphere selection cannot abstain and is less reliable for bilateral and midline-crossing abnormalities. We quantify this behavior and evaluate soft gating, which improves small-tumor DSC from 39.4% to 52.2%, with trade-offs for medium and large tumors.

  • The method assumes approximate left-right anatomical correspondence. Robustness to substantial midline shifts and non-rigid anatomical deformations remains to be investigated.

  • BraTS-2021 T1ce was used for inference-hyperparameter selection; we explicitly identify its results as tuning-set results. All other reported dataset-modality settings use the fixed configuration without further tuning.

  • Training-free refers to the absence of task-specific learned parameters, rather than the complete absence of ground-truth labels during methodological development.

Best regards,

The Authors

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.