acceptodds
Under review as a conference paper at ICLR 2027

Refining Plane Priors for Query-Conditioned Fetal Ultrasound Segmentation

Abstract

Text-guided fetal ultrasound segmentation requires locating the structure specified by a clinical query among multiple structures in the same image. Existing multimodal large language model (MLLM) approaches rely primarily on internal representations that lack fetal-ultrasound domain knowledge. A pretrained fetal-ultrasound vision-language model can provide domain-specific visual features and plane-level activation maps. However, the visual features do not directly provide an explicit spatial prior, while the activation maps remain coarse and unstable for query-specific localization. We introduce PRISM (Plane-prior Refinement and Integration for Segmentation with MLLMs), a framework that converts plane-level evidence into query-conditioned spatial prompts. PRISM uses the pretrained model through two complementary pathways: its domain-specific visual features are supplied to the MLLM, while its plane activation maps provide spatial evidence for refinement. Activation Prior Refinement (APR) learns a residual correction of the activation map from visual context and query semantics, producing a target-specific spatial prior. Patch-Adaptive Prior Composition (PAPC) then constructs experts from the refined prior, visual features, and query representation, selects an expert subset with a Set Router, and fuses their predictions spatially with a Patch Router. Extensive experiments on public and private fetal ultrasound benchmarks demonstrate that PRISM consistently outperforms the compared methods across model scales, reaching 80.55% and 71.41% cIoU on the public and private benchmarks, respectively, with a 13B language model.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
Medical MLLM, Fetal ultrasound, Medical Image Segmentation

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_h9FcOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes PRISM for query-conditioned fetal ultrasound segmentation. The main idea is to inject fetal-ultrasound-specific knowledge from a pretrained FetalCLIP model into an MLLM-SAM segmentation framework. PRISM uses FetalCLIP both as the MLLM visual encoder and as an external source of plane-level ScoreCAM priors. The proposed Activation Prior Refinement (APR) refines the coarse plane activation map using visual and query representations, while Patch-Adaptive Prior Composition (PAPC) adaptively combines the refined prior, visual features, and query semantics through set-level expert selection and patch-level fusion.

Experiments on one public benchmark assembled from five datasets and one private benchmark show consistent improvements over LISA, SESAME, READ, and Sim4Seg across 7B/13B model scales. The best model achieves 80.55 cIoU on the public benchmark and 71.41 cIoU on the private benchmark. Component and routing ablations generally support the contributions of CAM, APR, and PAPC.

Critical strengths
  1. The paper explicitly spatializes fetal-plane knowledge and refines it according to the requested anatomy. This is well motivated for fetal ultrasound, where imaging plane strongly constrains anatomical appearance.

  2. Gains are observed on both public and private benchmarks and at both 7B and 13B scales. The component ablation is particularly convincing: on the public 7B setting, the baseline improves from 38.41/70.01 gIoU/cIoU to 50.43/77.09 with the complete method.

Critical weaknesses
  1. At inference time, the query already specifies the imaging plane, yet the external branch independently predicts the plane from the image and uses that prediction to select the ScoreCAM target. I could not find quantitative plane-classification accuracy, an oracle-plane comparison, or an analysis of segmentation performance when the predicted plane is incorrect. This makes it difficult to determine how robust the proposed mechanism actually is and how much of the gain depends on correct plane recognition. The appendix states that prediction/annotation mismatches are recorded, but no quantitative analysis appears to be provided.

  2. Queries are generated from annotated plane/target names using a small number of wording variants. There is no evaluation on unseen paraphrases, omitted/incorrect plane descriptions, or unseen target/plane combinations. In addition, the private benchmark is described as being split at the source-image level, rather than explicitly at the patient/examination level. For ultrasound data, clarification that correlated images from the same patient/examination cannot cross splits is important.

Questions for authors
  1. What is the FetalCLIP plane-classification accuracy on the two benchmarks? Please report segmentation performance separately for correctly and incorrectly predicted planes, and ideally compare predicted-plane CAM vs. ground-truth/oracle-plane CAM. This would directly establish how sensitive PRISM is to the quality of its plane prior.

  2. Since the query already contains the plane name, why is the CAM target inferred independently from the image rather than selected from the query? How does performance change if the query-specified plane is used directly?

  3. Are train/validation/test sets separated at the patient or ultrasound-examination level, rather than only by source image? If multiple frames/images from the same patient or examination exist, can they appear in different splits?

  4. What is the computational overhead of PRISM at inference time? ScoreCAM operates over 1,024 activation channels and scores masked inputs for channel weighting, which appears substantially more expensive than a standard forward pass. Please report inference latency or FLOPs relative to LISA/Sim4Seg.

4 comments
AuthorsOpenReview ↗

Authors Rebuttal to Reviewer h9Fc [1/3]

We sincerely thank you for your careful and thorough review. We appreciate your recognition that PRISM spatializes fetal-plane knowledge according to the requested anatomy, a design well motivated for fetal ultrasound, and that its gains hold across both benchmarks and model scales, with the component ablation being particularly convincing. We address your questions below and have incorporated all feedback in the revised version. All new material added to the revised manuscript has been highlighted in pink text for better visibility.

Q1. At inference time, the query already specifies the imaging plane, yet the external branch independently predicts the plane from the image and uses that prediction to select the ScoreCAM target. I could not find quantitative plane-classification accuracy, an oracle-plane comparison, or an analysis of segmentation performance when the predicted plane is incorrect. This makes it difficult to determine how robust the proposed mechanism actually is and how much of the gain depends on correct plane recognition. The appendix states that prediction/annotation mismatches are recorded, but no quantitative analysis appears to be provided.

Response: Thank you for the insightful comment. The frozen FetalCLIP predictor used to select the ScoreCAM target achieves zero-shot accuracies of 80.89% and 59.83% on the public and private test sets, respectively. PRISM shows limited overall sensitivity to plane-prediction errors and retains its segmentation advantage over LISA when the predicted plane is incorrect.

For the oracle comparison, we use the same trained 7B model and segmentation queries, replacing only the predicted ScoreCAM plane with the annotated plane. For these explicit benchmark queries, the oracle also corresponds to using the query-specified plane directly. Prediction-True and Prediction-False indicate whether the actual FetalCLIP prediction matches the annotation; all methods use the same groups. The results are shown below (Appendix A.7.1, “Plane Classification and Sensitivity to Plane Prediction Errors”, Table 8).

Plane Classification Accuracy and Segmentation by Prediction Correctness.

DatasetMetric / SubsetLISAPRISM-PredictionPRISM-Oracle
PublicPlane accuracy—80.89100.00
PubliccIoU: All70.0177.0977.13
PubliccIoU: Prediction-True69.6676.8076.80
PubliccIoU: Prediction-False81.1786.1487.45
PrivatePlane accuracy—59.83100.00
PrivatecIoU: All60.5366.5667.66
PrivatecIoU: Prediction-True58.0465.5065.50
PrivatecIoU: Prediction-False68.4469.5373.71

Correcting the predicted plane to the annotated plane improves overall cIoU by only 0.04 and 1.10 percentage points on the public and private benchmarks, respectively. This indicates that PRISM’s overall segmentation gains do not rely entirely on correct plane recognition. On incorrectly classified images, PRISM still outperforms LISA by 4.97/1.09 points, although correcting the plane improves cIoU by 1.31/4.18 points. Thus, plane recognition affects segmentation, particularly on the private benchmark, while PRISM retains gains under the observed prediction errors.

We sincerely thank the reviewer for this insightful suggestion, which has helped strengthen the experimental validation of our work.

AuthorsOpenReview ↗

Authors Rebuttal to Reviewer h9Fc [2/3]

Q2. Queries are generated from annotated plane/target names using a small number of wording variants. There is no evaluation on unseen paraphrases, omitted/incorrect plane descriptions, or unseen target/plane combinations. In addition, the private benchmark is described as being split at the source-image level, rather than explicitly at the patient/examination level. For ultrasound data, clarification that correlated images from the same patient/examination cannot cross splits is important.

Response: Thank you for your valuable feedback. We address this concern through evaluations with implicit clinical queries, unseen query formulations, and a clarification of the patient-level data split.

1. Query understanding beyond explicit anatomical labels.

To examine whether PRISM can use query meaning beyond explicit plane and target names, under the guidance of experienced clinicians, we formulated implicit queries for five target categories. These queries describe the requested anatomy through its role in a clinical measurement. For example, the implicit query for the fetal head region is: “For fetal growth assessment using a cranial circumference measurement, segment the full region enclosed by the measurement contour.” All compared methods were retrained using the same implicit-query training data and evaluated on the corresponding held-out test split, retaining the original public-benchmark split assignments.

PRISM achieves the highest gIoU and cIoU among the compared methods under this shared setting, demonstrating effective segmentation from clinical descriptions without explicit plane or target names. The results are shown below (Main Text, Section 4.3, “Segmentation with Implicit Clinical Queries”, Table 2). The subset composition (Table 18), complete query formulations, and corresponding targets are provided in Appendix A.7.10, “Implicit Clinical Queries and Corresponding Segmentation Targets”.

Segmentation with Implicit Clinical Queries.

MethodgIoUcIoU
READ(CVPR2025)81.0686.47
SESAME(CVPR2024)83.9087.23
LISA(CVPR2024)83.9086.64
Sim4Seg(AAAI2026)84.4687.88
Ours(CLIP)85.4989.17

2. Generalization across query formulations.

We further evaluated whether performance generalizes beyond the query formulations used during training. In the Unseen-Query setting, the trained model is evaluated with a new inference-time template. In the Held-out Query setting, one original template is excluded from training and used only at test time.

PRISM maintains its advantage over LISA in both settings, with cIoU decreases of only 0.09 and 0.18 percentage points relative to the default setting. The results are shown below (Appendix A.7.3, “Generalization to Unseen and Held-Out Query Formulations”, Table 10).

Generalization to Unseen and Held-Out Query Formulations.

SettingPRISM gIoUPRISM cIoULISA gIoULISA cIoU
Default50.4377.0938.4170.01
Unseen-Query49.7777.0036.8068.12
Held-out Query49.9976.9136.3467.70

3. Patient-level separation.

The private training, validation, and test sets are patient-disjoint. All images from the same patient, together with their annotations and queries, remain in one split. Thus, correlated images from the same patient or examination cannot cross splits. We have clarified this in Main Text, Section 4.1.3, “Dataset” and Appendix A.5, “Dataset Details”, Tables 6–7.

We thank the reviewer for this helpful comment, which improved the rigor of the experimental analysis.

AuthorsOpenReview ↗

# Authors Rebuttal to Reviewer h9Fc [3/3]

Q3: Since the query already contains the plane name, why is the CAM target inferred independently from the image rather than selected from the query?

Response: Thank you for the valuable comment. Image-based plane prediction allows prior generation without an explicit plane name in the query, supporting the implicit clinical queries evaluated in Main Text, Section 4.3, “Segmentation with Implicit Clinical Queries”, Table 2.

Q4: Are train/validation/test sets separated at the patient or ultrasound-examination level, rather than only by source image? If multiple frames/images from the same patient or examination exist, can they appear in different splits?

Response: Thank you for your insightful comment. The private training, validation, and test splits are patient-disjoint. All frames and images from the same patient, including their annotations and queries, remain in one split, preventing overlap across patients or examinations (Main Text, Section 4.1.3, “Dataset”; Appendix A.5, “Dataset Details”, Tables 6–7).

Q5: What is the computational overhead of PRISM at inference time? ScoreCAM operates over 1,024 activation channels and scores masked inputs for channel weighting, which appears substantially more expensive than a standard forward pass. Please report inference latency or FLOPs relative to LISA/Sim4Seg.

Response: Thanks for your comment! We measured the computational cost of the 7B configurations, including parameters, FLOPs, peak GPU memory, and inference time. Although PRISM incurs higher FLOPs and inference latency, it achieves better segmentation performance than all compared methods on both benchmarks. The computational results are shown below (Appendix A.7.7, “Computational Cost and Inference Trade-Off”, Table 14).

Computational Cost and Inference Trade-Off.

MethodTotal params. (B)Trainable params. (M)FLOPs (T)Peak GPU memory (GB)Inference time (s)
LISA-7B7.7089288.2610.497717.1580.2506
Sim4Seg-7B (RVLS2M)7.7089288.2610.497717.1580.2650
PRISM-7B (Plane ScoreCAM)8.1424294.076176.57618.0381.4946

Dear Reviewer h9Fc

I hope this message finds you well.

I am one of the authors of Paper 11503, titled "Refining Plane Priors for Query-Conditioned Fetal Ultrasound Segmentation." As the author-reviewer discussion period is in its final stages, I wanted to gently follow up to see if our responses have satisfactorily addressed all of your concerns. If there are any additional points or feedback you'd like us to consider, please let us know. Your insights are invaluable to us, and we are eager to address any remaining issues to improve our work.

Thank you very much for your time and consideration.

Best regards,

The authors of Paper "Refining Plane Priors for Query-Conditioned Fetal Ultrasound Segmentation" (ID: 11503)

Reviewer_bEEPOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
3
Soundness
2(+1)
Read review
Paper summary

The paper proposes PRSIM which is a text-guided fetal ultrasound segmentation method that uses pre-trained fetal-plane information to improve segmentation performance. It combines FetalCLIP features, activation map refinement and an expert-routing module with an MLLM and the Segment Anything Model. Experiments are shown on public and private datasets with a 13B model.

Critical strengths
  1. The authors show useful use of domain-specific information. This is also supported by component ablations. For example, adding external CAM increases publich 7B cIoU from 70.01 to 74.11 and the other components raises it further to 77.09. This also shows the idea of using pre-trained fetal-ultrasound knowledge as an additional spatial prompt is reasonable and well motivated. Further, the authors show results across CLIP and FetalCLIP visual encoders suggesting that the approach is not entirely dependent on just replacing the MLLM visual backbone.
  2. The reported results are also strong. PRISM consistently does better than the compared methods on both datasets and with both 7B and 13B models. There is an improvement of almost 4% for both the public and private benchmarks using the strongest baseline. These are strong improvements especially given the difficulties in segmenting anatomical structures in fetal ultrasound images.
Critical weaknesses
  1. I am not sure about the novelty of the approach and believe there is insufficient evidence that the proposed components justify their complexity. For example, the paper misses comparisons with several related methods such as MedCLIP-SAMv2 [1] and F-TPA-SAM2 [2]. These already use medical VLMs to guide SAM-based segmentation. F-TPA-SAMP2 uses FetalCLIP for ultrasound segmentation and reports 85.9% Dice on PSFHS, a dataset used in the paper as well. Although the eval settings and metrics differ, a comparison would help establish the advantage of PRISM. If there is no code available for either, at least a strong motivation discussing the difference is needed. Also, the ablations while useful do not convince me of the need of both APR and PAPC modules. Simply adding raw CAM improves cIoU from 70.01 to 74.11. Adding APR to the CAM+PAPC improves cIoU by only 0.27. The pairwise routing only adds 0.222. These are incremental improvements given the additional complexity. The authors could instead just use simpler alternatives like feature concatenation, addition or attention-based fusion. Also, the model is quiet heavy. There is a FetalCLIP branch, an MLLM, SAM and then additional refinement and routing modules. ScoreCAM also needs repeated masked forward passes. The authors should provide inference time, memory usage, cost analysis etc. It is unclear if the improvements justify the cost.
  2. For a clinical application, there is very limited evaluation of query understanding and generalization. The paper claims query-conditioned segmentation, but the queries are generated from templates that explicitly specify structure and imaging plane. Does the model actually understand the query or simply learns to associate a fixed set of anatomical labels with segmentation masks? This is an important to distinguish, especially since F-TPA-SAM2 reports that fixed class-code prompts perform almost as well as learned text embeddings. The evaluation is also something I found not very clear. The private dataset uses source-image-level splits, is there a patient-level separation? The public benchmark combines several datasets but results are aggregate scores rather than separately per dataset or anatomical category.

[1] https://www.sciencedirect.com/science/article/abs/pii/S1361841525002968 [2] https://www.frontiersin.org/journals/medicine/articles/10.3389/fmed.2026.1889351/full

Questions for authors
  1. Can the authors compare PRISM with simpler FetalCLIP+SAM approaches including [1] and [2]? Of course in matched settings.
  2. Can the authors separately evaluate the contribution of APR and PAPC against simple feature-fusion baselines? Further, are the smaller reported improvements consistent across seeds?
  3. How does the method behave when the query uses wrong imaging plane, unfamiliar wording etc? Does it actually use query semantics?
  4. Could the authors report per-anatomy performance, address the split details and inference latency?

Most have been addressed in the response.

9 comments
AuthorsOpenReview ↗

Authors Rebuttal to Reviewer bEEP [1/6]

We sincerely thank you for your careful and thorough review. We greatly appreciate your recognition that our method makes useful use of domain-specific information, that using pre-trained fetal-ultrasound knowledge as an additional spatial prompt is reasonable and well motivated, and that the reported results are strong. We address your questions below and have incorporated all feedback in the revised version. All new material added to the revised manuscript has been highlighted in pink text for better visibility.

Q1. I am not sure about the novelty of the approach and believe there is insufficient evidence that the proposed components justify their complexity. For example, the paper misses comparisons with several related methods such as MedCLIP-SAMv2 [1] and F-TPA-SAM2 [2]. These already use medical VLMs to guide SAM-based segmentation. F-TPA-SAMP2 uses FetalCLIP for ultrasound segmentation and reports 85.9% Dice on PSFHS, a dataset used in the paper as well. Although the eval settings and metrics differ, a comparison would help establish the advantage of PRISM. If there is no code available for either, at least a strong motivation discussing the difference is needed. Also, the ablations while useful do not convince me of the need of both APR and PAPC modules. Simply adding raw CAM improves cIoU from 70.01 to 74.11. Adding APR to the CAM+PAPC improves cIoU by only 0.27. The pairwise routing only adds 0.222. These are incremental improvements given the additional complexity. The authors could instead just use simpler alternatives like feature concatenation, addition or attention-based fusion. Also, the model is quiet heavy. There is a FetalCLIP branch, an MLLM, SAM and then additional refinement and routing modules. ScoreCAM also needs repeated masked forward passes. The authors should provide inference time, memory usage, cost analysis etc. It is unclear if the improvements justify the cost.

Response: Thank you for the valuable and constructive comment, and for pointing us to these relevant studies. PRISM’s central contribution is to convert plane-level fetal-ultrasound knowledge into query-specific spatial prompts through refinement and adaptive composition. Following your suggestions, we conducted the comparisons and analyses below to evaluate these operations against related approaches and simpler alternatives, and to quantify their computational cost.

AuthorsOpenReview ↗

Authors Rebuttal to Reviewer bEEP [2/6]

First, we implemented the two cited approaches and evaluated them under the same protocols as PRISM. PRISM achieves higher gIoU and cIoU on both benchmarks. The results are shown below (Appendix A.7.2, “Comparison with FetalCLIP-SAM and Class-Embedding Baselines”, Table 9).

Comparison with FetalCLIP-SAM and Class-Embedding Baselines.

MethodPrivate gIoUPrivate cIoUPublic gIoUPublic cIoU
FetalCLIP-SAM6.188.3915.9516.13
Class-Embedding40.2264.7849.5776.66
PRISM-7B41.8966.5650.4377.09

Second, we compared PRISM with direct feature concatenation and element-wise addition to evaluate the roles of refinement and adaptive composition. Starting from CAM-only concatenation, APR and PAPC together improve public gIoU/cIoU by 3.86/2.98 percentage points, while PRISM improves cIoU over element-wise addition by 3.48 points. The results are shown below (Appendix A.7.6, “Comparison with Simple Feature Fusion”, Table 13).

Comparison with Simple Feature Fusion.

MethodgIoUcIoU
Features-cat (CAM-only)46.5774.11
Features-add49.5173.61
PRISM50.4377.09

The component ablations further identify how these gains arise. Adding APR to CAM+PAPC improves gIoU/cIoU by 1.18/0.27 points, demonstrating the benefit of target-specific refinement before composition (Main Text, Section 4.5.1, “Component Ablation”, Table 3). Pairwise scoring contributes 0.78/0.22 points, showing the benefit of accounting for expert combinations during selection (Main Text, Section 4.5.3, “Set-Routing Supervision and Utility Modeling”, Table 5).

Finally, we measured the computational cost of the 7B configurations, including parameters, FLOPs, peak GPU memory, and inference time. The additional computation and latency primarily arise from the repeated masked FetalCLIP forward passes used to generate Plane ScoreCAM. This supplies the spatial guidance whose segmentation benefit is demonstrated by the CAM ablation. Relative to LISA-7B, PRISM adds approximately 2.0% trainable parameters and 5.1% peak GPU memory, while achieving higher segmentation performance than all compared methods on both benchmarks. The full computational measurements are shown below (Appendix A.7.7, “Computational Cost and Inference Trade-Off”, Table 14).

Computational Cost and Inference Trade-Off.

MethodTotal params. (B)Trainable params. (M)FLOPs (T)Peak GPU memory (GB)Inference time (s)
LISA-7B7.7089288.2610.497717.1580.2506
Sim4Seg-7B (RVLS2M)7.7089288.2610.497717.1580.2650
PRISM-7B (Plane ScoreCAM)8.1424294.076176.57618.0381.4946

We thank the reviewer again for this insightful comment, which helped improve the completeness of the experimental analysis.

AuthorsOpenReview ↗

Authors Rebuttal to Reviewer bEEP [3/6]

Q2. For a clinical application, there is very limited evaluation of query understanding and generalization. The paper claims query-conditioned segmentation, but the queries are generated from templates that explicitly specify structure and imaging plane. Does the model actually understand the query or simply learns to associate a fixed set of anatomical labels with segmentation masks? This is an important to distinguish, especially since F-TPA-SAM2 reports that fixed class-code prompts perform almost as well as learned text embeddings. The evaluation is also something I found not very clear. The private dataset uses source-image-level splits, is there a patient-level separation? The public benchmark combines several datasets but results are aggregate scores rather than separately per dataset or anatomical category.

Response: We appreciate your thoughtful comment! PRISM uses the meaning of the query to identify the requested anatomy instead of associating fixed anatomical labels directly with segmentation masks. We evaluate this through implicit clinical queries and unseen or held-out query formulations: PRISM segments anatomy from clinical descriptions without explicit plane or target names and maintains its performance under new formulations. We also clarify patient-level separation and provide detailed public-benchmark results.

Reviewer_AbF5OpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes PRISM, a framework for query-conditioned fetal ultrasound segmentation. It converts plane-level evidence from a pretrained fetal ultrasound VLM into target-specific spatial prompts. Activation Prior Refinement (APR) uses visual context and query semantics to refine a coarse plane activation map through a residual correction. Patch-Adaptive Prior Composition (PAPC) then selects a subset of experts constructed from different input combinations, and spatially fuses their predictions into a dense prompt for SAM. Experiments on public and private benchmarks report consistent improvements over the compared methods across 7B and 13B models.

Critical strengths
  • The paper addresses a concrete gap between domain knowledge and spatial localization.
  • The framework supports multiple anatomical planes and segmentation targets within a unified formulation.
  • The paper is overall well organized and easy to follow.
Critical weaknesses
  • The evaluated queries specify predefined imaging planes and target structures, and Appendix A.6 describes template-based wording variations. Under this setting, it remains unclear how much of the task requires language understanding beyond identifying the requested category. A comparison with a class-conditioned segmentation model using learned target embeddings, as well as evaluations on held-out query formulations, would help establish the added value of the language-based interface and the MLLM.
  • How the model behaves when a requested structure is absent, outside the field of view, or inconsistent with the image? It would be useful to clarify whether the model can return an empty mask or otherwise reject an invalid request.
  • Requiring the query to specify both the imaging plane and the target structure might be a strong assumption.
  • The use of plane-level evidence is motivated by FetalCLIP’s image–plane alignment. However, the target structure is already provided in the query, raising the question of whether target-text-conditioned activation maps or patch–text similarity maps could provide a more direct spatial prior.
  • The necessity of the two-level routing design remains insufficiently established.
  • The teacher evaluates candidate priors against the ground-truth mask without decoding each candidate through SAM. However, a prior with better direct mask overlap need not produce a better final segmentation when combined with SAM’s image features and the sparse query prompt.
Questions for authors

Please sse the weakness part.

4 comments
AuthorsOpenReview ↗

Authors Rebuttal to Reviewer AbF5 [1/3]

We sincerely thank you for your positive assessment of our work and your thorough feedback: the paper is overall well organized and easy to follow. We address your questions below and have incorporated all feedback in the revised version. All new material added to the revised manuscript has been highlighted in pink text for better visibility.

Q1. The evaluated queries specify predefined imaging planes and target structures, and Appendix A.6 describes template-based wording variations. Under this setting, it remains unclear how much of the task requires language understanding beyond identifying the requested category. A comparison with a class-conditioned segmentation model using learned target embeddings, as well as evaluations on held-out query formulations, would help establish the added value of the language-based interface and the MLLM. How the model behaves when a requested structure is absent, outside the field of view, or inconsistent with the image? It would be useful to clarify whether the model can return an empty mask or otherwise reject an invalid request.

Response: Thank you for your valuable feedback. We address this concern through three aspects: generalization across query formulations, comparison with a class-conditioned alternative, and behavior under invalid requests. We also clarify the patient-level data split.

1. Generalization beyond explicit query formulations.

To assess generalization beyond the query formulations used during training, we evaluated two settings:

  • Unseen-Query: keep the trained model unchanged and use a new inference-time template: “Which pixels in this {plane} belong to the {target}? Return their segmentation mask.”
  • Held-out Query: exclude the third original template from training, retrain the model, and evaluate using the excluded template.

PRISM maintains its advantage over LISA, with cIoU decreases of only 0.09 and 0.18 percentage points relative to the default setting. The results are shown below (Appendix A.7.3, “Generalization to Unseen and Held-Out Query Formulations”, Table 10).

Generalization to Unseen and Held-Out Query Formulations.

SettingPRISM gIoUPRISM cIoULISA gIoULISA cIoU
Default50.4377.0938.4170.01
Unseen-Query49.7777.0036.8068.12
Held-out Query49.9976.9136.3467.70

More directly, the implicit clinical-query evaluation omits both the imaging-plane name and the explicit target name. Details of the query construction, training protocol, results, and target formulations are provided in our response to Q2 below and in Main Text, Section 4.3, “Segmentation with Implicit Clinical Queries”, Table 2.

2. Comparison with a class-conditioned alternative.

We compared PRISM with FetalCLIP-SAM and a Class-Embedding baseline under the same evaluation protocols. PRISM achieves higher gIoU and cIoU on both benchmarks, improving cIoU over Class-Embedding by 1.78 percentage points on Private and 0.43 on Public. The results are shown below (Appendix A.7.2, “Comparison with FetalCLIP-SAM and Class-Embedding Baselines”, Table 9).

Comparison with FetalCLIP-SAM and Class-Embedding Baselines.

MethodPrivate gIoUPrivate cIoUPublic gIoUPublic cIoU
FetalCLIP-SAM6.188.3915.9516.13
Class-Embedding40.2264.7849.5776.66
PRISM-7B41.8966.5650.4377.09

These evaluations show that PRISM maintains performance across query formulations, processes the tested clinical descriptions without explicit plane or target names, and improves over the class-conditioned alternative.

3. Empty-mask outputs for invalid requests.

To assess behavior under invalid requests, we evaluated query–image mismatches and targets cropped out of view; all requests should yield empty masks. PRISM and LISA have similarly low empty-mask rates in these tests. Segmentation hallucinations under invalid requests remain a challenging research problem, which we plan to investigate in future work. The results are shown below (Appendix A.7.4, “Empty-Mask Outputs for Invalid Requests”, Table 11).

Empty-Mask Outputs for Invalid Requests.

Negative-query settingLISAPRISM
Query-image mismatch5.00%4.83%
Target cropped out of view12.92%13.25%

We sincerely thank the reviewer for the valuable comments, which have helped us improve the quality of this work.

AuthorsOpenReview ↗

Authors Rebuttal to Reviewer AbF5 [2/3]

Q2. Requiring the query to specify both the imaging plane and the target structure might be a strong assumption.

Response: Valuable feedback! The original templates were developed through discussions with clinicians to reflect routine examinations, where the imaging plane and the anatomy of interest are typically known. PRISM also supports queries that describe the requested anatomy without explicitly naming either.

Under the guidance of experienced clinicians, we formulated implicit clinical queries for five target categories from the public benchmark, retaining the existing images and ground-truth masks. These queries identify anatomy through its role in a clinical measurement. For example, the implicit query for the fetal head region is: “For fetal growth assessment using a cranial circumference measurement, segment the full region enclosed by the measurement contour.” All compared methods were retrained using the same implicit-query training data and evaluated on the corresponding held-out test split, retaining the original public-benchmark split assignments.

PRISM achieves the highest gIoU and cIoU among the compared methods under this shared setting, demonstrating effective segmentation from clinical descriptions without explicit plane or target names. Together with class-embedding comparison and unseen/held-out-query evaluations, these results support the practical value of a language interface that accommodates both alternative wording and implicit clinical descriptions. The results are shown below (Main Text, Section 4.3, “Segmentation with Implicit Clinical Queries”, Table 2). The subset composition (Table 18), query formulations, and corresponding targets are provided in Appendix A.7.10, “Implicit Clinical Queries and Corresponding Segmentation Targets”.

Segmentation with Implicit Clinical Queries.

MethodgIoUcIoU
READ(CVPR2025)81.0686.47
SESAME(CVPR2024)83.9087.23
LISA(CVPR2024)83.9086.64
Sim4Seg(AAAI2026)84.4687.88
Ours(CLIP)85.4989.17

We sincerely appreciate the reviewer’s suggestion, which has helped us improve the clarity and completeness of the manuscript.

Q3. The use of plane-level evidence is motivated by FetalCLIP’s image–plane alignment. However, the target structure is already provided in the query, raising the question of whether target-text-conditioned activation maps or patch–text similarity maps could provide a more direct spatial prior.

Response: Thank you for the insightful comment! Our choice of plane-level evidence is motivated by FetalCLIP’s pretraining. Although Target ScoreCAM appears to provide a more direct spatial prior, FetalCLIP is trained for image–plane alignment.

To assess whether plane-level evidence provides a more informative spatial initialization than target-conditioned alternatives, we compared Plane ScoreCAM with Target ScoreCAM and target patch–text similarity. Plane ScoreCAM achieves the highest scores on all three localization metrics, and the qualitative examples show more concentrated responses on the relevant anatomy. The results are shown below (Appendix A.7.5, “Localization Quality of Alternative Spatial Priors”, Table 12 and Figure 5).

Localization Quality of Alternative Spatial Priors.

Prior sourcePixel AP ↑Pointing hit ↑Response enrichment ↑
Plane ScoreCAM16.87%20.44%4.56×
Target ScoreCAM14.48%16.90%3.64×
Target patch–text4.09%0.90%1.01×

We thank the reviewer for the insightful comments, which have helped enhance the rigor of our experimental evaluation.

AuthorsOpenReview ↗

Authors Rebuttal to Reviewer AbF5 [3/3]

Q4. The necessity of the two-level routing design remains insufficiently established.

Response: Thank you for your valuable insights. The two-level design addresses two distinct decisions: which experts are relevant to the current image–query pair, and how their contributions should vary across the image. The Set Router selects an expert subset using request-level context and pairwise utilities. Within this subset, the Patch Router assigns location-dependent weights, allowing different experts to contribute to different anatomical regions.

We directly ablated APR and PAPC under a common experimental setting. The results are shown below (Main Text, Section 4.5.1, “Component Ablation”, Table 3).

Component Ablation.

CAMAPRPAPCgIoUcIoU
✗✗✗38.4170.01
✓✗✗46.5774.11
✓✓✗48.0875.64
✓✗✓49.2576.82
✓✓✓50.4377.09

Adding APR to CAM+PAPC improves gIoU/cIoU by 1.18/0.27 points, while adding PAPC to CAM+APR improves them by 2.35/1.45 points. Together, APR and PAPC improve over the CAM-only configuration by 3.86/2.98 points, supporting their complementary roles in prior refinement and adaptive evidence composition.

Further analyses examine the routing design within PAPC. Top-4 outperforms using all seven experts (Main Text, Section 4.5.4, “Top-kk Sensitivity”, Figure 4), supporting selective expert composition. Adding pairwise utilities further improves gIoU/cIoU by 0.78/0.22 points (Main Text, Section 4.5.3, “Set-Routing Supervision and Utility Modeling”, Table 5), supporting interaction-aware subset selection.

We thank the reviewer again for this helpful suggestion.

Q5. The teacher evaluates candidate priors against the ground-truth mask without decoding each candidate through SAM. However, a prior with better direct mask overlap need not produce a better final segmentation when combined with SAM’s image features and the sparse query prompt.

Response: Thank you for the comment.

The component ablation shows that adding and refining the spatial prior improves the final segmentation. CAM improves gIoU/cIoU from 38.41/70.01 to 46.57/74.11, while the complete model reaches 50.43/77.09 (Main Text, Section 4.5.1, “Component Ablation”, Table 3).

Component Ablation.

CAMAPRPAPCgIoUcIoU
✗✗✗38.4170.01
✓✗✗46.5774.11
✓✓✗48.0875.64
✓✗✓49.2576.82
✓✓✓50.4377.09

We further ablated the proxy supervision used by the Set Router. Removing route distillation reduces performance to 48.96/76.14, whereas soft prior-proxy supervision achieves 50.43/77.09. These metrics are computed from the final SAM-decoded masks, showing that the proxy criterion improves downstream segmentation (Main Text, Section 4.5.3, “Set-Routing Supervision and Utility Modeling”, Table 5).

Set-Routing Supervision and Utility Ablation.

VariantTeacherSingletonPairwisegIoUcIoU
No route distillation✗✓✓48.9676.14
Hard prior teacherhard✓✓50.1376.42
Soft prior-proxy teachersoft✓✗49.6576.87
Soft prior-proxy teachersoft✓✓50.4377.09

Finally, using the same trained checkpoint, images, queries, SAM image features, and sparse prompts, we decoded all 35 candidate compositions. The Learned Set Router achieves 50.43/77.09, closely matching direct selection by the minimum prior BCE+Dice loss, which achieves 50.51/77.15 (Appendix A.7.9, “Candidate Selection Before and After SAM Decoding”, Table 17).

Candidate Selection Before and After SAM Decoding.

Selection rulegIoUcIoU
Learned Set Router50.4377.09
Proxy-loss minimum50.5177.15

The close agreement indicates that the learned router follows the objective-aligned proxy criterion and that this criterion remains effective after SAM decoding.

We appreciate the reviewer’s insightful suggestion once again.

Dear Reviewer AbF5

I hope this message finds you well.

I am one of the authors of Paper 11503, titled "Refining Plane Priors for Query-Conditioned Fetal Ultrasound Segmentation." As the author-reviewer discussion period is in its final stages, I wanted to gently follow up to see if our responses have satisfactorily addressed all of your concerns. If there are any additional points or feedback you'd like us to consider, please let us know. Your insights are invaluable to us, and we are eager to address any remaining issues to improve our work.

Thank you very much for your time and consideration.

Best regards,

The authors of Paper "Refining Plane Priors for Query-Conditioned Fetal Ultrasound Segmentation" (ID: 11503)

Comments on the paper

1 comment

Dear ICLR 2027 AC, SAC, and PC,

We sincerely thank you and the reviewers for the time and effort devoted to evaluating our work and supporting the review process. The constructive feedback has helped us strengthen the manuscript. As the discussion draws to a close, we briefly summarize our work and the substantive changes made in response to the reviewers’ feedback. All revisions and additional experimental results have been incorporated into the revised manuscript and highlighted in pink for ease of reference.

Our framework, PRISM, converts pretrained fetal-ultrasound knowledge into query-specific spatial prompts, supporting multiple anatomical planes and targets within one framework. The reviewers highlighted several aspects of this contribution:

  • AbF5 noted that the paper addresses “a concrete gap between domain knowledge and spatial localization.”
  • bEEP described the use of pretrained fetal knowledge as “reasonable and well motivated” and emphasized that “the reported results are also strong,” with consistent gains across both benchmarks and model scales.
  • h9Fc found the design “well motivated for fetal ultrasound” and the component ablation “particularly convincing.”

The rebuttal added evidence addressing the main experimental questions:

  1. Design justification. New matched FetalCLIP-SAM and class-embedding comparisons show higher performance for PRISM on both benchmarks. PRISM also exceeds simple concatenation and addition by 2.98 and 3.48 cIoU points, while three-seed ablations further evaluate APR and pairwise routing (Tables 9, 13, 19).
  2. Language-interface evaluation. Added implicit clinical-query experiments evaluate descriptions without explicit plane or target names under matched retraining. Separate unseen and held-out wording evaluations show smaller performance drops for PRISM than for LISA (Tables 2, 10).
  3. Reliability and transparency. New plane-accuracy, error-group, and oracle analyses quantify sensitivity to plane prediction. We also clarified patient-disjoint splits and added per-anatomy results and computational measurements (Tables 8, 14–16).

Following these additions, reviewer bEEP noted that the rebuttal had addressed the key concerns raised in the review and updated their score to 3.

We believe these results clarify the contribution, empirical performance, and design motivation of our framework.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.