acceptodds
Under review as a conference paper at ICLR 2027

EGMNet: Error-free and Global Memory Evolution for Breast Ultrasound Video Segmentation

Abstract

Breast ultrasound video segmentation remains challenging due to severe appearance variations and temporal discontinuity, where lesions may temporarily disappear, change their appearance, or be interfered by newly emerging targets, resulting in inaccurate lesion re-identification and slow segmentation recovery after reappearance. Existing memory-based methods alleviate this issue by storing historical representations, but recursive memory evolution may introduce accumulated update deviations and weaken previously learned lesion information during long-term propagation. Moreover, current-frame-driven memory updating lacks sufficient global temporal awareness, while redundant information in memory states limits representation efficiency. To address these challenges, we propose EGMNet, an efficient memory optimization framework for breast ultrasound video segmentation, consisting of three complementary components. First, we introduce EME, which employs continuous-time dynamics to derive an analytical memory transition process, enabling accurate spatial memory updating with reduced approximation deviation. Second, we develop GMA based on recursive least squares, which formulates memory updating as a projection optimization problem and incorporates global temporal information for adaptive memory evolution. Third, we propose CMC to exploit the intrinsic low-dimensional structure of memory updates, reducing redundant information storage while preserving informative representations. Extensive experiments on the BUS video segmentation dataset demonstrate that EGMNet achieves significant improvements over existing state-of-the-art methods. Besides, The proposed EGMNet provides stronger robustness under lesion disappearance and appearance variation scenarios, enabling rapid target re-capture and more reliable long-term ultrasound video segmentation.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
Video segmentation, Breast ultrasound video analysis

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 4 reviews, above 27% of the others.
Reviewer_PLxYOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

This paper proposes EGMNet to address the long-term memory modeling problem in breast ultrasound video segmentation. The method consists of three modules: EME, which is based on continuous-time analytical updates; GMA, which is based on recursive least squares; and CMC, which is designed to reduce redundant tokens. Experiments demonstrate that this method achieves superior segmentation performance on the BUV-1K and BUV2022 datasets and exhibits certain advantages in terms of robustness and efficiency.

Critical strengths

1.The research problem is of practical significance. This paper addresses issues such as the temporary disappearance of lesions, changes in appearance, and long-term temporal consistency, demonstrating a clear research motivation.

2.The experimental results are competitive. The method achieved good segmentation performance on both datasets and demonstrated advantages in certain boundary metrics and video memory usage.

3.The experimental evaluation is relatively comprehensive. The authors provide module ablation, noise robustness, and temporal visualization experiments, which preliminarily validate the effectiveness of the overall framework.

Critical weaknesses

W1. The method’s independent innovation is not sufficiently clear.

EME’s continuous-time analytical update bears a strong resemblance to existing EFLA methods; GMA primarily employs the classical RLS algorithm; and CMC is closely related to common token compression strategies. It remains unclear where EGMNet’s core methodological innovations lie compared to these existing techniques. We recommend further clarifying the technical differences and including targeted comparative experiments.

W2. Some mathematical derivations and complexity analyses require further refinement.

It remains unclear whether GMA, after introducing additional coefficients into the standard RLS update, still corresponds to the exact optimal solution for the defined least-squares objective. Furthermore, Eq. (16) in CMC exhibits an issue with inconsistent matrix dimensions, and the computational complexity analysis in the appendix may have omitted operations related to O(dK2n)O(d_K^{2n}). We recommend that the authors further review the mathematical derivations and clarify the actual computational cost.

W3. Direct experimental validation of the core mechanism is lacking.

The paper emphasizes that EME can prevent the accumulation of memory errors and improve re-recognition capabilities after lesion disappearance. However, the existing experiments primarily demonstrate final segmentation performance and noise robustness, without directly measuring long-term memory drift or the speed of target recovery. Furthermore, Figure 8 uses video splicing from different patients to simulate lesion disappearance and reappearance, which still differs from real clinical scenarios. We recommend supplementing the work with quantitative validation using real-world temporal events.

Questions for authors

1.What are the specific algorithmic differences between EME and existing EFLA methods?

2.Does the GMA update strictly satisfy the defined least-squares optimization objective? Could you further explain the related complexity analysis?

3.Could you provide experiments that directly quantify the accumulation of memory errors and the recovery rate for lesion re-identification?

Reviewer_TpDvOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
1
Soundness
1
Read review
Paper summary

This paper proposes EGMNet for breast ultrasound video segmentation. The framework consists of three components: Error-Free Memory Evolution (EME), which derives an analytical memory update from continuous-time dynamics; Global Memory Adaptation (GMA), which employs recursive least squares; and Compact Memory Compression (CMC), which reduces spatial redundancy before memory updating. Experiments on BUV-1K and BUV2022 demonstrate improved segmentation accuracy over the selected baselines, along with favorable results in inference efficiency and noise robustness.

Critical strengths

The paper addresses the practically important problem of maintaining lesion identity under appearance variations and temporary disappearance. The experiments cover both a relatively large private dataset and a public benchmark, with evaluations of region overlap, boundary accuracy, and inference efficiency. The ablation studies and noise robustness experiments also provide useful evidence of the empirical effectiveness of the proposed framework.

Critical weaknesses

The first concern is the limited methodological novelty of this work. The continuous-time formulation and analytical update of EME in Eqs. (3) to (7) closely follow the existing EFLA method. GMA builds on standard recursive least squares, while CMC mainly uses spatial token projection. I do not think the paper sufficiently establishes a new methodological contribution beyond combining these existing techniques. More controlled comparisons under the same backbone and training settings would help justify the proposed design. Secondly, the theoretical justification incomplete. Although EME provides an exact update under the assumed continuous-time dynamics, this does not necessarily guarantee drift-free long-term memory propagation. For GMA, the initialization of the inverse correlation matrix and the additional update coefficient in Eqs. (12) and (13) appear inconsistent with the exact solution of the unregularized least-squares objective in Eqs. (8) and (9), unless further assumptions are made. I would like the authors to clarify these theoretical claims. Finally, I find the experimental evidence insufficient to fully support the claimed lesion re-identification capability. Figure 8 simulates disappearance and reappearance by concatenating videos from different patients rather than evaluating naturally occurring events. On BUV2022, which contains only 63 sequences, the Dice improvement over the strongest baseline is 1.27 percentage points, without reported variance or confidence intervals. More importantly, the larger BUV-1K dataset remains private, making independent verification difficult. I believe public or controlled access to this dataset would substantially improve the reproducibility of the work.

Questions for authors

(1)Given the close similarity between EME and EFLA, what is the main methodological novelty of EME? Could the authors provide controlled comparisons using the same backbone and training settings to better demonstrate its contribution? (2)How do the initialization of the inverse correlation matrix and the additional update coefficient in GMA affect the claimed exact recursive least-squares solution? Under what conditions does the proposed update recover the exact solution? (3)Can the authors evaluate lesion re-identification on real disappearance and reappearance events and quantify the recovery time?

Reviewer_rrNJOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes EGMNet, a memory-based framework for breast ultrasound video segmentation. The method aims to improve segmentation consistency when lesions temporarily disappear or undergo appearance changes. It consists of three main components. First, Error-Free Memory Evolution (EME) uses a continuous-time formulation to update the memory state analytically. Second, Global Memory Adaptation (GMA) uses recursive least squares to incorporate historical information into memory updates. Third, Compact Memory Compression (CMC) reduces redundant information through spatial token compression. The two memory branches are combined using a learnable gate. Experiments on a private dataset (BUV-1K) and the public BUV2022 dataset report improved segmentation accuracy and robustness compared with existing methods.

Critical strengths

First of all, the paper addresses an important problem in breast ultrasound video segmentation. Changes in lesion appearance and temporary disappearance make temporal tracking difficult. I think improving the memory update mechanism is a reasonable direction for addressing these challenges.

Another strength is the consistent improvement on both datasets. As shown in Table 1, EGMNet improves Dice over LiVOS by 2.37 percentage points on BUV-1K and 1.27 points on BUV2022. The method also achieves 54.44 FPS with lower GPU memory usage than GDKVM. The ablation studies and noise robustness experiments provide additional evidence for the effectiveness of the proposed framework.

Critical weaknesses

First, I have some concerns about the novelty of Error-Free Memory Evolution (EME). The update in Eq. (7) is very similar to the one in the cited EFLA paper. Also, solving the continuous-time equation exactly does not necessarily prevent memory drift caused by inaccurate features or predicted masks. The authors should better explain what is new and why the proposed update improves memory stability.

Second, the theoretical explanation of Global Memory Adaptation (GMA) is not fully convincing. The update in Eqs. (12)–(14), particularly the additional update coefficient and covariance initialization, does not appear to give the exact least-squares solution claimed in Eq. (8). The authors need to clarify this inconsistency. The tensor dimensions in the CMC equations also need to be checked.

Finally, I think more experiments are needed to support the lesion recovery claim. The noise experiments are useful, but the disappearance and reappearance results are mainly based on selected examples, including a sequence created by combining two patient videos. A quantitative evaluation of recovery time and segmentation errors during lesion absence would make the claim more convincing.

Questions for authors

First, what is the main difference between the proposed EME update and the existing EFLA method? Could the authors provide a more direct comparison to clarify the contribution of EME?

Second, could the authors clarify whether the GMA update in Eqs. (12)–(14) actually solves the least-squares problem in Eq. (8)? Is the update coefficient fixed to one? The tensor dimensions in Eqs. (16)–(17) also seem inconsistent.

Finally, could the authors provide a quantitative evaluation of lesion disappearance and reappearance across different patients? In particular, recovery time and false positives during lesion absence would be useful. It would also help to clarify how the initial segmentation mask is obtained during inference.

Reviewer_xZCZOpenReview ↗
Rating
2
Confidence
3
Informativeness
1
Presentation
2
Soundness
1
Read review
Paper summary

This paper proposes EGMNet on the key-value linear memory framework of GDKVM to address the memory degradation caused by lesion disappearance, reappearance, and abrupt appearance changes in breast ultrasound video segmentation.

Critical strengths
  1. The problem has clinical significance and the dataset is valuable. The rapid recovery after the disappearance and reappearance of the lesion is the real pain point of ultrasound video segmentation. BUV-1K is of a large scale and is divided by patients. If it can be made public, it will contribute to this field.

  2. The derivation of EME in the rank-one case is correct, introducing new progress in linear attention to medical video segmentation is reasonable.

Critical weaknesses
  1. EME's Eq. 7 is essentially still the Delta rule, only with the update coefficient re-parameterized from βt\beta_t to ηt\eta_t. The term "Error-free" is only applicable to the assumed continuous-time dynamics and cannot prove that there is no error accumulation in the actual memory update. Additionally, Eqs. 5 and 6 are derived based on the rank-one matrix ktkt⊤k_tk_t^\top, but the paper does not clearly explain how to generalize when using multiple tokens.

  2. The adaptive mechanism of GMA is questionable. The RLS algorithm does not have a forgetting factor. As the number of historical observations increases, the influence of new information may gradually weaken, which is not entirely consistent with the long-term adaptive goal. The standard RLS also cannot directly address the problem of historical token contribution dilution proposed by the author. Moreover, P0=ϵIP_0 = \epsilon I and ϵ\epsilon being small implies a strong initial regularization, but this is not reflected in Eq. 8.

  3. The storage benefits of CMC lack basis. The size of the memory state S∈RdV×dKS \in \mathbb{R}^{d_V \times d_K} is independent of the number of tokens, so CMC cannot directly reduce the storage of the persistent memory state. Reducing the spatial tokens to m=4m=4 may also cause an information bottleneck, but the paper does not analyze this issue.

  4. The experimental protocol and baselines are insufficient. The paper does not specify the initialization method for the first frame mask. Methods such as Cutie and LiVOS typically rely on semi-supervised initialization. If different methods adopt different protocols, the comparison might be unfair. Moreover, relevant baselines such as SAM2, MemSAM, and Vivim are missing.

  5. The core claim lacks direct experimental support. The paper did not conduct quantitative measurements of cumulative recursive error, memory drift, or target recovery time. Fig. 8 only shows a single stitched sequence, making it difficult to support the conclusion of universal rapid recovery. The experiments also did not report inter-run variance or confidence intervals, and the partial ablation improvements were small, resulting in insufficient statistical reliability.

Questions for authors
  1. Does the key perform L2 normalization? If so, what is the essential difference between EME and "changing the activation function for β\beta"?
  2. When there are multiple tokens in each frame, how is the closed-form solution of EME calculated? Is it by serializing token by token, performing feature decomposition, or approximating directly using the rank-one formula?
  3. How are the first frame masks of EGMNet and each baseline obtained? Are they using the same initialization?
  4. Does GMA use the forgetting factor? If not, how does it adapt to newly emerging lesions in long videos? What is the specific value of ϵ\epsilon?
  5. The readout end only uses 4 compressed tokens. Where do the spatial details come from? How much does the performance decrease after removing the memory branch?
  6. What exactly is "baseline" in Table 2? Is it GDKVM?

Comments on the paper

2 comments
AuthorsOpenReview ↗

Response to Reviewer

We thank the reviewer for the comments. We have clarified the implementation details and added additional comparisons in our response. We would appreciate it if the reviewer could confirm whether these clarifications address the main concerns.

Thank you for the additional experiments and clarifications.

I found the lengthy and repetitive responses difficult to follow, particularly when trying to identify direct answers to my technical questions. Most of my original concerns remain unresolved, and I was expecting more direct quantitative evidence rather than large blocks of qualitative explanations.

  1. EME: The methodological novelty relative to EFLA remains unclear, and the rank-one derivation does not adequately justify the joint multi-token update.
  2. GMA: The adaptive update is still not adequately justified by the stated least-squares objective.
  3. CMC: The authors have not demonstrated that CMC actually reduces persistent memory storage. Although additional comparisons were provided, the requested quantitative evaluations of recovery time, cumulative memory drift, and statistical reliability remain insufficient.

I therefore maintain my original rating.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.