acceptodds
Withdrawn from ICLR 2027

Persistent Memory Bridges Occlusions in Video Object Segmentation

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Memory-based video object segmentation models such as SAM2 condition each frame on a fixed-size window of recent memory frames. We show that this fixed memory horizon is a substantial, previously unaddressed cause of a costly failure: once an object is occluded for longer than the window, every pre-occlusion memory frame has been evicted and the model falls back on a stale initialization it rarely recovers from. We introduce Persistent Memory Reinjection (PMR), a training-free mechanism that keeps a small buffer of pre-occlusion memory alive past the window and reinjects it into the model's own cross-attention when the recency window is incomplete, an exact and free occlusion signal whenever tracking is interrupted; for unconstrained video, where memory is written on every frame, we evaluate always-on and confidence-triggered variants of the same mechanism. On a controlled occlusion benchmark over 1500 MOSEv2 sequences, PMR improves post-occlusion region accuracy by up to 17 points (up to 6 on DAVIS-2017), bootstrap-significant at every occlusion length beyond the window, while leaving continuous-tracking accuracy exactly unchanged. Using only the four freshest pre-occlusion frames, PMR matches an infinite-horizon oracle that attends to all past memory, showing that of the re-acquisition accuracy recoverable without new supervision, the bounded horizon rather than memory content or capacity is the main limiting factor. PMR composes with fine-tuning, which improves continuous tracking but not re-acquisition, and extends to naturally occurring failures of several types.

Primary area:
foundation or frontier models, including LLMs
Keywords:
video-object-segmentation, occlusions, persistent-memory

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_iqpUOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper investigates re-acquisition failures in SAM2 when a long interruption makes useful pre-interruption memories inaccessible to its fixed recency window. It proposes Persistent Memory Reinjection (PMR), which retains previously encoded memories and re-keys the freshest entries into existing attention slots when the window is incomplete; separate always-on and confidence-triggered variants address continuously processed videos. The authors evaluate a controlled frame-skip protocol on MOSEv2 and DAVIS-2017, along with natural failure episodes, buffer-size ablations, fine-tuning, and a comparison with SAM2Long.

Critical strengths
  1. The paper identifies a concrete memory-access failure in a popular streaming segmenter. The fixed-window analysis explains why previously computed memories can become inaccessible following an interruption, and the controlled experiments test that diagnosis directly.

  2. PMR is an economical inference-time intervention. Its gap-length and buffer-size analyses provide evidence that access to a few recent pre-interruption memories can recover object identity without retraining the segmenter.

Critical weaknesses
  1. The main evaluation does not establish an improvement for occlusions encountered during ordinary video propagation. In Sec. 4.1, the controlled benchmark skips the entire interval, including the RGB frames, so missing entries are created in the recent-memory window. This directly enables the proposed structural gate. During continuous propagation, however, the model still writes memory on occluded frames, and the gate does not fire (Sec. 4.8). The gated method therefore produces exactly the SAM2 baseline results in Table 4. For the always-on variant designed for natural videos, Supplementary Table A2 reports J=0.7744 versus 0.7750 for SAM2, with no significant improvement in full-sequence accuracy. Recovery on episodes selected because the baseline already failed is useful evidence, but it does not replace evaluation of the net effect over complete videos. I am not convinced that the large gains in Table 1 support the broader occlusion-recovery claim.

  2. The comparisons are too limited to establish the incremental contribution of this memory policy. The authors do compare SAM2Long [1], but Supplementary Table A4 shows statistically comparable improvements across the 700 natural failure episodes, with +0.156 for PMR and +0.165 for SAM2Long. The significant PMR advantage is confined to 40 beyond-window occlusion episodes (Supplementary Table A5), mostly with relatively short absences. No corresponding comparison establishes an advantage on full continuous-video benchmarks, and the relevance of PMR to newer memory-based SAM 3 systems [3] is untested. Training-free occlusion-aware memory management also predates this work, including MA-SAM2 [2]. A simple last-valid-memory retention control would help isolate the benefit of re-keying from the benefit of preserving a reliable memory. Moreover, the learned and oracle retrieval ablations change buffer admission along with selection (Sec. 4.7), so they do not isolate memory-selection quality.

[1] SAM2Long: Enhancing SAM 2 for Long Video Segmentation with a Training-Free Memory Tree, ICCV 2025.
[2] Memory-Augmented SAM2 for Training-Free Surgical Video Segmentation, MICCAI 2025.
[3] SAM 3: Segment Anything with Concepts, arXiv 2025.

Questions for authors
  1. In Supplementary Table A2, the always-on variant improves the selected failure episodes but not the full-sequence mean. Can the authors quantify how many frames or sequences improve versus deteriorate, using the existing continuous-video predictions?

  2. Under the same FIFO bank and attention-slot budget, how does PMR compare with simply retaining the last reliable pre-gap memory? Does the improvement depend on the proposed temporal re-keying, rather than on keeping that memory accessible?

2 comments

We thank the reviewer for their comments and provide responses below.

Can the authors quantify how many frames or sequences improve versus deteriorate, using the existing continuous-video predictions?

Always-on PMR, MOSEv2 held-out, 800 videos, all objects: 175 videos better, 164 worse, 461 within 0.5 J (131 bit-identical); 10.3% of frames better and 10.0% worse (of 57,554, 1 J tolerance); full-video J +0.30 [-0.32, +0.91]. Confidence-gated PMR: 102 / 86 / 612 (494 identical), +0.39 [-0.13, +0.90]. The gated structural variant is bit-identical to SAM2 on all 800 videos.

Under the same FIFO bank and attention-slot budget, how does PMR compare with simply retaining the last reliable pre-gap memory? Does the improvement depend on the proposed temporal re-keying, rather than on keeping that memory accessible?

Corrected frame-skip protocol, post-gap J vs baseline:

variantg=10g=20g=40g=80
PMR (4 memories, re-keyed)+11.8*+13.0*+8.8*+8.9*
same 4 memories, not re-keyed+11.7*+12.9*+7.6*+9.0*
single last reliable memory, re-keyed+9.7*+9.7*+6.4*+9.1*
single memory, not re-keyed+11.2*+11.1*+7.2*+9.4*

The improvement comes from keeping pre-gap memory accessible: re-keying has no measurable effect (paired -0.1 / -0.1 / -1.2 / +0.1, all n.s.), and a single retained memory, not re-keyed, gives +11.2* / +11.1* / +7.2* / +9.4*. We will simplify the method accordingly.

The learned and oracle retrieval ablations change buffer admission along with selection, so they do not isolate memory-selection quality.

With admission held fixed (K = 16 FIFO), selecting 4 buffered memories at random gives +11.1* / +11.0* / +8.8* / +11.8*, against +11.8* / +13.0* / +8.8* / +8.9* for recency; recency is a sufficient rule.

Training-free occlusion-aware memory management also predates this work, including MA-SAM2. [...] the relevance of PMR to newer memory-based SAM 3 systems is untested.

MA-SAM2's released code depends on a modified sam2_base and a loader that were not released, so it could not be run; we will cite it. On SAM 3.1, PMR gives +4.4* / +8.1* / +5.3* at g = 10 / 20 / 40. SAM 3 as released already keeps confidently present frames in memory with no time horizon, so PMR does not add to it; gated PMR is bit-identical to each base model on continuous video.

Thanks for the rebuttal. I appreciate the additional comparisons, but my main concern remains, so I will keep my rating.

Reviewer_P4N6OpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

The paper studies re-acquisition failures of SAM 2–style video object segmentation (VOS) models. SAM 2 conditions each frame on the first frame plus a fixed window of the m = 6 most recent memory frames. The authors argue that once tracking is interrupted for more than m frames, no pre-interruption memory is attended, and the model falls back on the first-frame template. They propose Persistent Memory Reinjection (PMR), a training-free inference-time change:

  • PMR keeps a per-object FIFO buffer of recent memories with non-empty masks (K = 16).
  • When the recency window is incomplete, it re-keys the n = 4 most recent buffered memories into the oldest attention slots.
  • For ordinary video, where the window never has holes, the paper uses an always-on variant and a confidence-triggered variant (object score < 0.5).

The main evidence is a controlled protocol on held-out MOSEv2 training sequences and DAVIS-2017. Tracking is run to 30% of the video, a block of frames is skipped without running the model, and tracking then resumes. On this protocol PMR improves post-resumption J by +0.086 to +0.174 for gaps of 10–160 frames and is exactly inert on uninterrupted video. The paper also reports:

  • an "Oracle Reach" comparison;
  • ablations on buffer size and frame selection;
  • natural-failure episodes, with a comparison to SAM2Long;
  • results with a fine-tuned model;
  • transfer to other SAM 2–family models.
Critical strengths
  1. A clean, controlled diagnostic with careful statistics. The skip protocol isolates one factor, whether pre-interruption memory is reachable, from memory contamination. The paper reports paired bootstrap CIs over sequences. The effect is consistent across gap lengths and holds on all four Hiera sizes. This matters because it makes a narrow claim credible: when SAM 2's recency window is empty, reinjecting a few recent pre-gap memories recovers a meaningful part of the lost accuracy.
  2. A simple mechanism with no cost when inactive. PMR adds no parameters or training. Its structural gate leaves outputs bit-identical to SAM 2 when the window is full (per-sequence difference exactly 0 on MOSEv2 and DAVIS). Code is provided. This matters because the method is easy to adopt in any setting where the window can actually have holes.
Critical weaknesses
  1. The main evidence comes from a self-constructed protocol that matches neither a recognized benchmark nor real occlusion. All headline results come from the authors' own "controlled occlusion" protocol: the improvements of up to 17 J points, the significance at every gap length, and the "exactly zero cost" guarantee. In that protocol a block of frames is simply never processed. Standard VOS evaluation never does this: DAVIS, YouTube-VOS, MOSE/MOSEv2, LVOS and SA-V all run the model on every provided frame. The protocol uses held-out training sequences of MOSEv2 rather than the official splits, so the numbers cannot be compared with the literature.

    More importantly, it is unclear that skipping frames is a fair stand-in for occlusion:

    • The target is not occluded. The selection criteria only require the target to be visible (≥100 px) at both ends of the gap. During the skipped frames the target is typically still visible; it is simply never observed. The model never experiences the occluder or partial visibility.
    • Memory is never contaminated. No memory is written during the gap. Under real occlusion, SAM 2 writes a memory on every frame, including frames where the target is absent or mis-segmented. Contamination of the window by such frames is arguably the main failure mechanism in practice, and the protocol removes it by construction.
    • The window never has holes in real video. The structural gate fires only when the recency window has holes, which never happens under frame-by-frame inference. The paper acknowledges this (Sec. 4.8, Conclusion), so the component behind the clean results and the zero-cost guarantee is inactive on real video.

    On natural video the evidence is much weaker. The always-on variant changes full-sequence J by −0.0008 (CI [−0.009, +0.007]). On the sequences that contain failure episodes the change is +0.006 (CI [−0.013, +0.025]). The reported +0.156 is measured only on episodes selected because the baseline failed, which favors any method that differs from the baseline. Because the title and abstract frame the contribution as "bridging occlusions," this mismatch between the evaluated setting and the claimed setting is decisive for my assessment.

    The protocol is also under-specified relative to the code. The pre-gap pass tracks frame gap_start inclusively, so only g−1 frames are skipped. For shorter videos the gap is silently truncated (gap_end = min(gap_start + g, n − 10)), so the nominal g of the 40/80/160 rows is an upper bound.

  2. Positioning relative to prior work, and missing baselines. The claim that the fixed memory horizon is a "previously unaddressed" cause is not accurate:

    • SAM2Long (object-aware memory) and SAMURAI (motion-aware memory selection) already search past frames, keep those with a confident object/IoU MOSE/MOSEv2, LVOS and SA-V all run the model on every provided frame. The protocol uses held-out training sequences of MOSEv2 rather than the official splits, so the numbers cannot be compared with the literature.

    More importantly, it is unclear that skipping frames is a fair stand-in for occlusion:

    • The target is not occluded. The selection criteria only require the target to be visible (≥100 px) at both ends of the gap. During the skipped frames the target is typically still visible; it is simply never observed. The model never experiences the occluder or partial visibility.
    • Memory is never contaminated. No memory is written during the gap. Under real occlusion, SAM 2 writes a memory on every frame, including frames where the target is absent or mis-segmented. Contamination of the window by such frames is arguably the main failure mechanism in practice, and the protocol removes it by construction.
    • The window never has holes in real video. The structural gate fires only when the recency window has holes, which never happens under frame-by-frame inference. The paper acknowledges this (Sec. 4.8, Conclusion), so the component behind the clean results and the zero-cost guarantee is inactive on real video.

    On natural video the evidence is much weaker. The always-on variant changes full-sequence J by −0.0008 (CI [−0.009, +0.007]). On the sequences that contain failure episodes the change is +0.006 (CI [−0.013, +0.025]). The reported +0.156 is measured only on episodes selected because the baseline failed, which favors any method that differs from the baseline. Because the title and abstract frame the contribution as "bridging occlusions," this mismatch between the evaluated setting and the claimed setting is decisive for my assessment.

    The protocol is also under-specified relative to the code. The pre-gap pass tracks frame gap_start inclusively, so only g−1 frames are skipped. For shorter videos the gap is silently truncated (gap_end = min(gap_start + g, n − 10)), so the nominal g of the 40/80/160 rows is an upper bound.

  3. Positioning relative to prior work, and missing baselines. The claim that the fixed memory horizon is a "previously unaddressed" cause is not accurate:

    • SAM2Long (object-aware memory) and SAMURAI (motion-aware memory selection) already search past frames, keep those with a confident object/IoU score, and place the most recent qualifying ones into SAM 2's memory slots. Pre-occlusion memory therefore stays reachable however long the occlusion.
    • DAM4SAM skips memory updates when the predicted mask is empty and states the same diagnosis: SAM 2 fills its memory with frames lacking the target.
    • The MOSEv2 paper's RCMS promotes up to four reliable pre-disappearance memories when the target first disappears.
    • SAM 3 retains only frames in which the object is confidently present.

    PMR's natural-video variants (a confidence-filtered buffer reinjected always or on low confidence) are essentially the same mechanism. Several of these works are cited, but the only one compared is SAM2Long, and only on natural-failure episodes. There PMR is statistically tied overall (−0.009, CI [−0.028, +0.010]) and significantly worse on scale changes. There is no comparison with:

    • SAMURAI's memory selection;
    • DAM4SAM;
    • RCMS;
    • SAM 3's memory selection;
    • the built-in memory_temporal_stride_for_eval option, which widens the reach of SAM 2's memory without retraining;
    • SAM 3/3.1 as base models.

    Without these comparisons, it is unclear what PMR adds beyond the frame-skipping setting.

Questions for authors

Clarification (Weakness 1)

  1. Does frame skipping stand in for occlusion? In the skip protocol, in what fraction of skipped frames is the target actually occluded or absent according to the ground truth? Can you show that gains under frame skipping carry over to real occlusion of comparable length, for example on natural occlusion episodes stratified by ground-truth absence length? If the intended setting is interrupted tracking rather than occlusion (frame drops, low-rate inference), please evaluate one such realistic setting directly.

Experiments that could change my assessment

  1. Standard benchmarks (Weakness 1). Results under the standard protocols, on full sequences rather than failure episodes only:

    • MOSEv2 val/test via the official server, reporting J&Ḟ together with the disappearance (J&Ḟ_d) and reappearance (J&Ḟ_r) metrics;
    • LVOS v2;
    • SA-V.
  2. Comparison with prior baselines (Weakness 2). On these benchmarks, can PMR outperform existing training-free memory-selection methods that already keep pre-occlusion memory reachable? These include:

    • SAMURAI's memory selection;
    • DAM4SAM;
    • MOSEv2's RCMS;
    • SAM2Long's object-aware memory (without the memory tree);
    • SAM 3's memory selection.

    Does PMR still help when applied on SAM 3/3.1?

1 comment

We thank the reviewer for their comments and provide responses below.

In the skip protocol, in what fraction of skipped frames is the target actually occluded or absent according to the ground truth?

gntarget absent (% of skipped frames)target area < 50% of pre-gapany absence (% of videos)
108083.4%10.4%9.9%
206094.4%13.9%17.9%
402827.0%21.2%28.0%
80788.2%25.7%34.6%

The target is mostly visible during the gap, so the protocol measures recovery after an interruption in observation. We will describe it that way.

The pre-gap pass tracks frame gap_start inclusively, so only g-1 frames are skipped. For shorter videos the gap is silently truncated.

Both confirmed and corrected (skip exactly g frames; drop videos that cannot hold the gap). Baseline post-gap J and n on the corrected protocol: g = 10: 50.0 (808); 20: 45.5 (609); 40: 43.7 (282); 80: 42.7 (78). PMR: +11.8* / +13.0* / +8.8* / +8.9*.

Can you show that gains under frame skipping carry over to real occlusion of comparable length, for example on natural occlusion episodes stratified by ground-truth absence length?

Ground-truth absence events on MOSEv2 held-out (800 videos; target absent for L frames after at least five visible frames, then reappears; J over the 30 frames after reappearance; CI over events):

absence Ln eventsbaseline JPMR (always-on)
1-6 (inside the window)59353.9+0.4 [-1.3, +2.3]
7-1518957.0+1.6 [-1.2, +4.5]
16-4010236.8+9.2* [+2.4, +15.9]
> 404837.5+4.9 [-5.5, +15.6]
> 6 (beyond the window)33948.2+4.4* [+1.3, +7.4]

The gain appears for absences longer than the window, as the mechanism predicts, and is largest for absences of 16-40 frames.

If the intended setting is interrupted tracking rather than occlusion (frame drops, low-rate inference), please evaluate one such realistic setting directly.

Low-rate inference on 500 MOSEv2 videos (every K-th frame processed, true frame indices): PMR +3.9* at K = 8 (no significant change at K = 2 and 4). With bursty frame drops (bursts starting with probability 1/60 per frame, mean length 10 or 30 frames): +1.6* and +1.4* over the full video, +6.7* and +7.0* on the first frame after each drop.

The reported +0.156 is measured only on episodes selected because the baseline failed, which favors any method that differs from the baseline.

We will report ground-truth-defined events (previous comment) and full-video accuracy in place of the baseline-selected episodes. Full video, 800 MOSEv2 held-out videos: always-on PMR +0.30 [-0.32, +0.91]; gated PMR bit-identical to SAM2.

Results under the standard protocols, on full sequences [...] LVOS v2 [...] Comparison with prior baselines [...] SAMURAI's memory selection; DAM4SAM; MOSEv2's RCMS; SAM2Long's object-aware memory; SAM 3's memory selection; the built-in memory_temporal_stride_for_eval option.

All run from released code on the same checkpoint. LVOS v2 val, official J&F (140 videos): SAM2.1-L 83.7; PMR always-on 83.2 (+0.19, n.s.); PMR gated 84.4 (+1.75, n.s.); stride-4 84.8 (+1.29, n.s.); RCMS 84.6 (+2.57*); SAM2Long 85.5 (+2.62*); PRES 85.8 (+3.42*); SAMURAI 86.1 (+3.32*); DAM4SAM 86.5 (+4.52*). MOSEv2 held-out, ground-truth absences beyond the window (339 events): PMR +4.4*, RCMS +4.6*, SAM2Long +5.4*, PRES +5.5*, stride-4 +5.8*, DAM4SAM +6.4*, SAMURAI +9.6*. PMR's distinctive setting is interrupted tracking, where the tracker receives no frames during the gap and present-only admission has nothing to filter; there it gives +11.8* / +13.0* / +8.8* / +8.9* at g = 10 / 20 / 40 / 80.

Does PMR still help when applied on SAM 3/3.1?

On SAM 3.1, yes: +4.4* / +8.1* / +5.3* at g = 10 / 20 / 40. SAM 3 as released already keeps confidently present frames in memory with no time horizon (that selection alone is worth +14.6* / +12.0* / +7.4* on this protocol), so PMR does not add to it. Gated PMR is bit-identical to each base model on continuous video.

Reviewer_78LFOpenReview ↗
Rating
1
Confidence
4
Informativeness
1
Presentation
1(−1)
Soundness
2
Read review
Paper summary

Paper presents a memory mechanism for SAM2, targeting improvements during occlusion scenarios. Authors state the problem that SAM2 always updates the memory, even when the target is occluded or not present in the frame. This leads to the memory consisting of the initial frame of the video and the 'empty' frames (without target visible). To address this, they propose PMR, which is a training-free memory mechanism for SAM2 that operates when the target is occluded by reinjecting the frames from the most recent frames before occlusion into the memory and not updating the memory during the occlusion periods. SAM2+PMR shows improved performance, over SAM2, in occlusion scenarios.

Critical strengths

The proposed approach is simple, training-free, and improves the results in videos with occlusion.

Critical weaknesses

Weakness 1: Almost no novelty compared to DAM4SAM[1]. The central problem the paper aims to address is to recover from the occlusion of the object. The weakness of SAM2 there, as the authors state, is that it updates the memory when the target is not present, leading to 'empty' (without target visible) frames in the memory when the target is occluded. This problem has been addressed within DAM4SAM. In that paper, in architecture justifications, authors include model variant SAM2.1_PRES which includes in the memory only frames where the target is present (the predicted mask is not empty). The introduction of PMR in this ICLR submission is largely based on that idea. The claim that the problem was previously unaddressed is largely false.

Weakness 2: Very limited related work / baseline comparison. Out of all models that improve memory upon SAM2 (SAMURAI, DAM4SAM, SAM2Long), mentioned in Section 2, the method only compares against SAM2Long on a non-standard tracking evaluation. Furthermore, VOT datasets (e.g., LaSOT[2]) are likely more challenging with longer occlusion periods compared to VOS datasets. Also, the experiments on the size of K (comparison K=16 or K=32 or K=64) were likely not needed. As the authors state in the paper, PMR conditions only on the n=4 freshest entries. Similar holds for the experiments in Table 4, where by design there would be no injection of buffer frames if there is no occlusion, therefore the same results are expected.

[1] A Distractor-Aware Memory for Visual Object Tracking with SAM2, Jovana Videnovic, Alan Lukezic, Matej Kristan; CVPR, 2025

[2] LaSOT: A High-quality Benchmark for Large-scale Single Object Tracking, Heng Fan, Liting Lin, Fan Yang, Peng Chu, Ge Deng, Sijia Yu, Hexin Bai, Yong Xu, Chunyuan Liao, Haibin Ling; CVPR, 2019

Questions for authors

Question 1: How do you describe the novelty of proposed work compared to DAM4SAM and the ablations included in DAM4SAM paper?

Question 2: Could you evaluate all methods mentioned in Section 2 on an online tracking setup on at least LaSOT dataset?

2 comments

We thank the reviewer for their comments and provide responses below.

Weakness 1: Almost no novelty compared to DAM4SAM. [...] authors include model variant SAM2.1_PRES which includes in the memory only frames where the target is present. [...] Question 1: How do you describe the novelty of proposed work compared to DAM4SAM and the ablations included in DAM4SAM paper?

Thank you for pointing to the SAM2.1_PRES ablation; we will cite and discuss it. PRES and PMR act at different points. PRES filters what is written to memory while the target is absent; PMR keeps already-written memory reachable after the recency window has moved past it. The two coincide on ordinary video, where we find them comparable (we rebuilt PRES from the DAM4SAM code and ran it on the same checkpoint: paired PMR minus PRES on 800 MOSEv2 held-out videos, full video, +0.26 [-0.42, +0.95]). They differ when the tracker receives no frames during the gap (interrupted tracking, frame drops, low-rate inference): present-only admission then has nothing to filter, and retention is what helps, +11.8* / +13.0* / +8.8* / +8.9* post-gap J at g = 10 / 20 / 40 / 80 on the corrected protocol, on all four SAM2.1 sizes and the distilled students. We will present PMR as a reach mechanism for interrupted tracking, position it relative to PRES, and remove the "previously unaddressed" wording.

Question 2: Could you evaluate all methods mentioned in Section 2 on an online tracking setup on at least LaSOT dataset?

We ported SAMURAI, DAM4SAM, PRES, SAM2Long and RCMS from their released code and ran them on the same checkpoint. On ground-truth absences longer than the memory window in MOSEv2 held-out (339 events, J over the 30 frames after reappearance), the gains are PMR +4.4*, RCMS +4.6*, SAM2Long +5.4*, PRES +5.5*, stride-4 +5.8*, DAM4SAM +6.4*, SAMURAI +9.6*; on full videos all of these are within one J point of the baseline except SAMURAI (-1.6*).

The experiments on the size of K [...] were likely not needed. [...] Similar holds for the experiments in Table 4.

Agreed on both; we will condense them.

I thank the authors for their rebuttal. However, I will keep my score of clear reject. The rebuttal did not address my concerns. Also, the presentation of the results is very unclear.

  1. Following my request in the original review, the authors explicitly acknowledge that DAM4SAM-PRES and PMR coincide on the ordinary tracking videos and that the claim of a 'previously unaddressed' problem was inaccurate. The formulation of interrupted tracking is insufficiently motivated and not clearly defined, with insufficient benchmarking. I believe that existing models, like DAM4SAM (including DAM4SAM-pres) and SAMURAI, could be also adapted for this scenario, but the authors did not provide a precise task definition to assess this. Therefore, I remain unconvinced of the novelty of the proposed work.

  2. On online tracking, it seems that PMR achieves 2 points worse results than DAM4SAM and 5 points lower results than SAMURAI. The results here are unclear. The authors did not evalute on LaSOT (a dataset known for occlusion in tracking), but on MOSEv2.

Furthermore, the authors acknowledge that certain experiments and tables in the original paper were repetitive, raising additional concerns about the presentation. I will lower the score of presentation to 1.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.