acceptodds
Under review as a conference paper at ICLR 2027

: Multi-Modal Memory Evaluation through Cognitively-Grounded Video Tasks

Abstract

As multi-modal models advance towards long-form video understanding, memory emerges as a critical capability. Despite substantial efforts in developing video datasets and benchmarks, existing works primarily focus on perception and reasoning, without systematically evaluating memory: what models retain, how faithfully information is preserved, and how robust memory remains under interference. To address this gap, we introduce , the first comprehensive evaluation framework and benchmark for probing different memory dimensions in multi-modal models. Grounded in cognitive psychology, our design features carefully constructed tasks that probe key aspects of memory. Leveraging , we conduct extensive experiments across representative multi-modal models, revealing consistent weaknesses and distinctive behaviors. We find that models struggle to maintain disentangled representations when processing parallel video streams, exhibit interference patterns differing substantially from those observed in human memory, ground memory sources more reliably in the spatial domain than the temporal domain, and demonstrate limited symbolic memory. Collectively, our benchmark provides a valuable resource for future research, while our findings highlight memory as a fundamental yet underexplored capability and offer insights for designing more effective memory mechanisms in multi-modal models.

Primary area:
datasets and benchmarks
Keywords:
Multi-modal Memory, Multi-modal Models, Video Understanding

Reviews

Among:
Shape:
1234this paper, Top 32%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.50 over 2 reviews, above 68% of the others.
Reviewer_sT1dOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper introduces M³Eval, a benchmark for evaluating multimodal memory through four video-based paradigms inspired by cognitive psychology: divided attention, memory interference, interleaved events, and N-back. The authors evaluate several multimodal models and video agents alongside human participants, examining source identification, temporal organization, content retention, and cross-clip attribute matching. The experiments reveal substantial difficulties under these settings and interesting differences in how humans and models respond to interference and increasing sequence length. The main contribution is the structured evaluation design and its diagnostic findings.

Critical strengths
  • The four cognitively motivated paradigms provide a clear evaluation framework, with useful comparisons between different interference conditions.
  • The analyses reveal interesting patterns, including repetition benefits and different effects of temporal lag and sequence length.
  • The evaluation covers multiple models, agents, and human references, supported by single-video baselines, manual verification, and human agreement checks.
Critical weaknesses
  • The benchmark does not fully disentangle memory failures from perception and evidence retrieval. Although the single-video controls are helpful, reduced target-frame coverage or changes in effective resolution may also contribute to the performance drops.
  • Some mechanistic interpretations are stronger than the evidence supports. The observed human–model differences are interesting, but differing viewing conditions and limited causal controls warrant more cautious claims about the underlying memory mechanisms.
Questions for authors
  • Can the authors better disentangle memory interference from reduced visual evidence? For most models, the multi-video setting still uses a fixed total of 96 sampled frames, which may reduce the number of frames representing the target video compared with the single-video baseline.
  • For the False Memory Discrimination task, the construction makes “Neither video” always the correct semantic answer. Could the authors include balanced positive cases where the queried event appears in Video 1, Video 2, or both, to rule out this shortcut?
3 comments
AuthorsOpenReview ↗

Overall Response to Reviewer sT1d

We sincerely thank the reviewer for the positive assessment and constructive questions. Below we briefly summarize our key responses; details for each point are in the follow-up comment replies.

  • W1 & Q1 (Matched target evidence). Our single-video baselines already match target frames and per-stream resolution, and the main effects hold under this matched setting.
  • W2 (Mechanistic interpretations). Our human–model comparison follows standard practice, and the human interference asymmetry persists under input matched to the models. Our mechanism-level statements are empirical interpretations meant to inform future model design. Establishing causal mechanisms lies beyond the scope of this benchmark.
  • Q2 (Shortcut in False Memory Discrimination). The Neither video shortcut does not affect our reported results. On the balanced set requested by the reviewer, models still attribute most absent events to a video.

We have incorporated all these clarifications and supplementary results into the revised paper, highlighted in blue. All section, figure, and appendix numbers below refer to the revised paper.

AuthorsOpenReview ↗

Response to Reviewer sT1d (1/2): Matched Target Evidence and Mechanistic Interpretations

W1 & Q1: Matched target evidence

Our single-video baselines already match target frames and per-stream resolution, and the main effects hold under this matched setting.

Setting.

  • Divided Attention (split-screen). We place the two videos side by side at their original resolution, without resizing. Each stream thus has the same number of visual tokens per frame as the single-video input. The two streams play in sync. The target stream receives 96 frames in both the single-video and split-screen inputs.
  • Memory Interference (concatenation) and Interleaved Events (interleaving). The total number of input frames scales with the number of videos. We use 48 frames for one video, 96 frames for two videos, and 144 frames for the three-video setting in Fig. 8. The target video thus receives 48 frames in both the single-video and multi-video inputs.

Results. These matched comparisons use the same questions as the main evaluation, for seven models (Tables 6–8). Single-video accuracy exceeds multi-video accuracy by 19–29 pp for Divided Attention, 11–27 pp for Memory Interference, and 16–28 pp for Interleaved Events (per-model averages). We have added these input settings to Appendices B and H.


W2: Strength of mechanistic interpretations

Our human–model comparison follows standard practice, and the human interference asymmetry persists under input matched to the models. Our mechanism-level statements are empirical interpretations meant to inform future model design. Establishing causal mechanisms lies beyond the scope of this benchmark.

Fair comparison. Humans, models, and agents access video in different ways, such as which frames they see and whether they can revisit the video. This access is part of each method. It is not an external variable that can be controlled separately. Video-MME, MLVU, and InfiniBench also evaluate each model under its native input setting instead of a uniform frame count (Appendix H). Within this practice, we align the conditions where possible. Humans and models receive the same constructed videos. Humans may pause and rewind. This gives them the same capability as agents such as VideoLucy, which re-access the video during inference (Appendix C.2). Further, our key human–model comparisons are contrasts within each method, such as proactive versus retroactive interference (Table 2) and the effects of N and K on N-Back accuracy (Fig. 11). Each method keeps the same input access across all conditions of such a contrast. Differences in input access across methods thus do not directly enter these comparisons.

Matched human control. We also test 4 new participants under input matched to the models (a supplementary experiment requested by Reviewer vCcL; details are in our response to that reviewer). They see the same 96 uniformly sampled frames as the default model input, once and without pausing or rewinding, with counterbalanced conditions. On the 55 paired Memory Interference questions, human retroactive accuracy is still 16.36 pp lower than proactive accuracy, close to the 20.00 pp under the original protocol. Viewing conditions thus do not explain why humans show this asymmetry while models do not.

Scope of our interpretations. Our contribution is a benchmark that reveals where model memory differs from human memory under controlled paradigms. We present the mechanism-level statements as empirical interpretations of these differences, with wording such as "suggests" and "we hypothesize" (e.g., Sections 4.1 and 4.2). They aim to offer insights for future model design (Section 5). Establishing causal mechanisms would require interventions on model internals. We have stated this scope explicitly in Appendix C.3, together with the matched control.

AuthorsOpenReview ↗

Response to Reviewer sT1d (2/2): False Memory Discrimination

Q2: Answer shortcut in False Memory Discrimination

The Neither video shortcut does not affect our reported False Memory Discrimination (FMD) results. On the balanced set requested by the reviewer, models still attribute most absent events to a video.

Why the reported results are not affected. We query each question in a separate call. The model cannot see other questions, so it cannot learn that the FMD answer is always Neither video. In Table 3, all models and agents score at most 40.00% on FMD, far below the 100% of the shortcut.

Balanced experiment. Although the shortcut does not affect our reported results, we add a balanced experiment at the reviewer's request. For each FMD video, we add three new questions. In these questions, the queried event appears only in Video 1, only in Video 2, or in both videos. The new questions use the same template and the same four options as the original question. We report two error rates:

  • False positive rate (FPR): the fraction of absent-event questions (the original ones) answered with Video 1, Video 2, or Both videos.
  • False negative rate (FNR): the fraction of present-event questions (the new ones) answered with Neither video.

We have completed 16 of the 95 videos (64 questions) so far and will cover all 95 videos in the revision.

MethodAcc ↑FPR ↓FNR ↓
Random25.0075.0025.00
Always Neither video25.000.00100.00
Qwen3.5-4B32.8181.250.00
Qwen3-VL-8B-Instruct40.6387.502.08
InternVL3.5-8B34.38100.000.00
Qwen3.5-9B43.7581.250.00

Results. Always choosing Neither video now reaches only 25.00% accuracy, so the shortcut no longer works. All four models attribute 81.25%–100.00% of absent events to a video (FPR), above the random level of 75.00%. In contrast, they answer Neither video for at most 2.08% of present events (FNR). This indicates that the low FMD accuracy comes from false memory, not from a bias toward Neither video. Even with 64 questions, the gap between FPR and FNR is significant for each model (disjoint 95% Wilson intervals).

We have added this experiment to Appendix J.2 and the one-call protocol to Appendix H.


If our responses have addressed your concerns, we would appreciate it if you could consider raising your score.

Reviewer_jUt6OpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper introduces M³Eval, a benchmark for probing multimodal video memory through four paradigms adapted from cognitive psychology: divided attention, proactive/retroactive interference, temporal organization of interleaved events, and N-back scene/action judgments. The authors report 2,403 questions: 739 non-N-back questions based on 451 source videos and 1,664 N-back probes derived from 64 base sequence instances across 26 valid configurations. Questions are generated with Qwen3.5-27B and manually reviewed. Proprietary, open-weight, and agentic models are evaluated against human reference performance, with targeted measures of source attribution, order, content retention, intrusion, and false-memory discrimination. The results show substantial performance gaps and differing behavioral patterns under the reported evaluation protocols.

Critical strengths
  1. A useful, deliberately structured benchmark formulation. The four paradigms manipulate concurrent presentation, interference direction, interleaving, and symbolic lag/sequence length, with targeted error measures rather than a single aggregate QA score. This creates a promising framework for organizing and studying multimodal failure modes, even if the mechanistic interpretations need qualification.
  2. Substantial effort toward question quality and diagnostic comparisons. The authors report independent review by three co-authors, initial agreement of 91.4% with Fleiss’ κ = 0.84, and 137 discarded plus 125 revised questions. They provide matched-question single-video baselines for seven models, a five-person human baseline with reported participant variability, and a separate four-person agreement study on 200 shared questions. These are meaningful checks, although agreement and single-video performance do not resolve the construct-validity concern below.
Critical weaknesses
  1. The single-video controls do not yet isolate memory-related failure from changes in visual access. Unreported resizing, effective visual resolution, or token allocation may additionally change access to visual details. Tables 6–8 report substantial average performance costs under the changed input conditions, but do not establish Appendix B's stronger exclusion of perceptual explanations. The human baseline also uses complete videos with pause/rewind, whereas most models use sparse samples. The authors justify replay by reference to agentic methods; this is reasonable for a human reference, but does not establish equivalent access for every model.
  2. N-back reuses 64 base sequence instances across 26 configurations. Figure 11's caption describes ±1 standard-deviation bands, while its legend calls them confidence intervals; their calculation and treatment of sequence reuse are unclear. Robust K/N conclusions need sequence-aware analysis explaining the valid configuration grid and stimulus composition.
  3. Attention confusion is appropriately presented as a hypothesis in Section 4.1; stronger causal-attention explanations should retain that status unless tested.
Questions for authors
  1. If the stronger exclusion of perception is retained, can the authors provide target-evidence-matched single/multi-video comparisons on a representative subset? For concatenated/interleaved inputs, keep target timestamps/frames fixed while adding distractors; a same-frame concatenated-versus-interleaved comparison would help isolate ordering.
  2. Can the authors report paired item-level uncertainty for interference and sequence-aware analyses for N-back?
  3. Please also provide the Figure 9 source-memory counts, clarify whether spatial/temporal conditions use paired questions and the same target evidence, resolve the Figure 21 swap/prompt discrepancy, and reconcile Table 5's 647 questions with the reported 739. These details would materially affect interpretation of the key comparisons.
6 comments
AuthorsOpenReview ↗

Overall Response to Reviewer jUt6

We sincerely thank the reviewer for the careful reading and constructive suggestions. Below we briefly summarize our key responses; details for each point are in the follow-up comment replies.

  • W1 & Q1 (Single-video controls). The single-video baselines match target frames, per-stream resolution, and visual tokens per frame. They thus measure the cost of the competing video, with visual access to the target unchanged.
  • W1 (Human baseline access). Our human baseline follows standard practice and serves as a fair reference. Our key human–model comparisons are contrasts within each method. Under input matched to the models, the human interference asymmetry persists.
  • W2 & Q2 (Uncertainty). Our N-Back and Memory Interference conclusions hold under paired, sequence-aware uncertainty estimates.
  • W3 (Attention confusion). As the reviewer notes, our paper already presents attention confusion as a hypothesis.
  • Q3 (Details). We provide the requested details on Fig. 9, Fig. 21, and the question counts.

We have incorporated all these clarifications and supplementary results into the revised paper, highlighted in blue. All section, figure, and appendix numbers below refer to the revised paper.

AuthorsOpenReview ↗

Response to Reviewer jUt6 (1/2): Single-Video Controls and Human Baseline

W1 & Q1: Single-video controls

The single-video baselines match target frames, per-stream resolution, and visual tokens per frame. They thus measure the cost of the competing video, with visual access to the target unchanged.

Setting.

  • Divided Attention (split-screen). We place the two videos side by side at their original resolution, without resizing. Each stream thus has the same number of visual tokens per frame as the single-video input. The two streams play in sync. The target stream receives 96 frames in both the single-video and split-screen inputs.
  • Memory Interference (concatenation) and Interleaved Events (interleaving). The total number of input frames scales with the number of videos. We use 48 frames for one video, 96 frames for two videos, and 144 frames for the three-video setting in Fig. 8. The target video thus receives 48 frames in both the single-video and multi-video inputs.

Results. These matched comparisons use the same questions as the main evaluation, for seven models (Tables 6–8). Single-video accuracy exceeds multi-video accuracy by 19–29 pp for Divided Attention, 11–27 pp for Memory Interference, and 16–28 pp for Interleaved Events (per-model averages).

Scope (Q1). Fully separating memory from every perceptual factor is a broader open problem beyond the scope of this work. We have thus revised Appendix B to describe these baselines as measuring the cost of the competing video, without the stronger exclusion of perceptual explanations. The input settings above are now stated in Appendices B and H.


W1 (cont.): Human baseline access

Our human baseline follows standard practice and serves as a fair reference. Our key human–model comparisons are contrasts within each method. Under input matched to the models, the human interference asymmetry persists.

Why the human baseline is fair. Humans, models, and agents access video in different ways, such as which frames they see and whether they can revisit the video. This access is part of each method. It is not an external variable that can be controlled separately. Video-MME, MLVU, and InfiniBench also evaluate each model under its native input setting instead of a uniform frame count (Appendix H). Within this practice, we align the conditions where possible. Humans and models receive the same constructed videos. Humans may pause and rewind. This gives them the same capability as agents such as VideoLucy, which re-access the video during inference (Appendix C.2).

Contrasts within each method. Examples include proactive versus retroactive interference (Table 2) and the effects of N and K on N-Back accuracy (Fig. 11). Each method keeps the same input access across all conditions of such a contrast. Differences in input access across methods thus do not directly enter these comparisons.

Matched human control. We also test 4 new participants under input matched to the models (a supplementary experiment requested by Reviewer vCcL; details are in our response to that reviewer). They see the same 96 uniformly sampled frames as the default model input, once and without pausing or rewinding, with counterbalanced conditions. On the 55 paired Memory Interference questions, human retroactive accuracy is still 16.36 pp lower than proactive accuracy, close to the 20.00 pp under the original protocol. Human accuracy also still exceeds every model in Table 2 in each condition. This indicates that the human asymmetry does not come from the richer visual input or from replay.

We have added this discussion and the matched control to Appendix C.3.

AuthorsOpenReview ↗

Response to Reviewer jUt6 (2/2): Uncertainty, Attention Confusion, and Requested Details

W2 & Q2: Uncertainty for N-Back and Memory Interference

Our N-Back and Memory Interference conclusions hold under paired, sequence-aware uncertainty estimates.

Method. As requested by the reviewer, we estimate paired, sequence-aware uncertainty. We use a cluster bootstrap (10,000 resamples) over the independent units: the 55 item pairs of Memory Interference and the 64 N-Back base sequences. Each resampled sequence keeps all its configurations, so the estimates account for sequence reuse.

N-Back grid. Each base sequence has 12 clips, three from each of four video groups. Configuration (K, N) presents the first K clips and asks whether clip K matches clip K−N. Here K ∈ {2, 3, 4, 5, 6, 8, 10, 12} and 1 ≤ N ≤ min(4, K−1). This gives 26 configurations, each with balanced matches and mismatches. Since the grid is not rectangular, we estimate the K and N effects jointly.

Results.

  • N-Back. Model accuracy decreases with K (−2.2 pp per clip, 95% CI [−2.8, −1.6]) and does not decrease with N. In contrast, human accuracy decreases with N (−2.6 pp per step, 95% CI [−3.9, −1.3]). These results also hold on the fully crossed sub-grid with K ≥ 5.
  • Memory Interference. Humans show a significant asymmetry. Their proactive accuracy exceeds their retroactive accuracy by 20.00 pp (95% CI [7.3, 32.7]), and the intrusion rate shows the same pattern. For the nine-model average, this difference is not significant (95% CI [−7.5, 6.1]), and its CI does not overlap with the human CI. No model shows a significant asymmetry in the human direction.

Fig. 11. The shaded regions show ±1 SD of the fit residuals, as stated in the caption. We have corrected the legend label, which read "Confidence Interval". The CIs above account for sequence reuse.

We have added these analyses to Appendix J.1 and corrected the grid description in Appendix A.2.


W3: Attention confusion

As the reviewer notes, our paper already presents attention confusion as a hypothesis. Every mention keeps this status. The introduction and Section 4.1 state "we hypothesize", and Finding 1 states "probably due to attention confusion".


Q3: Details on Fig. 9, Fig. 21, and question counts

We provide the requested details below.

  • Fig. 9 counts and the 647 vs. 739 gap. Fig. 9 uses the two-option source-memory questions (Appendix G.1.3, Fig. 21). There are 92 such questions, 46 spatial and 46 temporal. They differ from the four-option Source Identification questions in Tables 1 and 3. Table 5 omits them, and 647 + 92 = 739. Finding 3 rests on the consistent direction of the gap. Spatial accuracy exceeds temporal accuracy for humans and for 8 of the 9 models (sign test across the 9 models, two-sided p = 0.039).
  • Pairing and target evidence. Yes. Each spatial question is paired with a temporal question on the same video pair and the same target evidence.
  • Fig. 21. The spatial condition uses frequent left/right swaps, as stated in the caption. The note under Question I was a figure error: it showed the note of the no-swap condition. We have corrected it.

We have added these details to Appendix J.4 and to the note of Table 5.


If our responses have addressed your concerns, we would appreciate it if you could consider raising your score.

Comments on the paper

13 comments
AuthorsOpenReview ↗

Overall Response to Reviewer vCcL

We sincerely thank the reviewer for the detailed review and constructive questions. Below we briefly summarize our key responses; details for each point are in the follow-up comment replies.

  • W1 & Q1 (Matched target evidence). Our single-video baselines already match target frames and per-stream resolution, and the main effects hold under this matched setting.
  • W2 & Q2 (Shortcut in False Memory Discrimination). The Neither video shortcut does not affect our reported results. On the balanced set requested by the reviewer, models still attribute most absent events to a video.
  • W3 & Q3 (Human input). Our human–model comparison follows standard practice. Under the matched setting requested by the reviewer, humans still show stronger retroactive interference than proactive interference.
  • W4 & Q4 (N-Back controls). Removing the attribute names does not change our N-Back conclusions. With text labels, all tested models score 100%, so the difficulty of N-Back lies in abstracting symbols from video. Processing without revisiting would evaluate a model plus a memory pipeline, not the model itself.
  • W5 & Q5 (Uncertainty and correlation). Our main conclusions hold under paired, sequence-aware uncertainty estimates. M3Eval correlates significantly with both downstream benchmarks, and its model ranking remains significantly consistent with them after removing Gemini.

We have incorporated all these clarifications and supplementary results into the revised paper, highlighted in blue. All section, figure, and appendix numbers below refer to the revised paper.

AuthorsOpenReview ↗

Response to Reviewer vCcL (1/4): Matched Target Evidence and False Memory Discrimination

W1 & Q1: Matched target evidence

Our single-video baselines already match target frames and per-stream resolution, and the main effects hold under this matched setting.

Setting.

  • Divided Attention (split-screen). We place the two videos side by side at their original resolution, without resizing. Each stream thus has the same number of visual tokens per frame as the single-video input. The two streams play in sync. The target stream receives 96 frames in both the single-video and split-screen inputs.
  • Memory Interference (concatenation) and Interleaved Events (interleaving). The total number of input frames scales with the number of videos. We use 48 frames for one video, 96 frames for two videos, and 144 frames for the three-video setting in Fig. 8. The target video thus receives 48 frames in both the single-video and multi-video inputs.

Results. These matched comparisons use the same questions as the main evaluation, for seven models (Tables 6–8). Single-video accuracy exceeds multi-video accuracy by 19–29 pp for Divided Attention, 11–27 pp for Memory Interference, and 16–28 pp for Interleaved Events (per-model averages). We have added these input settings to Appendices B and H.


W2 & Q2: Answer shortcut in False Memory Discrimination

The Neither video shortcut does not affect our reported False Memory Discrimination (FMD) results. On the balanced set requested by the reviewer, models still attribute most absent events to a video.

Why the reported results are not affected. We query each question in a separate call. The model cannot see other questions, so it cannot learn that the FMD answer is always Neither video. In Table 3, all models and agents score at most 40.00% on FMD, far below the 100% of the shortcut.

Balanced experiment. Although the shortcut does not affect our reported results, we add a balanced experiment at the reviewer's request. For each FMD video, we add three new questions. In these questions, the queried event appears only in Video 1, only in Video 2, or in both videos. The new questions use the same template and the same four options as the original question. We report two error rates:

  • False positive rate (FPR): the fraction of absent-event questions (the original ones) answered with Video 1, Video 2, or Both videos.
  • False negative rate (FNR): the fraction of present-event questions (the new ones) answered with Neither video.

We have completed 16 of the 95 videos (64 questions) so far and will cover all 95 videos in the revision.

MethodAcc ↑FPR ↓FNR ↓
Random25.0075.0025.00
Always Neither video25.000.00100.00
Qwen3.5-4B32.8181.250.00
Qwen3-VL-8B-Instruct40.6387.502.08
InternVL3.5-8B34.38100.000.00
Qwen3.5-9B43.7581.250.00

Results. Always choosing Neither video now reaches only 25.00% accuracy, so the shortcut no longer works. All four models attribute 81.25%–100.00% of absent events to a video (FPR), above the random level of 75.00%. In contrast, they answer Neither video for at most 2.08% of present events (FNR). This indicates that the low FMD accuracy comes from false memory, not from a bias toward Neither video. Even with 64 questions, the gap between FPR and FNR is significant for each model (disjoint 95% Wilson intervals).

We have added this experiment to Appendix J.2 and the one-call protocol to Appendix H.

AuthorsOpenReview ↗

Response to Reviewer vCcL (2/4): Input Differences between Humans and Models

W3 & Q3: Input differences between humans and models

Our human–model comparison follows standard practice. Under the matched setting requested by the reviewer, humans still show stronger retroactive interference than proactive interference.

Why the original comparison is fair. Humans, models, and agents access video in different ways, such as which frames they see and whether they can revisit the video. This access is part of each method. It is not an external variable that can be controlled separately. Video-MME, MLVU, and InfiniBench also evaluate each model under its native input setting instead of a uniform frame count (Appendix H). Within this practice, we align the conditions where possible. Humans and models receive the same constructed videos. Humans may pause and rewind. This gives them the same capability as agents such as VideoLucy, which re-access the video during inference (Appendix C.2). Further, our key human–model comparisons are contrasts within each method, such as proactive versus retroactive interference. Each method keeps the same input access across both conditions of such a contrast.

Matched setting. Although the original comparison is fair, we run a matched control at the reviewer's request. We recruit 4 new participants who did not take part in the original study. They answer the 55 paired questions of Memory Interference under three controls:

  1. Matched input. Participants see the same 96 uniformly sampled frames as the default model input.
  2. Single viewing. The frames play once, with no pausing or rewinding.
  3. Counterbalancing. We assign conditions with a Latin square. Each participant sees each video pair in only one order, with roughly equal numbers of proactive and retroactive trials in random order.
Human settingAcc Pro ↑Acc Retro ↑ΔIR Pro ↓IR Retro ↓Δ
Original protocol (full video, pause/rewind; Table 2)94.5574.5520.003.6420.00−16.36
Matched setting (controls 1–3)80.0063.6416.3616.3630.91−14.55

Results. Δ is proactive minus retroactive, as in Table 2. Under the matched setting, human accuracy drops in both conditions but still exceeds every model in Table 2 in each condition. The asymmetry persists. Accuracy is 16.36 pp lower in the retroactive condition than in the proactive condition, and the intrusion rate is 14.55 pp higher. Both gaps remain close to those under the original protocol (20.00 and 16.36 pp). This indicates that the human asymmetry does not come from the richer visual input or from replay.

We have added this control and the discussion above to Appendix C.3.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.