acceptodds
Withdrawn from ICLR 2027

CorsiBot: A Cognitive Robotic Model of Serial Spatial Recall

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Humans can reproduce a recently observed sequence of locations by selecting its items in order. The Corsi block-tapping task is widely used to test this ability and measures visuospatial working memory span.Most studies of this task have explored how span depends on sequence length and spatial layout, and have treated recall as a readout of the stored sequence. However, recall in this task is carried out through action. Whether and how the visual feedback from actions takes part in step-by-step retrieval remains unclear. In this research, we introduce CorsiBot, an embodied cognitive robot model that observes pointing sequences and recalls them through successive selections, allowing the visual feedback after each selection to be systematically varied. CorsiBot reproduces key features of human recall, including a decline in accuracy with increasing sequence length and characteristic error patterns. The results show that the model's internal state carries a stable signal of the next location to be recalled, and that altering only the timing of the visual feedback of a selection, with the network and visual content unchanged, shifts both this signal and the next choice, while the model still retains which location it has just selected. We found that the visual feedback of action do not merely record what has been done but shape which location is retrieved next. These findings suggest that step-by-step retrieval from visuospatial working memory may be coupled to the sensory feedback of human actions. In addition, CorsiBot provides an embodied robotic platform for testing how action shapes sequential memory retrieval.

Primary area:
applications to neuroscience & cognitive science
Keywords:
spatial sequence memory, working memory, cognitive robotics, behavioral modeling

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_rdMPOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

CorsiBot is a CNN-LSTM model of spatial serial recall trained on simulated robot-pointing videos. It compares zero, symbolic and visual feedback, then analyzes recurrent states and recall errors. In the visual condition, rearranging feedback frames changes the next choice even when the initial state, frame counts, transition counts and final inputs are matched. Frozen probes show an accompanying change in next-target decodability.

Critical strengths

S1. The endpoint intervention is carefully controlled. The two branches share the complete initial state, selected item, clip instance, twelve updates and final two inputs (Section 6; Appendix D). Next-choice accuracy changes from 52.22% to 69.95%, consistently across three seeds. The last frame or elapsed time alone cannot explain this difference.

S2. The analysis respects its limits. Natural-state probes are frozen across interventions and distinguished from the response head. The authors do not claim causal mediation, and report repeated responses, seed variation and conditions where visual feedback performs worse.

Critical weaknesses

W1. Feedback sensitivity is not yet a distinctive account of recall. Both endpoint schedules depart from native video and reduce accuracy; the forward-motion pair changes endpoint exposure and reverses the mean direction (Table D1). The paper establishes dependence on input history, but connects it to no specific recall-error prediction that separates a retrieval-context mechanism from generic disruption of recurrent dynamics.

W2. The human comparison does not establish a particular advantage for visual feedback. Shape normalization removes absolute discrepancies, and SF-null shows similar profile features (Appendix E). SF-vision repeats on 71.33% of nine-item lists. The cited human study reports about 1% repeated responses (Hurlstone and Hitch, 2015, p. 317). These denominators differ; a matched comparison is needed to assess this important aspect of recall.

Assessment. I recommend weak rejection (2/4). The controlled model-level effect is convincing, but its added explanatory value for human serial recall remains limited.

Questions for authors

Q1 (W1). In the existing endpoint branches, which errors increase: re-selection of the previous item, a consistent list-position shift, or diffuse errors? A breakdown of the saved choices would test whether feedback changes retrieval context in a specific way.

Q2 (W2). What are repetition rate per response and repetition lag for SF-vision and SF-null? Comparing these existing outputs with the human reference would clarify whether visual feedback improves cognitive correspondence beyond normalized curve shape.

Reviewer_2iVDOpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
3
Soundness
3
Read review
Paper summary

Authors examined how providing visual feedback to an LSTM as it performs a visual sequential recall and copying task affects task execution. They make the following claims:

  1. Their CNN-LSTM model recapitulates some facets human behaviour on this task (e.g., some error statistic distributions)
  2. The next location in the sequence can be linearly decoded from the model’s internal state.
  3. Adjustment of the visual feedback sequence affects model behaviour.
Critical strengths

None

Critical weaknesses
  1. It is clear that SF-vision is better than S0-null, but SF-null is better than SF-vision; this suggests that the LSTM primarily benefits from receiving a reminder of where it is in the sequence. In principle, SF-vision could be using the visual content itself somehow but since no real attempt is made to see how the visual information actually propagates through the circuit. this is impossible to attest.
  2. In particular, it is surprising that almost no attempt is made to ablate the actual visual content of the feedback. While S0-null and SF-null clearly are intended as ablations of feedback modality, I would have expected the authors to attempt to manipulate visual content to assess whether and how the LSTM is using the visual feedback. For example, i) irrelevant feedback such as pointing to the wrong block, ii) either only showing the start or the end block, iii) temporal shuffling not holding the endpoints fixed as in Fig. 4.
  3. The timing manipulation is not particularly convincing — okay, so an LSTM is sensitive to the input order. For this analysis to be meaningful, I would expect the authors to characterize more fully and carefully the mechanism through which this sensitivity parses out. Does the manipulation create a bias for a particular item? Does it confuse the network’s understanding of its index in the list? Is it basically just injecting noise into the network dynamics?
  4. The decoding analyses are not especially ambitious. Authors show that training a next-target classifier from the network activity just before choice yields an accuracy curve that increases from the first feedback from to the last — but this doesn’t necessarily show that the network decision develops over this interval (as authors, I think, acknowledge). In principle, one should also train the next-target classifier from the very first feedback frame and show that the accuracy of it is low. Furthermore, it was confusing to me that the authors attempted to decode the ground truth label instead of the network’s choice? If you want to assess whether and how the internal activity predicts network behaviour, you should try and decode its actual output, not the label. Indeed, the fact that the probe accuracies and the model accuracies at various experiments are nearly identical suggests that the probe is simply re-discovering the output head (both are linear readouts from the same LSTM state, aren’t they?). Generally speaking, I would have expected these analyses to really carefully interrogate the LSTM recurrent state for various decision variables but this was not really done.
Questions for authors

None

Reviewer_1FusOpenReview ↗
Rating
2
Confidence
1
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes CorsiBot, a cognitive robotic model for investigating visuospatial working memory and sequential recall. Using an LSTM-based architecture, the model observes spatial pointing sequences and reproduces them through successive location selections. The authors compare different feedback conditions and demonstrate that visual feedback timing influences both internal memory representations and subsequent recall decisions. The model also reproduces certain human-like serial recall patterns, suggesting a potential interaction between sensory feedback and sequential memory retrieval.

Critical strengths
  1. The paper investigates how action-related visual feedback influences spatial memory retrieval, offering an interesting perspective on the interaction between perception, memory, and action.

  2. The experiments manipulate visual feedback timing while keeping network weights, initial recurrent states, and final visual inputs fixed. This provides meaningful evidence that feedback history influences subsequent decisions in the trained model.

Critical weaknesses
  1. Although the experiments demonstrate that visual feedback timing affects subsequent recall, they do not establish that this behavior reflects a human-like memory retrieval mechanism rather than the temporal sensitivity of the LSTM. The comparison with human behavior is primarily based on qualitative error patterns, without direct human experiments under matched feedback manipulations. Stronger mechanistic analysis or human behavioral validation is needed to support the cognitive interpretation.

  2. Despite being presented as an embodied robotic model, CorsiBot produces symbolic location selections rather than executing actual robotic actions during recall. Moreover, experiments are restricted to a fixed nine-location board and a single simulated environment. It remains unclear whether the observed feedback effects generalize to different spatial layouts, action trajectories, or robotic configurations. These limitations weaken the broader claims about embodied cognition.

Questions for authors

see the weakness

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.