acceptodds
Under review as a conference paper at ICLR 2027

STaR-VL: Capability-Aligned Reinforcement Learning for Visual-Temporal Memory

Abstract

Vision-language models can recognize objects in individual images yet struggle to retain and compare observations across a changing scene. We present STaR-VL, a staged post-training framework for visual-temporal memory on a frozen Qwen2.5-VL backbone. Three LoRA adapters learn per-frame perception, symbolic temporal memory through a recurrent memory module, and transfer to visual histories, respectively; each stage combines supervised fine-tuning with a capability-aligned reinforcement-learning reward. Our contributions are a capability-aligned training framework and a controlled evaluation separating its gains from in-domain joint training, persistent-memory augmentation, and inference-time prompting. On SpaMEM, STaR-VL-3B reaches a V-score of 47.60, compared with 21.85 for the untrained backbone, 40.80 for a joint rank-48 LoRA, and 38.90 for an adapted ReMEmbR system on the same backbone. A structured memory-table prompt raises the untrained model to 22.21. The same trained checkpoints transfer to held-out benchmarks: the 7B model improves VSI-Bench from 33.3 to 46.4, with smaller gains on EgoSchema and TempCompass. Visual-history and symbolic-history controls reveal complementary effects of frame coverage and accessible scene state. These results support staged capability-aligned training for object recall, event detection, and cumulative memory over explicitly ordered observations, while identifying fine-grained grounding and temporal localization as remaining challenges.

Primary area:
foundation or frontier models, including LLMs
Keywords:
vision-language models, visual-temporal memory, reinforcement learning, curriculum learning, parameter-efficient fine-tuning

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_dSFCOpenReview ↗
Rating
3
Confidence
3(+1)
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper presents STaR-VL, a staged post-training framework for visual-temporal memory. It trains three LoRA adapters on a frozen Qwen2.5-VL backbone. The stages cover perception, symbolic temporal memory, and visual adaptation. A recurrent module compresses observation histories into memory tokens. Each stage combines supervised fine-tuning and reinforcement learning. Experiments cover SpaMEM and four external benchmarks. The main improvements appear in object recall, event detection, and cumulative memory.

Critical strengths

1.It compares per-frame perception, visual-history reasoning, and symbolic-history reasoning. These experiments reveal a gap between object recognition and the effective use of past observations. Additional controls identify history coverage and accessible scene state as important factors in visual-temporal performance.

2.Experiments on the same backbone cover joint training, persistent-memory augmentation, and structured prompting. The results demonstrate the framework’s advantages over these alternative approaches to improving visual-temporal memory.

Critical weaknesses

1.The paper reports a 6.8-point gain over joint LoRA in Table 3. The authors do not establish matched memory architectures, optimization objectives, or total training budgets for this comparison.

2.The paper does not verify temporal answer correctness through its stated QA-dict reward. The paper defines the dictionary reward as key-set F1 in Section 3. The authors already supply all eight required keys in the QA-dict prompt. The paper therefore permits full task reward for answers with correct keys and incorrect counts, trajectories, or timestamps under the stated definition.

Questions for authors

1.The authors should compare staged and joint training with the same recurrent memory module, comparable adapter capacity, matched objectives, and matched total training budgets. The authors should train each configuration independently.

2.The authors should specify the implemented QA-dict reward. The authors should report the reward assigned to an answer with correct keys and incorrect field values, and clarify any discrepancy with the stated definition. And authors should compare key-set scoring with scoring that also checks field values.

3.The authors should keep the training settings fixed and report the corresponding temporal localization, tracking, and cumulative memory scores.

Reviewer_2wXkOpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
1
Soundness
1
Read review
Paper summary

STaR-VL, is a post-training method that learns per-frame perception, symbolic temporal memory and transfer to visual histories. Each of this capabilities is learned in an independent module (LoRA adapter) and can combine supervised fine-tuning and RL techniques. STaR-VL trains on a subset of the SpaMEM benchmark, and results in improved performance in a validation subset of SpaMEM.

Critical strengths

None

Critical weaknesses

The entire evaluation setup of this paper is flawed, this includes the training data, validation split, and metrics. None of the results in this paper follows a sound evaluation protocol aligned with the original benchmark.

Training Data: This paper trains in a subset of the SpaMEM benchmark. According to the SpaMEM paper, the benchmark is an evaluation only set and it provides no dedicated training set. The mentioned “house-disjoint split” (line 312) does not exist on the original SpaMEM paper or on its official release (https://huggingface.co/datasets/mill-ct-liao/SpaMEM). Validation Data: The “held-out evaluation” is another arbitrary subset of SpaMEM made on this paper. Moreover this held-out evaluation includes less than 5% of the houses (5 out of 104) and about 20% of the total frames (10.304 out of 53.664). No information is provided on the sampling of this subset, or is potential alignment with the full dataset. Missing Benchmark Data: As a consequence of the construction of two artificial training and validation sets, 99 houses and about 20.768 frames (roughly 40% of the released data) are missing in the experiments of this paper.

As outlined in line 312, the authors train STaR-VL in the domain of Procedurally Embodied Environments and validate in the very same domain. This creates an unfair knowledge gap between the STaR-VL and any other VLM tested in this paper. None of the main results of this paper (tables 1 and 2) represent a fair direct comparison between the baselines and STaR-VL. Simply put, STaR-VL works as an in-distribution model, and everything else operates out-of-distribution. The improved results do not represent any technical contribution, rather, the domain knowledge obtained during the training of STaR-VL in SpaMEM.

Due to these reasons, I can not find a single result in this paper that can be considered a direct and fair comparison to the baseline, none of the paper results can support the claimed contributions in the submission.

Questions for authors

Train and validation data: why would the authors create an arbitrary training split in an evaluation only dataset? Inconsistencies with released data: Could the authors outline which houses were used to construct the train and validation set. After a quick check on the released data I can not find any house with over 800 images, how could the authors obtain 22.592 images from only 10 houses? Likewise, what houses were selected for the validation split, again houses with over 2.000 images would be required, and I don't seem to find them. Additional Inconsistencies with released data: Given the release date of SpaMEM data (14 october 2026) I do not understand how the authors could obtain access to the benchmark before the submission deadline. Was there a private data release for the authors before the conference submission? If that was the case, how does the private release compare with the current open release at (https://huggingface.co/datasets/mill-ct-liao/SpaMEM)?

1 comment
Reviewer_2wXkOpenReview ↗

Second round reply

I thank the authors for the reply, as it has clarified some of my concerns. I address the items clarified and the items open After the revision, the results in the paper appear far more useful for understanding and validating the claims of StaR-VL. I am willing to raise my score if the authors provide sufficient empirical evidence regarding the full dataset distribution, model sizes and relevant results for the M-score

I must remind the authors to update the PDF. The discussion forum is informative, but what gets incorporated into the manuscript matters.

Comparison with the full dataset Since the authors train with a small portion of the dataset the alignment of this subset to the full data must be understood. I will request the author to provide the following statistics for the whole set so we can compare the distribution of the annotations used to calculate M-score and V-score

Action distribution ( percentage of spawn, place, remove, etc.) object category distribution (Apple, Pen, Egg, ...) distribution of source and target receptacles. I would like to see the distribution of train, validation and the excluded set directly compared In addition I want to understand if the trained network generalizes to the excluded data the same way it generalizes to the validation set. I want the authors to sample at least two more validation sets of the exact same size in houses as the current one. And perform Inference of at least 2 models. I fully understand the results are not going to be identical but the variability must be bounded. For this particular comparison I would like to see all the metrics (columns) presented in table 1

Dataset specification and reproducibility I don't see a clear specification of the dataset used in the PDF. It is clear to me that the images in the selected subset are identical, but the authors have already acknowledged that there are differences between the datasets. Clearly stating the version of the dataset is critical for the reproducibility of future work. The script to count the images was not provided. I had to write it myself, although I obtained the same result. I also had to write my own script to check the SHA hashes of the image files. I confirm they match.

V-score and model size The V-score results look reasonable for small models, but what about larger models? The original manuscript includes results in models up to 72B parameters. While the comparisons at 2B–4B look sound, providing only three results between 8B and 10B limits the evidence for the generality of the proposed method. Figure 2 should be updated to include the methods with in-domain pretraining.

M-score results Do the authors have results for the Symbolic-history regime (M-score)? To my understanding, the new results presented in the rebuttal cover only the V-score.

Reviewer_1WncOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

This paper proposes STaR-VL, what tunes VLM in 3 strages: symbolic-memory, and visual-adaptation. Three LoRA adapters and a recurrent memory module are trained with SFT and capability-specific rewards. The reported 3B V-score is 47.60, exceeding joint rank-48 LoRA and an adapted persistent-memory baseline, with additional transfer gains on VSI-Bench.

Critical strengths
  1. The proposed three stage tuning pipeline is a new and reasonable design. Comprehensive ablation studies verifies the effectiveness of this design.
  2. The experiments show the methods can outperform multiple types of baseline in both in-domain and out of domain scenarios.
Critical weaknesses
  1. Memory-specific training baselines are limited. Joint LoRA and ReMEmbR are useful controls, but do not establish competitiveness with specialized learned-memory or memory-oriented RL methods. Representative comparisons, such as MA-LMM or the cited LMEE/MemoryExplorer, need comparable training data and history access, with input/task differences explained rather than assuming a universal SOTA ranking.
  2. The recurrent memory design needs a clearer explanation in the main text. Although Figure 1 outlines attention pooling, GRU updates, and memory-token projection, detailed implementations are not clearly explained in main text. Please clarify the recurrent inputs and update procedure.
Questions for authors

Please refer to my two concerns in the weakness section.

Comments on the paper

2 comments
Reviewer_2wXkOpenReview ↗

The response regarding in-domain training is unsatisfactory

The authors confirm that the main comparisons (Tables 1 and 2) assess the capabilities of a smaller model trained in-domain against those of a larger model with no in-domain training. The mismatch in both model size and training data makes it impossible to draw a clear conclusion about the performance of StaR-VL relative to other models.

Tables 1 and 2 are not informative, as they contain two mismatches: StaR-VL models differ from the baselines in both size and in-domain pretraining.

To achieve a meaningful comparison, the authors should first provide a table comparing models with exactly the same parameter count—for example, only 3B models with no training on SpaMEM. This would allow us to quantify any initial advantages of each model. Following this comparison, all models should be fine-tuned on exactly the same data and compared again. This second comparison would allow us to analyze which method performs better with in-domain training.

The second comparison is the most fundamental one. If the proposed capabilities and LoRA modules are indeed effective, StaR-VL should outperform every other model. In fact, ablating the LoRA modules and capabilities should reduce StaR-VL's performance toward that of the most similar baseline.

Once these two baseline scenarios have been established and the superiority of StaR-VL has been demonstrated, comparisons across different model sizes and levels of supervision become relevant.

Any other empirical evaluation procedure will fail to identify the exact source of the improvements and will provide no informative results. Without this empirical evaluation, my rating remains unchanged.

Thank you for the response and the new experiments. My main concerns are addressed.

For the revised paper, please make two small updates clear:

  1. Please fix the QA-dict reward formula and text in Section 3.1 and Appendix C.1.

  2. Please add the new controlled experiments to the paper, and clearly note the setup differences from the original system (a single persistent rank-16 LoRA and 4 RMM memory tokens, instead of K=64 in the original paper).

I will keep my rating as Weak acceptance.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.