- Rating
- 1
- Confidence
- 4
- Informativeness
- 1
- Presentation
- 2
- Soundness
- 1
Read review
Paper summary
The paper proposes a memory-augmented framework for laboratory video understanding (RA-Lab). It divides videos into segments and uses an MLLM to combine segments corresponding to the same step into experiences. Each experience is then compressed through 'visual abstraction', where an MLLM decides what information to preserve and how to organize it. Based on this, an image-generation model produces an abstract visual memory. Each experience has associated images and structured text descriptions that are stored in memory and retrieved to support reasoning required to answer queries. To evaluate this, the paper proposes a benchmark (LabVQA). The proposed method is compared against prior streaming models, open-source and closed-source MLLMs, and other memory-augmented multimodal agents. RA-Lab shows improved performance on most categories. The paper also ablates the proposed memory retrieval against text-only memory, raw-frame memory, and no-memory settings.
Critical strengths
I find the laboratory setting interesting, especially since losing visual details through text-based memory compression is a well-known problem. The idea of preserving these details through visual abstraction is a useful direction. The ablations in Table 1 and Figure 4 are informative and help show how the memory representation and abstraction choices affect performance.
Critical weaknesses
-
A proper benchmark statistic is missing in the main paper, and the details given in the main paper are unclear and inconsistent with the appendix. Since the main QA evaluation uses only one benchmark (LabVQA), which is proposed in the paper itself, the main paper should provide a clear benchmark statistics and report the number of videos, QA pairs, and distribution of video durations. The main paper describes 80 videos from three sources: FineBio, BioVL2, and EgoExoLearn. However, Table 7 in the appendix reports approximately 50 videos from only FineBio and BioVL2, without explaining what happened to EgoExoLearn. It also mentions 1050 queries, but is this the set used throughout the main QA experiments? (Not mentioned in the main paper, and unclear). What is the average duration of the videos for the 1050 queries? What is the distribution of video duration in the final benchmark? This is important to understand how much long-term memory retrieval is actually needed.
-
The method section explains some central components only at a very high level, making it difficult to understand how to reproduce them. For example, Section 3.2 introduces A_plan, but does not explain how the model is prompted to generate the abstraction plan. Section 4 mentions that GPT-5.5 is used, but how is it instructed to decide what to preserve and how to organize it? What does the resulting plan look like, and how is it converted into image-generation instructions? This is one of the main components of the method, but the main paper does not provide a concrete prompt or clearly point to one.
-
It is unclear how the image-generation step preserves fine visual details without modifying or fabricating information. In laboratory understanding, details such as instrument readings, object counts, and text on tubes can directly affect the answer. The method generates or edits images, so how is it ensured that these details remain faithful to the original observations? The paper acknowledges that abstraction can lose information, but changing or inventing information is another concern. This is particularly relevant in a scientific setting, where small changes in details can alter the interpretation of an experiment. There is no clear verification step or evaluation showing whether such details are preserved.
-
The evaluation does not sufficiently establish whether the proposed memory pipeline offers an advantage over stronger long-video models. The paper evaluates Qwen2.5-VL-7B and Gemini 1.5 Pro, but does not explain why newer alternatives, such as Qwen3-VL and newer Gemini Pro or Flash models, are not considered. This matters because the proposed method itself uses frontier models for understanding and image generation. A central question here is what information we lose when compressing the observations, and whether a strong model given the video directly can retain and retrieve more of those details. Comparisons with stronger models, given comparable access to historical evidence, would help establish when the proposed abstraction is useful.
Questions for authors
Please address the questions raised in the weaknesses above, particularly the benchmark inconsistencies, the implementation of abstraction planning, the faithfulness of generated visual abstractions, and why comparisons with stronger long-video models are not included.