- Rating
- 2
- Confidence
- 4
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper proposes AD-Memo, a VLA-based autonomous driving framework that uses language-based memory to retain historical driving information. The model periodically generates textual descriptions of important objects and events, which are reused as inputs at subsequent steps. The authors also introduce Da Capo, a semi-closed-loop RL method that optimizes memory generation using future driving and VQA rewards. Experiments on all-way-stop and general driving datasets evaluate trajectory prediction, scene understanding, and memory transferability.
Critical strengths
The paper proposed to handle the problem in long-horizon driving by utilizing text language for history representation. The Da Capo training strategy looks interesting, since it is trained on the driving reward. The paper is generally easy to follow and understand.
Critical weaknesses
-
From the perspective of the reviewer, language memory works better than latent memory is because that language memory comes from well-pretrained LLMs (e.g., chatgpt), but latent memory comes from small memory module trained by small amount of data (e.g., Orion). Therefore, it is hard to say whether your memory mechanism works or the LLMs are too powerful. So the reviewer think this paper is similar to GPTDriver, which describe the scene in language. The reviewer does not think using language itself for memory representation is a good idea, since it loses a lot of information, such as grounding, spatial, geometric, motion, etc. Suggestions: you can even use a graph for such a memory representation, or you can research how to utilize LLMs great powers for better latent memory representation (like ChatGPT boosted latent representation).
-
A big problem is that your experiment results have very marginal improvements, which is not sufficient to support your claims, e.g., no-memory RL baseline, Avg. ADE only improves from 1.883 to 1.864. In addition, no close-loop results provided.
-
The visualizations are difficult to follow, and the differences between consecutive frames are not clearly illustrated.
-
Does this memory mechanism accumulate errors? Not like latent memory that can directly be trained in a end-to-end manner with the planning task, the language memory might accumulate errors (e.g., the vehicle position, the traffic light color, etc.). Does this happen? How to handle it? More analysis should be provided.
Questions for authors
Please refer to the weakness part for raised questions.
3 comments
Rebuttal to Reviewer t4ke (2/3)
W2 (1). Improvements.
As both the language memory and RL are our contributions, the performance should be studied for (1) models before and after RL for each memory variant (no memory / CoT-as-memory / ours), and also for (2) memory variants that go through the same stage of training (SFT / SFT+RL).
-
The performance gain from RL training is significant for all variants of memory, as acknowledged by the reviewer ZNwV (“which also substantially improves the no memory baseline”). While some metrics did not improve, such as minADE for general driving, we explained in the paper (Sec. 4.2; now line 405-407) that the minimum metrics (minADE, minFDE, and corner distance) do not improve because RL sharpens policy distribution towards prediction in a single direction; thus, RL models suffer more when the driver’s intention cannot be determined at a crossing. This does not necessarily mean worse driving quality. On scenarios with clear intention such as all-way stop, our experimental result in Sec. 4.1 shows that such a phenomenon does not happen. We also do achieve a significantly lower most-likely driving metric (ML. ADE/FDE) in the original Tabs. 1 and 2 (now Tabs. 1 and 3).
-
Our memory also achieves significant performance gains within the SFT models on the current Tabs. 1 and 3, which serves as an ablation showing the validity of language memory. On the general driving dataset, the best performance on every metric belongs to either our SFT model or the RL model.
-
Furthermore, Besides control advantage, our proposed method also shows significantly better scene understanding ability as it achieves much higher MCQ accuracy on both all-way stop and general driving datasets. It also shows versatile 0-shot portability to other models in Sec. 4.3 that prior vector-based methods cannot achieve. This is especially important with the recent success of GPT-6 Astra which translates language reasoning to robot policy, but it is largely underexplored in prior works. Thus, we also kindly disagree with the point “not sufficient to support your claims”.
W2 (2). Closed-loop evaluation results.
To address the reviewer’s question, we test 396 unseen all-way stop scenes using Alpasim (https://github.com/NVlabs/alpasim) with our model trained on the all-way stop data, each with 6 rollouts. Controls are applied every 0.1s with memory generated every 0.4s; the length is on average 14 seconds. Memories generated between each keyframe separated by 0.4s are discarded, but the performance is evaluated on each control frame.
We list the closed-loop evaluation results below, where higher success rates are better, distance traveled is an auxiliary sanity check metric, and lower is better for the remaining metrics:
| Models | Stop success rate ↑ | Go success rate ↑ | Roll through rate ↓ | ↓ | Offroad ↓ | Collision at fault ↓ | Wrong lane ↓ | Plan deviation ↓ | Distance traveled |
|---|---|---|---|---|---|---|---|---|---|
| Base Model | 44.4% | 35.1% | 4.7% | 4.78 | 0.04 | 0.03 | 0.51 | 0.24 | 74.48 |
| Alpamayo 2 Super | 57.3% | 38.1% | 3.2% | 3.44 | 0.04 | 0.02 | 0.45 | 0.28 | 71.24 |
| No mem. (SFT only) | 48.8% | 40.5% | 1% | 4.22 | 0.04 | 0.03 | 0.54 | 0.24 | 78.46 |
| CoT mem. (SFT only) | 47.3% | 39.2% | 0.9% | 4.39 | 0.04 | 0.05 | 0.55 | 0.24 | 77.08 |
| AD-Memo (SFT only) | 48.8% | 38.3% | 1% | 4.13 | 0.04 | 0.04 | 0.51 | 0.23 | 74.19 |
| No mem. (SFT+Da Capo) | 57.0% | 42.3% | 1.3% | 3.53 | 0.04 | 0.03 | 0.56 | 0.20 | 82.57 |
| CoT mem. (SFT+Da Capo) | 53.0% | 39.4% | 1.7% | 3.85 | 0.04 | 0.04 | 0.57 | 0.19 | 79.02 |
| AD-Memo (Ours) | 59.2% | 39.5% | 1.2% | 3.31 | 0.04 | 0.04 | 0.58 | 0.18 | 79.35 |
The result shows that AD-Memo achieves higher stop success rate and lower with better or similar safety metrics (offroad, collision at fault, wrong lane, plan deviation). We hypothesize that such safety metrics can be further improved if we integrate these terms into the RL reward (for now we only use distance to ground truth for driving reward).
Rebuttal to Reviewer t4ke (3/3)
W3. Visualizations.
Thanks for pointing this out, and sorry for the confusion. We assume this point is about our video, since we use red boxes to emphasize objects of interest in the paper such as Fig. 1, 12, 13. We modified the video to clarify.
W4. Analysis on the accumulation of errors in language memory.
-
Great question. In fact, preventing propagation of incorrect memory is the reason why we develop our RL algorithm to recover from inaccurate memory generated by the agent itself, because RL optimizes the agent’s driving quality conditioning on the agent’s own, imperfect output instead of ground truth perfect memory. We mentioned this in line 231, and now modified the wording to further emphasize this.
-
We do have an ablation on propagation of incorrect memory, which uses the ground truth memory (SFT+ref. mem.) in Tab. 1 and Tab. 2 (now Tab. 3). The result shows that there is indeed a large gap due to incorrect memory propagation between SFT and “SFT+ref. mem.” models. The result also shows that RL can level or even go beyond this gap by outperforming the “SFT+ref. mem” results.
-
We also updated our paper to include Figs. 20 and 21 as ablations on how the maximum number of memory entries affects performance on the same trained model. The result shows that some metrics degrade due to compound error in memory, while others keep improving (especially MCQ accuracy). This being said, our method consistently outperforms CoT-with-memory, which shows the validity of our memory.
-
To further study how incorrect memory affects driving predictions, we conduct an ablation where we substitute self-generated memory with memory generated from a random wrong episode on the all-way stop dataset. The result is listed below and also added in Appendix D.13. The result shows that the driving quality notably degrades, but does not collapse; meanwhile, MCQ accuracy largely decreased. The result indicates that 1) our method is reliably robust with inaccurate memory, and 2) the scene-understanding gain entirely relies on the memory content received at test time, while the driving performance gain from memory is less reliant on test-time input.
| Method | minADE | Avg. ADE | ML ADE | stop succ. | go succ. | roll-through | MCQ |
|---|---|---|---|---|---|---|---|
| AD-Memo | 0.951 | 1.866 | 1.915 | 89.3% | 45.0% | 5.3% | 51.7% |
| AD-Memo w./ wrong memory input | 1.011 | 1.909 | 1.966 | 89.2% | 43.8% | 5.4% | 34.9% |
| no-memory (SFT + Da Capo) | 0.944 | 1.952 | 2.005 | 88.9% | 43.8% | 5.2% | 35.3% |
Rebuttal to Reviewer t4ke (1/3)
We thank the reviewer for their time and feedback. We appreciate the reviewer highlighting the clear presentation and interesting RL training strategy. Here are our responses:
W1. Benefits of using language memory.
We agree that language memory works well because it comes from well-pretrained LLMs. But “LLMs are too powerful” is not a limitation. In contrast, our very motivation is to best utilize the reasoning capability of an LLM which we feel are not sufficiently unleashed in current VLA agents. We believe leveraging this ability is promising. Indeed, the recent success of GPT-6 Astra proved that LLMs with stronger reasoning ability yield better control policies.
To extend to larger models such as Alpamayo 2 Super and real-life visual input, the memory module is not necessarily generalizable with a small amount of data. Orion works well on the CARLA simulator, but real-life data is out-of-distribution; this is stated in their paper (Appendix C.1), where their method is suboptimal to other VLM-based methods on real-life scenarios (nuScene) due to out-of-distribution trajectories.
As we state in line 77-78, our motivation is to build an explainable and portable memory, which we emphasize in our teaser and show in Sec. 4.3. Compared to language-based memory, vector-based memory is much harder, if at all possible, to comprehend by humans or plug into other models in a zero-shot manner.
We agree that language memory, in its current form, cannot fully recover information such as grounding, spatial, geometric and motion; future augmentation may be needed to fully capture this information. However, there is also no guarantee that vectors can capture all this information. Compared to vectors, language merits include:
Concentrated: By recording in language, the model already conducts reasoning on whether an object is important to current driving, and which kind of spatial relation is the most important to notice, which helps simplify the decision-making process.
Extendable: Language can easily extend to minutes or even hours of information that is indeed important (e.g., traffic regulations such as speed signs) without introducing an additional architecture design; related discussion is listed in Appendix F. Also, data for language memory can be shared across all language-based models; latent vectors, on the contrary, might be attached to specific architectures. Thus, applying language memory leads to minimal training infrastructure and no model architecture change, which is more convenient for iterating production-size models.
While both works use language to describe the scene, there is one key difference between our language in memory and GPT-Driver: GPT-Driver’s input contains coordinate numbers, while we intentionally remove such numbers (“NEVER output a numeric ego speed or units” in Appendix C.3). The reason is that LLMs are pretrained strong enough for reasoning, but not pretrained to deal with these numbers quickly and properly. By only keeping the high-level relation for the agent to consider, we force the agent to focus on reasoning and prevent hallucination. We fully leverage the reasoning ability of LLM, but also avoid leveraging them where there is no such ability.
Thanks for the suggestions. We already use decision graphs during data curation (Appendix C.2.1) and it helps produce better memory. As for utilizing the LLM’s power for better latent memory representation, we agree that it is an interesting direction to explore especially with the current progress on embedding models (https://arxiv.org/abs/2412.12591v2) and research agent (https://arxiv.org/abs/2505.19955). However, we did not include it in our scope as we value the explainability and portability of the memory, which latent memory cannot provide.