acceptodds
Under review as a conference paper at ICLR 2027

Breaking the Historical Action Shortcut: Mitigating Visual Modality Collapse in Gaming VLA Models via Historical Action Noise Injection

Abstract

Interleaved multimodal inputs—streams of multi-frame images alternating with historical action tokens—have become a primary data format for Vision-Language-Action (VLA) models used in Embodied AI and gaming agents. Under Supervised Fine-Tuning (SFT), we identify a failure mode of this paradigm: because expert gameplay produces action sequences with strong temporal correlation and low conditional entropy, the autoregressive transformer tends to route gradient updates through the low-entropy historical pathway, which can starve the high-dimensional visual encoder—a phenomenon we term visual modality collapse. Unlike the classical copycat/inertia shortcut in unimodal behavioral cloning, this failure is (i) conditional—it persists even after perfect marginal class balancing because within any trajectory, and (ii) cross-modal—both modalities are read by the same causal transformer, so the shortcut manifests as gradient starvation of the visual pathway rather than a data-imbalance artifact. We provide a first-principles account of this collapse under standard simplifying assumptions from information-bottleneck theory and gradient dynamics, and introduce two directly measurable diagnostics—Modality Information Gain (IG) and KL Divergence Shift—that isolate which modality actually drives predictions. Building on this analysis, we propose Historical Action Noise Injection (HANI): with probability , historical action tokens are replaced by uniformly sampled tokens from the discrete action vocabulary, keeping the ground-truth target intact. This discrete, semantically mismatched signal is harder to recover from than continuous noise, which drives up the historical pathway's prediction error and routes larger corrective gradients back onto the visual encoder. On the extended MiniWorld WallGap benchmark with Qwen3-VL-8B-Instruct as backbone, HANI outperforms Standard SFT, NEFTune, and Modality Dropout on task success and modality diagnostics (: ; : ). Additional experiments on three visually richer 3D environments—ViZDoom (Deadly Corridor), Minecraft (MineRL Navigate), and RoboTHOR (ObjectNav)—suggest that the shortcut is not tied to a low-texture testbed, and that HANI's relative improvement is largest in the environment (Minecraft) whose expert data most closely matches the low-entropy regime our analysis targets. HANI is a single-stage, one-line data-loader intervention with zero inference overhead.

Primary area:
applications to robotics, autonomy, planning
Keywords:
Vision-Language-Action models, shortcut learning, modality collapse, imitation learning, gaming agents, multimodal learning

Reviews

Among:
Shape:
1234this paper, Top 32%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.50 over 2 reviews, above 68% of the others.
Reviewer_e9ywOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper studies a failure mode in interleaved vision-language-action (VLA) training for gaming and embodied tasks, where highly correlated historical actions can dominate prediction and reduce reliance on visual input. The authors characterize this issue as visual modality collapse under supervised fine-tuning, and introduce two diagnostics—Modality Information Gain and KL Divergence Shift—to probe modality dependence. To mitigate the problem, they propose Historical Action Noise Injection (HANI), which corrupts historical action tokens during training to weaken the shortcut and encourage stronger visual grounding. Experiments on MiniWorld and three additional 3D environments show consistent improvements over standard SFT and several regularization baselines.

Critical strengths
  1. The paper addresses a practically important and underexplored training pathology in interleaved VLA models. The paper identifies a plausible failure mode in supervised fine-tuning of gaming and embodied VLA models: when historical actions are highly temporally correlated, the model may over-rely on the low-entropy action-history pathway and under-utilize visual input. It targets a real weakness of current interleaved VLA training pipelines, rather than simply improving benchmark metrics.
  2. The proposed intervention is simple, lightweight, and practically appealing. HANI is easy to implement, requires no architectural changes or additional parameters, and adds no inference-time overhead.
Critical weaknesses
  1. The methodological novelty appears somewhat targeted and incremental. HANI is well motivated and practically useful, but methodologically it remains close to the broader family of input-corruption or modality-perturbation techniques. The paper makes a reasonable case that discrete, semantically mismatched history corruption is better aligned with the proposed failure mode than generic continuous noise or dropout. Still, the overall contribution feels more like a targeted refinement than a fundamentally new training principle.
  2. The empirical support is somewhat narrow relative to the paper’s broader framing. The paper appropriately acknowledges many of its limitations, but the main evidence is still concentrated on a single VLA backbone and a relatively narrow family of discrete-action gaming and embodied settings characterized by strong action inertia. As a result, the current results more convincingly support the claim that HANI is effective in this regime than the broader claim that the diagnosed shortcut is a general failure mode of interleaved VLA SFT.
  3. The results on richer environments are informative but comparatively coarse-grained. The MiniWorld experiments include task success, modality-level diagnostics, and ablations, whereas the ViZDoom, Minecraft, and RoboTHOR results are reported only as aggregate SFT-versus-HANI reward comparisons. These additional experiments therefore broaden the empirical scope of the paper, but they do not provide the same level of statistical or mechanistic support as the main benchmark.
  4. The current experiments do not fully isolate which components of HANI are necessary for its gains. The paper includes a useful ablation over the perturbation probability p, but it remains unclear how much of the improvement depends specifically on HANI’s full design—namely, replacing the entire action history with discrete semantically incorrect tokens—rather than on more modest ways of weakening the historical pathway.
Questions for authors
  1. Clarification on mechanism isolation. Beyond the Modality Dropout comparison and the ablation over p, do the authors have additional evidence that distinguishes HANI’s proposed mechanism from a more generic regularization effect induced by corrupting historical actions during training? For example, have the authors evaluated simpler variants such as perturbing only part of the history, shortening the history window, or partially masking action tokens?
  2. Clarification on cross-backbone generality. Do the authors have any preliminary evidence that the same historical-action shortcut, and the benefit of HANI, also appear in VLA backbones other than Qwen3-VL-8B-Instruct?
  3. Clarification on the richer 3D environments. For ViZDoom, Minecraft, and RoboTHOR, can the authors report variance estimates and, if available, at least one modality-level diagnostic analogous to the MiniWorld analysis? This would help clarify whether the observed gains in those environments arise from the same proposed mechanism.
Reviewer_eogaOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper identifies a historical-action shortcut in gaming VLA models: temporally correlated expert actions allow SFT models to predict from action history while underusing vision, causing Visual Modality Collapse. It proposes two modality-dependence diagnostics and Historical Action Noise Injection (HANI), which randomly replaces historical action tokens during training. Experiments on MiniWorld and three additional 3D environments report improved performance and stronger visual reliance.

Critical strengths
  • The historical-action shortcut and Visual Modality Collapse are meaningful problems for gaming and embodied VLA models.
  • HANI is simple, adds no inference overhead, and consistently improves over standard SFT in the reported experiments.
Critical weaknesses
  1. The theoretical claims are substantially stronger than what the analysis establishes.

    • A small data-level I(Y;V∣H)I(Y;V\mid H) does not imply that SFT must ignore vision or inevitably cause gradient starvation. Theorem 2 also assumes independent noise, although the replacement depends on the original action, while Theorem 3 does not establish a positive lower bound on visual gradients. The analysis offers useful intuition, but claims such as “inevitability,” “mathematically compelled,” and “strictly bounded away from zero” should be weakened.
  2. The experiments neither directly verify the proposed “gradient awakening” mechanism nor compare against the most relevant shortcut-mitigation baselines.

    • Final IG/KL values show changed modality reliance but do not verify that HANI restores visual gradients; training-time gradient norms or controlled freezing experiments are needed. Moreover, NEFTune and Modality Dropout are generic regularizers. Comparisons with keyframe reweighting, repeated-action downsampling/loss decay, and corruption-matched controls are needed to establish that the gains are specific to HANI.
Questions for authors

See weakness

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.