acceptodds
Under review as a conference paper at ICLR 2027

Appearance-free Video Generation for Measuring a Perception Gap in Motion-Defined Forms

Abstract

VLMs have achieved remarkable progress on video understanding benchmarks, yet it remains unclear whether they are truly as capable as humans. A key aspect of human perception is persistence of vision. We can identify motion patterns in videos whose individual frames appear as pure noise (without any appearance or recognizable content). To quantify this capability for VLMs, we propose an algorithm to generate appearance-free videos from any given source of motion. Each frame is indistinguishable from noise, and only motion contains the information. We then introduce AppearFreeVQA, a benchmark with ≈5,300 appearance-free videos spanning four motion categories and three difficulty levels. We demonstrate that humans achieve approximately 93% accuracy, while frontier VLMs (GPT-5.1) reach only 41.7%, slightly above random chance. Oracle experiments show that supplying target optical flow improves Qwen3.5-9B from 14.6% to 66.8%, indicating that motion extraction is a major bottleneck. Finally, we show that training on AppearFreeVQA can improve VLMs' capabilities on camouflaged object detection, a real-world application where appearance cues are limited.

Primary area:
foundation or frontier models, including LLMs
Keywords:
VLM, motion perception

Reviews

Among:
Shape:
1234this paper, Top 15%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.00 over 3 reviews, above 85% of the others.
Reviewer_bUD8OpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This study investigated whether Visual Language Models (VLMs) can replicate the human visual ability to recognize content in videos based solely on visual motion. To this end, the authors proposed an algorithm that generates "appearance-free" stimuli (videos in which the textures have been replaced with i.i.d. Gaussian noise) from arbitrary videos, and used this to create a benchmark called AppearFreeVQA. Using this benchmark, the authors tested multiple VLMs and confirmed that these models cannot perceive appearance-free stimuli in the same way humans do, and that this bottleneck lies in the extraction of motion information. Furthermore, they demonstrated that using AppearFree stimuli as training data improves performance on camouflaged object detection tasks.

Critical strengths

The most significant contribution is the proposal of AppearFreeVQA. Compared to SpookyBench, it offers a wider variety of stimuli and includes more natural scene motion. The fact that the task was expanded from simply "What" (whether objects can be recognized) to "How" (motion direction or action) is also appreciated.

Critical weaknesses

Given what was already known from the SpookyBench paper, the additional differences seem incremental. It was already known that the perception of appearance-free motion is catastrophically impaired and that the bottleneck lies in extracting boundaries from motion. The gains obtained from the expanded tasks and stimuli in this new benchmark do not seem particularly significant. While the extension of the task from "What" to "How" is appreciated, it remains unclear whether this extension actually led to a deeper understanding. Perhaps the most noteworthy aspect is the "Application" (fine-tuning for camouflaged animal detection), but since these results are based on a single model, they seem somewhat preliminary to serve as a main contribution.

Another critical point concerns the generation of appearance-free stimuli, in that the necessity of MotionOptim is not sufficiently justified. While this explanation is provided in Appendix 1.8, the description in A 1.8 seems incorrect. If both foreground and background noise are independently sampled from the same i.i.d. Gaussian distribution, then even if the holes are filled with background noise, a single-frame still image would remain strictly i.i.d. Gaussian noise, and there would be no reason for discrepancies in boundary lines or local variance to occur. The authors should clarify why appearance leakage occurs in still images.

Additionally, the authors should cite the related work below and discuss how their method differs from it. While this paper highlights the ability to handle arbitrary video motions beyond hand-crafted rule-based motion, the method in the cited study already shares this characteristic.

Filip Ilic, Thomas Pock, and Richard P Wildes. Is appearance free action recognition possible? In European Conference on Computer Vision, pages 156–173. Springer, 2022.

Questions for authors

In the "Motion Misaligned" experiment, the model achieves relatively high accuracy even though the optical flow is replaced with that of a different video. Could the authors provide further explanation or intuition for why this occurs?

In the "Appearance Level Control" experiment, the stimuli are generated by solving a least-squares optimization problem. How does this formulation fundamentally differ from a simple alpha blend between the appearance-free image and the original RGB image? The current optimization approach seems unnecessarily complicated.

In Section 4.4, why was only NVILA-Lite-2B used? Would other models also show improvements? Also, wouldn't it be more effective to fine-tune the model so that optical flow information can be calculated separately and used as an additional input?

Reviewer_RHUyOpenReview ↗
Rating
3
Confidence
3
Informativeness
3
Presentation
3
Soundness
3
Read review
Paper summary

This paper studies a topic that is rarely discussed: the persistence of vision for appearance-free video understanding. The authors propose methods to generate training data, build a benchmark that reveals the weaknesses of leading VLMs, and conduct extensive experiments to verify plausible downstream application. Overall, a fun paper to read.

Critical strengths
  1. The paper is very coherent and well-presented, it is a pleasure reading it.
  2. The takeaways, albeit not very surprising, are useful to validate.
  3. The data pipeline is sound and comprehensive.
Critical weaknesses

The only major concern is regarding the practical significance of the research. As far as I know, appearance-free videos have only been used as an innovative CAPTCHA against bots. In the later section, the authors point out that training on the proposed dataset also benefits camouflaged animal detection, which is also a topic that holds less significance, compared to mainstream video understanding tasks, in the field. A much more convincing experiment would be applying it to understanding general videos with fast-moving scenes.

Questions for authors
  1. I find the dataset construction might be the cause of limited transferability to general capabilities. Despite the four categories, all objects are blended into noisy backgrounds. There might be other ways to construct such data. But this point is less significant as I myself can't think of a better way so I think it's unfair to request the authors to do so.

  2. Following 1, training on AppearFreeVQA even hurts the performance on general video understanding benchmark such as MVBench. However, I have to point out that reporting negative results is just as valuable as the positive ones, and the authors should be rewarded instead of penalized. Hence, I would only encourage the authors to discuss further on how can this study benefit video understanding on at large scale, and this point will not undermine the rating.

  3. I am curious why the model outperform humans at the medium level only? Some discussions would be helpful.

Reviewer_snrrOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
2
Soundness
3
Read review
Paper summary

This work investigates whether VLMs can perceive motion in the absence of static appearance cues in a manner similar to humans. It proposes a two-stage video generation method, ForwardWarp + MotionOptim: first, the target optical flow is applied to a random noise image, followed by an optimization process that generates frames exhibiting both the statistical properties of noise and the motion structure. Based on above, the paper introduces AppearFreeVQA, a benchmark designed to evaluate VLM performance across three levels of increasing difficulty: motion detection, moving entity identification, and fine-grained motion understanding.

Critical strengths
  1. Technical advantages: (1) The combination of ForwardWarp and MotionOptim integrates motion states and noise-appearance constraints into a controllable synthesis pipeline. (2) The multi-level question-answer pairs, coarse, medium, and fine, can evaluate model performance at progressively finer levels, assessing whether motion exists, what is moving, and how the motion occurs.
  2. Comprehensive experiments: The evaluation covers multiple open-source and closed-source models, and the appendix provides additional experiments and in-depth analyses.
Critical weaknesses
  1. Experimental inconsistency: The results reported for Symbol and Animal are inconsistent between the main paper and the appendix.
  2. Methodological issues: 1) The statistical details of the human baseline need further clarification, such as the number of annotators who answered each question. 2) The random baseline needs to be more clearly defined, for example, clarifying whether the calculation approaches for coarse, medium, and fine difficulties are uniform.
Questions for authors
  1. Why do the reported statistics for Symbol and Animal differ between the main paper and the appendix? What are the corresponding results in the final released version?
  2. How does this work quantify the extent to which appearance information has been eliminated?
  3. Are frame sampling strategies and input resolutions consistent across different VLMs? If not, how to eliminate the potential impact of preprocessing differences?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.