- Rating
- 2
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
The paper introduces OctVideo, an adaptive spatiotemporal octree representation whose leaves encode local RGB values and first-order gradients. Error-guided subdivision allocates tokens to complex regions, while sparse processing supports video reconstruction and recognition. A video VAE combines this representation with a regular latent grid and hierarchical decoding. Experiments demonstrate favorable reconstruction–compute trade-offs, supported by controlled architectural ablations and a complementary action-recognition evaluation.
Critical strengths
- A coherent representation with useful analytical structure. The first-order leaf descriptors support direct reconstruction, and reconstruction error provides a natural criterion for adaptive subdivision.
- Strong internal evidence for the design choices. The controlled ablations distinguish the effects of gradient descriptors, adaptive allocation, sparse decoding, and residual pathways. In particular, comparison with larger dense models supports a meaningful reconstruction–compute advantage.
Critical weaknesses
- The scope of the practical efficiency advantage needs tighter qualification. The reported encoder timing excludes octree construction, which is important considering the small encoding-time difference from LeanVAE. The high-resolution evaluation also uses different native frame counts and execution protocols across methods. Moreover, OctVideo has higher peak memory than LeanVAE and worse DAVIS LPIPS/rFVD despite faster inference. These results support a useful trade-off, but do not yet establish a uniformly preferable operating point.
- All compared baselines are video VAEs built for latent video generation, so a good latent space should be not only accurate in reconstruction but also suitable for generation. The paper trains no diffusion model on the OctVideo latent and provides no evaluation of latent-space quality, leaving the suitability of the representation for its baselines' primary use case unverified.
Questions for authors
- Could the authors provide additional analysis of the latent space's suitability for generation? I would like to see a small-scale generation experiment on the latent if possible.
- What are the complete input-to-reconstruction latency and peak memory, including tree construction and descriptor preparation, under a matched device, precision, and batch size?
- The compared classifiers differ in clip resolution, temporal extent, and evaluation views, so token count alone provides incomplete evidence for the computational benefit in recognition. What are the total inference FLOPs and measured throughput for action recognition when all evaluation views are counted, ideally under a matched clip/view protocol?