acceptodds
Under review as a conference paper at ICLR 2027

Learned Image Compression for Vision-Language-Action Models

Abstract

Vision-language-action (VLA) models increasingly rely on high-frequency multi- camera observations, making visual communication a major bottleneck for real- time robotic control in bandwidth-constrained or distributed deployment settings. Existing image and video codecs, however, are designed to preserve generic visual fidelity rather than the control performance of downstream VLA policies. In this work, we introduce SPARC (SPatially Adaptive Rate Control), a learned image compression framework tailored for VLA-driven robots. Our key observation is that the importance of visual information varies substantially across both camera views and spatial regions within an image. Based on this observation, SPARC employs a lightweight temporal mask selector that adaptively allocates bitrate over latent representations according to task relevance while leveraging temporal context. We further introduce a tilted rate loss that stabilizes training by reducing the tendency of entropy-based objectives to over-suppress rare yet task-critical visual patterns. Experiments on diverse robotic benchmarks, including RoboCasa365, VLABench, and LIBERO, show that SPARC consistently achieves stronger control performance than conventional image/video codecs and recent learned compression methods under the same bitrate budget. We additionally demonstrate real-world deployment benefits in remote-control settings, where our method substantially improves the bitrate-success tradeoff.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
Neural Image compression, Vision-Language-Action models

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_TghtOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

This paper proposes an image compression framework, SPARC, for vision-language-action models. Unlike typical image compression methods that are optimized for preserving signals/pixels, this framework is designed to preserve important visual information or semantics that are required to perform robotic actions. They propose a temporal mask selector and a specialized loss function to allow adaptive bitrate allocation between camera streams and image regions, where masks are generated by leveraging hyperprior features from the current and previous camera observations. The authors conduct evaluations on both simulation benchmarks and real-world environments, where bitrate, success rate, and latency are measured and compared. The proposed method outperforms several conventional codecs (like JPEG) and neural codecs (like MS-ILLM, DCVC-RT).

Critical strengths
  1. The application scenario is concrete, as described in the introduction: communication-constrained environments, robotic systems that have many cameras, and multi-robot systems. This makes the work meaningful and useful.
  2. Successfully design a neural codec for VLA tasks. Propose the temporal mask selector for selecting important features and tilted rate loss for stabilizing training.
  3. Rather sufficient evaluations. The real-world experiments are highly encouraged. Results are good.
Critical weaknesses
  1. For VLA tasks, the topic to study should be video compression instead of image compression. These tasks can be regarded as extension of typical recognition/perception tasks (classification, detection, segmentation, etc.) in the field of image/video coding for machines (ICM/VCM). For those tasks, studying image compression is sometimes meaningful, as a single frame could provide useful information for simple applications. However, VLA tasks are complicated and highly depend on temporal context to make decisions and perform actions, which indicates a dynamic environment and long-term interactions between robots and environments. This work focuses on image compression, while the introduced temporal mask selector utilizes temporal information. In the evaluations, video codecs are also compared. These indicate obvious contradictory that should be addressed. I understand that studying image compression for a new task is always the first step and can provide solid basis for further video compression. However, if the research topic is image compression, I suggest focusing on a more static part of VLA tasks.
  2. The novelty and contribution of the method are limited. Selecting and preserving information that are only necessary for given tasks by masking latent features to reduce bitrate is a common approach in the field of ICM/VCM.
  3. The proposed method does not take text instruction inputs as priors. The temporal mask selector takes hyperprior-derived visual features, temporal embeddings, and camera embeddings as inputs. It does not directly receive the language instruction or the current action objective. Therefore, it produces the same bitrate allocation regardless of whether the robot is instructed to manipulate different objects.
  4. The evaluation does not include ICM/VCM codecs. The simulation experiments include several other codecs, while the real-world ones simply compares JPEG (even not JPEG2K used in simulation experiments), where more methods should be compared. The end-to-end latency evaluation contains ambiguities and potentially inconsistent assumptions: in table 3, the total encoding and decoding latency of SPARC is 124.7ms, which matches the numbers in table 4, while the results of the two tables are obtained under different bitrate settings, which are 0.0454 for table 3 and 0.15 for table 4.
Questions for authors

Please refer to weaknesses.

4 comments

Thank you for your careful review. We address your concerns as follows. Following your suggestion, we added our revised version of the manuscript. Our revision is marked with blue in the manuscript.

W1 — Image vs. Video Compression for VLA Tasks

W1. For VLA tasks, the topic to study should be video compression instead of image compression. These tasks can be regarded as extension of typical recognition/perception tasks (classification, detection, segmentation, etc.) in the field of image/video coding for machines (ICM/VCM). For those tasks, studying image compression is sometimes meaningful, as a single frame could provide useful information for simple applications. However, VLA tasks are complicated and highly depend on temporal context to make decisions and perform actions, which indicates a dynamic environment and long-term interactions between robots and environments. This work focuses on image compression, while the introduced temporal mask selector utilizes temporal information. In the evaluations, video codecs are also compared. These indicate obvious contradictory that should be addressed. I understand that studying image compression for a new task is always the first step and can provide a solid basis for further video compression. However, if the research topic is image compression, I suggest focusing on a more static part of VLA tasks.

We believe this is a misunderstanding. We did not use image compression merely as a stepping stone toward video compression—our central argument is that image compression is a better pathway for the communication of visual signals for robotics tasks, especially for manipulation tasks, because of action chunking (i.e., image inputs to VLAs are temporally discontinuous). The design of SPARC revolves around incorporating a necessary amount of temporality inside the image compression framework; thus we don’t believe there is any contradiction.

W2 — Novelty of Temporal Masking for VLA Control

W2. The novelty and contribution of the method are limited. Selecting and preserving information that are only necessary for given tasks by masking latent features to reduce bitrate is a common approach in the field of ICM/VCM.

The novelty of SPARC lies in how to determine masks for VLA-based robotic control. The unique characteristics of VLA, unlike downstream tasks in prior ICM/VCM literature [1,2,3,4], is that the model should make sequential predictions with subsampled frames (due to action chunking), often involving long-term interaction between robots and dynamic environments. Thus, the masking strategy should account for the temporal relevance between distant frames. Noticing this unique nature of coding for VLAs, SPARC adopts image compression and conditions the mask on temporal context, so that bit allocation adapts to the task phase and to each camera in multi-camera robots.

W3 — Language Instruction Conditioning

W3. The proposed method does not take text instruction inputs as priors. The temporal mask selector takes hyperprior-derived visual features, temporal embeddings, and camera embeddings as inputs. It does not directly receive the language instruction or the current action objective. Therefore, it produces the same bitrate allocation regardless of whether the robot is instructed to manipulate different objects.

On the one hand, we agree that instruction conditioning could further reduce bitrate by discarding regions irrelevant to a given instruction.

On the other hand, we empirically discovered that it is very challenging to incorporate such information, especially under the stringent encoder-side on-device constraints. In fact, our early versions included language conditioning via FiLM, but it turned out that it does not add statistically significant gain. We are open to the possibility that language conditioning can indeed help, but we suspect that it may require heavier forms of encoding, which falls outside the scope of this project.

Finally, we note that SPARC indeed captures a notion of (instruction-agnostic) task-relevance, as presented in Figure 1.

W4-1 — Comparison with ICM/VCM Codecs

W4-1. The evaluation does not include ICM/VCM codecs.

As requested, we add experiments on ICM/VCM. We select SA-ICM [1] as an ICM baseline and SVC [2] as a VCM baseline, based on the availability of publicly released code and checkpoints, as well as their compatibility with latency requirements for VLA. SPARC outperforms both baselines.

ICM/VCM codec comparison

Benchmark / BackboneMethodbppSucc. (%)bppSucc. (%)bppSucc. (%)
RoboCasa, π0.5SVC——0.05627.80.08539.8
RoboCasa, π0.5SA-ICM————0.220*28.0
RoboCasa, π0.5MS-ILLM0.0150.20.05934.30.09843.9
RoboCasa, π0.5SPARC (Ours)0.01514.40.05357.20.08061.3
RoboCasa, π0SVC——0.05520.60.08632.0
RoboCasa, π0SA-ICM————0.199*25.3
RoboCasa, π0MS-ILLM0.0144.10.05819.40.09624.4
RoboCasa, π0SPARC (Ours)0.01412.90.04535.20.06636.3
RoboCasa, GR00T N1.6SVC——0.05524.80.08347.3
RoboCasa, GR00T N1.6SA-ICM————0.163*50.9
RoboCasa, GR00T N1.6MS-ILLM0.0110.70.04440.00.07348.8
RoboCasa, GR00T N1.6SPARC (Ours)0.01110.70.04246.50.07051.5
VLABench, π0.5SVC——0.03432.80.05534.2
VLABench, π0.5SA-ICM————0.094*32.7
VLABench, π0.5MS-ILLM0.0086.50.03635.90.05834.6
VLABench, π0.5SPARC (Ours)0.0077.40.03036.50.05136.5
LIBERO, π0.5SVC——0.03892.60.06696.0
LIBERO, π0.5SA-ICM————0.136*97.3
LIBERO, π0.5MS-ILLM0.00925.60.03997.00.06597.2
LIBERO, π0.5SPARC (Ours)0.00944.70.03897.10.06197.2

*: Note for high bpp

W4-2 — Additional Latency Comparisons

W4-2. The simulation experiments include several other codecs, while the real-world ones simply compare JPEG (even not JPEG2K used in simulation experiments), where more methods should be compared.

We believe the reviewer is referring to the end-to-end latency table (Table 4), as the accuracy results already report results for MS-ILLM other than JPEG (Fig 6).

We have added MS-ILLM and JPEG-2000 to the revised latency table (also see below); the accuracies for JPEG-2000 will be added too as soon as all our tasks are done running.

Finally, we also note that we have also added an accelerated version of SPARC (opt.)—via ONNX & OpenVINO optimizations—which decreases the latency from 124.7ms to 58.0ms.

SettingJPEG (q95)JPEG2000MS-ILLMSPARCSPARC (opt.)
Enc/Dec (ms)0.516.7123.6124.758.0
Varying bandwidth C (N = 1)
250 kbps——421.4*169.3102.6
500 kbps985.3*393.9*226.5145.678.9
750 kbps331.3199.3185.8138.371.6
1 Mbps199.3137.1168.2134.868.1
2 Mbps77.167.7144.5129.763.0
Varying streams N (C = 100 Mbps)
N = 11.717.6124.0124.858.1
N = 1282.918.0124.1124.858.1
N = 25629.019.5124.2124.858.1
N = 512——124.6124.858.1
N = 1024———124.858.1
N = 2048———124.958.2
N = 3072———125.759.0
Max. streams N_max2673768473,3903,390
* : Exceed the deadline

W4-3 — Consistency of Latency Across Bitrate Settings

W4-3. The end-to-end latency evaluation contains ambiguities and potentially inconsistent assumptions: in table 3, the total encoding and decoding latency of SPARC is 124.7ms, which matches the numbers in table 4, while the results of the two tables are obtained under different bitrate settings, which are 0.0454 for table 3 and 0.15 for table 4

We clarify that the bitrate settings reported in the manuscript are correct: Table 3 reports latency at 0.0454 bpp, while Table 4 uses 0.15 bpp. The bitrate has a negligible effect on latency, with an encoding/decoding latency difference of less than 1 ms between 0.0454 bpp and 0.150 bpp.

This is because most of the latency is spent on neural-net pipelines, which is shared among all bitrates. The only rate-dependent part of the pipeline is entropy coding, accounting for less than 14% of the total encoding latency; its computational cost varies only slightly across different bitrates.

References

[1] Takahiro Shindo, Kein Yamada, Taiju Watanabe, and Hiroshi Watanabe. Image coding for machines with edge information learning using Segment Anything, ICIP, 2024.

[2] Hadi Hadizadeh and Ivan V. Bajić. Learned scalable video coding for humans and machines. EURASIP Journal on Image and Video Processing, 2024.

[3] Nam Le, Honglei Zhang, Francesco Cricri, Ramin Ghaznavi-Youvalari, Hamed Rezazadegan Tavakoli, and Esa Rahtu. Learned image coding for machines: A content-adaptive approach. In IEEE International Conference on Multimedia and Expo, 2021.

[4] Ruoyu Feng, Yunpeng Qi, Jinming Liu, Yixin Gao, Xin Li, Xin Jin, and Zhibo Chen. Diff-ICMH: Harmonizing machine and human vision in image compression with generative prior. NeurIPS 2025.

AuthorsOpenReview ↗

Kind Reminder

Dear Reviewer Tght,

We hope this message finds you well.

We have carefully addressed the questions and concerns you raised. In summary:

  • W1: We clarified that our focus on image compression is a deliberate design choice rather than a stepping stone toward video compression since VLA observations are temporally subsampled, due to action chunking and SPARC incorporates the necessary temporal context within an image compression framework.
  • W2: We clarified the novelty of SPARC in determining masks for VLA control, where the mask is conditioned on temporal context so that bit allocation adapts to the task phase and to each camera.
  • W3: We respectfully clarified that language instruction conditioning is currently outside the scope of our project under the stringent on-device encoder constraints.
  • W4-1: We added ICM/VCM baselines (SA-ICM and SVC) across multiple benchmarks and VLA backbones, where SPARC achieves higher or comparable success rates at substantially lower bitrates.
  • W4-2: We added MS-ILLM and JPEG2000 to the latency table, introduced an accelerated version of SPARC (opt.), and added real-world JPEG2000 results (see our general comment).
  • W4-3: We clarified that the encoding/decoding latency differs by less than 1 ms between 0.0454 and 0.15 bpp, as most of the computation lies in the bitrate-independent neural network pipeline.

We sincerely appreciate your constructive feedback, which has been very helpful in improving our work. As only about 36 hours remain until the end of the discussion period, we would be happy to provide any further clarification if any concerns remain unresolved. If our responses have addressed your concerns, we would greatly appreciate it if you could consider reflecting this in your evaluation.

Thank you again for your time and thoughtful evaluation of our work.

Best regards,

The Authors

Reviewer_o45oOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper presents SPARC, a learned image compression framework optimized for vision-language-action policies. Starting from a pretrained image codec, SPARC learns a temporal latent selector that determines which entries to transmit using current and previous hyperprior features. The receiver reconstructs the mask from shared information, avoiding explicit mask transmission. The method also introduces a tilted rate objective that reduces differences between local bit penalties, aiming to prevent the selector from discarding expensive but action-relevant information. Evaluation covers three simulation benchmarks, three VLA backbones, and real-robot manipulation. Additional experiments examine temporal history, training choices, backbone transfer, visual perturbations, and network conditions.

Critical strengths
  1. A practically important problem with promising empirical results. The paper addresses a meaningful problem at the intersection of learned image compression and robotic control. The experimental evaluation covers multiple robotic benchmarks and VLA architectures, and SPARC demonstrates substantial bitrate savings compared with existing compression methods. The inclusion of real-world robotic experiments further strengthens the practical relevance of the work.
  2. An effective and reasonably well-motivated adaptive compression framework. The temporal mask selector provides a practical mechanism for allocating bits to task-relevant latent features without explicitly transmitting the masks. The use of hyperprior features for mask generation is an interesting design choice that can reduce additional communication overhead.
Critical weaknesses
  1. Limited isolation of methodological contributions. The proposed approach builds on existing ideas in task-aware compression and adaptive latent masking. Although Appendix C.2 includes an action-based fine-tuning baseline, it only fine-tunes the decoder, making it difficult to isolate the benefits of the proposed masking mechanism. Moreover, the masking ablations in Table 1 replace masks only at inference time rather than training alternative strategies under comparable conditions.
  2. Insufficient justification of the tilted rate loss. The proposed loss is largely heuristic. While Table 8 shows improvements for certain values of α, the paper provides limited evidence supporting its claimed training stability benefits.
  3. Unclear latency analysis. Table 3 reports encoding/decoding latency at 0.0454 bpp, while Table 4 uses 0.15 bpp but appears to adopt the same computation cost. This discrepancy requires clarification, particularly since the practical latency advantage is a key claim.
Questions for authors
  1. How much of SPARC's improvement comes from the proposed masking mechanism rather than task-aware fine-tuning? Could the authors provide a comparison with a simpler learned masking baseline trained under the same action-based objective?
  2. Could the authors provide direct evidence that the tilted rate loss improves training stability, beyond the final success rates reported in Table 8?
  3. In Table 4, is the encoding/decoding latency measured at 0.15 bpp, or reused from the 0.0454 bpp setting in Table 3? How does this affect the end-to-end latency conclusions?
  4. Could the authors compare against alternative masking policies trained with their own adapted decoders, rather than changing masks only at inference? For the tilted loss, would its advantage persist after independently tuning λ for α=1 to obtain matched actual bitrates? Please clarify how α was selected without using the final evaluation results.
5 comments

Thank you for your careful review. We address your concerns as follows. Following your suggestion, we have updated our revised version of the manuscript. Our revision is marked with blue in the manuscript.

W1 / Q1 / Q4-1 — Isolating the Contribution of Masking

W1. Limited isolation of methodological contributions. The proposed approach builds on existing ideas in task-aware compression and adaptive latent masking. Although Appendix C.2 includes an action-based fine-tuning baseline, it only fine-tunes the decoder, making it difficult to isolate the benefits of the proposed masking mechanism. Moreover, the masking ablations in Table 1 replace masks only at inference time rather than training alternative strategies under comparable conditions.

Q1. How much of SPARC's improvement comes from the proposed masking mechanism rather than task-aware fine-tuning? Could the authors provide a comparison with a simpler learned masking baseline trained under the same action-based objective?

Q4-1. Could the authors compare against alternative masking policies trained with their own adapted decoders, rather than changing masks only at inference?

Thank you for pointing this out. To isolate our methodological contribution, we conducted a mask training experiment, verifying that SPARC indeed improves over alternative masking strategies. The result below isolates the contribution of masking with temporal mask selector from simple codec training with task-aware fine-tuning. We will add this result to the manuscript.

MaskingKeepbppSucc. (%)KeepbppSucc. (%)KeepbppSucc. (%)
Current-only0.2350.01513.10.6670.05354.70.6440.08057.5
Saliency0.2360.0142.60.6670.05453.90.6350.09057.4
Spatially Uniform0.2360.0080.40.6670.04134.80.6350.06345.8
Random0.2360.0080.20.6670.0412.30.6350.06423.6
SPARC (Ours)0.2360.01514.40.6670.05357.20.6350.08061.3

W2 / Q2 — Justification of the Tilted Rate Loss

W2. Insufficient justification of the tilted rate loss. The proposed loss is largely heuristic. While Table 8 shows improvements for certain values of α, the paper provides limited evidence supporting its claimed training stability benefits.

Q2. Could the authors provide direct evidence that the tilted rate loss improves training stability, beyond the final success rates reported in Table 8?

Thank you for the comment.

First, we note that we mistakenly choose an inadequate expression: “training stability” is not the most appropriate word to describe how tilted loss affects better training. Its primary purpose is to remove latents that are high-bit-yet-task-irrelevant.

Without tilting (α=1\alpha=1), the rate loss is heavily driven by local bit entries, bjb_j, making the overall training guided toward masking out high-bit regions. Since much of the task-relevant regions also have high bjb_j, the training ends up masking out much of the task-relevant regions.

In contrast, with α=0\alpha=0, the rate loss is agnostic to bjb_j of each pixel, making the overall training mostly task-accuracy-guided. As a result, the training ends up retaining the pixels that require too much bjb_j compared to the amount of information they contain.

The tilted rate loss with 0<α<10<\alpha<1 is designed to strike the gracious balance between these two extremes by suppressing bjb_j of high-bit regions. The training ends up masking out pixels that contribute to excessive bjb_j compared to their relevance with task-accuracy.

Supporting experiment. We measure the point-biserial correlation between the fraction of bits removed by the mask at each latent position and a binary indicator of high-bit regions using 5,745 policy-camera frames from 48 rollouts from RoboCasa365 benchmark under the same bit savings and VLA backbone (π0.5\pi_{0.5}, bpp = 0.080).

Rate termRemoval on background with high bjb_j
α=0\alpha=0+0.09
α=0.4\alpha=0.4+0.35
α=1\alpha=1+0.24

The result clearly shows that α=0.4\alpha=0.4 exhibits the highest correlation between bit removal and high-bit background regions among the tested settings, supporting our hypothesis that the tilted rate loss effectively removes task-irrelevant high-bit regions while preserving task-relevant information.

W3 / Q3 — Consistency of Latency Across Bitrate Settings

W3. Unclear latency analysis. Table 3 reports encoding/decoding latency at 0.0454 bpp, while Table 4 uses 0.15 bpp but appears to adopt the same computation cost. This discrepancy requires clarification, particularly since the practical latency advantage is a key claim.

Q3. In Table 4, is the encoding/decoding latency measured at 0.15 bpp, or reused from the 0.0454 bpp setting in Table 3? How does this affect the end-to-end latency conclusions?

We clarify that the bitrate settings reported in the manuscript are correct: Table 3 reports latency at 0.0454 bpp, while Table 4 uses 0.15 bpp. The bitrate has a negligible effect on latency, with an encoding/decoding latency difference of less than 1 ms between 0.0454 bpp and 0.150 bpp.

This is because most of the latency is spent on neural-net pipelines, which is shared among all bitrates. The only rate-dependent part of the pipeline is entropy coding, accounting for less than 14% of the total encoding latency; its computational cost varies only slightly across different bitrates.

Q4-2 — Matched-Bitrate Comparison and Alpha Selection

Q4-2. For the tilted loss, would its advantage persist after independently tuning λ\lambda for α=1\alpha=1 to obtain matched actual bitrates? Please clarify how α was selected without using the final evaluation results.

As suggested, we checked whether applying tilted rate loss is better than independently tuning λ\lambda for α=1\alpha=1 to obtain matched actual bitrates.

MethodbppSucc. (%)bppSucc. (%)bppSucc. (%)
α=1\alpha=1, λ\lambda re-tuned0.01513.80.05355.70.07958.1
SPARC (Ours)0.01514.40.05357.20.08061.3

On selecting α\alpha. We select α\alpha using 50 validation episodes from each benchmark, apart from training episodes, based on success rates. Each validation episode is randomly initialized. The final chosen α\alpha is either 0.4 or 0.6 across all benchmarks and a family of VLA models.

AuthorsOpenReview ↗

Kind Reminder

Dear Reviewer o45o,

We hope this message finds you well.

We have carefully addressed the questions and concerns you raised. In summary:

  • W1 / Q1 / Q4-1: To isolate the contribution of our masking mechanism, we trained alternative masking strategies (current-only, saliency, spatially uniform, and random) under comparable training conditions, and showed that SPARC consistently outperforms them.
  • W2 / Q2: We clarified that the primary role of the tilted rate loss is to remove high-bit yet task-irrelevant latents, and provided supporting evidence through a correlation analysis between bit removal and high-bit background regions.
  • W3 / Q3: We clarified that the encoding/decoding latency differs by less than 1 ms between 0.0454 and 0.15 bpp, as most of the computation lies in the bitrate-independent neural network pipeline.
  • Q4-2: We showed that the tilted rate loss outperforms a baseline with independently re-tuned λ at matched actual bitrates, and clarified that α is selected on randomly initialized validation episodes.

We sincerely appreciate your constructive feedback, which has been very helpful in improving our work. As only about 36 hours remain until the end of the discussion period, we would be happy to provide any further clarification if any concerns remain unresolved. If our responses have addressed your concerns, we would greatly appreciate it if you could consider reflecting this in your evaluation.

Thank you again for your time and thoughtful evaluation of our work.

Best regards,

The Authors

Reviewer_D3b1OpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes SPARC, a learned image compression framework for vision-language-action models. It introduces a temporal mask selector that uses historical hyperprior features to selectively preserve latent representations, together with a tilted rate loss to balance bitrate reduction and task-relevant information preservation. The method is evaluated on RoboCasa365, VLABench, and LIBERO with multiple VLA backbones, as well as on a real AgileX PiPER robot. Results demonstrate improved bitrate-success tradeoffs compared with existing compression methods.

Critical strengths
  1. Practical problem and meaningful formulation. The paper addresses an important challenge in VLA deployment by optimizing visual compression for downstream action performance rather than generic image reconstruction quality. The temporal masking mechanism is a reasonable approach to preserving task-relevant visual information under limited bandwidth.
  2. Comprehensive experimental evaluation. The authors evaluate SPARC across multiple benchmarks and VLA backbones, with extensive ablations and real-world robotic experiments. These results provide encouraging evidence of its effectiveness and practical relevance.
Critical weaknesses
  1. Limited task conditioning. The temporal mask selector relies on visual and temporal features without explicitly incorporating language instructions. Thus, its masking decisions are not directly conditioned on the current task, which may limit its adaptability when different instructions require attention to different regions in the same scene. This also raises questions about the scope of the task-aware compression claim.
  2. Incomplete validation of the method and system-level claims. The contribution of temporal masking is not fully isolated from action-based codec fine-tuning, as the corresponding comparison is limited to one benchmark and backbone. Additionally, the real-world evaluation uses only 10 trials per task, while the latency analysis relies on estimated network transmission times. The assumptions behind the large-scale concurrent-stream results in Table 4 also require clarification.
Questions for authors
  1. Does the mask selector receive language instructions during training or inference? If not, how does the method handle different task instructions under the same visual observation?
  2. Could the authors extend the action-based MS-ILLM baseline comparison to other benchmarks or VLA backbones to better isolate the contribution of temporal masking?
  3. Could the authors clarify the bandwidth-sharing assumptions and latency calculation in Table 4, particularly the result with 4096 concurrent streams?
5 comments

Thank you for your careful review. We address your concerns as follows. Following your suggestion, we added our revised version of the manuscript. Our revision is marked with blue in the manuscript.

W1 / Q1 — Language Conditioning and Task Scope

W1. Limited task conditioning. The temporal mask selector relies on visual and temporal features without explicitly incorporating language instructions. Thus, its masking decisions are not directly conditioned on the current task, which may limit its adaptability when different instructions require attention to different regions in the same scene. This also raises questions about the scope of the task-aware compression claim.

Q1. Does the mask selector receive language instructions during training or inference? If not, how does the method handle different task instructions under the same visual observation?

The mask selector does not receive language instruction during training or inference. Please see our response to W3 of Reviewer Tght for why we did not include instruction conditioning. We are willing to clarify that the scope of tasks we mention in the paper is each benchmark, rather than every general robotic task.

W2-1 / Q2 — Isolating Temporal Masking across Benchmarks and Models

W2-1. Incomplete validation of the method and system-level claims. The contribution of temporal masking is not fully isolated from action-based codec fine-tuning, as the corresponding comparison is limited to one benchmark and backbone.

Q2. Could the authors extend the action-based MS-ILLM baseline comparison to other benchmarks or VLA backbones to better isolate the contribution of temporal masking?

To isolate temporal masking from action-based fine-tuning, we demonstrate additional results on the impact of SPARC in additional benchmarks and off-the-shelf VLA models. SPARC’s gain over action-based tuning is maintained at different settings, especially in terms of bit preservation.

MethodbppSucc. (%)bppSucc. (%)bppSucc. (%)
π0.5 on RoboCasa365
MS-ILLM0.0150.20.05934.30.09843.9
Action-based tuning0.0154.00.05839.50.09949.2
SPARC (Ours)0.01514.40.05357.20.08061.3
π0 on RoboCasa365
MS-ILLM0.0144.10.05819.40.09624.4
Action-based tuning0.01413.20.05235.20.08636.3
SPARC (Ours)0.01412.90.04535.20.06636.3
GROOT N1.6 on RoboCasa365
MS-ILLM0.0110.70.04440.00.07348.8
Action-based tuning0.0119.80.04445.90.07352.0
SPARC (Ours)0.01110.70.04246.50.07051.5
π0.5 on VLABench
MS-ILLM0.0086.50.03635.90.05834.6
Action-based tuning0.0118.30.03832.10.07635.8
SPARC (Ours)0.0077.40.03036.50.05136.5
π0.5 on LIBERO
MS-ILLM0.00925.60.03997.00.06597.2
Action-based tuning0.00942.10.03996.40.06596.8
SPARC (Ours)0.00944.70.03897.10.06197.2

W2-2 — Number of Real-World Evaluation Trials

W2-2. Additionally, the real-world evaluation uses only 10 trials per task,

Following your suggestion, we extend the number of episodes from 10 trials per task to 20 trials per task. We have initially chosen to conduct 10 trials per task, following [1,2]. We have updated our paper with new real-world experiments results to reflect our revision.

MethodAverage bppAverage subtask scoreFull-task success rate
Uncompressed24.000065.83%48.33% (29/60)
MS-ILLM0.01521.67%0.00% (0/60)
MS-ILLM0.053036.67%21.67% (13/60)
MS-ILLM0.154152.50%36.67% (22/60)
MS-ILLM0.623155.83%41.67% (25/60)
SPARC0.017246.67%28.33% (17/60)
SPARC0.052045.83%28.33% (17/60)
SPARC0.137865.83%50.00% (30/60)
SPARC0.504566.67%51.67% (31/60)

W2-3 / Q3 — Bandwidth-Sharing Assumptions and Real-world Network Latency

W2-3. The latency analysis relies on estimated network transmission times. The assumptions behind the large-scale concurrent-stream results in Table 4 also require clarification.

Q3. Could the authors clarify the bandwidth-sharing assumptions and latency calculation in Table 4, particularly the result with 4096 concurrent streams?

Analytical model. The analytical model used for estimating network transmission time is essentially from [3]. In particular, the model considers the following setup: we assume NN camera streams are statistically multiplexed over a single shared link of capacity CC (one FIFO queue without per-stream bandwidth reservation), ignoring propagation delay and protocol overhead. Since the superposition of many independent periodic streams approaches a Poisson process (Palm–Khintchine theorem), we model the link as an M/M/1 queue with NN concurrent camera streams that share a link of capacity CC, modeled as an M/M/1 queue: λ=Nf\lambda = Nf, μ=C/Sˉ\mu = C/\bar{S} with Sˉ=bpp×H×W\bar{S} = \mathrm{bpp} \times H \times W, and T=1/(μ−λ)T = 1/(\mu - \lambda). With N=3072N = 3072, C=100C = 100 Mbps, and f=3f = 3 fps (one observation per 10-step chunk at 30 Hz), SPARC (0.15 bpp) yields ρ≈0.91\rho \approx 0.91 and T≈1.0T \approx 1.0 ms.

Supplementary real-world measurements. We measure the real-world network latency in this experiment. In this experiment, each observation is sent as one TCP message over a link rate-limited with tc netem. Two Linux network namespaces were connected by a veth pair, and the link rate was limited to 0.1–10 Mbps in both directions using tc netem rate control. The actual bitstreams produced by each codec for one observation were sent over TCP with netcat, and latency was defined as the time from the start of transmission until the receiver had received the full payload. A new TCP connection was opened per transfer, and we report the median over 30 runs per setting.

We note that we have also added an accelerated version of SPARC (opt.)—via ONNX & OpenVINO optimizations—which decreases the latency from 124.7ms to 58.0ms.

SettingJPEG (q95)JPEG2000MS-ILLMSPARCSPARC (opt.)
Enc/Dec (ms)0.516.7123.6124.758.0
Varying bandwidth C (N = 1)
100 kbps3,138.5*2,403.7*950.6*345.7*279.0
250 kbps1,261.5*975.7*456.6*215.7149.0
500 kbps634.5*499.7*294.6174.7108.0
1 Mbps320.5260.7211.6152.786.0
2 Mbps164.5143.7168.6138.772.0
5 Mbps70.571.7143.6132.766.0
10 Mbps35.546.7137.6134.768.0
* : Exceed the deadline

References

[1] Physical Intelligence, Kevin Black, Noah Brown, James Darpinian, Karan Dhabalia, Danny Driess, Adnan Esmail, Michael Equi, Chelsea Finn, Niccolo Fusai, et al. π0.5: a vision-language-action model with open-world generalization. arXiv preprint arXiv:2504.16054, 2025

[2] Wang, Y.-J., Jiang, H., Deng, S., Xue, H., Ye, W., Duan, R., Haghtalab, N., Sastry, S. S., Abbeel, P., & Qi, H. Reconstruct, practice, go real: Guided self-improvement for embodied, 2026.

[3] D. Bertsekas and R. Gallager, Data Networks, 2nd ed., Prentice-Hall, 1992.

AuthorsOpenReview ↗

Kind Reminder

Dear Reviewer D3b1,

We hope this message finds you well.

We have carefully addressed the questions and concerns you raised. In summary:

  • W1 / Q1: We clarified that the mask selector does not use language instructions, explained why we decided not to include lanaguage instructions (see our response to W3 of Reviewer Tght), and specified that the scope of our task-aware compression claim is defined per benchmark.
  • W2-1 / Q2: We extended the comparison with action-based tuning to additional benchmarks (RoboCasa365, VLABench, LIBERO) and VLA backbones (π0, π0.5, GR00T N1.6) to better isolate the contribution of temporal masking and updated the manuscript accordingly.
  • W2-2: We increased the real-world evaluation from 10 to 20 trials per task and updated the manuscript accordingly.
  • W2-3 / Q3: We clarified the bandwidth-sharing assumptions and the latency calculation, and added supplementary real-world network latency measurements, including an accelerated version of SPARC (opt.).

We sincerely appreciate your constructive feedback, which has been very helpful in improving our work. As only about 36 hours remain until the end of the discussion period, we would be happy to provide any further clarification if any concerns remain unresolved. If our responses have addressed your concerns, we would greatly appreciate it if you could consider reflecting this in your evaluation.

Thank you again for your time and thoughtful evaluation of our work.

Best regards,

The Authors

Comments on the paper

1 comment

Dear Reviewers,

As noted in our response to Reviewer Tght, we have added real-world experiment results for JPEG2000. Following the suggestion of Reviewer D3b1, each task is evaluated over 20 trials.

MethodbppAverage subtask scoreFull-task success rate
JPEG20000.03327.5%10.0%
JPEG20000.53151.7%31.7%

We have also updated the manuscript to reflect these new results.

Thank you again for your time and thoughtful evaluation of our work.

Best regards,

The Authors

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.