- Rating
- 1
- Confidence
- 4
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
The paper proposes Scalable Feature Coding (SFC) for distributed large vision-language model inference. Visual tokens are ordered using a diversity-based traversal and partitioned into a base layer and enhancement layers. Different cumulative token budgets serve image captioning, object detection, and instance segmentation. Layer-specific learned codecs use hyperpriors, cross-layer attention-based entropy context, and two-pass intra-layer context. Experiments use an InternVL2.5-based model and compare rate–accuracy performance against feature-coding baselines, along with latency comparisons against image-transmission pipelines. Additional experiments evaluate semantic segmentation and visual grounding.
Critical strengths
(1) The framework offers explicit cumulative decoding access points. Earlier reconstructed latents are available before enhancement-layer decoding, and the two-pass intra-layer context avoids fully sequential autoregressive decoding. This provides a coherent implementation of progressively accessible visual features.
(2) The paper evaluates both task accuracy and compression efficiency. Table 3 provides useful controlled evidence that cross-layer and intra-layer contexts improve the proposed codec, while Appendix A.4 explicitly discusses position-signaling overhead. These details strengthen the technical credibility of the implementation.
Critical weaknesses
(1) The experiments do not isolate the benefit of scalable coding from token reduction and stronger entropy modeling.
The proposed system changes several factors simultaneously: the number of retained tokens, their ordering and partitioning, the number of separately parameterized codec branches, and the entropy-context architecture. Figures 5 and 8 demonstrate improvements for the complete system, but do not establish which gains specifically require a shared scalable bitstream.
A particularly important missing control is to apply the same token selection and cumulative budgets to the existing feature codecs. Without this comparison, the captioning and detection gains may substantially reflect encoding fewer tokens rather than the proposed hierarchy. TOFC already combines token merging with learned feature coding, so its inclusion alone does not establish a matched-token comparison; the adaptation and token-budget settings need to be specified explicitly.
The paper also lacks independently optimized, non-scalable codecs for the same access points. Such a comparison would quantify the rate–accuracy cost or benefit of supporting several prefixes within one bitstream. The context ablations in Table 3 demonstrate improvements within the layered design, but do not answer whether this design is preferable to selecting an appropriately sized non-scalable representation when the task is already known.
This distinction matters because the reported experiments evaluate each task separately, whereas a distinctive benefit of scalable coding is reuse of previously transmitted information when additional detail is subsequently requested. The paper does not evaluate this progressive-request setting or quantify its advantage over task-selected independent representations.
Furthermore, the fixed allocation of tasks to token budgets is motivated by a small set of task-level sensitivity curves. The ordering algorithm optimizes geometric diversity, and Appendix A.1 appropriately acknowledges that its guarantee does not establish task accuracy or semantic disentanglement. Consequently, the interpretation of the layers as global, object-level, and pixel-level semantics is stronger than the evidence for a fixed token-budget hierarchy.
Overall, the paper establishes a promising combined system, but does not sufficiently establish the distinctive contribution of scalability over simpler alternatives built from its component techniques.
(2) The latency evaluation is insufficient to substantiate the claimed distributed-inference advantage over relevant alternatives.
Figure 7 compares SFC against centralized JPEG/BPG pipelines. These comparisons change both representation coding and computation placement: the centralized baseline performs visual encoding in the cloud, while SFC performs visual processing and feature coding at the edge. The result is a valid comparison between complete pipelines, but it does not isolate the latency benefit of the proposed codec or establish superiority over the distributed feature-coding baselines in Figure 5.
Table 2 likewise compares reduced-token SFC against the original full-token model. It establishes a cloud-side benefit from input compaction, but does not show that scalable coding is necessary to obtain that benefit. A matched-token baseline could obtain similar prefill and KV-cache reductions.
The operating points used for Figure 7 are also insufficiently specified. The manuscript does not clearly identify the exact rate–accuracy points selected for each method or establish that their task accuracies are matched. Consequently, the reported latency reductions cannot be fully assessed as equal-quality comparisons.
Appendix A.3 states that all experiments use RTX-3090 GPUs, and the transmission analysis uses three low-bandwidth settings. This supports a particular GPU-based, bandwidth-constrained operating regime. It leaves the applicability to resource-limited edge deployment uncertain, especially because the edge must execute the visual encoder, diversity ordering, and learned feature encoder. The ordering includes sequential selection rounds, whose contribution should be measured explicitly.
A complete timing protocol should identify whether the reported latency includes token ordering, all codec transforms, actual entropy encoding and decoding, position signaling, cloud prefill, and full output generation. These details are especially important when comparing tasks with substantially different output lengths and when transmission becomes less dominant.
The current results therefore demonstrate potential latency savings against selected image-transmission pipelines, but provide insufficient evidence for the broader efficiency advantage claimed over relevant distributed feature-coding alternatives.
Questions for authors
(1) Do the rate–accuracy improvements remain when LaMoFC, DT-UFC, and TOFC use the same token ordering and the same cumulative token budgets as SFC? Please specify the token counts, training data, adaptation procedure, and model capacity used for each baseline.
(2) How does SFC compare with independently trained non-scalable codecs at each access point under matched token budgets and comparable codec capacity? What is the cost of supporting multiple prefixes, and what benefit appears when a second task requests enhancement information after the base layer has already been transmitted?
(3) How sensitive are the conclusions to the fixed layer allocations? Were these allocations selected using validation results, and do alternative allocations or simpler ordering methods achieve comparable performance under the same budgets?
(4) Which exact operating points are used in Figure 7, and are task accuracies matched across methods? What are the corresponding payload sizes and complete latency breakdowns?
(5) Does the timing include visual encoding, diversity ordering, learned transforms, actual entropy coding, position signaling, prefill, and complete output generation? How do the feature-coding baselines perform under the same hardware, bandwidth, and equal-accuracy protocol?
(6) In Table 1, does “unseen tasks during training” refer specifically to codec training? Please clarify the task exposure of the underlying Tang et al. checkpoint and distinguish codec transfer from generalization to tasks unseen by the LVLM itself.