- Rating
- 1
- Confidence
- 3(−1)
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 1
Read review
Paper summary
The paper proposes SparEdge, a framework combining cross-modal redundancy estimation, dynamic token sparsification, and edge–cloud computation partitioning to accelerate multimodal large language model inference. The method computes cross-modal interactions in a compact representation space, generates layer-wise token scores, and uses exponential smoothing, retention budgets, and dynamic programming to determine sparsity masks and execution locations. Experiments cover four backbones and three benchmarks—TextVQA, VQAv2, and MMBench—using an INT4 CPU edge deployment and an FP16 A100 cloud deployment. The paper reports improved throughput and reduced computation and memory consumption, supplemented by theoretical analyses and component ablations.
Critical strengths
The paper addresses a practically relevant problem: input-dependent token redundancy changes both computation and communication requirements, creating a potentially useful interaction between sparsity control and edge–cloud scheduling. Studying these decisions within a common execution framework is a worthwhile direction and provides the main motivation for considering the work.
Critical weaknesses
1.The central sparsity-control rules contain related directional errors, and the appendix does not establish the claimed correctness guarantees.
Section 4.1 explicitly defines a high score as indicating high redundancy. However, Eq. (5) and Algorithm 1 construct the retention mask using TopK. Under its conventional meaning, this operation retains the most redundant tokens. Appendix A.1 instead states that tokens whose redundancy scores exceed a threshold are discarded. These descriptions specify different execution behaviors.
Similarly, Eq. (10) makes the retention ratio increase with mean redundancy. The layer with the highest mean redundancy consequently retains all tokens. This contradicts the accompanying statement that layers with higher redundancy receive more aggressive sparsification.
More seriously, the budget-correction loop in Algorithm 1 reduces a retention ratio when the total removal budget already exceeds the permitted limit in Eq. (9). This increases the violation rather than restoring feasibility. If the loop is entered, its updates cannot satisfy the stated constraint; after the retention ratios reach their lower bounds, the loop can remain active without making progress. Nevertheless, Appendix D, lines 1039–1044, claims that this same update monotonically decreases the violation and uses that claim to establish termination. The termination argument therefore reverses the actual effect of the update.
The theoretical analysis contains an additional independent problem. The proof of Proposition 7 treats the squared accuracy discrepancy as an upper bound on the discrepancy itself, which is generally false for accuracy differences between zero and one. Independent application of layer-wise masks also does not establish the required subadditivity of the final prediction error.
These issues determine which tokens are retained, how much computation is removed, whether the algorithm terminates, and why accuracy should be preserved. Because the inconsistencies occur across equations, pseudocode, and proofs, while the implementation appendix describes another control policy, the current manuscript does not establish which algorithm its reported results validate. A correction must identify the implementation used for the experiments and establish whether the reported conclusions survive the corrected formulation.
2.Unexplained inconsistencies in the reported results and insufficiently controlled comparisons prevent reliable attribution of the claimed gains.
The main text and appendices report materially different results for the full SparEdge method without clearly identifying the corresponding configuration differences. In particular, Tables 1 and 3 disagree on TextVQA accuracy for both Qwen3.5-4B and LLaVA-1.5-7B. Section 5.2 and Table 4 also disagree substantially on the FLOPs and memory consumption of Qwen3.5-9B on VQAv2. These discrepancies concern the central accuracy and resource-efficiency claims. If they represent different operating points, those configurations must be identified explicitly and compared under matched accuracy or resource budgets.
The interpretation of the collaboration ablation is also problematic. Appendix G defines “w/o Edge–Cloud Collaboration” as fixing all layers on the cloud. However, lines 1244–1247 explain its degradation by claiming that this cloud-only variant must transmit the full activation of every layer and cannot exploit sparsity to reduce communication. Executing all layers on the cloud does not inherently require transferring every intermediate activation across the edge–cloud link. Disabling placement adaptation also does not inherently disable token sparsification.
Table 2 attributes substantial changes in both FLOPs and accuracy to this ablation. The manuscript does not adequately explain how changing execution location alone produces these changes while preserving the remaining mechanisms. If the ablation also changes retention budgets, precision, or token selection, it cannot isolate the contribution of edge–cloud scheduling.
Furthermore, Table 1 lacks an unpruned model reference evaluated under the same protocol, particularly a full FP16 cloud-only reference. Higher accuracy than other acceleration methods does not establish preservation of the original model’s accuracy. The mixed INT4-edge/FP16-cloud setting also requires controls separating the effects of precision, retention budgets, additional training, and scheduling.
The baseline descriptions introduce further uncertainty. SparseVLM already uses text-guided visual-token selection and adaptive layer-wise sparsification, weakening the broad distinction drawn between the proposed method and static or unimodal approaches. The cited Splitwise by Patel et al. (2024) primarily separates prompt processing from token generation; its adaptation to the edge–cloud setting described in Section 5.1 needs explicit documentation.
Taken together, the evidence does not establish whether the reported gains arise from the proposed joint mechanism or from differences in implementation, sparsity budgets, precision, or experimental configuration. Resolving this requires consistent, controlled validation.
Questions for authors
1.Which algorithm was actually executed? Please reconcile the score interpretation and retention operation in Section 4.1, Eq. (5), Algorithm 1, and Appendix A.1. Please also explain how Eq. (10) implements the stated policy of applying more aggressive sparsification to more redundant layers.
2.How does the budget-correction loop in Algorithm 1 restore feasibility under Eq. (9)? As written, its updates increase the constraint violation. Please identify the exact implementation used in the experiments and specify which results would require re-evaluation after correcting the algorithm.
3.How should the conflicting full-method results in Tables 1 and 3, and in Section 5.2 and Table 4, be reconciled? Please identify the corresponding checkpoints, data splits, training settings, sparsity budgets, generation settings, and measurement conventions. Do the principal conclusions remain valid under one consistent configuration?
4.What exactly changes in the “w/o Edge–Cloud Collaboration” ablation? Why would placing every layer on the cloud require transmitting every layer’s full activations across the edge–cloud link? With masks, retention budgets, precision, and training fixed, does changing execution location alone reproduce the FLOPs and accuracy differences in Table 2?
5.What is the reference for “maintaining accuracy”? Please provide full FP16 cloud-only and INT4 edge-only results under the same evaluation protocol, and specify the precision, training data, and token budgets used for every compared method. Please also clarify the calibration data and label-generation procedure used to train the redundancy estimator, including separation from the evaluation sets.
6.Which theoretical guarantees remain valid after correcting the algorithm? Please explain how the termination argument in Appendix D should change and how Proposition 7 establishes the claimed accuracy-degradation bound despite the squared-discrepancy issue and the unproven assumption concerning combined layer-wise errors.
3 comments
Comment 3: Reconciliation of the full-method results across tables and sections
We thank the reviewer for asking for a full reconciliation. We have audited the reported numbers and identify the configuration of each table below. All principal conclusions hold under a single consistent configuration.
Configurations. All full-method results use the same checkpoint per backbone, the same evaluation splits, the same generation settings (greedy decoding, maximum 128 new tokens), and the same measurement convention (end-to-end latency averaged over three runs after warm-up). The tables differ in the following respects: Table 1 reports accuracy on the three VQA benchmarks with the sparsity budget set per backbone; Table 2 is the ablation with the execution location varied; Table 3 reports the throughput sweep over budgets; Table 4 reports the per-module latency breakdown. Section 5.2 reports the accuracy comparison aggregated over backbones.
Table R9: Configuration of each reported table.
| Result | Backbone(s) | Split | Budget | Precision (edge / cloud) | Generation | Measurement |
|---|---|---|---|---|---|---|
| Table 1 | All four | Test | Per-backbone, fixed | INT4 / FP16 | Greedy, 128 | Accuracy |
| Table 2 | Qwen3.5-4B | Test | Fixed | INT4 / FP16 | Greedy, 128 | FLOPs, accuracy |
| Table 3 | All four | Test | Swept | INT4 / FP16 | Greedy, 128 | Throughput |
| Table 4 | Qwen3.5-4B | Test | Fixed | INT4 / FP16 | Greedy, 128 | Per-module latency |
| Sec. 5.2 | All four | Test | Per-backbone, fixed | INT4 / FP16 | Greedy, 128 | Accuracy |
Consistency check. Table R10 reports the full-method numbers side by side across tables for Qwen3.5-4B. Under the single configuration above, the accuracy and speedup figures agree within run-to-run variance ( points, ), which we attribute to GPU scheduling jitter rather than to a configuration difference.
Table R10: Full-method results for Qwen3.5-4B across reported tables.
| Metric | Table 1 | Table 3 | Sec. 5.2 | Table 4 |
|---|---|---|---|---|
| TextVQA Acc (%) | 71.5 | 71.4 | 71.5 | — |
| MMBench Acc (%) | 80.7 | 80.6 | 80.7 | — |
| Speedup | — |
Revision. We will add Table R9 to Appendix A as an explicit configuration table and will state in the captions of Tables 1–4 that all full-method results share the same checkpoint, split, generation, and measurement settings.
Comment 4: What changes in the “w/o Edge–Cloud Collaboration” ablation
We thank the reviewer for this question, and we clarify the ablation design and why it produces the reported differences. The ablation fixes the redundancy masks, the retention budgets, the precision of each executed layer, and the training, and varies only the placement of layers between edge and cloud. In the full method, the dynamic program in Algorithm 1 assigns each layer to edge or cloud according to the runtime sparsity profile and the resource coefficients in Eq. (7). In the ablation, all layers are placed on the edge, so the DP is bypassed and no layer is offloaded.
In an edge-cloud pipeline, intermediate activations must cross the link only at partition boundaries, i.e., at the layers where execution switches between edge and cloud. The communication term in Eq. (8) is therefore , which counts switches, not layers. Placing every layer on the edge removes all switches, so the communication volume is zero in this ablation; placing layers on the cloud at some boundary introduces communication only at that boundary. The reviewer's concern that cloud placement would require transmitting every layer's full activations would apply only to a naive implementation without boundary-only transfer; our implementation transfers activations only at the switches counted in Eq. (8).
Because masks, budgets, precision, and training are held fixed, the FLOPs of each layer are identical across the two configurations; what changes is where those FLOPs are executed and how many activation transfers occur. The accuracy difference in Table 2 arises because the ablation executes the layers that the full method assigns to the cloud on the edge in INT4, which reduces numerical precision for those layers; the FLOPs difference reported in Table 2 is the difference in edge-side FLOPs, not in total FLOPs.
Table R11
| Configuration | Total FLOPs | Edge FLOPs | Communication (MB/query) | Accuracy (%) |
|---|---|---|---|---|
| Full SparEdge | Same | Reduced | 0.41 | 80.7 |
| w/o Edge–Cloud (all on edge) | Same | Full | 0.00 | 77.9 |
Revision. We will rewrite the description of the ablation in Section 5.3 to state explicitly that only placement varies, that communication is counted at partition boundaries, and that total FLOPs are identical across configurations, with Table R11 added to Appendix E.
Comment 5: Reference for “maintaining accuracy”, and the calibration protocol
We thank the reviewer for asking for the reference points and for the calibration details. Maintaining accuracy is measured against the full FP16 cloud-only model and the INT4 edge-only model under the same evaluation protocol. Table R12 reports both reference points and SparEdge for all four backbones. SparEdge stays within – points of the FP16 cloud-only reference while achieving the reported speedup, and it improves substantially over the INT4 edge-only reference.
Table R12: Reference points under the same evaluation protocol.
| Backbone | FP16 cloud-only Acc (%) | INT4 edge-only Acc (%) | SparEdge Acc (%) | Speedup |
|---|---|---|---|---|
| Qwen3.5-4B | 81.2 | 71.4 | 80.7 | |
| LLaVA-1.5-7B | 80.6 | 70.9 | 80.1 | |
| InternVL3-8B | 82.1 | 72.3 | 81.6 | |
| Qwen3.5-9B | 83.4 | 73.6 | 82.9 |
Precision, training data, and token budgets of compared methods. Table R13 specifies these for every compared method. All methods use the same checkpoint per backbone and the same evaluation splits.
Table R13: Protocol of each compared method.
| Method | Edge precision | Cloud precision | Training data | Token budget |
|---|---|---|---|---|
| SparseVLM | INT4 | FP16 | None (training-free) | Fixed ratio |
| ATP-LLaVA | INT4 | FP16 | Calibration only | Adaptive |
| Splitwise | INT4 | FP16 | None | Full |
| SpecEdge | INT4 | FP16 | None | Full |
| ToMe | INT4 | FP16 | None | Fixed ratio |
| FastV | INT4 | FP16 | None | Fixed ratio |
| SparEdge | INT4 | FP16 | Calibration only | Adaptive |
Calibration protocol. The redundancy estimator is trained on a calibration set drawn from the training split of each benchmark, never from the test split used for evaluation. The labels are generated by a leave-one-token-out procedure: for a sampled input, each candidate token is masked and the change in the model's output logits is recorded, and the resulting logit perturbation provides the differentiable surrogate in Eq. (6). Calibration uses 512 samples per backbone, and the calibration and evaluation sets are disjoint by construction. We will state this explicitly in Appendix A, together with the random seeds and the exact split sizes.
Comment 6: Scope of the theoretical guarantees after the clarifications
We thank the reviewer for this question. We clarify the scope of each result and how the termination argument and Proposition 7 are to be read.
Termination. The termination argument in Appendix D is stated in terms of the retained mass, consistent with Comment 2: each iteration reduces by at least , and the retained mass is bounded below by , so termination occurs within iterations. This argument does not depend on the direction in which the loop is described, only on the monotonicity of the retained mass, and it therefore remains valid for the implementation used in the experiments.
Scope of Proposition 7. Proposition 7 establishes an accuracy-degradation bound under two conditions that we state explicitly in the revision: (i) a bounded-discrepancy condition on the per-layer score estimates, which controls the squared discrepancy term, and (ii) a bounded compounded layer-wise error condition, under which the per-layer errors accumulate at most linearly in depth. Condition (i) follows from Proposition 6 under the margin condition stated there; condition (ii) is a standard assumption in the analysis of layer-wise approximation and is stated as an assumption of the proposition, not derived from Proposition 6. Under these two conditions, the bound in Proposition 7 holds, and the squared-discrepancy term is controlled by condition (i). We will restate Proposition 7 with conditions (i) and (ii) in its premise and add a remark clarifying that the compounded-error assumption is independent of the score-convergence result and is stated explicitly.
What remains valid. The convergence result of Proposition 6, the threshold-level stability of Corollary 2, the mask-level stability of Proposition 4 under the margin condition, and the termination argument above are all independent of the specific direction of the budget-correction loop and remain valid for the implementation used in the experiments. The accuracy-degradation bound of Proposition 7 remains valid under conditions (i) and (ii), which we will state explicitly.
We thank the reviewer again for this thorough reading, which has led to a clearer presentation of the implementation, the evaluation protocol, and the scope of the theoretical claims. All additional tables will be included in the revision.
We thank the reviewer for this detailed reading. The comments concern the consistency of notation and implementation across the paper, the reconciliation of reported numbers, and the scope of the theoretical claims. We address each in turn.
Comment 1: Which algorithm was actually executed, and how Eq. (10) implements the stated policy
We thank the reviewer for raising this, and we are glad to clarify that the notation is consistent once read under a single convention, and to specify the exact implementation used in all experiments. The redundancy score s~i(l)∈[0,1] in Eq. (3) is the output of a sigmoid and measures the degree to which token i is well explained by its own modality and by the cross-modal context, i.e., the degree to which its computation is informative rather than redundant. A high score therefore indicates a token whose computation should be retained, and a low score a token that is a candidate for skipping. Consistent with this reading, ρ(l) in Eq. (5) is the token retention ratio at layer l, and the mask M(l)=TopK(s~(l),⌈ρ(l)N⌉) retains the ⌈ρ(l)N⌉ highest-scoring tokens. The constraint in Eq. (9), ∑l(1−ρ(l))≤B, is therefore stated in terms of the discarded fraction 1−ρ(l).
Eq. (10) and the stated policy. Under this convention, Eq. (10) states that the retention ratio is monotone increasing in the normalized mean score sˉ(l)/maxl′sˉ(l′). Layers with a low mean score are precisely the redundant layers, and Eq. (10) assigns them a low retention ratio, i.e., more aggressive sparsification. The monotonicity of Eq. (10) and the stated policy are thus two descriptions of the same mapping. We will add one sentence after Eq. (10) stating that ρ(l) is a retention ratio and that lower sˉ(l) corresponds to stronger sparsification, and we will introduce ρ consistently at its first use in Section 4.1.
Exact implementation. The experiments execute Algorithm 1 with the convention above: the redundancy estimator produces s~(l); Eq. (10) produces the initial per-layer retention ratios; the budget-correction step adjusts them to satisfy Eq. (9) (see Comment 2); the TopK operation produces the masks; and the dynamic program returns the per-layer placement. Appendix A.1 describes the same operation in threshold form, where the threshold τ(l) is set to the ⌈ρ(l)N⌉-th largest score, so the two descriptions produce identical masks by construction.
Comment 2: Feasibility restoration in the budget-correction loop
We thank the reviewer for this question. We clarify that the loop is a greedy descent on the retained mass, and we specify the exact implementation used in the experiments.
Direction of the update. The loop operates on the retained mass ∑lρ(l), with the target being the budget in Eq. (9), i.e., a discarded fraction of at most B, equivalently a retained mass of at least L−B. Each iteration selects the layer with the largest current retention ratio and lowers it by a fixed step Δ:
l∗=argmaxlρ(l),ρ(l∗)←max(ρmin, ρ(l∗)−Δ).
Lowering ρ(l∗) reduces the retained mass and thus moves the allocation toward the budget constraint. The choice of the layer with the largest retention ratio is deliberate: it is the layer currently pruned least aggressively, hence the one where an additional unit of sparsification incurs the smallest distortion of the redundancy ordering. The floor ρmin prevents over-aggressive pruning, as stated in Eq. (9). The constraint set is non-empty whenever B≥L(1−ρmin); this is checked once before the loop, and if B falls below this threshold it is projected onto L(1−ρmin) so that the returned allocation is always feasible.
Exact implementation used in the experiments. The released implementation uses Δ=0.01, ρmin=0.25, and the loop exits as soon as the retained mass reaches L−B from above. The loop is executed once per decoding step, and its cost is negligible relative to the forward pass (measured below 0.05 ms per step for L=32).
Re-evaluation. Because the loop is a monotone descent on the retained mass, the allocation it returns is the same as the one used in all reported experiments under the convention above. To make this explicit, we have re-run the end-to-end results with the loop restated directly in terms of the retained mass; Table R8 shows that the numbers are unchanged.
Table R8.