acceptodds
Under review as a conference paper at ICLR 2027

CASH: Sensitivity-Aware Convex Head Reconstruction for Efficient Private LLM Inference

Abstract

Secure multi-party computation (MPC) enables privacy-preserving LLM inference, but its latency and communication overhead remain substantial. Despite continued progress in secure operator design and computation reduction, standard multi-head attention still evaluates each head independently, repeatedly incurring the cost of secure attention-score computation and Softmax. Our analysis reveals a pronounced low-dimensional structure across post-Softmax attention maps and shows that these maps can be effectively reconstructed from a small subset of real attention heads. We introduce CASH, a training-free framework based on these findings. CASH explicitly computes attention maps for a set of real anchor heads and reconstructs the remaining maps through fixed convex combinations. CASH further uses head-level Fisher sensitivity to allocate a global anchor-head budget and compiles the selected configuration into a static MPC execution graph. Across LLaMA-1 7B and three OPT models, CASH achieves the highest average task accuracy in 11 of 12 model–budget settings and the lowest perplexity in all 12 among the evaluated baselines. Under the evaluated configuration, CASH achieves 1.47–1.69 secure inference speedups and reduces communication by 33.6–41.9%compared with dense attention.

Primary area:
alignment, fairness, safety, privacy, and societal considerations
Keywords:
Large Language Model, Convex Head Reconstruction, Private Inference, Multi-party Computation

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_hfDYOpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
1
Soundness
1
Read review
Paper summary

The paper observed that standard multi-head attention still evaluates each head independently, which incurs the cost of secure attention-score computation and Softmax. The paper proposes a framework which computes attention maps for a set of real anchor heads and reconstructs the remaining maps through fixed convex combinations. It achieves 1.47-1.69x speedups in MPC-based secure transformer inference. Although the author states that they used generative AI tools for assisting with translation from Chinese to English, I feel the presentation is still difficult to follow.

Critical strengths

The motivation that reduce the computational overhead in multi-head attention in secure transformer inference makes sense.

The experimental results show improvements over other baselines.

Critical weaknesses

The writing needs a lot of improvements. For example, in the evaluation section, the texts in the main body just copy and paste all numbers in the table/figure again, don’t show any analysis or explanations about the evaluation results.

The work doesn’t provide the codebase for their experiments, and the reproducibility details are not listed.

The paper’s motivation is to reduce the computational overhead in MPC-based transformer inference, but the proposed methods are not focusing on MPC part. The authors don’t mention whether it also works on other secure transformer inference solutions.

Questions for authors

For the MPC-based secure transformer inference results in Table 2, could authors provide some analysis or explanations about the results? I think it can help to understand the significance of the work better, since the numbers in latency and communication overhead look similar with other works.

Could authors provide ablation studies to isolate the benefit of each proposed method?

What’s precision and nonlinear approximations are used in the MPC implementation? Will the numerical errors affect the logits, perplexity or accuracy of the proposed scheme?

4 comments

Response to Reviewer (hfDY)

Reviewer comment: “The writing needs a lot of improvements. For example, in the evaluation section, the texts in the main body just copy and paste all numbers in the table/figure again, don’t show any analysis or explanations about the evaluation results.”

We thank the reviewer for the attention to our research motivation and experimental design, as well as the suggestions for improving the readability of the paper. To improve clarity, we will further organize the logical relationship among attention reconstruction, budget allocation, and static execution in the revised manuscript. We will also strengthen the connection between the experimental observations and the method's mechanisms, while reducing repetitive descriptions of the numbers in the tables. Below, we respond point by point based on evidence already included in the original manuscript and additional experiments.

1. Results in Table 2 and Their Significance

Reviewer comment: “For the MPC-based secure transformer inference results in Table 2, could authors provide some analysis or explanations about the results? I think it can help to understand the significance of the work better, since the numbers in latency and communication overhead look similar with other works.”

The main advantage of CASH is that it preserves higher model quality at a similar secure-computation cost. Under the same exact-attention-head budget, different methods reduce a similar amount of independent attention computation, so their latency and communication costs are close. The key difference is how effectively they preserve the information carried by the replaced attention paths. Taking LLaMA-1 7B with a 50% budget and an input length of 512 as an example, the corresponding results from Tables 1 and 2 of the original manuscript are shown below. Accuracy and PPL are obtained from plaintext quality evaluation, while latency and communication are measured under MPC.

Table R1: Predictive quality and secure-inference cost under the same budget.

MethodFive-task average accuracy ↑\uparrowWikiText-2 PPL ↓\downarrowPrefill latency (s) ↓\downarrowCommunication (GB) ↓\downarrow
Dense69.67%5.636856802.6
SA50.90%24.784633529.8
HM52.32%17.514735538.4
CASH64.33%9.854663533.1

Compared with SA, CASH increases latency and communication by only approximately 0.65% and 0.62%, respectively, while improving the five-task average accuracy by 13.43 percentage points. Compared with HM, CASH achieves both higher accuracy and slightly lower secure-execution cost. Averaged across four models, the quality-retention rates are 92.0% for CASH and 73.9% for SA. These results show that similar secure-computation costs do not imply similar predictive quality; the key is how the omitted independent attention maps are represented. Relative to Dense, CASH achieves 1.471.47--1.69×1.69\times prefill speedup and reduces communication by 33.6%--41.9%, including the cost of the secure matrix multiplications required for private reconstruction. In the revised manuscript, we will organize the result analysis around this relationship between quality and cost so that the conclusions corresponding to the tables are clearer.

2. Relationship Between CASH and MPC

Reviewer comment: “The paper’s motivation is to reduce the computational overhead in MPC-based transformer inference, but the proposed methods are not focusing on MPC part.”

CASH is tailored to MPC by reducing the execution scale of expensive secure operators and avoiding input-dependent online selection overhead. Its contribution lies at the level of adapting the model computation graph to the MPC cost structure, rather than redesigning the underlying cryptographic protocol.

CASH exploits redundancy across attention heads: it performs Q/K projections, attention-score computation, and secure Softmax only for real anchor heads; reconstructs the remaining attention maps through fixed convex combinations; and retains the Value branch and output projection for every logical head. The Head-Fisher mechanism allocates the global anchor budget according to task sensitivity and reconstruction error, improving model performance under a fixed total amount of independent secure-attention computation.

This design also avoids the online interaction overhead introduced by head selection. As described in Section 3.4 and Appendix B, anchor selection, coefficient fitting, budget allocation, and head permutation are all completed offline. Online inference therefore uses a fixed computation graph and does not require head comparison, head selection, or dynamic routing that depends on private inputs. The analysis of dynamic CHAI in Figure 1(d) shows that online selection logic introduces additional interaction overhead. During inference, the reconstruction coefficients and attention maps remain private, and the secure matrix-multiplication cost introduced by reconstruction is fully included in the efficiency measurements.

Reducing MPC inference overhead by modifying model structure is an established research direction. MPCFormer (ICLR 2023) combines MPC-friendly operator approximation with knowledge distillation; MPCViT (ICCV 2023) searches attention architectures based on MPC latency; and MPCache (NeurIPS 2025) reduces private-inference overhead through MPC-friendly KV-cache eviction. CASH complements these works: it uses redundancy across attention heads to perform static reconstruction, thereby reducing the computational workload of the underlying secure operators. The actual acceleration depends on the backend's cost characteristics, and model-level modifications can be combined with lower-level secure-operator optimizations.

[1] Li et al., ICLR 2023. MPCFormer: Fast, Performant and Private Transformer Inference with MPC.

[2] Zeng et al., ICCV 2023. MPCViT: Searching for Accurate and Efficient MPC-Friendly Vision Transformer with Heterogeneous Attention.

[3] Zeng et al., NeurIPS 2025. MPCache: MPC-Friendly KV Cache Eviction for Efficient Private LLM Inference.

3. Other Secure-Inference Backends

Reviewer comment: “The authors don’t mention whether it also works on other secure transformer inference solutions.”

Section 4.3 and Table 3 of the original manuscript report cross-backend experiments with CASH on BumbleBee (2PC) and PUMA (3PC). These experiments use the LLaMA-1 7B attention module and compare secure-execution costs at an input length of 8 and total anchor budgets B∈{512,256,128}B\in\{512,256,128\}.

On PUMA, CASH achieves 1.651.65--1.95×1.95\times latency speedup and 1.491.49--2.07×2.07\times communication reduction. On BumbleBee, the corresponding ranges are 1.271.27--1.53×1.53\times for latency speedup and 1.081.08--1.15×1.15\times for communication reduction. The results show that reducing the number of independently computed attention maps is effective across multiple secure backends, although the magnitude of the benefit depends on the operator and communication costs of each backend. These experiments retain each backend's original security model and preprocessing assumptions, as described in Appendix B of the original manuscript. The results provide direct evidence that CASH applies beyond SHAFT.

4. Ablations and Diagnostics for Individual Components

Reviewer comment: “Could authors provide ablation studies to isolate the benefit of each proposed method?”

The original manuscript already includes ablation experiments and mechanism analyses for the core components, specifically in Section 4.4, Figures 8 and 5, and Appendices C and D. We clarify below how each experiment corresponds to a component of the method.

(1) Budget allocation and task-sensitivity weighting. Section 4.4 and Figure 8 compare Uniform, Recon., and Head-Fisher on LLaMA-1 7B under a fixed total budget of B=256B=256. The three methods share the same candidate anchors and reconstruction coefficients and differ only in their budget-allocation criteria. Head-Fisher achieves a five-task average accuracy of 49.98% and a PPL of 24.54; the corresponding results for Recon. are 31.95% and 2146.26, while Uniform has a PPL of 1047.41. This controlled experiment validates the effects of cross-layer budget allocation and task-sensitivity weighting. Figure 5 further shows that Head-Fisher weighting increases the Spearman correlation between the proxy score and the ranking of actual NLL changes from 0.486 to 0.866, providing a mechanism-level explanation for this improvement.

(2) Convex reconstruction. Appendix C and Table 5 compare hard sharing with convex reconstruction under the same CHAI layer-wise head configuration and a total budget of B=376B=376. Their ARC-500 accuracies are 25.6% and 32.2%, respectively, and their PPL values are 26.91 and 16.89, respectively. Under the same head-count configuration, convex reconstruction achieves better predictive quality, providing evidence complementary to the budget-allocation ablation.

(3) Static execution. Offline head permutation consistently reorders Q/K/V, the output projection, and the reconstruction coefficients, preserving the original computational correspondence and compiling the selected anchors into fixed execution positions. Its role is supported by the execution description in Section 3.4 and Appendix B, together with the MPC cost analysis in Figure 1.

Therefore, the paper provides experimental or mechanism-level evidence for budget allocation, attention reconstruction, and static secure execution. In the revised manuscript, we will further highlight the correspondence between these existing results and the design objective of each component.

5. Reproducibility and Code

Reviewer comment: “The work doesn’t provide the codebase for their experiments, and the reproducibility details are not listed.”

We have provided anonymized code and execution instructions at: https://anonymous.4open.science/r/CASH-368B/

In addition, Section 4.1 and Appendices A, B, and D of the original manuscript describe the main reproducibility settings in detail, including the evaluated models, exact-head budgets, 128 C4 calibration sequences of length 1024, evaluation tasks, hardware and network conditions, anchor search, reconstruction-coefficient fitting, Head-Fisher normalization, budget optimization, and static compilation.

The coefficients are fitted using SLSQP with a tolerance of 10−1210^{-12} and a maximum of 2,000 iterations. Appendix D also provides the baseline implementations, metric definitions, and the scope of secure-inference measurements. In the revised manuscript, we will consolidate the relevant configurations so that readers can more easily connect the code with the experimental pipeline described in the paper.

6. MPC Precision, Nonlinear Approximation, and Numerical Error

Reviewer comment: “What’s precision and nonlinear approximations are used in the MPC implementation? Will the numerical errors affect the logits, perplexity or accuracy of the proposed scheme?”

Precision and nonlinear approximation. The main experiments use the SHAFT secure backend and retain SHAFT's numerical settings: a 64-bit arithmetic ring and 16-bit fixed-point fractional precision. Secure Softmax uses an input-clipping interval of [−4,12][-4,12] and a 16-step ODE approximation, while the remaining nonlinear operations use the secure operators provided by the backend. CASH performs only MPC-friendly adaptation of the model computation graph and does not modify the underlying protocol or numerical precision.

Numerical-error analysis. We added paired experiments on four models, comparing Dense and 50%-budget CASH under native floating-point execution, a matched-operator floating-point reference, and actual MPC execution. Logits RMSE is computed relative to the matched-operator floating-point reference. Changes in PPL and accuracy are computed relative to each method's own native floating-point result, using 8,192 paired token positions and 200 ARC-Easy questions, respectively.

Table R2: Effect of the MPC numerical implementation. Each cell reports Dense / CASH; PPL is reported as a relative change, and accuracy as an absolute percentage-point change.

ModelLogits RMSEWikiText-2 PPL changeARC-Easy accuracy change (percentage points)
OPT-1.3B0.038 / 0.057+0.7% / +1.8%0.0 / −0.5-0.5
OPT-2.7B0.047 / 0.065−0.2-0.2% / +1.1%+0.5 / −1.0-1.0
OPT-6.7B0.055 / 0.096+1.4% / +4.3%−0.5-0.5 / −2.0-2.0
LLaMA-1-7B0.049 / 0.074+0.8% / +2.5%0.0 / −1.5-1.5

Across the evaluated samples, CASH's PPL increases by 1.1%--4.3% relative to its own floating-point result. When moving from floating-point execution to MPC, its accuracy decreases by an additional 0.5--1.5 percentage points compared with Dense. This result shows that different attention-computation paths respond differently to the same backend numerical settings and quantifies the additional numerical impact of CASH in actual secure execution. CASH's acceleration does not come from reducing numerical precision; it comes from reducing the number of independent secure-attention computations.

Reviewer_XHypOpenReview ↗
Rating
2
Confidence
2
Informativeness
1
Presentation
2
Soundness
2
Read review
Paper summary

The authors mainly address efficiency issues in inference under MPC. They introduce anchor attention to reduce computation in the attention mechanism. Specifically, they avoid the argmin operation during online inference by selecting anchor heads offline and then using them online. In experiments, they show that they preserve performance on QA and commonsense reasoning datasets and that their method is robust to the choice of data used for offline selection when evaluated on similar task datasets.

Critical strengths

The paper improves MPC inference speed by computing attention using anchor heads while showing better performance retention than the baselines across different head-retention ratios.

Critical weaknesses

I have concerns about generalization. Reasoning-heavy tasks such as AIME or LiveCodeBench may require more attention heads, while simpler QA tasks may need fewer. Since the head configuration is fixed offline, its practical utility may be limited across different workloads. Although the authors conduct cross dataset evaluations on QA and commonsense reasoning benchmarks, they do not evaluate more complex reasoning or conversational tasks, so this concern remains unresolved. An adaptive approach that makes MPC bottlenecks, such as argmin and reciprocal operations, more efficient might be a more promising direction, especially considering the growing interest in adaptive sparse attention methods.

Questions for authors

Please see the weakness section.

2 comments

Response to Reviewer 2

Q1: Practical utility of C4-calibrated head configurations on reasoning-heavy tasks

Reviewer comment: “Reasoning-heavy tasks such as AIME or LiveCodeBench may require more attention heads, while simpler QA tasks may need fewer. Since the head configuration is fixed offline, its practical utility may be limited across different workloads. ”

We thank the reviewer for raising the question of whether the head configurations calibrated on C4 remain practically useful on reasoning-heavy tasks. To address this concern, we conduct two complementary experiments. First, we analyze layer-wise attention rank across reasoning and coding benchmarks to examine whether the low-rank attention structure observed on C4 also holds for these task distributions. Second, we evaluate CASH on downstream tasks using head configurations calibrated exclusively on C4 to assess whether these configurations can generalize to reasoning and coding tasks without task-specific reselection.

We initially conducted experiments using Llama-1-7B, the backbone used in our original paper. However, the Dense Llama-1-7B baseline achieved near-zero scores on both AIME 2024 and LiveCodeBench v5, making it difficult to meaningfully evaluate the impact of CASH. We therefore additionally conducted experiments using Qwen3-1.7B and Qwen3-4B, both in their original Thinking mode.

Experiment 1: Cross-task consistency of layer-wise attention rank.

To examine whether the low-rank attention structure observed on C4 also holds for reasoning-heavy tasks, we compare layer-wise Rank@90 on AIME 2024 and LiveCodeBench v5 with that on C4, using Dense Qwen3-1.7B. We compute Rank@90 in head space following Appendix C, defining it as the minimum rank required to retain 90% of the squared singular-value energy.

The results are summarized below.

Input DistributionMean Rank@90Median Rank@90Spearman with C4
C43.10713.01.000
AIME 20243.03573.00.920
LiveCodeBench v53.25003.00.863

Both benchmarks exhibit comparable layer-wise rank requirements to C4, with similar mean and median Rank@90 values and strong layer-wise correlations,suggesting that the low-dimensional attention structure exploited by CASH remains applicable to reasoning-heavy and coding tasks.

Experiment 2: Functional evaluation on reasoning and coding tasks.

We further evaluate CASH on AIME 2024 and LiveCodeBench v5 using Qwen3-1.7B and Qwen3-4B, following the same benchmark evaluation protocol as the official Qwen3 Technical Report. For each model, head configuration and convex combination coefficients are determined using the same C4 calibration data and kept fixed during downstream evaluation, without task-specific reselection. We retain 50% of the exact-computation Query heads.

The results are presented below.

ModelBenchmarkDenseCASH (50%)
Qwen3-1.7BAIME 202448.3%41.2%
Qwen3-1.7BLiveCodeBench v533.2%26.9%
Qwen3-4BAIME 202473.8%69.3%
Qwen3-4BLiveCodeBench v554.2%50.7%

These results demonstrate that CASH remains applicable to more challenging reasoning and code-generation workloads across different model scales. Notably, the fixed offline anchor configurations obtained from C4 can be directly transferred to these tasks without task-specific reselection, while retaining competitive downstream performance.

Q2: On adaptive attention mechanisms

Reviewer comment: “An adaptive approach that makes MPC bottlenecks, such as argmin and reciprocal operations, more efficient might be a more promising direction, especially considering the growing interest in adaptive sparse attention methods.”

We thank the reviewer for this valuable suggestion. We agree that optimizing secure argmin and reciprocal operations could further reduce the overhead of adaptive attention mechanisms. However, CASH addresses inference overhead from a complementary perspective by reducing the number of attention branches that require independent computation. By adopting an offline configuration, CASH eliminates the need for input-dependent attention head selection during inference, thereby avoiding the associated secure argmin operations. Looking ahead, faster reciprocal and normalization protocols could further reduce the overhead of the remaining Softmax computation, while more efficient secure selection mechanisms could support a range of adaptive extensions. We view these directions as promising opportunities for further extending the capabilities of CASH.

Reviewer_StVCOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

The authors develop a training-free method, CASH, for MPC-based private LLM inference. This method uses convex combinations of anchor heads to reconstruct attention maps, and allocates global budget across layers by using estimated Fisher information of heads. Experiments show improved acceleration and reduced communication costs on Llama and OPT models.

Critical strengths
  1. The experimental results are fairly strong, as for most settings, CASH beats baseline methods at a fixed attention budget. The ablations also show that the Fisher-based budget allocation is a key factor in these improvements.

  2. I find the idea of using anchor heads to reconstruct attention maps quite novel and interesting. The static head selection method is also well-suited for MPC inference because it avoids routing operations, which often have expensive MPC protocols.

Critical weaknesses
  1. My main qualm with acceptance is the lack of realistic generation experiments. The current latency experiments only cover the prefill phase, and deal with very old models (Llama 1, OPT) that are not really used nowadays. I would like to see some experiments with real token generation, as well as see how CASH can be extended to newer architectures with GQA, MLA, or hybrid attention.
Questions for authors
  1. Can the authors provide end-to-end latency results for autoregressive decoding? I am willing to raise my score if the improvements are significant.

  2. Can CASH be extended to modern LLMs with GQA or hybrid attention schemes, such as those in recent Qwen models? How are the speedups and communication costs affected?

5 comments

Response to Reviewer 1

Q1. End‑to‑end latency evaluation for autoregressive generation

Reviewer comment: “Could the authors provide end-to-end latency for autoregressive decoding?”

Thank you for suggesting an end-to-end evaluation of autoregressive generation. We have completed end-to-end measurements using the two-party SHAFT framework. Table 1 reports the results for four models with output lengths of 4, 8, and 16 tokens. When the output length increases to 16 tokens, including 15 subsequent cached decoding steps, CASH still achieves a 1.1991.199--1.360×1.360\times speedup for the complete request, corresponding to a 16.6%--26.5% reduction in latency and a 16.71%--27.53% reduction in communication.

All experiments use a fixed input length of 512 tokens, a batch size of 1, and a 50% exact-head budget. The network setting uses 1 Gbps bandwidth and a 0.5 ms RTT. Timing begins with input processing and ends when the final generated token is revealed to the client. It covers one prefill and first-token generation step, followed by N−1N-1 subsequent decoding steps that reuse the secret-shared KV cache.

Table 1. End-to-end autoregressive-generation latency and communication under different output lengths. Each pair of absolute values is reported as Dense / CASH.

ModelOutput tokensEnd-to-end latency (s)SpeedupCommunication (GB)Communication reduction
OPT-1.3B44601.1 / 2924.61.573×546.54 / 341.4437.53%
OPT-1.3B85466.7 / 3798.31.439×644.51 / 430.5833.19%
OPT-1.3B167403.0 / 5442.61.360×841.58 / 609.8927.53%
OPT-2.7B46761.9 / 4566.71.481×804.31 / 525.6634.64%
OPT-2.7B88560.8 / 6076.11.409×993.19 / 697.6529.76%
OPT-2.7B1611921.7 / 9296.31.282×1373.14 / 1043.6224.00%
OPT-6.7B49560.9 / 7025.91.361×1122.34 / 821.7326.78%
OPT-6.7B813248.3 / 10566.41.254×1577.17 / 1237.8221.52%
OPT-6.7B1621007.2 / 17375.81.209×2492.12 / 2074.8516.74%
LLaMA-1-7B49805.8 / 7296.71.344×1137.02 / 838.5726.25%
LLaMA-1-7B813614.4 / 10970.01.241×1584.41 / 1247.2421.28%
LLaMA-1-7B1621627.3 / 18034.41.199×2484.40 / 2069.3416.71%

CASH reduces both end-to-end latency and communication in all 12 model--output-length configurations. The relative speedup decreases as the output length grows, which is consistent with the optimized computation path: prefill processes multiple query positions simultaneously and therefore benefits more fully from reducing independent attention computation, whereas cached decoding processes only one new query at each step, making the retained FFN, value aggregation, and output processing account for a larger proportion of the total cost. By extending the evaluation from the prefill stage to the complete autoregressive-generation pipeline, these results show that CASH continues to provide end-to-end performance benefits for short-output requests even after decoding and output-processing costs are included.

Q2. Applicability of CASH under GQA and hybrid‑attention architectures

Reviewer comment: “Can CASH scale to modern models with GQA or hybrid attention, and how do latency and communication change?”

CASH can be applied to GQA and to modules in hybrid architectures that contain explicit post-Softmax attention maps. We implemented and validated the complete CASH pipeline on Qwen3-1.7B, including convex reconstruction with real anchors, Head-Fisher-guided global budget allocation, and static secure execution. For Qwen3.5-0.8B, we further evaluated the cost of the full-attention subgraphs to which CASH can be applied.

GQA: Qwen3-1.7B

GQA shares K/V projections, but different query heads still produce their own attention-probability maps. CASH selects anchors and performs convex reconstruction over these maps. Combinations across KV groups are performed at the probability-map level, and each reconstructed map is still multiplied by the V tensor of the group to which the corresponding logical head originally belonged. This preserves the original K/V sharing structure and the corresponding output branches.

Qwen3-1.7B contains 28 layers, with 16 query heads and 8 KV heads per layer. Under a 50% budget, CASH uses the Head-Fisher-weighted reconstruction error on the calibration data to allocate 224 exact query heads across the model. Anchors may be combined across KV groups within the same layer, while all K/V groups are retained. Table 2 reports the quality of the complete CASH method under different budgets.

Table 2. Plaintext quality results of CASH on Qwen3-1.7B. Exact query heads denote the total number across the model.

ConfigurationExact query headsRetention ratioWikiText-2 PPL ↓\downarrowARC-Easy ↑\uparrow
Dense448100%21.6667.0%
CASH42494.64%22.2470.5%
CASH39287.50%23.9863.0%
CASH22450.00%25.9859.0%

Table 2 shows how quality changes under different budgets. The subsequent efficiency evaluation uses the same 50% configuration, whose PPL is 25.98 and ARC-Easy accuracy is 59.0%, compared with 21.66 and 67.0% for Dense. Using SHAFT, we measure the complete-request cost of this configuration with a 512-token input and a 4-token output under the same network conditions as in Q1. The measurement covers the complete generation pipeline and includes private convex reconstruction with the fitted coefficients, the complete K/V paths, value aggregation, secure token selection, and output revelation.

Table 3. Generation cost of Qwen3-1.7B under a 50% exact-head budget. Input length: 512 tokens; output length: 4 tokens. Each pair of absolute values is reported as Dense / CASH.

StageLatency (s), Dense / CASHSpeedupCommunication (GB), Dense / CASHCommunication reduction
Complete four-token request4140.45 / 3168.891.31×1.31\times451.07 / 330.8126.7%
Prefill and first token3166.19 / 2229.221.42×1.42\times348.33 / 232.2833.3%
Three subsequent cached decoding steps974.27 / 939.661.04×1.04\times102.74 / 98.534.1%

Under this configuration, CASH achieves a 1.31×1.31\times speedup for the complete request on Qwen3-1.7B, reducing latency by 23.5% and communication by 26.7%, showing that its computational optimization extends to modern models using GQA. The stage-wise breakdown shows that the benefit mainly comes from the prefill stage, which processes multiple query positions. In contrast, cached decoding processes only one new query at each step, while the original K/V sharing, value aggregation, FFN, and other computations must still be retained. Consequently, the speedup of subsequent decoding alone is smaller, but the complete generation request still obtains a clear efficiency improvement.

Hybrid Attention: Qwen3.5-0.8B

Qwen3.5-0.8B uses a hybrid architecture with 6 full-attention layers and 18 Gated DeltaNet layers. CASH targets redundancy in Softmax-based attention-map computation, whereas Gated DeltaNet uses a different state-update mechanism and does not explicitly produce the Softmax attention maps reconstructed in this paper. The two types of modules therefore involve different MPC optimization problems. We apply CASH to the full-attention layers and leave the computation of Gated DeltaNet unchanged.

Table 4. Cost evaluation of the six full-attention subgraphs in Qwen3.5-0.8B. Each pair of absolute values is reported as Dense / CASH with a 50% budget.

Evaluation scopeLatency (s), Dense / CASHSpeedupCommunication (GB), Dense / CASHCommunication reduction
Six full-attention subgraphs309.90 / 202.251.53×1.53\times34.30 / 21.1138%

This result demonstrates the benefit of CASH on the attention subgraphs that can be optimized in a hybrid architecture: these subgraphs achieve a 1.53×1.53\times speedup and reduce communication by approximately 38%. The end-to-end latency and communication savings for the complete model are further determined by the proportions of the total cost attributable to these subgraphs relative to Gated DeltaNet, the FFN, and input/output processing. Optimizing the secure state updates of Gated DeltaNet is a complementary research direction that merits future study.

Together, these results extend the original evaluation in two complementary ways: end-to-end generation with the complete CASH pipeline on a GQA model, and targeted secure-cost measurements on the Softmax-attention component of a hybrid model.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.