acceptodds
Under review as a conference paper at ICLR 2027

Scaling Up FHE-Based LLM Inference: Higher Throughput and Longer Inputs

Abstract

Fully homomorphic encryption (FHE) enables non-interactive confidential inference of large language models (LLMs), but existing solutions scale poorly: they are limited to small models or to a few input tokens, and activation outliers force high-degree polynomial approximations of non-linear layers. We scale up FHE-based LLM inference in two directions. First, for fully encrypted prompts, we suppress outliers without retraining via attention-sink prefixes and additional orthogonal rotations, introduce a numerically stable sum-of-squares polynomial evaluation for sparsely packed ciphertexts that speeds up SoftMax, and combine these with fast BLAS-based homomorphic linear algebra whose encoding conversions are mostly fused into bootstrapping. Second, when a long context is public and only the query is sensitive, we process the public prefix in the clear and encrypt only the query; the resulting wide plaintext-ciphertext attention is evaluated without bootstrapping on its wide path, using a depth-1 matrix multiplication and a shallow SoftMax. Our implementation, Sylph, runs the prefill of Llama-3.1-8B on a 128-token encrypted prompt in 20s on eight RTX PRO 6000 GPUs, versus 134s for the prior state of the art on eight B200 GPUs (5.8x faster on the same four B200 GPUs), and processes a 4096-token prompt whose last 128 tokens are encrypted in 64s.

Primary area:
alignment, fairness, safety, privacy, and societal considerations
Keywords:
Privacy-preserving machine learning, homomorphic encryption, LLM inference

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_4KrkOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper targets on scaling up FHE-based LLM inference to decoder models like Llama-3.1-8B. The paper scales up it in two directions: 1) fully encrypted prompts and 2) a long context is public and only the query is sensitive. The implementation results show the proposed scheme runs the prefill of Llama-3.1-8B in 20 seconds on 8 GPUs. The motivation is good, but I feel the presentations of technical contributions are not very clearly.

Critical strengths
  1. The presentation is clear. The figures are well designed and helpful for understanding the scheme.

  2. Although the source code is not allowed to redistribute, the paper provided executable binaries and detailed descriptions of proposed algorithms.

Critical weaknesses
  1. The baselines are limited. For end-to-end evaluation, as far as I know, there are few works also work on FHE-based inference of decoding model, like Cachemir[1]. For the matrix multiplications, authors compared their scheme with JKLS algorithm and Powerformer, but some works contributed in this part are not mentioned and compared, like Tricycle[2]

[1] Cachemir: Fully homomorphic encrypted inference of generative large language model with kv cache. arXiv preprint arXiv:2602.11470, 2026.

[2] Tricycle: Private transformer inference with tricyclic encodings. CCS 2026

  1. The comparisons with the baseline are running in different library and different GPU, while the authors don’t provide a benchmark on the hardware and library, so the comparison results are not clear to me.

  2. The work builds on existing works and the improvement is incremental, such as the ring switch and the coeff-encoding PCMM are all directly used from existing works. And this work is also not the first one to considering using coeff-encoding PCMM in transformer inference, which is explored in LEAF [3].

[3] LEAF: A low-latency evaluation architecture for feedforward block in privacy-preserving transformer inference. Cryptology ePrint Archive, Paper 2025/956, 2025.

Questions for authors
  1. The paper only provides overall run time and a simple analysis of where the speedup comes from, which looks not clear to me. I think the time cost, ciphertext depth, and comparison with existing works of each step in an end-to-end evaluation flow can help to understand the improvement better, like the one in NEXUS and MOAI.

  2. N=2^8 degree is used in the coeff-encoding PCMM, which is much smaller than widely adopted parameters like N=2^16. I’m interested in what the plaintext modulus, ciphertext modulus and scale are used in this case and how about the error of this step.

1 comment

We thank you for your careful reading. We will add more kernel-level comparison and move Table 2 to the main text. As Cerium (SOSP 2026) explains, FHE-based Llama inference raises significant system problems; to our knowledge, Sylph is the first to run the complete chain of prefill, decoding and token feedback (homomorphic token selection and embedding lookup) end to end on real GPUs.

W1. We will add both works as baselines: 'Cachemir' measures individual kernels and derives its end-to-end figure from them: 3.02 s per decoder block on one A100-80GB (customized Phantom library, sequence length 512), i.e. ~100 s per token. It does not cover the token feedback, and it reports 694 GB of encoded weights without describing how a full run fits on an 80 GB GPU. Sylph’s figures are measured on end-to-end encrypted runs. Per token, the 32 layers take 24 s on one B200 or 31 s on one RTX PRO 6000 after a 128-token prompt, and 22 s on eight RTX PRO 6000 with 3968 further public tokens (Table 7); the final RMSNorm, LM head, encrypted token selection and embedding lookup add ≈2.2 s (App. F). The converted weights take 58 GiB (Table 9).

Tricycle. Its tricyclic encoding also gives depth-1 slot-encoded matmuls with O(n)O(\sqrt{n}) rotations, so it is the right comparison for our depth-1 PCMM (Sec. 4.2); the asymptotics are comparable, and we will add it next to JKLS and Powerformer. Unlike tricyclic encodings, which need their own packing with coprime dimensions, ours keeps the slot layout of the rest of the pipeline.

Tricycle is end-to-end only on BERT; at Llama-2-7B dimensions it reports matmul micro-benchmarks (one layer, 32 tokens, one A100, HEaaN v1): 7.3–12.1 s for QKV and 0.5–1.0 s for QKᵀ, noting that its PCMM cost grows superlinearly with the matrix dimension. In Sylph (BLAS-based PCMM for projections), at 128 tokens on one RTX PRO 6000, QKV takes 326 ms , QK 316 ms, and a complete layer including bootstrapping 3.55 s (Table 11). As GPUs and libraries differ, we will report both works side by side with their settings, not as speedups.

W2. The paper has three controls:

  • Same hardware. On four B200 GPUs, Sylph’s prefill takes 26 s vs. 152 s reported by Cerium (5.8×; Table 7).
  • Same library and GPU. Fig. 4a runs the linear-algebra algorithms taken from Cerium's released artifact on our library; ours are 1.8–3.3× faster.
  • Library alone: at identical bootstrapping shape on one RTX PRO 6000, 16.96 ms vs. 23.5 ms with Cerium's artifact (1.39×, line 437).

W3. We do not claim coefficient-encoded PCMM or its ring switching. LEAF adopts the same PCMM for BERT-base's feed-forward block in isolation (BFV, on CPU), and notes that using it inside a slot-encoded CKKS pipeline is hard because the conversions offset much of the gain. We will cite it.

Our contribution is the global integration that lets each step use its best algorithm. Given the rigidity of FHE ciphertexts, this requires: layouts and conversions: packing layouts that absorb the bit reversal (App. E) and SinC-preserving ring switching (App. G), both new; circuit design (levels, bootstrapping placement) driven by range calibration, which speeds up RMSNorm by 15× and SoftMax by 2.2×, plus slim sum-of-squares evaluation (a further 1.25×); deployment: GPU memory, parallelism and communication (6.5× per layer on 8 GPUs). Correctness is part of this: Sylph matches FP16 perplexity under FHE, while the released approximations of Cerium and MOAI receive out-of-domain inputs on Llama-3 (App. L).

Q1. Per-layer prefill at 128 tokens on RTX PRO 6000. Step times (Table 11) include the format conversions and bootstrapping attributed to each step. Kernel times (Fig. 4a–b; one GPU, same library) exclude them; "prior" is Cerium's algorithm for projections, JKLS for QK + ScoreV, and prior-work input ranges for non-linear layers.

StepStep, 1 / 8 GPUs (ms)Kernel, ours (ms)Kernel, prior (ms)
RMSNorm (×2)40 / 3910 each147 each
QKV proj.326 / 6574135
RoPE35 / 12
QK + ScoreV768 / 10462203
SoftMax440 / 813711040
O proj.315 / 4050100
Gate/up proj.1040 / 132248450
SiLU198 / 27
Down proj.391 / 50102315
Total3550 / 550

Levels: PCMM runs at the lowest level, the depth-1 PCMM uses one level, the SoftMax main track 8, and bootstrapping returns 14; we will add a level column. We compare kernels within one library because papers attribute bootstrapping to steps differently.

Q2. N = 2⁸ is used only for the PCMM, on Module-LWE ciphertexts of rank k = 16, so the underlying lattice dimension is kN = 2¹². CKKS has no plaintext modulus; the precision is set by the scaling factor, Δ = 2³⁸ on a 53-bit base modulus, with log PQ = 106 at this step (≥128-bit security, lattice estimator). The measured precision of this step is 15 bits; 12 bits suffice to match FP16 perplexity (Table 4).

Reviewer_q35xOpenReview ↗
Rating
3
Confidence
1
Informativeness
3
Presentation
3
Soundness
3
Read review
Paper summary

The authors mainly address the bottlenecks in inference under FHE. They fuse encoding conversions with bootstrapping, make polynomial evaluation more efficient by reducing the approximation ranges, and optimize attention over public contexts. WIth these methods, they reduce latency significanlty.

Critical strengths

I'm not familiar with FHE protocols, but as I understand it, the paper reduces bottlenecks through conversion fusion, makes polynomial evaluation more efficient by reducing the approximation ranges, and speeds up public attention through further optimizations.

Critical weaknesses

The idea of prefilling public context in plaintext and reusing it is not novel. Similar approaches have already been proposed in MPC, where non-sensitive parts are prefilled in plaintext and only the sensitive parts are processed securely. [1]

[1] Song, J., Park, J., & Yang, E. (2026). Secure autoregressive inference with prompt separation via key-value caching.

Questions for authors

Please see the weakness.

1 comment
AuthorsOpenReview ↗

Thank you for the positive evaluation

We thank you for your positive evaluation of our work. We do not claim the idea of processing a public context in the clear: Sec. 4.1 notes that it is standard prefill applied to two chunks, and we will cite and discuss Song et al. (2026) in the final version.

Reviewer_jtULOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

Sylph runs Llama-3.1-8B prefill and decoding under CKKS on multiple GPUs, using HEaaN2. To keep activation ranges below 40, it adds an attention-sink prefix and fuses in QuaRot rotations. The linear algebra uses BLAS-based PCMM and batch CCMM, with encoding conversions fused into bootstrapping and a packing layout that absorbs the bit reversal. The SoftMax auxiliary-track polynomials are evaluated on sparse ciphertexts using a sum-of-squares decomposition. Finally, the paper introduces "heterogeneous prefill". A public prefix is prefilled in the clear and only the private query is encrypted, with the PC-attention computed through a depth-1 JKLS variant that avoids wide bootstrapping.

The reported numbers are 20 s for a 128-token prefill on 8x RTX PRO 6000 (compared with Cerium's reported 134 s on 8x B200), 64 s for 3968 public plus 128 private tokens, and a perplexity that matches FP16 (6.13).

Critical strengths
  • These are strong end-to-end numbers for encrypted 8B prefill. This includes multi-GPU scaling (5.8x on 8 GPUs) and fully homomorphic token selection and embedding lookup for decoding.
  • The paper is upfront in Table 2 about which components are reused, which I appreciate.
  • Appendix L replays the Cerium and MOAI approximations on real Llama-3 activations and shows that prior systems silently feed out-of-domain inputs to their approximations. This is a valuable sanity check that the community needs.
  • Calibration is validated on held-out corpora (C4 and UltraChat), and the precision requirement is justified using both noise simulation and actual FHE perplexity.
  • The authors compare bootstrapping across libraries at the same shape (1.39x) to separate library speed from algorithmic gains.
Critical weaknesses
  1. The novelty is mostly in putting existing pieces together. The authors' own Table 2 shows that nearly all of the expensive operations are reused or adapted from prior work. What is left is small. The new ring switching is a simple re-indexing of a standard step. The data layouts are careful engineering, but not a new idea. The new SoftMax evaluation relies on a classical math result and speeds up SoftMax by only 1.25x. The new matrix multiplication is a standard trick that concurrent work also uses. None of these is a learning contribution, and the tricks for handling large activations are taken as-is from prior work on model quantization.

  2. The "long inputs" claim is overstated. The title and abstract promise longer inputs, but only a public prefix can be long, while the private part is still 128 tokens. Prefilling public text in the clear and encrypting only the query is chunked prefill, and the paper itself calls it "exact in the clear" and "standard prefill applied to two chunks." Unfortunately, many of the target use cases, such as private documents in RAG or a private chat history, involve a private long context, and Sylph does not handle that.

  3. The headline numbers rest on cross-paper hardware comparisons. The 6.7x and 5.8x figures compare Sylph on HEaaN2 against Cerium's reported timings. The algorithm-level comparison in Fig. 4a is against a re-implementation of "Cerium's algorithms", and I have no way to check how faithful it is. HEaaN2 is closed-source, only binaries are provided, and Cerium's algorithms may well be tuned differently in their original setting. The 1.39x bootstrapping gap suggests that the library itself contributes a meaningful share, and other primitives (NTT, key switching) are not compared at all.

  4. Reproducibility is weak. The core library cannot be redistributed, and the review artifacts are binaries only, so independent verification of either correctness or timing is effectively impossible.

  5. Calibration comes with no guarantee. The intervals are the WikiText-2 min and max, widened by fixed factors (1.2x, or 0.3x-3x for RMSNorm). As the conclusion admits, adversarial or OOD inputs will silently corrupt the outputs. Since only the client is harmed, this is not a security issue, but it is a correctness one.

  6. The accuracy evaluation is thin. It consists of one perplexity number on WikiText-2 with 128-token windows. There are no downstream task accuracies, no comparison of instruction-following or generation quality with plaintext, and the token-match check for the selector uses only 37 inputs. A matching perplexity of 6.13 can still hide errors in the tail.

  7. I am not convinced ICLR is the right venue. As with similar submissions, the contribution is cryptographic engineering, and the ML-facing ideas (outlier mitigation reused from quantization, range calibration) are not new to the ML community. I am not convinced that this would be considered even favorably at security or cryptography venues.

Questions for authors
  1. What is actually new beyond the integration? Could you quantify the end-to-end contribution of each new component (SOS evaluation, SinC ring switching, the layouts, and the depth-1 PCMM) compared with the reused ones?
  2. How faithful is the re-implementation of "Cerium's algorithms"? Were its parameters tuned? Could you run Cerium's released artifact on B200 in your setup?
  3. How would Sylph handle a private long context, say 4096 encrypted tokens, and how does the cost scale?
  4. Can the leakage of the prefix identity in RAG be mitigated, for example with a fixed prefix set or with PIR?
  5. Could you report downstream task accuracy, and greedy-generation agreement with plaintext over longer generations and more than 37 inputs?
  6. How well do the calibrated intervals hold on code, math, or multilingual inputs, and on contexts longer than 4096 tokens?
3 comments

We thank you for the careful reading, and for highlighting App. L, the held-out calibration, and the same-shape bootstrapping comparison. This comment covers novelty, the comparison with Cerium, reproducibility, prefix leakage, and venue. The next one covers long inputs, calibration, and accuracy, with new experiments.


Novelty (W1, Q1)

Sylph is a system paper, and Table 2 states what is reused. Our claim is that making the pieces work together correctly at 8B scale is the contribution: as released, Cerium's and MOAI's approximations receive out-of-domain inputs on Llama-3 (App. L), whereas Sylph matches FP16 perplexity under FHE.

Effect of the new components (128-token prefill, one GPU, 3.55 s per layer):

ComponentEffect
Range calibrationRMSNorm 15×, SoftMax 2.2×, SiLU 1.8×: ≈1.0 s saved per layer, i.e., at least 1.3× end to end
Layouts, SinC ring switchingMake the reused BLAS kernels usable without explicit conversions, saving about 2,300 key switches per layer (~10% of the total)
Slim sum-of-squares2.2–2.5× on each inverse-square-root polynomial (e.g., 62.5 → 28.0 ms), 1.25× on SoftMax (≈3% end to end); numerically stable and general-purpose
Depth-1 PCMM3968 public + 128 private tokens in 64 s on eight GPUs, vs. 74 s with 3-level JKLS

The outlier techniques are reused from quantization, as Table 2 says; the FHE-specific step is using them to bound approximation and bootstrapping ranges.


Comparison (W3, Q2)

  • Same hardware, reported: On four B200 GPUs, Sylph takes 26 s vs. 152 s reported by Cerium (5.8×).
  • Same machine, measured: We ran Cerium's released artifact on our own machine: one transformer block takes 10.6 s on one RTX PRO 6000 (66 GiB of GPU memory). Its encoded weights are 28 GB per block—about 900 GB for the model, which is more than our host holds (Sylph's converted weights take 52 GiB; Table 9). We therefore extrapolate 32 blocks to about 338 s, against 116 s for Sylph on the same GPU: 2.9×, the same ratio as on one B200 (87 s vs. 253 s).
  • Same library, per algorithm: Fig. 4a is not a from-scratch re-implementation. We took the linear-algebra algorithms from Cerium's released artifact, ran them on our library, and optimized their FHE parameters; ours are 1.8–3.3× faster. We will reword this clearly in the text.
  • Library alone: At identical shape, bootstrapping is 1.39× faster in our library. On the same RTX PRO 6000 (N=216N = 2^{16}), a forward NTT over a 1.7-kbit modulus takes 23 µs in our library and 29 µs in Cerium's; a key switch takes 164 µs vs. 162 µs at a 670-bit evaluation level, and 294 µs vs. 436 µs at a 1350-bit bootstrapping level (Cerium's figures come from its own micro-benchmark and include the tensor product).

Putting these together: of the 2.9× speedup on one GPU, at least 2.1× is not from the library. The rest of the 5.8× comes from multi-GPU scaling (3.3× vs. 1.7× from one to four GPUs), which our partitioning and communication schedule provides (App. K).


Reproducibility (W4)

The binaries let reviewers re-run timings and check decrypted outputs against plaintext inference; the algorithms are library-independent and fully specified (App. E–K). We will release the calibration code, the noise simulator, and the polynomial coefficients.


Prefix Leakage (Q4)

Sec. 2.3 lists the prefix identity as leakage. A fixed prefix (system prompt, manual) is independent of the query and reveals nothing about it. For RAG, since long public contexts are cheap, the client can retrieve coarsely (a whole manual) or add decoys.

Hiding the prefix fully turns PC-attention into CC-attention, which is what the chunked prefill of our second comment computes: a client that retrieves the document itself (locally or by PIR) submits it as encrypted context at that cost. For a server-side corpus, the server could precompute every document's K/V in plaintext and return the selected one as ciphertexts through PIR; the query then pays only the attention part (7.4× a 128-token prefill for 3968 hidden tokens, measured) plus the PIR. We have not implemented this selection.

1. Long inputs and private K/V cache (W2, Q3)

Scope of the submitted paper. We agree that the scope should be clearer. The long inputs in the submission are public contexts. We will revise the abstract and introduction to specify long public contexts with short private queries.

New experiment with up to 4,096 private tokens. We implemented chunked prefill (Sec. 4.1) with an encrypted K/V cache. Each 128-token chunk attends directly to the encrypted K/V ciphertexts from earlier chunks. The cached ciphertexts are never decrypted or bootstrapped again.

Private tokensChunksTotal cost vs. 128-token prefill
1,024813.5×
4,09632136×

These measurements cover all 32 layers on one GPU. The cache uses 0.8 GB per chunk (16 ciphertexts per layer), and peak VRAM at 4,096 tokens was 84 GiB. Chunk jj, indexed from zero, cost approximately (1+0.20j)(1 + 0.20j) times a 128-token prefill through chunk 31. By comparison, extrapolating Table 11 to one-shot prefill at 4,096 tokens gives 1,024× the cost for QKTQK^T, SoftMax, and ScoreV, and 32× elsewhere. This would take roughly 12 hours on one GPU and require 8,192 ciphertexts for the attention state, exceeding GPU memory.

At both lengths, the top-1 token and top-5 token set matched plaintext after every chunk. At 4,096 tokens, decrypted logits retained 10 bits of precision. Table 4 indicates that 12 bits are needed to match FP16 perplexity, so we report top-k agreement at this length. When a public prefix is appropriate, we measured 64 s on eight GPUs for 3,968 public tokens and 128 private tokens.

2. Calibration across domains and sequence lengths (W5, Q6)

The interval coverage in Sec. 3.1 is empirical. On held-out C4 and UltraChat, we saw zero out-of-range values in more than 10⁷ observations per activation. For the new study, we evaluated 2,438 finer-grained per-layer intervals fitted on WikiText-2 and C4 training text, using 256-channel blocks for SiLU.

Evaluation dataOut-of-range rate
Held-out WikiText-2 and C46 × 10⁻¹⁰
English Wikipedia (24 articles, 4,139–11,890 tokens, 1.2 × 10¹² values)3–6 × 10⁻¹⁰
Math (224 MATH and GSM8K problems)4.5 × 10⁻⁹
Multilingual Wikipedia (8 other languages)1.9 × 10⁻⁸
Python source code (64 files)4.9 × 10⁻⁸

Long English Wikipedia articles had out-of-range rates at or below the in-domain baseline. Rates were higher on math, multilingual text, and code. Covering all values observed in the new domains required no change to the layer-level SiLU intervals and a 1.5× margin for RMSNorm. In a few SoftMax layers, we needed to expand the exponent input range by up to 21% of its original width. The normalizer range needed to be 89× wider in one layer on math and 5.5× wider on code. We will include this analysis and state that deployment to a new domain requires calibration data from that domain.

3. End-to-end FHE accuracy (W6, Q5)

End-to-end FHE matched plaintext greedy decoding on all 1,011 generated tokens across 240 inputs.

EvaluationInputs / generated tokensResult
RTE and MRPC (zero-shot Yes/No, first token)100 pairs per taskAll answers matched. Accuracy was 85% on RTE and 71% on MRPC for both runs.
SAMSum summarization16 dialogues / 351 tokensAll tokens matched. ROUGE-1/2/L was 49.1 / 21.5 / 41.7 for both runs.
Held-out WikiText-2, C4, UltraChat, and SAMSum20 prompts / 100 tokensAll tokens matched.
Longer generations4 runs / 11, 101, 129, and 119 tokensAll tokens matched.

Input selection. For each GLUE task, we took the first 100 pairs whose prompts fit within 128 tokens and passed our range pre-check. Of 277 RTE pairs, 217 fit and 3 of those failed the check. All 408 MRPC pairs fit and passed.

Near ties. The token selector has a resolution of 5.2 × 10⁻³, and the maximum logit error in App. F is 6.5 × 10⁻⁴. One RTE example matched despite a top-two logit gap of 3.9 × 10⁻³. Our check stopped the 119-token run at a gap of 3.3 × 10⁻³. In plaintext decoding across all 819 SAMSum test dialogues (48,154 greedy steps), gaps this narrow occurred in about 0.1% of steps. At such gaps, the selector can return the runner-up when its logit lies within the selector's resolution of the maximum.

Additional same-library baseline (W3)

Since our reply above, we also ported Cachemir, a concurrent FHE decoding system, to our library (details in our reply to Reviewer 4Krk). For decoding, Cachemir's 32 layers are faster per token (8.34 s compute-only vs. 31.0 s) but need 560 GB of encoded weights in host memory, against [[52 / 58]] GiB for Sylph on a single GPU, and it does not cover token feedback. Together with the Cerium artifact measurement, every baseline in the paper now has a same-machine comparison.

Venue (W7)

Privacy is in ICLR's scope, and ICLR 2026 published MOAI, an FHE transformer system.

AuthorsOpenReview ↗

Revised PDF uploaded; request for re-evaluation

We have uploaded a revision (changes in blue) that implements what our two replies above promised. For convenience, here is where each of your points is addressed:

  • W2/Q3 (long inputs). The introduction and conclusion now state that the long inputs are long public contexts with short private queries. App. O adds the fully private case: chunked prefill over an encrypted K/V cache, 13.5× (1,024 tokens) and 136× (4,096 tokens) the cost of a 128-token prefill, with top-1/top-5 agreement after every chunk.
  • W3/Q2 (comparison). Fig. 4a is reworded: the baselines are the algorithms of Cerium's released artifact run on our library. Sec. 5 and App. N add the artifact measured on our machine (2.9× on one GPU, at least 2.1× of it not from the library), primitive timings (NTT, key switching, bootstrapping), and a same-library port of Cachemir, which we credit as faster for decoding at ~10× the weight footprint.
  • W5/Q6 (calibration). App. P reports out-of-range rates on math, multilingual text, code and articles of up to 11,890 tokens, and states that a new domain requires calibration data from that domain.
  • W6/Q5 (accuracy). App. Q: FHE matches plaintext greedy decoding on all 1,011 generated tokens across 240 inputs (RTE, MRPC, SAMSum, generations of up to 129 tokens), with a discussion of near ties.
  • W1/Q1 (novelty). Table 2 is now in the main text, and App. M quantifies each new component. We agree that Sylph is a system paper; our claim is that making these pieces work together correctly at 8B scale is the contribution, as the released approximations of prior systems do not hold on Llama-3 (App. L).
  • W4, Q4. A reproducibility statement and App. R (prefix leakage) are added.

If these changes address your concerns on soundness and evaluation, we would be grateful if you could reconsider your assessment. If any concern remains, we will answer it within the discussion period.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.