- Rating
- 3
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This research introduces TELEX, a non-interactive framework for privacy-preserving autoregressive LLM inference using Fully Homomorphic Encryption (FHE). To address the computational overhead and numerical errors associated with vocabulary-dependent operations under FHE, the authors introduce an FHE-friendly latent reasoning transfer mechanism. The server performs encrypted reasoning in a continuous latent space, while a lightweight client model generates the final answer using the decrypted reasoning representations. Additionally, the authors develop a layout-compatible KV-cache mechanism to accelerate encrypted autoregressive decoding. Experimental results demonstrate substantial decoding speedups while largely preserving the accuracy of the server model.
Critical strengths
- Effective integration of latent reasoning and FHE
- Efficient KV-cache design
- The presentation and organization are generally strong.
Critical weaknesses
- Limited task diversity and uncertain generalization beyond arithmetic reasoning. Although the authors evaluate TELEX on several benchmarks, these datasets evaluate related mathematical reasoning capabilities and may not sufficiently demonstrate the general applicability of the framework.
- Insufficient attribution of the performance gains from latent reasoning transfer The proposed framework combines several components. However, the current results do not sufficiently isolate their individual contributions. For example, Table 1 reports that TELEX outperforms SIM-CoT on GSM8K-Aug. However, the comparison involves different inference settings: SIM-CoT uses a 1B model, whereas TELEX additionally receives latent representations from a 3B server. Therefore, the performance difference cannot be attributed solely to the proposed transfer mechanism.
- Limited analysis of the fixed latent reasoning budget. One of TELEX's main efficiency advantages comes from reducing the number of encrypted decoding steps through fixed-length latent reasoning. However, the relationship between the latent budget, reasoning complexity, and final answer accuracy is insufficiently explored.
Questions for authors
- SIM-CoT comparison: Could the authors clarify the differences between the SIM-CoT baseline and TELEX in Table 1, particularly regarding model capacity, training supervision, and inference configuration?
- Contribution of latent transfer: Could the authors provide a more detailed ablation comparing the client with matched server latents, mismatched latents, zero latents, and no latent input? How much of the final improvement is attributable to the transferred representations rather than additional supervision and the dynamic selector?
- Latent budget: How sensitive is TELEX to the number of latent reasoning steps? Is there a consistent accuracy–efficiency trade-off when increasing or decreasing the latent budget?
3 comments
Q2. Effectiveness of Latent Transfer and Component-wise Ablations
Thank you for raising this question. We provide evaluation results under different latent-input conditions in Table 4. As shown in Table 4, replacing matched latents with mismatched or zero-valued latents, or removing the latent input altogether, substantially reduces accuracy. These results demonstrate TELEX indeed leverage reasoning latents to improve the performance. We also report the ablation results to separate HP/L2C and selector contribution. As shown in Table 5, both HP and L2C receive reasoning latents, with answer-only and CoT+answer supervision, respectively. L2C alone achieves 61.0% on GSM8K-Aug and 61.3% OOD average. Adding the frozen selector increases these to 62.5% and 61.6%, corresponding to gains of 1.5 and 0.3. An oracle selector achieves 67.6% and 63.3% accuracy, suggesting that HP direct answer readout and L2C-based CoT expansion provide complementary predictions that there is room to improved selection.
Table 4. Accuracy under different latent-input conditions.
| Latent-input condition | GSM8K-Aug | GSM-Hard | MultiArith | SVAMP | OOD Avg. |
|---|---|---|---|---|---|
| Matched | 62.47 | 16.30 | 98.89 | 69.67 | 61.62 |
| Mismatched | 3.41 | 0.76 | 8.33 | 30.00 | 13.03 |
| Zero | 0.30 | 0.08 | 0.00 | 0.33 | 0.14 |
| No latent input | 6.60 | 1.59 | 12.78 | 26.67 | 13.68 |
Table 5. Ablation results for HP, L2C and the selector
| Method | GSM8K-Aug | OOD Avg. | Training Supervision |
|---|---|---|---|
| HP | 51.0 | 54.0 | answer-only |
| L2C | 61.0 | 61.3 | CoT+answer |
| TELEX | 62.5 | 61.6 | - |
| TELEX (oracle) | 67.6 | 63.3 | - |
Dear Reviewer yTMf,
Thank you again for the time and care you have put into reviewing this paper. I sincerely appreciate your thoughtful feedback and your contribution to the review process.
The authors have now responded to your initial comments. When you have a chance, could you please take a look at their responses and engage with them in the discussion? Your follow-up feedback would be very helpful in clarifying any remaining questions and informing our final assessment.
Many thanks for your continued engagement and support!
Best wishes,
Your AC
W1. Additional Evaluation Beyond Mathematical Reasoning
Thank you for your suggestion. We additionally evaluate TELEX on ProsQA[1], a benchmark for multi-step logical reasoning and planning. We use Coconut-GPT2 as the server model to provide reasoning latents, and Pythia-14M as the client model that consumes these latents. As shown in Table 1, TELEX achieves 95.4% accuracy, retaining 97.3% of the server model's accuracy. We further compare three Pythia-14M configurations trained with different forms of supervision. As shown in Table 2, by leveraging server's reasoning latents, TELEX's 95.4% outperforms No-CoT's 62.4% and SFT-CoT's 69.8% . These results show that TELEX can successfully leverage the server's reasoning latents to produce high quality answers than the small-model baselines do.
Table 1. Recovery of server accuracy on ProsQA
Table 2. Accuracy comparison on ProsQA (Pythia-14M)
W2. Performance Gains from Server Reasoning Latents
Thank you for raising this concern. We would like to emphasize that TELEX's latent transfer mechanism is designed to enable a lightweight client model to leverage reasoning latents generated by a more capable server model. The results in Tables 1 and 2 of TELEX main paper are intended to demonstrate that server provided reasoning latents enable a lightweight client model to approach the server's accuracy, substantially outperforming the client-only baselines. For example, TELEX enables a 1B client model to approach the accuracy of a 3B server model by leveraging server reasoning latents. Table 3 further quantifies the accuracy gap between the 3B server and different 1B client. As shown in Table 3, TELEX 1B achieves comparable performance with server 3B model by using server's reasoning latents while SFT-CoT 1B and SIM-CoT 1B have larger gaps of −4.1 and −6.2, respectively.
Table 3. Accuracy comparison between the 3B server and 1B client configurations.
W3 & Q3. Latent Budget Sensitivity and Accuracy–Efficiency Trade-off
Thank you for this insightful question. Figure 3 of SIM-CoT [2] shows that model performance is sensitive to the latent reasoning budget, with additional latent steps generally improving accuracy. This observation motivates a corresponding analysis of the accuracy–efficiency trade-off in TELEX. Such an analysis, however, requires end-to-end retraining for every latent-budget setting, and each training run for 3B-4B models takes approximately one week with our current computational resources. We are therefore unable to complete a controlled sweep within the rebuttal period. We commit to including this experiment in the final version of the paper. Specifically, we will evaluate multiple latent-step settings under matched training conditions and report their effects on task accuracy, encrypted inference latency and computational cost. This analysis will clarify how the latent budget should be selected for reasoning tasks of different complexity.
Q1. Clarification of the SIM-CoT Comparison
Thank you for your valuable question. Both the SIM-CoT 1B baseline and the TELEX 1B client model are initialized from the same base model, Llama-3.2-1B-Instruct. SIM-CoT uses step-level CoT supervision and the TELEX client is trained to leverage reasoning latents, with HP using answer-only supervision and L2C using CoT-plus-answer supervision. At inference, SIM-CoT uses the question as input, while TELEX uses the question and server's reasoning latents.
[1] Hao, S., Sukhbaatar, S., Su, D., Li, X., Hu, Z., Weston, J., & Tian, Y. (2025). Training Large Language Models to Reason in a Continuous Latent Space. Conference on Language Modeling (COLM).
[2] Wei, Xilin, et al. "Sim-cot: Supervised implicit chain-of-thought." International Conference on Learning Representations. Vol. 2026. 2026.