- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper proposes POND, a framework in which a frozen large model (Sender) only prefills the input and a small model (Receiver) performs all autoregressive decoding. A trainable bridge maps the Sender’s hidden states into additional attention keys/values of the Receiver, and an periodic-review mechanism lets the Sender re-read the Receiver’s partial output and refresh this guidance. With a Qwen3.5-4B Receiver and three frontier-scale Senders, the authors report improvements over the Receiver alone on math, code and knowledge benchmarks.
Critical strengths
- The problem setting is clear. It turns the asymmetry between parallel prefilling and sequential decoding into a division of labor across different models.
- The performance improvements are consistent, with all three senders improving the receiver on all nine benchmarks.
- To some extent, the results suggest that the accuracy gain does not come at the cost of efficiency: even without review, the outputs are shorter and the latency is lower compared with the receiver alone.
Critical weaknesses
- The paper lacks comparisons with other baseline methods, such as C2C[1], which is mentioned in the paper.
- Some experimental results mentioned in the appendix are missing from the paper, such as constant/shuffled message, matched-budget text advising, and bridge-only experiments.
- Only one receiver model (Qwen3.5-4B) is evaluated, with no experiments on models of other sizes or from other model families.
- The paper lacks a comparison with majority voting of the receiver alone under an equivalent resource budget.
- Earlier work, such as LLM-to-SLM[2], has already proposed a similar idea (a frozen large model encodes the prompt once, and a small model does all the decoding), but the paper neither cites nor discusses it.
[1] Tianyu Fu et al. Cache-to-Cache: Direct Semantic Communication Between Large Language Models. arXiv: 2510.03215, 2025.
[2] Benjamin Bergner et al. Think Big, Generate Quick: LLM-to-SLM for Fast Autoregressive Decoding. arXiv: 2402.16844, 2024.
Questions for authors
N/A
2 comments
Response to Reviewer i8Kg (2/2)
W4: Comparison with majority voting of the receiver alone under an equivalent budget.
We agree this is a useful reference point.
- Latency budget. Single-transfer POND is already faster than one receiver-alone sample on every benchmark (Tables 1–2), so a latency-matched voting baseline gets at most one sample unless samples run in parallel.
- Compute budget. We compare against receiver maj@k with k = 3, chosen to match POND's total compute (sender prefill plus receiver decoding):
| Method | AIME | OlymMath-Hard | GPQA-Diamond | MMLU-Pro |
|---|---|---|---|---|
| Receiver maj@3 (compute-matched) | 83.33 | 51.0 | 78.3 | 79.2 |
| POND (single sample) | 85.33 | 53.0 | 79.29 | 80.93 |
| POND maj@3 | 87.33 | 54.0 | 80.8 | 81.8 |
A single POND sample outperforms compute-matched voting on all four tasks. Voting is also orthogonal to POND and adds further gains on top of it. Voting over programs is not directly defined without execution-based selection, so we report math and multiple-choice tasks only.
W5: Relation to LLM-to-SLM (Bergner et al., 2024).
Thank you for pointing out this work; we will cite and discuss it. LLM-to-SLM shares POND's high-level motivation: a frozen large model encodes the prompt in parallel and a small model performs all decoding. POND differs in four respects:
- Decoder-only frontier senders. LLM-to-SLM mainly uses encoder–decoder LLMs (T5 encoders up to ~11B). Its authors report that decoder-only features worked poorly in their framework and leave them to future work. POND uses the final-layer states of decoder-only frontier models and improves the receiver consistently.
- Injection mechanism. LLM-to-SLM adds the projected LLM representation to the SLM's prompt embeddings, once, at the input layer; sequence alignment then requires the SLM to adopt the LLM's tokenizer and embedding matrix and to train a new output head. POND injects a separate gated KV memory at every full-attention receiver layer (Sec. 3.3–3.4). The receiver keeps its own tokenizer, vocabulary, and text KV cache, and senders with different tokenizers are handled by character-offset alignment (Eq. 9, Appendix B).
- Periodic review. LLM-to-SLM calls the LLM once on the prompt; its authors list this as a limitation and suggest dynamic invocation as future work. POND's review realizes this direction: the sender re-reads the partial response through delta-only prefill, without rewriting the receiver's cache (Sec. 3.5–3.6).
- Task regime. LLM-to-SLM evaluates translation, summarization, and instruction following. POND targets long-form reasoning (AIME, OlymMath-Hard, LiveCodeBench, GPQA-Diamond) with outputs of thousands of tokens, where review matters most.
POND additionally shows cross-domain transfer: trained on math/code only, it raises the knowledge average from 84.77% to 86.44%, whereas matched receiver-only SFT lowers it to 83.45% (see our response to Reviewer UMZc, W1).
Response to Reviewer i8Kg (1/2)
We thank the reviewer for recognizing the clarity of the problem setting, the consistency of the improvements across all senders and benchmarks, and that the accuracy gains do not come at the cost of efficiency. We address each weakness below.
W1: Comparison with C2C.
We agree that C2C is a direct and important baseline. We trained and evaluated C2C and POND with the same sender (Qwen3.8-Max), receiver, training data, and training budget:
Single-transfer POND is more accurate and faster than C2C on both suites, and review adds a further 0.88 / 0.74 pp. Both methods transfer representations without intermediate text; POND differs in three ways:
W2: Experiments described in the appendix are missing from the main paper.
Thank you for this careful observation. These controls were completed (Appendix F) but omitted from the main text for space. We will add them to the revision:
W3: Only one receiver model is evaluated.
Our study varied the sender (three frontier models, two of them cross-family with different tokenizers) while fixing the receiver, in order to isolate the effect of sender representations. To address this concern, we add Gemma 3 4B IT as a second receiver family (Qwen3.8-Max sender; SFT and POND use the same math/code data and training budget):