- Rating
- 1
- Confidence
- 4
- Informativeness
- 1
- Presentation
- 1
- Soundness
- 1
Read review
Paper summary
The paper proposes frequency-division multiplexed (FDM) memory: each of 40 categorical KEY=VALUE facts is encoded as an amplitude-shift-keyed sinusoidal carrier, and the carriers are summed, sampled at 256 points, quantized and mapped to vocabulary tokens, with an interleaved copy appended to form a 512-token block placed in the prompt. Four small models (GPT-2 Medium, Qwen3-0.6B, LFM2.5-1.2B, Hermes3-3B) are fully fine-tuned on 115K synthetic single-block examples and then 100K hop-structured examples to answer templated questions and write out all 32 context channels as NAME=VALUE pairs. The paper reports 93–100% per-channel accuracy, near-ceiling FACT accuracy up to 4 hops (82.8–83.4% at 5 hops), decoding at a second position range after additional fine-tuning, tolerance to a learned K/V prefix, and, with general-text replay, 8–10 of 10 held-out natural-language questions answered correctly, and concludes that attention can learn to retrieve from a frequency-addressed in-context memory.
Critical strengths
S1: The schema and templates are specified, and much of the code is released, so several claims can be checked.
The 40-channel schema (Table 8) and the n-hop question templates (Table 9) are listed in the appendix. The linked repository contains data generators, evaluation scripts and result files. Single-block training code is withheld, and the coexistence experiment of Table 7 is not included. This made it possible to verify several claims directly; as W3 and W4 show, however, that verification mostly undermines the paper's conclusions.
S2: The decoding results are consistent across four architectures.
GPT-2, Qwen3, LFM2.5 (a hybrid convolution–attention model) and Hermes3 all learn to decode the synthetic signal. Per-channel accuracy is high everywhere except LFM2.5's last channels (see W3). The prefix-tolerance experiment (Table 6) uses multiple seeds and reports Wilson intervals. These are the most carefully reported results in the paper, but they do not address the missing baselines.
Critical weaknesses
W1: The submission appears to violate double-blind review.
The Reproducibility Statement (p. 10, l. 486–489) links to a GitHub repository under a personal, non-anonymized account, which may reveal the author's identity. Anonymous code hosting was available and should have been used. I leave the policy decision to the AC.
W2: The presentation is poor, and there is not a single explanatory figure.
The paper contains no figures at all. The full pipeline (channel values → ASK carriers → composite signal → quantization → token mapping → interleaving → prompt and output format) is described only in prose and two equations, which makes the method hard to follow. There is also no Related Work section.
Several key details are missing:
- The number of quantization levels L is not given in the method section; it can only be inferred from "log2 64" in §3.4.
- The single-block question templates are never described. According to the released code, there are three: proceed, risk assessment, and share the secret.
- The 10-channel mixed-domain configuration is not defined.
The text also contains inconsistencies:
- Table 4 reports only three of the four architectures and omits Qwen3 without explanation.
- The claim that joint accuracy "remains at 100% through 4 channels for all three architectures" (l. 297) conflicts with GPT-2's 99.8% at N=4 in the same table.
- The Discussion refers to "up to three architectures tested (Table 2)", although Table 2 has four.
- Hermes3's EXTRA accuracy is 99.8% in Table 2, but the note to Table 4 reports a single error on ch37 in 1,500 samples, which implies about 99.998%.
W3: The released code shows signs of insufficient verification and does not match the method and conclusions in the paper.
Signs of unverified generated code. The AI Use Statement says that LLM tools were used for "organization and editing of manuscript drafts" and that "Python experimental scripts were generated by these tools as were the evaluation of results of running these scripts". The released code shows signs that it was not carefully checked:
- The LFM2.5 data-generation script (
scaling_replication/eidetic_lfm2_fdm_e2e.py) loads its tokenizer from a non-existent model ID,"Qwen/LiquidAI LFM2.5-1.2B-Base". The Hermes3 script likewise uses"Qwen/Hermes-3-Llama-3.2-3B". - The GPT-2 "plaintext baseline" result file in
nhop_composition/results/is byte-identical to the GPT-2 FDM result file.
Encoding. Eq. (1) assigns each channel one of M amplitude levels over the whole signal. Every encoder in the released code, including the one in scaling_replication/ whose released per-model results match Table 3, instead does something different: it converts each value into bits and modulates each bit with binary ASK in a separate time segment of the signal. Part of the information is therefore carried by position, which is in tension with the title's claim of "frequency, not position".
Token mapping. The paper describes a fixed bijection τ from quantization levels to tokens. The code instead decodes the token IDs into a string and re-tokenizes it inside the prompt; the paper mentions re-tokenization only in passing in §3.2. Using the released encoder and tokenizers, I found that this round trip changes the token sequence in most samples: 182 of 300 for Qwen3, 282 of 300 for LFM2.5 and all 300 for Hermes3, with lengths of 510–548 tokens. In general, then, the model does not receive τ(q(x)).
Scoring. The paper says EXTRA requires the full NAME=VALUE string to appear verbatim. The scripts that reproduce Tables 2–4 (scaling_replication/eval_*.py, frequency_gradient/perchannel_eval_*.py, joint_retrieval/verified_repro/*/eval.py) only check whether the bare value appears anywhere in the response. Only the n-hop script checks NAME=VALUE.
High-frequency errors. The paper states that generation is capped at 250 new tokens (§2.1.3). It also notes that the high-frequency channels are the last ones in the Context block (§3.1.1). It does not test whether this explains the errors. With the released LFM2.5 tokenizer, every reference answer is 256–289 tokens long, so even a perfect model is cut off within the last channels.
Under the released scorer, truncation alone predicts the following, close to the reported values:
| ch35–39 | EXTRA | Joint accuracy, N=32 | |
|---|---|---|---|
| Truncated perfect model | about 99 / 88 / 46 / 24 / 17% | about 93% | about 2% |
| Reported | 98.9 / 86.1 / 47.5 / 27.3 / 15.9% | 93.0% | 2.1% |
The "high-frequency carrier degradation" on LFM2.5 is therefore most likely a truncation artifact.
W4: The experiments are toy-scale and lack the necessary baselines.
Task. The task is a fixed synthetic schema of 40 categorical facts with 2–5 values each, about 75 bits in total, answered through fixed templates. Single-key retrieval is tested only through four fixed 1-hop templates (STATUS, THREAT, RULE, LOCATION). The 32 context channels are never queried individually; they are always transcribed as a whole block in a fixed order.
Missing baselines.
- There is no baseline that places the same facts in context as plain text. Written as
NAME=VALUElines, the 40 facts take roughly 220–310 tokens with these tokenizers, fewer than the 512-token FDM block. - There is no comparison with learned memory or context compression (gist tokens, ICAE, prefix-tuning).
- Apart from the 25% RULE chance level in the Table 7 caption, no chance-level or ceiling reference is given. For example, in the released scorer, the FACT check for the single-block "Should {AGENT} proceed with the mission?" template passes whenever the agent name, which appears in the question, is copied into the answer.
- Natural-language competence is judged from only 10 questions, with no measurement before fine-tuning, as the paper acknowledges.
Unreported plaintext baseline. The linked repository contains a matched plaintext baseline that the paper does not report, and its results are mixed. In the clean condition (corruption_experiments/README.md, 150 samples per condition):
| Model | Plaintext context accuracy | FDM context accuracy |
|---|---|---|
| Qwen3 | 100% | 82.0% |
| LFM2.5 | 91.0% | 99.6% |
FDM is ahead on both models only under stride deletion. The plaintext-baseline README (nhop_composition/plaintext_baseline/README.md) concludes that "under clean memory conditions, plaintext and FDM achieve comparable compositional accuracy when given matching compositional training". It argues that FDM's benefit lies in robustness under stress, which the paper does not evaluate.
The 5-hop drop. The n-hop plaintext baseline's 5-hop FACT score on Qwen3 is 4022/4825 (83.36%), exactly the same as FDM on Qwen3. With the released templates and scorer, even a perfect output reaches only about 83% at 5 hops, because the reference answer of the META-chain template omits some of the consulted values. The reported 5-hop drop is therefore a scoring artifact rather than a limit of FDM.
Questions for authors
Q1: Why is the plaintext baseline in the repository not reported, and in what setting does FDM outperform plain text?
This is a requested experiment that could change my assessment. Please report the plaintext baseline next to every FDM result, under the same training budget and generation limit, together with the token cost per fact. Please also include the corruption conditions in which FDM is claimed to help, and add learned-memory baselines (e.g., gist tokens, ICAE, prefix-tuning). Without them, the paper does not show a benefit of FDM over plain text.
Q2: Which encoder produced the reported results: the M-ary ASK of Eq. (1), or the time-segmented binary ASK in the released code?
This is a clarification request. If it is the time-segmented version, please explain how the "frequency, not position" claim still holds. Please also state how the decode-and-re-tokenize step mentioned in §3.2 changes the sequence the model actually sees.
Q3: Are the LFM2.5 high-channel errors caused by the 250-token generation limit?
This is a requested experiment. Please re-run Tables 2–4 with a substantially larger generation limit and with the Context block emitted in a shuffled channel order.
Q4: What are the chance-level and maximum achievable scores for each metric?
This is a clarification request. Please give chance and ceiling scores for each metric beyond the 25% given for RULE. Please also report the 5-hop FACT accuracy when only values present in the reference answer are scored, and explain the inconsistencies noted in W2. Finally, please state what is trained for the "4K steps" per seed in Table 6.
10 comments
Part 2
The reviewer is correct about the released encoder (Q2). Equation (1) in the submission does not describe what the code does. The encoder writes each value index as B=3 bits, divides the 256-sample signal into three symbol intervals of 85 samples, and in each interval keys channel k’s carrier to A_high or A_low according to the corresponding bit. In this way, bit 1 of every fact lives in the first 85 samples, bit 2 in the next 85, bit 3 in the last 85. So once the system has found the right carrier, its amplitude level in those three places is read in order. Therefore this latter part is positional; the addressing of the fact is by frequency. Further, the 40 carriers are summed within every interval. This is binary ASK, bit-serial, and it is what produced the results presented in the original draft of the paper. We have corrected the revision accordingly: Equation (1) and §2.1 now state the bit-serial scheme explicitly, with the symbol interval T_b = 85/f_s; the captions of Figures 1, 2 and 4 describe the value as a three-bit on/off pattern across the intervals rather than a single amplitude; and the resolution argument in the Discussion has been corrected, since it used the full signal length N where the symbol interval N/B applies. Over one 85-sample interval the Fourier resolution is f_s/85 ≈ 1.18 Hz, which is above the 1 Hz carrier spacing, so the hosts’ 99–100% per-channel accuracy is not explained by carriers being resolvable within a single interval. The revision states this as an open question rather than a property of the design.
The reviewer is correct. The generator decodes the mapped token IDs to a string and the prompt is re-tokenized, so the host receives a re-tokenization of τ(q(x)), not τ(q(x)) itself. The string is determined by the levels and the tokenizer is deterministic, so the model’s input is still a fixed function of the signal; the model learned to read this re-tokenized version, and every number in the paper was measured on it. We have corrected §2.1 to state what the host actually receives, replaced “512 tokens” by “≈512” where an exact count was implied, and are checking whether the re-tokenization is lossless; the result will be stated in the revision.
The reviewer is correct. The scripts behind Tables 2–4 score EXTRA by checking whether the bare value appears anywhere in the response, and only the n-hop script checks the full NAME=VALUE string. The paper’s description of EXTRA is therefore wrong for those tables and has been corrected. Since several values are shared across channels (LOW/MEDIUM/HIGH, CLEAR, NONE, OPEN, CRITICAL, UNKNOWN), the bare-value check can credit a wrong channel when another channel carries the same value, so it can overstate accuracy. We are re-scoring the same 1,500 test samples per host with the strict NAME=VALUE check and will report the strict numbers in Tables 2–4 of the revision, with the lenient numbers alongside for comparison.
The reviewer’s truncation analysis is being tested directly (Q3): the same test samples are being re-run with a 512-token generation cap and scored at both caps. Pending that result, every reference to “high-frequency carrier degradation” has been removed from the revision and the LFM2.5 ch36–39 errors are described neutrally as errors in the last channels emitted.
W4. We accept the characterisation. The task is a fixed synthetic schema of 40 categorical facts with 2–5 values each, queried through fixed templates. As Appendix B shows, the FDM-encoded token sequence the model receives bears no surface relationship to the KEY=VALUE facts it encodes; to a reader it is a string of unrelated word fragments. The paper is a feasibility study of one question: can a transformer retrieve facts from such a sequence when the facts are addressed by carrier frequency rather than by position, at high accuracy and across architectures? On that question the results are positive and, as the reviewer’s own checks confirm, reproducible. The paper does not show, and the revision does not claim, that FDM is more accurate or more token-efficient than plain text on this task. The authors did not design the study to pit the technique against plaintext, and the rewritten Introduction and Abstract state the exact reason for having done it: to demonstrate that a single frequency-based technique, a simple modulation scheme known throughout a wide and mature communications-engineering field, can be used by four transformer language models for KEY=VALUE fact retrieval, which is relevant to the communication capabilities of language models and agents and to how information is distributed to humans.
Part 3
The importance is in showing that the technique, rooted in decades of theory and laws outside machine learning and built on frequency for efficient transmission between transmitters and receivers, works and applies to language modelling, not that it is better than plaintext retrieval. The natural-language competence experiments have been removed from the paper, as they require a robust evaluation that has not been conducted.
On Q1 (why the plaintext baseline in the repository is not reported, and in what setting FDM outperforms plain text): the authors ran the plaintext comparison early in the experimental pipeline on Qwen3 and LFM2.5 (the GPT-2 run was never completed, as noted above), but the value of the technique is not as a replacement for plaintext retrieval in transformers, and we do not claim a setting in which it outperforms plain text. The result files are retained in the repository and the revision reports them in an appendix next to the FDM results, with the token cost of each representation. The FDM retrieval technique as designed is based on one mature frequency-based modulation technique, which offers a different way to organise tokens for language-model retrieval, without any FFT or demodulation module, resting on the idea of separable carriers, each carrying one fact, composed into one signal. Rather than compare to plaintext, the authors have attempted to demonstrate the extent of FDM as an accessible retrieval technique for transformers in context, and have begun to extend it toward the parametric regime: the revision introduces Table 6, a split-substrate experiment showing that KEY=VALUE facts injected into the K/V cache are read correctly while FDM-modulated KEY=VALUE facts in context are read accurately in the same forward pass.
We also agree with Q4 that META is not a retrieval measure as scored: 60% of samples are credited without reading the response, which is why all four hosts show 79.8–79.9%. This is stated in the caption of Table 2 and the column is not interpreted.
We greatly appreciate the thorough review by the reviewer of our work.
Author response: method corrections (Eq. 1, tokenization, scoring), baselines, and revised PDF
We thank the reviewer for the very helpful comments and thorough review. The authors have rewritten the Introduction and Abstract for clarity of the technique, which comes from a field outside machine learning, and have added four figures in consideration of the reviewers’ comments about a lack of clarity. In particular Figure 4 outlines the full design and flow from channel values → ASK carriers → composite signal → quantization → token mapping → interleaving → prompt and output format. The prose, adjusted particularly with regard to Equation 1, is now supported by the added figures. The number of quantization levels is 64 and we have indicated it in the prose and in Figure 4’s caption. The single-block question templates are listed with the full channel definitions in Appendix A. In the interest of space and to contribute to the clarity of the paper’s main point, the 10-channel mixed-domain configuration has been removed. We thank the reviewer for pointing out the missing Qwen3 result in Table 4 and have added “This experiment was not conducted on the Qwen3 model and is not reported here”; it was not conducted for lack of resources and time. “Remains at 100%” has been adjusted to “remains at near 100%”. Hermes3’s EXTRA accuracy is corrected to 100.0%; the 99.8% was a transcription error (every Hermes3 run gives one error in 1,500 samples, on ch37). The reviewer is correct, and we thank them for checking the code. The three scaling_replication/eidetic_* scripts were templated from the Qwen3 script and the MODEL_ID strings (and the architecture notes in their docstrings) were never updated. This has been corrected in the repository. The data used in the paper was produced by the generators in joint_retrieval/verified_repro/{lfm2,hermes3}/generator.py, which carry the correct IDs (LiquidAI/LFM2.5-1.2B-Base, NousResearch/Hermes-3-Llama-3.2-3B); the MODEL_ID constant is used only for tokenizer loading in generate, and the token-map bijection depends only on VOCAB_SIZE and the seed, which were correct, so no reported result is affected. The eval paths load the finetuned checkpoints by path and were unaffected. The reviewer is also correct that the GPT-2 plaintext result file is a copy of the GPT-2 FDM n-hop result; the GPT-2 plaintext baseline was never completed, and the file has been removed.