acceptodds
Under review as a conference paper at ICLR 2027

Receiver-Conditioned State Reconstruction in Hybrid Recurrent Language Models

Abstract

Language-model programs can compute several candidate branches before selecting one to continue. Saving a full internal state for each candidate increases storage and transfer costs. The retained state of one completed branch can instead serve as a reference for compact records of the other computed branches. The selected branch can then be restored without retaining their shared starting state. For fixed affine branch maps, the theory identifies missing information and corrections that limit error amplification. In short Gated DeltaNet branches, decay matching represents the recurrent state as a scaled sibling state plus a compact residual. A stored lossless bitwise repair corrects numerical discrepancies to recover the native state exactly. On Qwen3.5-4B, repaired records reproduce native cache bits, fresh logits and 128-token continuations at all 64 evaluated endpoints in the same runtime. Across 48 short endpoints, median paired byte ratios are 29.1% relative to canonical lossless Full and 30.1% relative to direct sibling exclusive OR (XOR).

Primary area:
foundation or frontier models, including LLMs
Keywords:
Hybrid recurrent language models, State reconstruction, Lossless compression, Low-rank residuals, Branching inference

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_d3Y9OpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper studies how to reconstruct the internal state of an executed branch in a hybrid recurrent language model using a completed sibling branch without access to their shared initial state. Building on the affine update structure of Gated DeltaNet, the authors propose Direct Factor Difference (DFD) and Scalar Subspace Residual (SSR) representations, together with headwise fallback and lossless repair mechanisms.

Experiments on Qwen3.5-4B demonstrate exact state recovery with lossless repair and reduced storage requirements for short branches, at the cost of additional computation. The authors also evaluate reconstruction fidelity, scalability with branch length, transfer costs, and downstream continuation.

Critical strengths

1. A well-motivated reconstruction problem with a clear structural solution.

The paper addresses a practical problem in multi-branch LLM execution: recovering a branch from an available sibling state without retaining their common starting state. By exploiting Gated DeltaNet's update structure, the method enables compact state reconstruction without inverting the target transition operator.

2. Strong lossless recovery results.

The repaired DFD-H method achieves bit-exact recovery and identical continuations across all 64 evaluated endpoints. For short branches, its median record size is only 29.1% of canonical lossless Full. The evaluation includes relevant baselines and fidelity analyses, while acknowledging additional computation and reduced compression efficiency for longer branches.

Critical weaknesses

1. The end-to-end practical benefit remains unclear.

Despite reducing record size, repaired DFD-H incurs 1.044 times the local cost of canonical lossless Full and 6.638 times that of Raw Full. The reported latency improvements concern approximate variants under different protocols, rather than the final lossless method. Actual network transfer costs and the memory overhead of resident sibling states and temporary buffers are also not fully evaluated.

2. The methodological novelty and broader applicability need stronger justification.

The theoretical results largely build on established principles, with the main distinction appearing to lie in the receiver-conditioned setting. However, the prevalence of this setting in realistic serving workloads is not established. The paper's own comparisons also suggest that retaining the common fork may be more efficient when multiple candidates are involved.

Questions for authors

End-to-end efficiency.

Under what deployment conditions would repaired DFD-H outperform the strongest baselines? Could the authors estimate the break-even bandwidth while accounting for all computation and memory costs?

2 comments
AuthorsOpenReview ↗

Final exact efficiency, reference costs and the contribution of the factors

Thank you for emphasizing the link between compact records and complete recovery cost. This helped focus the implementation changes and the new comparisons.

Implementation improvements. The implementation now groups FP32 prediction, packs tensors contiguously, compresses in parallel and overlaps prediction with repair decoding. Owned buffers remove duplicate copies. The recovery principle is unchanged: decay matching, captured branch factors and lossless repair under the same sibling-only receiver condition. Applicable optimizations also serve the baselines (Table 50, p.49).

Measured gains. On top of packing, grouped prediction and parallel coding, owned buffers and receiver overlap reduce lifecycle time at 5/8 matched endpoints at 1 Gbit/s. Reductions are 15.5% at the fixed two-token endpoint and 13.6% at the four-token endpoint, with identical wire streams and predictor hashes (Table 51, p.50). At 500 Mbit/s with a resident receiver sibling, the optimized implementation has median paired time reductions of 15.9% versus raw Full, 25.0% versus canonical lossless Full and 41.4% versus decay-scaled sibling XOR. It beats all five baselines on 40/48 short endpoints (Table 53, p.50).

Added experiments. New comparisons cover exact transfer, a 100-request handoff, complete candidate libraries and the decay-scaled sibling control.

1. Complete recovery cost and the break-even range.

The optimized DFD-H implementation with lossless repair (DFD-R) gives these median paired lifecycle ratios, including encoding, actual transfer, restoration and fresh forward:

Offered rateDFD-R / raw FullDFD-R / canonical FullDFD-R / scaled XOR
100 Mbit/s0.4050.4390.413
500 Mbit/s0.8410.7500.586
1 Gbit/s1.2720.9570.692

The measurements bracket the change in ranking against raw Full between 500 Mbit/s and 1 Gbit/s. At 1 Gbit/s, DFD-R retains lower paired median cost than both compressed comparators shown above. Timing charges required branch execution/capture and uses paced TCP loopback, starting with a resident receiver sibling (Table 3, p.7; Table 53, p.50).

All 64 endpoints preserve native bits, fresh logits and continuations.

2. Are staging, reference storage and the fork alternative accounted for?

The new 100-request replay charges candidate generation, initial staging, selection, capture, transfer and restoration. Eight requests trigger sibling-only recovery, all exact. Within those eight at 1 Gbit/s, paired workflow time ratios are 1.295 to raw Full, 1.116 to canonical Full and 0.917 to scaled XOR (§U.1, Table 58, p.53).

The handoff anchor occupies approximately 55.7 MB of logical cache state (§T.6, p.52). Separately, complete-library accounting includes references, shared fields, the index and every candidate record. Across eight four-candidate groups, totals are 760.4 MB for DFD-R, 1,678.3 MB for raw Full and 1,435.6 MB for scaled XOR. Temporary-buffer peak memory was not separately measured (§T.5, Table 56, p.52).

Fork layouts have lower aggregate storage when reference placement is free, consistent with the submitted §6.2 (p.9). The new exact comparison includes fork-plus-token replay and fork-plus-factors with repair under the same storage accounting. In the measured handoff, the fork stays at the producer while the receiver holds a completed candidate. DFD restores another branch from that available candidate without obtaining the fork (§U.1, p.53).

3. What is the added methodological and practical value?

The construction turns a completed sibling into an inverse-free reference for exact branch recovery. Decay matching cancels shared history; captured factors encode branch-specific updates, and lossless repair restores native bits (§3, pp.3–5).

The added decay-scaled sibling XOR control (SX) tests the factors directly. Complete DFD-R records have a median paired byte ratio of 0.300 to SX, with a 95% prompt-cluster interval of [0.287, 0.329], and are smaller at every short endpoint (§T.1, Table 49, p.48).

The ALFWorld archive study finds storage benefits on 25/100 tasks using complete action candidates. Within those 25, median complete-library savings are 17.59% against canonical Full and 19.22% against SX, including anchor costs. Fig.2 (p.10) and §U.2 (pp.53–54) report the per-task measurements.

Data and code: Display inputs are in data/rebuttal/ and rebuttal_artifact/, with regeneration scripts in scripts/. rebuttal/EVIDENCE_LOCATIONS.txt lists the raw measurements and execution code. Revised page numbers refer to the 54-page manuscript.

The new ALFWorld results show storage savings on 25 of 100 tasks after accounting for the reference state, and the 100-request replay directly addresses my question about end-to-end latency. For the eight requests that required sibling-only recovery, the workflow was still slower than both full-state baselines at 1 Gbit/s, so I am keeping my rating.

Reviewer_xpYEOpenReview ↗
Rating
1
Confidence
2
Informativeness
1
Presentation
2
Soundness
1
Read review
Paper summary

This paper studies how to recover a completed branch state in a hybrid recurrent language model when the shared fork state has been discarded but a sibling branch remains resident. It first characterizes when the sibling state contains sufficient information for reconstruction, and derives the minimum supplementary rank under both information-loss and bounded-gain criteria. For Gated DeltaNet with scalar per-head decay, matching cumulative decay cancels the shared history, yielding a compact low-rank residual added to a scaled sibling state. The paper develops direct-factor and shared-subspace encodings with per-head dense fallback, plus a lossless XOR repair that converts the approximate reconstruction into the exact native state.

Critical strengths

The paper clearly separates two orthogonal problems—information loss and error amplification. The method requires no inversion of the target transition matrix, and the headwise fallback guarantees robustness.

Critical weaknesses

All experiments are conducted on a single model (Qwen3.5-4B), so the method's generality to other models remains unestablished; for channel-wise-decay variants (KDA, Gated DeltaNet-2), the paper provides only a different theoretical analysis without any evaluation. Second, the method is never validated on any real-world workload—all evidence comes from constructed endpoints and controlled templated tasks, while the frequency of resident siblings in deployment, a key premise of the method, remains unmeasured. Finally, the gains are confined to short branches and slow networks: beyond branch length 32 the records exceed all lossless baselines, and at unpaced rates DFD-H costs up to 1.2× Full, underperforming even naive baselines in fast-network, long-branch scenarios.

Questions for authors
  1. Does the method generalize to other models?
  2. How long do sibling branches remain resident on average from completion until selection, and how frequently does this actually occur in practice?
2 comments
AuthorsOpenReview ↗

Model validation, measured sibling availability and application benefits

Thank you for emphasizing model generality and practical relevance. Your questions helped focus both the implementation work and the additional experiments.

Implementation improvements. The implementation now packs tensors contiguously, groups FP32 prediction, compresses in parallel and overlaps receiver prediction with repair decoding. The recovery principle remains unchanged: decay matching, captured branch factors and lossless repair under the same sibling-only receiver condition. Applicable optimizations also serve the baselines (Table 50, p.49).

Measured gains. On top of packing, grouped prediction and parallel coding, owned buffers and receiver overlap reduce lifecycle time at 5/8 matched endpoints at 1 Gbit/s. Reductions are 15.5% at the fixed two-token endpoint and 13.6% at the four-token endpoint, with identical wire streams and predictor hashes (Table 51, p.50). At 500 Mbit/s with a resident receiver sibling, the optimized implementation has median paired time reductions of 15.9% versus raw Full, 25.0% versus canonical lossless Full and 41.4% versus decay-scaled sibling XOR. It beats all five baselines on 40/48 short endpoints (Table 53, p.50).

Added experiments. The revision adds a second-model test of the final exact method, a decay-scaled sibling baseline and transfer timings that include encoding and restoration. It also adds a 100-request handoff replay and a 100-task ALFWorld candidate-storage study.

1. Model generality.

The submitted paper alreadly included the Qwen3.5-9B and Falcon-H1 studies reported in Appendices M.2–M.3 (p.32; revised p.33):

Model and studyReported result
Qwen3.5-9B, shared-base factual updates900/900 outputs match Full; both methods answer 894/900 correctly.
Falcon-H1-1.5B, local readout diagnosticsAUC 0.9354 across 183 completed diagnostic conditions.
Falcon-H1-1.5B, factor-based continuation96 conditions, up to 64 tokens, with a 5% tensor-payload budget. Source and target-difference factors match Full on 48/96 and 51/96 conditions.

For the final lossless DFD method (DFD-R), the revision adds 40 Qwen3.5-2B conditions, all matching native cache bits, fresh logits and 128-token continuations. Across the 32 short conditions, compact-head coverage is 98.8%, and the median paired record size is 17.5% of raw Full (§T.4, Table 55, p.51).

The final DFD-R construction uses scalar per-head decay. Section C.1 (p.15) notes that channel-wise decay in KDA and Gated DeltaNet-2 requires a different invariant-space argument.

2. Recovery triggers and sibling residency.

Recovery is triggered on 8/100 requests in the measured handoff, and all eight restorations are exact. The producer computes two complete four-token candidates and stages one at the receiver before selection. Selecting the other triggers recovery; the remaining 92 requests continue the staged candidate. The fork stays at the producer and is never transferred to the receiver.

Across the eight recovery requests, mean completion-to-selection time is 769.7 ms. Mean receiver residency before selection is 146.5 ms, and installation-to-replacement time is 1,190.9 ms, excluding audit overhead. All 100 staged siblings remain valid at selection (§6.4, p.10; §U.1, p.53).

3. Benefits across branch lengths and link rates.

The new application study shows storage savings with complete ALFWorld actions, including branches longer than four tokens. Generated actions span 3–14 tokens. DFD libraries are smaller on 25/100 tasks; within those 25, median savings are 17.59% against canonical lossless Full and 19.22% against decay-scaled sibling XOR (SX). Each total includes the anchor and all candidate records (Fig.2, p.10; §U.2, pp.53–54).

At 1 Gbit/s, median paired time ratios are 0.957 to canonical Full, 0.692 to SX and 1.272 to raw Full. Record ratios to raw Full are 0.389/0.681 at 8/16 tokens and exceed one at 32/64 (Tables 53–54, pp.50–51).

These timings charge required execution/capture, encoding, actual paced TCP loopback transfer, restoration and fresh forward, with a resident receiver sibling.

Data and code: data/rebuttal/application_followup_snapshot.json contains the task and handoff records; scripts/regenerate_application_tables.py reproduces their statistics. rebuttal/EVIDENCE_LOCATIONS.txt locates the original Qwen/Falcon results, raw logs and execution code. Revised page numbers refer to the 54-page manuscript.

Dear Reviewer,

With the November 18 (AoE) discussion deadline approaching, I’ve summarized how the original submission and the revised experiments address your questions. The revised results show bit-exact recovery on Qwen3.5-2B and Qwen3.5-4B, along with storage and latency gains measured over complete candidate libraries and end-to-end recovery.

Review DimensionEvaluated SettingKey ResultReference
Model generalityQwen3.5-9B factual updates900/900 matched Full, with 894 correctOrig. App. M.2
Cross-family evidenceFalcon-H1-1.5B (Mamba-2 hybrid)0.9354 diagnostic AUC across 183 conditionsOrig. App. M.3
Exact recovery on 2BQwen3.5-2B, 40 endpoints40/40 bit-exact at 17.5% of raw Full storageRev. Table 55
Exact recovery on 4BQwen3.5-4B, 64 endpoints64/64 bit-exact, with matching 128-token continuationsRev. Table 3
Short-branch storageQwen3.5-4B, 48 short endpoints29.6% of canonical lossless Full and 30.2% of sibling XOR storageAbstract / Table 3
Factor ablationComparison with decay-scaled XOR (SX)30.0% of SX storage, with savings at 48/48 endpoints and 29.8 MB savedRev. Table 49
Medium-branch storage8- and 16-token branches38.9% and 68.1% of raw Full storage, respectivelyRev. Table 54
Complete library storageEight groups of four candidates760.4 MB vs. 1,678.3 MB for raw FullRev. Table 56
Multi-token task archives100 tasks with 3–14-token actionsSmaller complete libraries on 25/100 tasks, with reported median savings of 17.59% vs. canonical lossless FullRev. Sec. U.2
Handoff validity100-request staged replay100% of staged siblings remained valid. Exact recovery was triggered in 8% of requests.Rev. Sec. 6.4
Sibling residencyReceiver hold before selection146.5 ms mean receiver hold and 769.7 ms total completion timeRev. Sec. U.1
Latency at 100 Mbit/sFull TCP lifecycle for exact recovery0.405× raw Full, 0.439× canonical lossless Full, and 0.413× SXRev. Table 53
Latency at 500 Mbit/sFull TCP lifecycle for exact recovery15.9% lower than raw Full, 25.0% lower than canonical lossless Full, and 41.4% lower than SXRev. Table 53
Win rate at 500 Mbit/sComparison with all five exact baselinesLower latency than all five baselines on 40/48 endpointsRev. Table 53
Latency at 1 Gbit/sComparison with lossless baselines0.957× canonical lossless Full and 0.692× SX, with a 15.5% reduction in the optimized comparisonRev. Tables 3 and 51
Channel-wise decayKDA and Gated DeltaNet-2Formal derivation of the distinct invariant-space formulationOrig. Sec. C.1
  1. Evidence across four models and two hybrid families. The original appendices report factual-update results on Qwen3.5-9B and diagnostic results on Falcon-H1-1.5B, a Mamba-2 hybrid. The revised Qwen3.5-2B study adds bit-exact cache and continuation recovery at all 40 endpoints, with 98.8% compact-head coverage. The Qwen3.5-4B evaluation confirms bit-exact recovery at all 64 endpoints.

  2. Handoff validity and complete library storage. The shared fork stays at the producer, and the receiver stages a logical anchor of about 55.7 MB. All 100/100 staged siblings remained valid through selection, with a mean installation-to-replacement time of 1,190.9 ms. Counting anchors, shared fields, indices, and every candidate record, DFD-R used 760.4 MB across the eight four-candidate groups, compared with 1,678.3 MB for raw Full and 1,435.6 MB for SX. Complete 3–14-token libraries were smaller on 25/100 tasks, with reported median savings of 17.59% vs. canonical lossless Full and 19.22% vs. SX.

  3. End-to-end exact recovery latency. The measurements include branch capture, grouped FP32 prediction, parallel compression, paced TCP transfer, overlapped repair decoding, and a fresh forward pass. Applicable optimizations are shared across baselines. At 100 Mbit/s, raw Full takes 2.47× as long as DFD-R. At 500 Mbit/s, DFD-R has lower latency than all five exact baselines on 40/48 endpoints. At 1 Gbit/s, the end-to-end handoff workflow takes 0.917× the latency of SX (Rev. Table 58).

Please let me know if you have any remaining questions or concerns about the paper’s soundness, informativeness, or scope before the discussion window closes.

Reviewer_r9eWOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper reconstructs a branch's recurrent state using a retained sibling state, decay matching, and a low-rank residual. Because floating-point reconstruction is not necessarily exact, it also stores a lossless repair. The evaluation distinguishes approximate reconstruction, exact restoration, continuation agreement, and downstream task success.

Critical strengths

The algebra is clean and avoids inverting the target transition. The paper explains the conditions under which receiver-side state is useful, rather than treating all recurrent models as interchangeable. This gives the method a reasonably clear scope.

The strongest empirical result is exact restoration using roughly 29% of canonical lossless Full's bytes on short branches. I also appreciate the negative results: approximate numerical agreement does not always preserve trajectories, and longer branches can erase the compression benefit. Reporting these distinctions makes the exact-restoration result easier to assess.

Critical weaknesses
  • The useful operating regime is narrow. Branches need to be short, and a suitable sibling must remain resident and unchanged. Repaired records become larger than the lossless baselines at branch lengths 32 and 64. The constructed ALFWorld forks demonstrate feasibility, but not how frequently a natural workload meets these conditions. A trace-based estimate of sibling availability and branch lengths would help establish relevance.

  • A stronger baseline would clarify the contribution. Direct XOR uses an unscaled sibling. Please also compare a decay-scaled sibling followed by the same lossless repair, without the captured low-rank factors. This would separate the benefit of decay matching from the additional information stored in the factors.

  • Fewer bytes do not yet establish faster serving. Table 2 reports about 2.33 seconds of local work for exact repair versus 0.35 seconds for raw Full. The transfer experiments use an earlier protocol. Please measure the final exact method with encoding, transfer, and restoration included, and identify the bandwidth or reuse conditions at which it breaks even.

Questions for authors

See weaknesses

2 comments
AuthorsOpenReview ↗

Added scaled-XOR control, final exact timings and application measurements

Thank you for suggesting the decay-scaled sibling control. It provides a direct way to distinguish the factors' contribution from decay matching.

Implementation improvements. Per-head operations and serial coding are replaced by grouped FP32 prediction and parallel compression. Typed packing and owned buffers reduce copying; receiver prediction overlaps repair decoding. These changes preserve decay matching, captured branch factors and lossless repair under the same sibling-only receiver condition. Applicable optimizations also serve the baselines (Table 50, p.49).

Measured gains. On top of packing, grouped prediction and parallel coding, owned buffers and receiver overlap reduce lifecycle time at 5/8 matched endpoints at 1 Gbit/s. Reductions are 15.5% at the fixed two-token endpoint and 13.6% at the four-token endpoint, with identical wire streams and predictor hashes (Table 51, p.50). At 500 Mbit/s with a resident receiver sibling, the optimized implementation has median paired time reductions of 15.9% versus raw Full, 25.0% versus canonical lossless Full and 41.4% versus decay-scaled sibling XOR. It beats all five baselines on 40/48 short endpoints (Table 53, p.50).

Added experiments. The revision adds your proposed scaled-XOR control, transfer measurements of the final exact method, a 100-request handoff replay and a 100-task candidate-storage study.

1. When does the recovery setting occur?

The new handoff experiment produces eight actual sibling-only recoveries in 100 requests, all exact. The receiver stages one completed candidate; recovery is triggered when selection chooses the other. The fork stays at the producer. The remaining 92 requests continue the staged candidate directly (§6.4, p.10; §U.1, p.53).

Storage benefits also extend beyond the 1–4-token tests. A separate ALFWorld archive study uses complete action candidates of 3–14 tokens. 25/100 tasks have smaller DFD libraries; within those 25, median savings are 17.59% against canonical lossless Full and 19.22% against scaled XOR. These totals include the anchor and all records (Fig.2, p.10; §U.2, pp.53–54). Record ratios to raw Full are 0.389/0.681 at 8/16 tokens and exceed one at 32/64 (Table 54, p.51).

2. Do the factors add value beyond decay matching?

Yes: the median paired byte ratio to the requested baseline is 0.300. Decay-scaled sibling XOR (SX) uses the same repair codec without factors. Repaired DFD-H, denoted DFD-R, reduces complete bytes at all 48 short endpoints, with a 95% prompt-cluster interval of [0.287, 0.329]. Repair savings exceed the additional non-repair fields by 29.813 MB on average. This directly measures the factors' contribution after charging their storage (§T.1, Table 49, p.48).

3. Does the final lossless implementation deliver a time benefit?

The final exact comparison measures one recovery transaction, including encoding, transfer and restoration. Median paired time ratios are:

Offered rateDFD-R / raw FullDFD-R / canonical FullDFD-R / SX
100 Mbit/s0.4050.4390.413
500 Mbit/s0.8410.7500.586
1 Gbit/s1.2720.9570.692

The observed ranking against raw Full changes between 500 Mbit/s and 1 Gbit/s. Timing starts with a resident receiver sibling and charges required execution/capture, encoding, actual paced TCP loopback transfer, restoration and fresh forward. All 64 endpoints match native cache bits, fresh logits and continuations (Table 3, p.7; Table 53, p.50).

Thank you again for suggesting a control that makes the source of the compression gain much clearer.

Data and code: data/rebuttal/ and rebuttal_artifact/ contain display inputs; scripts/ reproduces the tables. rebuttal/EVIDENCE_LOCATIONS.txt lists raw logs, analyses and execution code. Page numbers refer to the revised manuscript.

I've read the response and I have no further questions. I will keep my score as-is.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.