acceptodds
Under review as a conference paper at ICLR 2027

Memory-Carrier Contrast Set: Isolating Temporal Carry in Frozen Language-Model Control

Abstract

Long-horizon control with a frozen language model poses an attribution problem: a recurrent adapter may win because it transports evidence across decisions, because it changes the policy's current-feature access, or because its training contract differs from the retained-context baseline. The Memory-Carrier Contrast Set (MCS) turns these alternatives into explicit interventions on carry and access while retaining backbone features, supervision, head geometry, optimizer, and interaction budget. We instantiate MCS with Layered State Bottleneck (LSB), which mixes stop-gradient features from three transformer depths into a 128-dimensional GRU state that alone drives the decision heads. Across prespecified persistent-history tasks in MiniGrid-S13, TextWorld, and ALFWorld, the matched carry-by-access factorial assigns 4.1 success points (95% CI [2.9, 5.3]) to temporal carry; without bypass, the task-wise effects are 4.7–5.5 points, while the S9/S11 effects remain inside a two-point equivalence margin. Under a matched LLaMA-2-7B contract, LSB also exceeds a native FLC-4096 controller by 5.4 points ([3.9, 6.9]); the measured uncached full-prefix implementation processes 17.6 times as many retained tokens per decision and has 2.8 times the p95 latency. A final-layer GRU reproduces the concentration on persistent-history tasks, and task-only LSB retains a 2.6-point advantage over projected bypass while remaining within 0.3 points of the DTQN-style encoder. MCS therefore identifies temporal transport as the active ingredient in this frozen-controller regime and places bounded recurrence on a favorable success–resource operating point.

Then back it, or bet against it.

Related papers

Open the market on this paper to see 7 more related papers.