acceptodds
Under review as a conference paper at ICLR 2027

Jue: An Energy-Based Concept Field for Online Relational Memory on Frozen Transformers – What Addressing Suffices For, What Propagation Adds, and a Matched-Interface Protocol for Telling Them Apart

Abstract

A deployed language model is frozen: retrieval and fine-tuning fetch or overwrite information, but nothing accumulates inside a system that could be said to be learning. We present Jue, an energy-based concept field attached to a frozen transformer that accumulates knowledge online through local Hebbian updates—no gradients reach the base model, and language enters and leaves through explicit provenance contracts. Disabled, the system is exactly the base model. All results are reported under a falsification-first protocol: preregistered thresholds and conclusion wordings, frozen materials, constant provenance with automatic staleness, and a permanent generation-level endpoint CI. The contribution is a three-act mechanism study. Act one: across eight non-conditioned geometric readout variants, accuracy on a divergent multi-relation testbed pins at exactly —the first-taught relation permanently captures a concept's output. Act two: relation-conditioned addressing repairs this completely () and, as our own strongest baseline, sweeps the capability matrix—addressing suffices for single-hop disambiguation, and the field adds nothing measurable there. Act three: under isolation-merge teaching, cut-edge causality holds at the string level on blind-registered chains spanning 2–4 hops with zero shortcut contamination (), while without isolation chain formation collapses (). But soft multi-candidate retrieval reaches where propagation reaches , and a matched provenance-aware typed graph memory reproduces every measured property of the field and exceeds it on provenance completeness. That comparison runs under a matched-interface protocol we state separately and consider reusable—identical write contract, readout, provenance and intervention interfaces, matched budget, and blind materials with both architectures frozen at registration—six conditions without which an equivalence result is uninterpretable. Three discriminative probes on the field's native ground returned one tie (a zero-relaxation ablation isolates the dynamics' contribution at ) and two instrument blocks we localize here. The energy dynamics are not yet doing irreplaceable work on any surface we could measure; the contribution is accordingly the audit discipline, these measured equivalences, and the complete falsification record—nineteen stop-gates, four hard stops, four retired claims, and the author hypotheses struck down by the project's own data.

Primary area:
neurosymbolic & hybrid AI systems (physics-informed, logic & formal reasoning, etc.)
Keywords:
online learning, associative memory, energy-based models, frozen language models, relational memory, graph memory, mechanistic evaluation, falsification, negative results, Hebbian learning

Reviews

Among:
Shape:
1234this paper, Bottom 2%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.33 over 3 reviews, above 2% of the others.
Reviewer_wDFiOpenReview ↗
Rating
1
Confidence
4
Informativeness
1
Presentation
1
Soundness
1
Read review
Paper summary

This paper studies Jue, an external relational memory for a frozen Qwen3-0.5B. It combines hidden-state concept nodes, a Hebbian quadratic energy field, relation-conditioned controller lookup, and provenance anchors that replay stored answer tokens. On small synthetic tests, unconditioned field readout scores 0.333 while controller lookup reaches 1.000; a typed graph matches or exceeds the field, and disabling relaxation affects only 3/117 degraded-cue cases. Sequential teaching, conflicting updates, scaling, and natural-language realization remain unreliable. The evidence therefore indicates that structured addressing and token replay—not attractor dynamics—explain most successes, making the falsification analysis stronger than the proposed memory mechanism.

Critical strengths
  • The authors expose negative results, instrument failures, repairs, and scope boundaries rather than hiding them. The causal edge-cut criterion and explicit separation of retrieval, field state, and realization are scientifically healthier than relying on endpoint accuracy alone and make the narrow negative conclusions easier to interrogate.

  • The paper includes an ablation that isolates relaxation and a matched typed-graph comparator, then retires its own accuracy and irreplaceability claims when those tests fail. This is consequential evidence against the authors' preferred mechanism and is the strongest reason to view the submission as an honest mechanism audit rather than a collection of cherry-picked wins.

Critical weaknesses
  • The claimed system does not demonstrate the proposed capability. Its headline success is direct oracle-structured key-value addressing in the controller, while the answer is realized by replaying a stored gold token sequence. Relation type and direction do not reside in the symmetric energy field, and the field is neither necessary nor beneficial on any measured surface. Its actual online behavior fails at modest scale, under conflict, under ordinary sequential chain teaching, and in all three natural demonstrations. Repairing this would require a different end-to-end system and evaluation, not a small revision.

  • The empirical and reproducibility standards are insufficient. Results come from one very small model and tiny synthetic, internally generated suites, with weak bespoke baselines, no established benchmark or distributional replication, and a severely incomplete treatment of close prior work. The artifact omits the core implementation and runnable environment, while its self-hashes, self-authored registrations, and stored outputs cannot independently validate the many post-failure repairs. This prevents verification of the narrow results and makes the broader trustworthiness claims substantially stronger than the available evidence.

Questions for authors
  1. For every headline result, can the authors identify the exact answer-selection path and state whether changing W or running relaxation can alter the selected answer after holding the controller dictionary fixed? In particular, please provide a no-field/no-Hebbian ablation across all controller-level tables, not only the 3/117 attractor probe. If those results are unchanged, what learned relational content remains in Jue rather than in relations[(source, relation_type)]?

  2. Can the authors release the complete implementation at the cited commits, including the missing exp07_phase1_protocol and src/* modules, dependency lockfile, checkpoint identification, and one-command scripts that regenerate the decisive tables from raw materials? Can the claimed preregistrations be tied to an independently timestamped history rather than unverifiable hashes embedded in the final snapshot?

  3. Does Jue retain a measurable advantage or even reliable operation when entity/relation fields are inferred from raw language rather than supplied by the controller, answers are generated without per-position gold-token scheduling, memory is realistically sized, and comparisons use strong contemporary retrieval/graph-memory baselines across multiple model sizes and independently sampled tasks? These tests are necessary to support the current motivation; otherwise the claims should be limited to a small synthetic controller audit.

Reviewer_LTc6OpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
1
Soundness
1
Read review
Paper summary

The paper proposes Jue, an energy-based memory module attached to a frozen language model, which stores associations through local Hebbian updates and supports retrieval through activation propagation. The experiments investigate relation-conditioned addressing and multi-hop memory retrieval. However, the results do not establish a clear advantage over simpler retrieval and graph-based alternatives.

Critical strengths

None

Critical weaknesses

Limited empirical evaluation. The evaluation relies on small, controlled testbeds rather than established benchmarks. For example, the ambiguous-relay comparison contains only 20 constructed examples. These experiments provide insufficient evidence of robustness, scalability, and practical usefulness.

Unclear and incomplete method presentation. Although individual components are described, the paper lacks a clear, unified account of the complete method. Important operations are scattered across the main text and appendices, making the overall pipeline difficult to follow and reconstruct.

No demonstrated advantage over simpler alternatives. The paper includes several baselines, but the proposed energy-based mechanism does not show a convincing benefit. Relation-conditioned addressing matches its single-hop performance, while soft retrieval and typed graph memory match or exceed its results on other evaluated tasks. Consequently, the additional architectural complexity is not sufficiently justified.

Questions for authors

None

Reviewer_Un9ZOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
1
Soundness
2
Read review
Paper summary

This paper introduces an "energy field" layered atop a frozen Qwen3-0.5B model: concept nodes are activated via relaxation dynamics, relational knowledge is encoded in symmetric weights WW updated by local Hebbian rules, and both inputs and outputs are governed by explicit "provenance contracts." The authors organize their findings into three acts. Act I: Unconditional geometric readout yields a constant value of 0.333 on a test set comprising 4 entities and 3 relations. Act II: Conditional addressing—using (entity, relation) pairs as keys—achieves a score of 1.000; the authors' custom "A-relcond" baseline performs on par with their proposed method, "Jue," across all capability dimensions. Act III: Field propagation demonstrates "interruptible" causality within multi-hop chains (16/18 chains formed; 22/22 conditional causal links), though its accuracy is surpassed by a soft-retrieval baseline (15/20 vs. 20/20). Under a "matching interface protocol," a typed graph memory with provenance reproduces all measurable properties of the field while offering superior provenance integrity. An ablation study with zero relaxation reveals that the net contribution of the dynamics is merely +3/117.

Critical strengths

As for the strengths, I believe the author is candid about the limitations of their work; an entire section of the paper is dedicated to discussing "what Jue cannot yet do," and Figure 1 (on the right) explicitly lists four claims that have since been retracted. Furthermore, the conclusion clearly states that energy dynamics does not currently play an indispensable role.

Critical weaknesses
  1. In my view, the current manuscript is in a draft state and does not meet the readability standards required for the main proceedings. Crucially, key results are relegated to the appendix, while the main text focuses primarily on narrative and limitations, rendering the text itself internally inconsistent. Furthermore, Figures 5 and 6 still bear the label "(draft)" in the submitted manuscript. The abstract is overly long, discusses the results excessively, and lacks a relevant review of the existing literature.

  2. The authors conclude that single-hop disambiguation is achieved through "conditional addressing," while performance on multi-hop tasks falls short of that attained by "soft retrieval" methods. Other features of the "typed graph memory" were successfully replicated, though the "dynamics" component was found to contribute only 3/117th of the overall performance. In other words, the "energy field" mentioned in the title plays no meaningful role according to any measurable metric. The authors have essentially repackaged their rigorous internal review and falsification process as a research contribution. In my view, this pertains only to the quality of the research process itself, rather than constituting a scientific or technical achievement that others could build upon. While "negative results" are certainly worthy of publication, such papers must address questions of genuine interest to the research community. Given that the primary function of this paper is to debunk the very architecture the authors themselves originally proposed, there is little for an reader to learn from it.

Questions for authors
  1. Is there any task where the field is expected to succeed—and which can be measured at the current scale? If there is no such expectation, what is the design rationale for including an "energy field" in this system?
  2. How does a minimalist system—consisting of frozen LM hidden-state encodings, a key-value table, an explicit graph, and forced decoding—perform across the tasks presented in this paper?
  3. If forced decoding is removed, is there any positive evidence that the base model can naturally leverage this memory within the context to generate multi-token responses?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.