acceptodds
Under review as a conference paper at ICLR 2027

The Circle Was Hiding a Helix: Helical Reduced Representations for Exact Vector Symbolic Binding

Abstract

Vector Symbolic Architectures (VSAs) store symbols as vectors and combine them by binding and bundling. A bundle of bound pairs in dimensions holds only about pairs, where is the codebook size. The cause of this limit lies in the binding operation, not in the vectors. Binding is multiplication in a group , and bundling is addition in the group algebra . If is compact, every representation is unitary. Every bound term then has magnitude 1, superposed terms are separable only by statistics, and the capacity bound follows. Bundling cannot escape to the Lie algebra of either, because distributivity forces any such operation to be a representation. We introduce Helical Reduced Representations (HeRR) bind on the non-compact group . Their vectors are Laurent polynomials. Binding is polynomial multiplication, bundling is polynomial addition, and nesting depth is polynomial degree. Circular convolution is this ring modulo , so existing VSAs are quotients of HeRR. Storing a HeRR in fixed space is a choice of characters of , and three independent capacity budgets follow: dimension, number of evaluation points, and numerical precision. Non-unitary characters, absent from compact groups, supply the third. Two exact decoders recover a bundle without observing crosstalk, so the codebook need not be random. At 128 complex numbers of storage, HeRR retrieves 96 role-filler pairs at accuracy 1.00, against 0.13 for FHRR at equal storage. With a correlated codebook, FHRR reaches 0.00 at 8 pairs while HeRR remains exact. The binding capacity limit is therefore a property of compact binding groups, not of vector representations.

Primary area:
neurosymbolic & hybrid AI systems (physics-informed, logic & formal reasoning, etc.)
Keywords:
vector symbolic architectures, hyperdimensional computing, holographic reduced representations, binding capacity, group algebra, representation theory, compositional representations, sparse interpolation

Reviews

Among:
Shape:
1234this paper, Bottom 2%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.25 over 4 reviews, above 2% of the others.
Reviewer_QtzrOpenReview ↗
Rating
1
Confidence
4
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper extends a new variant of Vector Symbolic Architectures (VSAs), which store and process symbols (e.g., by means of a binding operation that acts as group multiplication) in a distributed vector representation format. The paper proposes Helical Reduced Representations (HeRR), a non-compact group Z. Binding is implemented as polynomial multiplication. Synthetic experimental results with random and correlated codebooks show better recall performance compared to the previous variants of VSAs.

Critical strengths

Proposing a VSA on the non-compact group Z, providing extra capacity budgets.

Critical weaknesses
  1. The paper has barely touched on the crosstalk noise and its consequences. For practical use cases of VSA inside neural networks where codebooks and exact recovery methods are not typically available/applicable, the crosstalk noise is the dominant factor. The cosine similarity sharply drops with 1/sqrt(K), independent of D. Could HeRR binding provide any improvement here?

  2. On reducing the crosstalk effect, the state-of-the- art factorizers for distributed sparse block codes are missing (https://journals.sagepub.com/doi/10.3233/NAI-240713)

  3. What is the computational and memory complexity of the proposed decoders? How does it compare with prior decoders?

  4. The text of the paper seems to be heavily written by AI. Its language needs refinement to be more readable for humans.

Questions for authors

The paper is currently missing real-world experimental results to clarify its significance (if any). Here are some domains in which HeRR can be benchmarked against other VSA variants:

  1. On abductive reasoning tasks, e.g., for solving Raven's progressive matrices (https://www.nature.com/articles/s42256-023-00630-8). The current reasoner is based on FHRR and fractional power encoding and runs in real time on CPU. It could be great to apply HeRR in this reasoner and cross-compare the quality and speed as well.

  2. Neural network tasks like eXtreme multi-label classification (https://arxiv.org/pdf/2109.02157), which at that time was difficult for HRR. How does HeRR compare here?

Reviewer_rwjeOpenReview ↗
Rating
1
Confidence
3
Informativeness
2
Presentation
1
Soundness
1
Read review
Paper summary

This work targets interference and retrieval erros in vector symbolic architectures, aiming to store and accurately recover more symbolic associations within a fixed-size representations.HeRR encodes symbols using Laurent polynomials with integer heights and use Fourier separation, magnitude levels, and sparse interpolation for retrieval, supported by algebraic analysis and claimed exact-recovery guarantees. On controlled synthetic tasks, the paper reports strong retrieval performance with correlated codebooks and nested structures, including 100% retrieval accuracy for 96 stored pairs using 128 complex values, compared 13% for FHRR.

Critical strengths
  • Synthetic results provided are promising in several controlled comparisons with matched bundle-storage sizes.
  • Stress tests are relevant and help identify the method's benefits and operating constraints
Critical weaknesses

First of all, the writing of this work is very hard to read. I believe all the maths and jargons should be written in an intuitive and easily readable narratives, especially considering that ICLR is a general AI conference. Maybe for domain specific venues and journals it shouldn't be an issue, but the writing of this work is a major hurdle for delivering its content to generic ICLR audience.

Second, I am not convinced that the problem addressed in this work is anywhere interesting nor impactful. Why are the authors doing this research? What is of interest to the AI community?

Also, I find substantial faults in the maths of the manuscript, grave enough for me to be skeptical on the technical soundness of this paper.

Maths fault list

  • page 3 lines 141-147, Proposition 2.9: "Equivalent to unitary" does not imply norm preservation in the original coordinates. A compact group representation can be made unitary by choosing a suitable inner product, but its matrices need not have operator norm one under the original Euclidean norm. The proof skips this distinction when making its norm-one conclusion.
  • page 3 lines 149-154: The claimed capacity ceiling does not follow from compactness. Equal sized terms do not force interference to behave randomly or require statistical decoding, as structured unitary codebooks can separate terms exactly through Fourier decoding. So the claimed DlnN ceiling does not come from compactness alone.
  • page 3 Table 1 line 112, line 124-126: Ordinary HRR is incorrectly covered by the compact group argument. General HRR binding uses a weighted combination of cyclic shifts, whose Fourier eigenvalues need not have unit magnitude. The fact that the individual shifts are unitary therefore does not make arbitrary HRR binding unitary.
  • page 3 line 161, page 4 line 163-173: Proposition 2.10 changes the binding operation during its proof. With the stated additive binding, setting K=1 gives psi(X+A)=A+psi(X), but the proof instead writes psi(X+A)=phi(A) * psi(X). Why does it change here?
  • page 4 lines 199-200: HeRR cannot reproduce every existing VSA through folds or quotients. HeRR’s multiplication is commutative, and every algebraic quotient or homomorphic image of it remains commutative. It therefore cannot reproduce general non-commutative matrix binding, such as that permitted by MBAT and GHRR, while preserving the binding operation.
  • page 4 lines 192-193, 199: Polynomial degree is incorrectly identified with nesting depth. Binding adds assigned heights, whereas nesting depth depends on how operations are grouped: four height-one atoms give degree four for both (ab)(cd) and ((ab)c)d, although their tree depths differ. Degree can represent depth under a specially designed height-assignment scheme, but it does not intrinsically measure arbitrary nesting depth.
  • page 5 lines 246-249 Eq (6): Decoder A starts from an incorrect sampling identity. Equation (6) incorrectly places r^{h_k} inside the exponential node.
  • page 5 lines 246-249, 263: Decoder A does not guarantee unique recovery. At r=1, which the decoder explicitly permits, the different one-term bundles R(t)=1 and R(t)=t^M produce identical samples. Thus unrestricted integer heights cannot be recovered uniquely, regardless of the decoder; additional height restrictions or a different sampling design are necessary. Appendix A specifies a different grid on p. 10, line 500, which requires its own recovery justification.
  • page 5 Table 2 line 218 and 253-263, page 10 lines 515-516" The specified algorithm cannot attain the advertised M/2 recovery capacity. The algorithm removes one row from a signal-subspace matrix with L=floor M/2 floor rows, so recovering K nodes requires L-1 >= K, hence M>=2K+2. Although M>=2K can be a valid necessary bound, it is insufficient for this implementation, whose appendix explicitly caps the recovered rank at L-1.
  • page 5 lines 261-262: The height-recovery formula ignores complex-logarithm branches. Even assuming the intended relationship z=(romega)^h, the principal-logarithm quotient log z / log(romega) need not equal h, because phase wraps around.
  • page 5 lines 267-269, page 6 line 284: Decoder B mixes original and shifted query coordinates. Unbinding by t^{-h_q} moves the target to height zero, but subsequent steps still select its original coil c_q and request its original level l_q. Unless these indices are explicitly updated, the algorithm can filter out its own target or return the wrong level, even for a single stored item.
  • page 6 lines 281-286: Correct phase rounding does not guarantee correct level rounding. The argument controls phase error but overlooks the magnitude rounding used to identify a term’s level.
  • page 6 lines 277-278, 315-317, page 10 lines 504-506: Distinct integer heights do not guarantee exact unitary-coil recovery. Fourier filtering requires heights to be distinct modulo M, since heights zero and M have identical sampling signatures. The appendix’s sampling of distinct heights does not ensure this so the stated protocol does not justify the exact recovery guarantee.
  • page 4 line 190-197: The claimed HRR reduction leaves an unaccounted dimension. Because every polynomial coefficient already has D components, imposing t^D=1 produces an algebra with D^2 complex dimensions, corresponding to D parallel HRR channels. Identifying this with one ordinary D-dimensional HRR requires scalar coefficients or an explicit channel restriction.
  • page 3 Table 1 line 114, 119-120: A one-parameter subgroup of a compact torus need not be compact.
Questions for authors
  • why is this problem interesting? Why solving this problem is to be impactful? What are the impacts the authors aim to make for AI community with this work?
  • The maths are so sloppy overall, honestly I smell LLM generated maths without proof reading. Please address to my concerns above, I may be wrong on some of them but I am confident this work is not technically sound. Unless authors convince me otherwise, I will retain my rating to the end.
  • I don't see any analysis on computational cost and comparison to baselines. Can authors provide it?
  • I feel baselines are insufficient, could the authors include other related baselines such as structured VSA baselines or conventional indexed symbolic storage? The following prior work seems to be relevant: "Variable Binding for Sparse Distributed Representations: Theory and Applications" Paxon Frady et al.
Reviewer_BmwsOpenReview ↗
Rating
1
Confidence
2
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

The paper studies how to increase the capacity of fixed-size vector memories for role–filler pairs while supporting binding and bundling. HeRR stores evaluations of a polynomial whose integer exponents encode the positions assigned to the pairs; two decoders use the resulting phase and magnitude differences to recover them. With 128 complex numbers of storage, the authors report accuracy of 1.00 for 96 pairs, compared with 0.13 for FHRR at the same budget. The paper claims that changing the algebraic structure used for binding can overcome the capacity limits of existing methods.

Critical strengths

The paper uses polynomials to describe binding and bundling within one framework and stores pairs at different magnitude levels. This is an interesting modeling perspective and illustrates the potential of representation theory for designing symbolic memory encodings.

Critical weaknesses

1. The main experiment changes the query requirements: queries need an additional address, but the cost of obtaining it is omitted.

FHRR aims to store KK role–filler pairs in a single fixed-length vector. This memory occupies O(1)O(1) space with respect to KK. A query supplies only the role, and readout and filler matching also take O(1)O(1) time with respect to KK. Let SFS_F be the stored FHRR vector, and let rkr_k and jkj_k be the role and filler ID for pair kk. The query inputs and target output are:

Query⁡(SF,rk)⟶jk.\operatorname{Query}(S_F,r_k)\longrightarrow j_k.

Along with the role, HeRR's main experiment requires the target pair's address: the position assigned to that pair during encoding. Let SHS_H be the stored HeRR memory and hkh_k the target address. Its query interface is:

Query⁡(SH,rk,hk)⟶jk.\operatorname{Query}(S_H,r_k,h_k)\longrightarrow j_k.

This difference is explicit in Decoder B in Section 3.4 and in run_spiral, the supplementary code for Table 5. Addresses are assigned randomly, and the query code directly uses the target address without showing how to obtain it from the role alone. The two methods therefore use memory vectors of the same size, but their queries require different information.

One way to restore the original query interface would be to store a mapping from roles to addresses, requiring O(K)O(K) additional index entries in a direct implementation. This storage must be counted. In fact, once these entries are allowed, each role can be mapped directly to its filler ID, eliminating the need for HeRR decoding. Another option would be to search through addresses at query time, incurring a worst-case O(K)O(K) cost per query. The additional space or time cost of either approach must be included in the comparison.

The main experiment does not explain how to avoid this overhead while maintaining comparable query efficiency, and it excludes the storage and computational costs of obtaining the address. Therefore, the accuracy advantage in Table 5 does not establish that HeRR improves on FHRR's original associative memory task with fixed storage and queries that provide only the role. This is my main reason for recommending rejection.

2. The proof of Proposition 2.10 is inconsistent with the definition of additive binding.

Let X be the input vector, A the vector bound to it, and ψ the encoding function. For a single term, Eq. (1) gives an additive relation, while the first step of the proof uses a multiplicative one:

  • From the definition: ψ(X + A) = A + ψ(X).
  • First step of the proof: ψ(X + A) = φ(A)ψ(X).

Here, φ is a function introduced in the proof. The proof does not justify this transition, so its subsequent derivation of the exponential form is invalid.

Questions for authors
  1. How is the target address obtained when a query provides only the role? Are the storage and computational costs of obtaining it included in the comparison with FHRR? If the application itself provides the address, what advantage do the current experiments demonstrate over storing the filler IDs directly at those addresses?
  2. How does the first step of the proof of Proposition 2.10 derive the multiplicative relation above from additive binding?
Reviewer_bc9LOpenReview ↗
Rating
2
Confidence
3
Informativeness
3
Presentation
2
Soundness
2
Read review
Paper summary

This paper presents a new vector symbolic architecture based on helices, Helical Reduced Representations (HeRR). The paper supports this endeavor with proofs arguing limitations with other solutions as well as algorithms and experiments showing its capacity.

Critical strengths

The main strength of this paper is its introduction of Helical Reduced Representations (HeRR) VSA. This is supported by a mathematical description as well as two algorithms for the more complex unbind operations. The motivation is well supported with short proofs, and includes definitions for readers that might not be familiar with many necessary definitions. The benefit is limited by its uneven expectations from the reader. The proofs seem to assume stronger knowledge with linear algebra and Lie algebra, and less knowledge of abstract algebra basics. The proofs are also sparse.

Critical weaknesses

The organization of the paper is poor. For example, much of the information in the appendix A would be better in a ‘background’ section that introduces other works. This is also seen in the proofs using ‘facts’ from section 2.1. It would be easier to understand the proofs if the information was presented in-line. In general, the proofs should be expanded to be easier to read. Section 4 (Results) needs more clarity in the introduction to each test and in the evaluation of the results shown. The protocol for the experiments is based on the restrictions of double-precision, but the related information is spread across earlier sections (e.g. 3.3). Again, some information from the Appendix may be better placed in the main paper. Furthermore, the paper appears to only compare against FHRR due to its intrinsic similarities, but with the dimensionality fixed, it should be possible to compare against more distant VSAs.

Questions for authors

For many VSAs, there are three functions: binding, bundling, and permutation. This paper focuses on binding and bundling, but does HeRR have a natural permutation function? For future work, it may be useful to generate a visualization of the HeRR algebra to aid the reader.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.