- Rating
- 1
- Confidence
- 4
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 1
Read review
Paper summary
The paper proposes FastOmniTMAE, a modified Omni TM-AE embedding model that replaces the global class-sum update probability with a clause-local rule intended to remove synchronization and expose clause-level parallelism. It also presents CPU, CUDA, OpenCL, and SoC-FPGA implementations and introduces two evaluation suites covering classification diagnostics, word similarity, clustering, and cross-platform training time. The paper reports up to a training speedup in the classification experiment, approximately in the word-similarity experiment, and – FPGA speedups over a single local ARM core. It further argues that GPUs are poorly matched to TM training and that dedicated FPGA logic is a more suitable platform.
The submitted package contains substantial C, CUDA, OpenCL, Python, and RTL code. However, the central comparisons are not currently valid enough to support the conclusions. The classification evaluation has a reachable stale-cache error, exposes the target feature directly in the input, and reports clause-level diagnostics rather than independent downstream classification. The H100 result comes from an implementation with extremely low launch parallelism, so it cannot support a platform-level conclusion about GPU suitability. Finally, the proposed method changes several learning and implementation choices simultaneously without the controls needed to attribute gains to the local probability rule.
Critical strengths
None that currently supports acceptance. The submission contains real engineering work, including C, CUDA, OpenCL, and RTL implementations with clause engines, state buffers, and AXI integration. However, implementation breadth alone does not validate the algorithmic or platform-level conclusions, and the current experimental comparisons do not isolate those conclusions cleanly.
Critical weaknesses
-
The main embedding-quality evidence is not a valid independent evaluation of the claimed learning improvement.
The classification experiment in Figure 4 has multiple consequential problems.
First, the submitted incremental prediction path caches a reverse literal-to-clause index and false-literal counts. This cache is initialized from the automata states on the first prediction, but the training method subsequently updates those states in place without invalidating the cache. In an execution test using the submitted Python training method and recompiled native C code, training changed the automata states while the cache remained marked as initialized. The same input then produced prediction 1 with the stale cache and prediction 0 after only forcing cache reconstruction. Thus, the reported epoch-wise classification path can evaluate clauses that no longer correspond to the current trained state. This does not prove that the entire Figure 4 curve is caused by the bug, but it prevents that curve from supporting claims about training stability or sustained classification superiority until the relevant version is identified or the experiment is rerun.
Second, the example generator constructs positive examples only from documents containing the target token and negative examples only from documents not containing it, while retaining the target feature in . Therefore, for nondegenerate targets, the generated task satisfies
Calling the submitted native generator on 256 synthetic examples confirmed that directly reading this one input bit recovered all 256 labels. This permits correct prediction without learning semantic context.
Third, the reported precision, recall, and F1 are not obtained by training an independent downstream classifier on frozen embeddings. The benchmark calls per-clause
clause_precisionandclause_recall, averages the non-NaN clause statistics, and computes their harmonic mean. Training and metric computation receive the sameX_train, while the metric functions regenerate examples from the supplied corpus. Consequently, the stated “500 inference examples” do not, from the submitted path, establish an independent held-out embedding-classification evaluation.The other quality comparisons do not repair this problem. FastOmniTMAE’s similarity function returns the coordinate of word 2 in word 1’s learned vector, whereas Word2Vec, FastText, and GloVe are evaluated using cosine similarity. The former can be directional: under a synthetic invocation of the submitted function,
score(alpha,beta)=9andscore(beta,alpha)=2, while the cosine baseline was symmetric. Different representation-specific scores may be legitimate, but they do not constitute a common geometric evaluation without clear interpretation and additional controls. Moreover, when the original token cache is absent, the submitted baseline path reconstructs “sentences” from sparse row indices, losing word order and repeated-token frequency; two different bags with different original orders both became the same deduplicated sequence. The package does not contain the cache or result snapshot needed to determine whether the reported baseline runs avoided this path.Together, these issues mean that the paper has not established that FastOmniTMAE preserves or improves embedding quality under a common, leakage-resistant, independent evaluation.
-
The algorithmic and hardware comparisons are not designed to identify the claimed mechanism or platform conclusion.
FastOmniTMAE is presented primarily as removing a global synchronization dependency while preserving the original learning behavior. However, Algorithm 1 replaces the global class sum with a single local contribution , which changes the stochastic learning dynamics rather than merely rescheduling an equivalent computation. In the submitted configurations, clause weights are initialized to fixed signs and are not updated. Negative-weight clauses are skipped, while only positive-weight clauses train. With positive weight and , the local update probability is approximately fixed at 0.5:
- 0.5 for an inactive clause;
- 0.499975 for an active positive example;
- 0.500025 for an active negative example.
A native 512-update test confirmed that weights remained fixed, negative-weight clause states did not change, and positive-weight clause states did. Therefore, a matched fixed- control is essential. Without it, the experiments do not show that the proposed local formula is necessary. Nor do they isolate removing synchronization from simultaneously removing weight learning, changing clause gating, altering feedback behavior, and moving execution into different native implementations. The paper also lacks substantive empirical comparison with directly relevant prior asynchronous and local-feedback TM approaches that it cites.
The GPU conclusion is even less supported. The appendix itself states that each profiled H100 update launches one 64-thread block, which can occupy at most one of 132 SMs. The default submitted hardware runner uses 128 threads and blocks; with 32 clauses, this is still one block, and only 32 threads perform clause work. Target tokens are processed sequentially, and one kernel is launched per example. Under the Table 2 configuration of 44 targets and 8,000 examples per target, this implies 352,000 sequential single-block update launches. This implementation is structurally incapable of using the H100 at meaningful scale. The statement that CUDA exhibits poor performance “despite full GPU utilization” and that the result demonstrates a general mismatch between GPUs and TM logical operations is therefore not justified. At most, the experiment shows that this particular under-parallelized CUDA implementation performs poorly.
The FPGA results demonstrate execution of a hardware design, but they do not establish broader efficiency. The reported ZCU104 speedup is relative to one ARM core on the same board; the ZCU104 time of 119.9 seconds remains approximately the desktop i9 single-thread time of 57.7 seconds. The designs also consume 86.54% and 75% of available BRAM on ZCU104 and Zybo, respectively. No power, energy, timing report, or bitstream artifact is supplied, so “small hardware footprint” and lower-power implications are not supported by the reported DSP count alone.
Thus, the experiments currently establish only that the submitted FastOmniTMAE implementation outperforms particular baseline implementations under selected configurations. They do not identify the local probability rule as the cause, establish equivalence to the original learning mechanism, or show that GPUs are inherently unsuitable relative to FPGAs.
Questions for authors
-
Was Figure 4 generated with the submitted incremental clause-evaluation cache implementation? If not, what exact code revision or cache-invalidation fix was used? Do existing run artifacts show that the clause cache was rebuilt after each training update or epoch?
-
Was the target-token feature masked from in the classification experiment? If so, where does that occur in the experimental version? If not, how should the classification result be interpreted given that the submitted generator exposes ?
-
Are Figure 4’s precision, recall, and F1 values clause-level diagnostics computed from the training corpus, or results from an independent downstream classifier evaluated on held-out examples? How are the stated 500 inference examples used in the submitted metric path?
-
Is there an existing fixed- control using the same native implementation, training budget, clause gating, and weight behavior? This would materially affect whether the local probability formula, rather than the surrounding implementation changes, explains the result.
-
Which existing comparison isolates removal of the class sum from removal of weight learning, the
wj ≥ 0gate, feedback changes, and movement of loops into native code? -
How does FastOmniTMAE differ empirically from the cited asynchronous TM architecture and alternate local-feedback mechanism? Were either implemented as a matched baseline?
-
Why should the H100 result support a platform-level conclusion when the profiled and submitted CUDA paths launch only one block for 32 clauses and process target tokens sequentially? Was any implementation with concurrent targets, multiple literal chunks, or enough blocks to occupy the H100 evaluated?
-
What does “full GPU utilization” mean in the main text, given the appendix’s stated maximum active-SM coverage of approximately 0.758% for the profiled kernel?
-
Were Word2Vec, FastText, and GloVe trained from the original ordered token sequences, or from sequences reconstructed from sparse row indices? Was
tokenized_sentences.picklepresent in the runs used for Table 1? -
Should the FastOmniTMAE word-pair score be interpreted as directional association rather than vector cosine similarity? What existing common scoring experiment supports direct comparison with the cosine-based baselines?
-
Which reported memory figures refer to one target’s temporary automata state, and which refer to a deployable full-vocabulary embedding table? Was the claimed 8-bit vector representation actually used and evaluated in the supplied software and hardware paths?
-
Can the authors provide the exact code revision, configurations, corpus-cache provenance, and existing raw outputs corresponding to Figures 2–5 and Tables 1–2? A version mismatch that resolves the cache and preprocessing issues could materially change my assessment.