acceptodds
Under review as a conference paper at ICLR 2027

TC-CRP: Budgeted Integer Learning through Zero-Sum Weight Transfers

Abstract

Biological synapses have bounded capacity, and their changes are constrained by the units that constitute them. Programmable synaptic devices face corresponding limits in conductance range and write endurance, with each parameter change requiring programming operations. These constraints motivate learning rules that turn desired changes into useful actions within a finite write allowance. We introduce TC-CRP, which converts class-discriminative learning signals into budget-constrained, zero-sum integer weight transfers. Votes identify coordinates worth modifying, and an executor realises their priorities as paired ±1 updates. Each pair costs two unit writes; a batch quota and a cumulative ledger control parameter modification throughout training. Complete-pair commits preserve integer weights, exact row balance and coordinate bounds, making the write allowance enforceable as learning proceeds. With full unobstructed execution, ranked selection maximises per-row alignment with the learning signal over the prescribed unit-action set, providing a principled basis for allocating limited writes. Because coordinates are bounded and the budget is spent across batches, a batch’s proposed changes may require decisions about what to carry forward. Historical accumulation and execution subtraction let later priorities reflect past signals while retaining the same legal actions. At matched write counts on the two primary image tasks, directional selection outperforms a random-coordinate control. TC-CRP thus connects controllable parameter-modification cost with supervised learning and provides a framework for examining how update selection and historical memory use a limited write allowance.

Primary area:
optimization
Keywords:
spiking neural networks, first-spike time (TTFS), integer-weight learning, zero-sum weight transfer, budget-constrained plasticity, residual tracking

Reviews

Among:
Shape:
1234this paper, Top 15%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.00 over 3 reviews, above 85% of the others.
Reviewer_fNW4OpenReview ↗
Rating
3(+1)
Confidence
4
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

The paper addresses learning in constrained environments for spiking classifiers whose synapses are programmable devices with limited range and write endurance. It builds on integer SNN training (I-LIF, full-integer online SNN training) and integer-only network training (NITRO-D). The proposed rule, TC-CRP, trains a single linear readout on integer time-to-first-spike (TTFS) latencies. It turns class-discriminative, perceptron-like votes into paired ±1 integer weight transfers that keep each row zero-sum and respect a weight bound and a cumulative write budget. Three priority signals are compared: the current vote (A), accumulated demand (CUM), and accumulated demand minus committed writes (H2, akin to error feedback). Feasibility and conditional per-row optimality are proven. At equal write counts, ranked selection clearly beats a random-coordinate control on MNIST and Fashion-MNIST. Memory gives higher mean accuracy than A on MNIST, Fashion-MNIST, N-MNIST and DVS128 Gesture, but the gain is statistically clear only on N-MNIST.

Critical strengths

S1 - The authors propose a clear and well-formalised method. The abstract and introduction give an efficient overview of the problem and the method. Sec. 3.2 defines the algorithm compactly and without ambiguity, including ties, the activity gate, pair dropping, budget truncation and resumption. The contribution is simple but interesting, and it is easy to understand and re-implement.

S2 - The theoretical analysis is well-formulated and appreciated. Props. 1–3 are stated with explicit conditions. App. A gives counterexamples showing exactly when Prop. 2 stops holding. This makes the guarantees of the rule transparent.

Critical weaknesses

W1 - The authors provide only experimental and theoretical evidences on very simple architecture, with no evidence of scaling. All experiments use a single linear layer on first-spike latencies, and absolute accuracies are low: 88.1–88.9% on MNIST, 76.2–79.3% on Fashion-MNIST, and about 16–17% on CIFAR-10 (App. E.2). Nothing indicates how the rule (votes, row-wise ranking, memory) would extend to multi-layer networks. It is therefore hard to judge how the method would perform in realistic scenarios.

W2 - The comparison framework proposed by the authors is limited. The write-matched comparisons are only internal (A/CUM/H2) or against RAND. RAND is a uniform random control designed after A's results were known (Sec. 5.3), and it performs at chance level (11.6% and 9.6%). The external references are a zero-sum floating-point linear model and a multiclass perceptron (Sec. 5.7, App. F), plus H1 (App. E.5). They are not write-matched, and they all outperform TC-CRP. For example, the perceptron reaches 90.45/84.26% best validation accuracy against 88.66/81.35% for H2. There is no comparison under the same write budget with learning rules from the neuromorphic and in-memory computing literature that also limit device writes. Examples are sign-based fixed-step (Manhattan rule) updates for memristive crossbars, and mixed-precision training that accumulates updates in a high-precision variable and programs the device only when a threshold is crossed (Nandakumar et al., 2020), which is close in spirit to CUM/H2. There is also no comparison with TTFS-based SNN learning rules such as S4NN or the cited EventProp.

S. R. Nandakumar, M. Le Gallo, C. Piveteau, V. Joshi, G. Mariani, I. Boybat, G. Karunaratne, R. Khaddam-Aljameh, U. Egger, A. Petropoulos, T. Antonakopoulos, B. Rajendran, A. Sebastian, E. Eleftheriou. “Mixed-precision deep learning based on computational memory.” Frontiers in Neuroscience 14:406, 2020. doi:10.3389/fnins.2020.00406

Questions for authors

Q1 - What motivates the row-conservation (zero-sum) requirement from the hardware side? The introduction mainly gives invariance to a common latency shift, and Sec. 6 calls signed row balance "a modelling choice". How would accuracy and write cost change with unpaired ±1 writes?

Q2 - How were J = 3,187,200, 6,374,400 and 12,748,800 chosen? They appear to correspond to 3, 6 and 12 epochs of maximum write capacity (cf. App. E.6). Why use B_w = 10^9 in the main experiments? Since max|W| ≤ 5467 (Table 9), the bound is never active in the headline results.

Q3 - CUM and H2 store a C×d int64 memory, the same size as W but at higher precision. How large is this overhead in storage and compute (cf. Table 6 for time), and is it compatible with the on-device setting that motivates the work?

Q4 - Do you see a path to multi-layer networks, and do you have preliminary results?

Q5 - What does "TC-CRP" stand for? The acronym is not expanded anywhere in the paper or the appendix.

7 comments

We thank you for highlighting our formal clarity and explicit guarantees. In our revised PDF, Sections 1-6 remain on pp. 1-9, and we added three appendices covering external comparisons (Appendix J), multi-layer and CNN tests (Appendix K), and memory and write overhead (Appendix L), along with an English supplement/README.md. Under OpenReview's 5,000-character limit, Part 1 covers W1/Q4 and W2; Part 2 covers Q1-Q3, Q5, and Ethics.

W1 & Q4. Multi-Layer Feasibility and Frozen-Feature Readouts (Appendix K)

Our new tests establish preliminary, task-dependent feasibility for hidden-layer training rather than finished deep scaling:

  1. Why nonlinear hidden layers are required: Stacking two pure linear layers stays within the linear class and sends 98.60%-98.75% of writes to the hidden matrix, lowering validation accuracy by 5.69-12.27 pp below single-layer baselines.
  2. Five-seed two-layer A: Sharing 2M training writes under Bw=127B_w=127 with clipped hidden units and separate initialization accounting, two-layer A reaches 89.70% on MNIST. It beats identically initialized frozen hidden weights (78.86%, positive on all 5 seeds) and single-layer A (85.77%). On Fashion-MNIST, hidden clipping triggers dead-zone collapse: 52.54%-57.42% of validation inputs produce all-zero hidden activations, dropping accuracy to 37.51% vs. 69.85% frozen.
  3. Exploratory H2 probe: On seed-101 internal validation, jointly switching to piecewise-linear +8/+1 activation, H2 tracking, k_hid=5, k_out=64, Bw=109B_w=10^9, and 12,748,800 writes avoids collapse and beats frozen controls on both MNIST (93.76% vs. 83.44%) and Fashion-MNIST (77.88% vs. 76.00%). Both prototypes propagate output error signals to the hidden layer and apply row-wise ranked pair updates under a shared write budget; A uses an STE, while H2 uses piecewise-linear derivatives. Full deep TTFS scaling remains future work.
  4. Constrained heads on frozen CNN features: On CIFAR10-DVS, we froze a conventionally trained 3.28M-parameter CNN, quantized its 2048-D features to int32, and trained only integer linear heads with CNN training outside budget JJ. At 250k writes over 5 seeds, H2 (70.78%), CUM (70.68%), and A (69.92%) beat paired Manhattan (64.66%) and approach tuned thresholds (71.94%, vs. 76.50% float L2 reference). Only the linear head obeys our constraints, so we make no integer front-end or chip-energy claim.

W2. Write-Matched External Comparisons (Appendix J)

RAND tests coordinate ranking inside A rather than acting as prior work. Adapting the requested Manhattan and threshold rules into our linear setup gives controlled algorithmic benchmarks showing where TC-CRP helps:

  1. Manhattan & Mixed-Precision Threshold Accumulation (Nandakumar et al., 2020; 5-seed test):
    • Wide bound (Bw=109B_w=10^9, 750k writes): A beats paired Manhattan on N-MNIST (78.33% vs. 71.08%) and matches on Fashion-MNIST (72.75% vs. 72.64%). H2 beats tuned Threshold-pair on Fashion-MNIST (78.75% vs. 77.02%) while trailing on N-MNIST (83.04% vs. 85.92%), where equal write counts let Threshold-pair see nearly 2.8x as many sample exposures.
    • Tight weight bound (Bw=4B_w=4, ∣w∣≤4|w|\le 4, 100k writes): Across 5 seeds, H2 keeps zero row sums and achieves 76.53% on Fashion-MNIST and 83.99% on N-MNIST, matching Threshold-pair at 74.90% and 84.05%. By contrast, Threshold-single breaks row balance on Fashion-MNIST (66.62%, max row sum 576) and stalls early on N-MNIST (21.32% vs. 80.11% against Threshold-pair at common J=50,000J=50,000 on an expanded grid). Unlike Nandakumar et al., TC-CRP turns H2 residuals directly into conditionally optimal +/-1 pair transfers (Prop. 2).
  2. Low-Write Integer S4NN (5-seed test): We selected nine low-write operating points from TC-CRP accuracy-write curves across MNIST, Fashion-MNIST, and 256-bin N-MNIST to examine the trade-off between retaining high accuracy and reducing writes, and evaluated our 400-hidden integer S4NN adaptation (with initialization writes counted separately) at the same write counts. Appendix J.4 details the selection procedure. TC-CRP outperforms both S4NN scorings (author tied/fallback and strict earliest-spike) at all nine points: for instance, at 109k writes on MNIST, H2 reaches 84.23% vs. 65.59% author and 3.97% strict, and at 241k on Fashion-MNIST 76.74% vs. 57.71% and 44.23%. At a larger budget of 12,748,800 writes on seed 101, strict S4NN reaches 94.80% MNIST in 1,022 epochs and 66.39% Fashion-MNIST in 24 epochs, showing S4NN fits large budgets while TC-CRP performs better at the tested low-write operating points (EventProp remains open).

Continuing our response to Reviewer fNW4 (Part 2/2: Q1-Q3, Q5 & Ethics).

Q1. Motivation for Zero-Sum Rows & Unpaired Ablation (Appendix J)

Hardware limits alone do not require zero row sums: signed row balance is a modeling choice providing exact invariance to common latency shifts. To test when it helps, we compare A against an unpaired rule (Unpaired-topabs) at equal total writes, trading two writes per pair for one write per unpaired step:

Setting (Budget)Conserved Pair A (%)Unpaired +/-1 (%)+/-128 Shift Flips (A / Unpaired)
Fashion-MNIST raw latency (750k)72.7538.970.00% / 33.09% & 25.62%
N-MNIST raw latency (750k)78.3317.300.00% / 77.88% & 25.08%
Frozen CIFAR10-DVS CNN (250k)69.9268.72N/A

The tested paired rule substantially outperforms Unpaired-topabs on these raw-latency inputs and is comparable to unpaired writes on learned non-negative CNN features; at 500k writes on CNN features, unpaired reaches 71.22% vs. 70.84% for A. Across physical -10 to +10 ms window shifts on 10,000 raw N-MNIST streams, three threshold baselines vary by at most 0.13 pp, while centering Threshold-single cuts +10 ms prediction flips from 1.116% to 0.852% at a 3.82 pp accuracy cost (62.47% vs. 66.29%).

Q2. Origin of Budgets JJ and Bw=109B_w=10^9 vs. Active Bounds (Appendix E.3 & Table 23)

  • Why {3,187,200; 6,374,400; 12,748,800}: Your reading is exact. With 53,100 training samples at batch size 128 (415 batches per pass), one full-capacity pass allows up to 1,062,400 writes, so our budgets equal 3x, 6x, and 12x this pass capacity rather than standard epochs, since inactive rows skip writes.
  • Wide vs. active bounds: Setting Bw=109B_w=10^9 isolates update ranking from saturation. We also tested active bounds in 54 TC-CRP runs across Bw∈{16,64,256}B_w \in \{16, 64, 256\} (all reaching JJ with zero row sums) and in our new Bw=4B_w=4 runs. At Bw=16B_w=16 on MNIST, A, CUM, and H2 reach 85.56%, 86.44%, and 86.39% despite 45.1%-90.7% pair rejections, and at Bw=4B_w=4, H2 keeps 76.53% on Fashion-MNIST and 83.99% on N-MNIST.

Q3. Memory, Speed, Write Savings & Hardware Fit (Appendix L)

  • Footprint & compute: Alongside int32 weights, CUM and H2 keep an int64 history tensor twice as large in bytes (61.25 KiB on images; 184,960 B on N-MNIST), A requires no persistent history tensor, and all variants discard histories at inference. Core training times on our i7-13620H/RTX 4050 system are 14.141-15.912 s on MNIST and Fashion-MNIST and 20.090-24.084 s on N-MNIST.
  • Why this controller RAM pays off: A avoids this additional history-storage requirement. In exchange for modest controller RAM and int64 state writes, CUM and H2 cut committed weight writes needed for target accuracy by 87.5% on Fashion-MNIST (75% target) and 50% on N-MNIST (80% target), and cut direction reversals at 750k writes from 40.94%-56.18% under A down to 0.41%-1.37%, though N-MNIST top-1% write share rises from 5.04% to 7.56%-7.59%. Against integer S4NN on Fashion-MNIST at 241k writes, H2 reduces 32B block writebacks from 6.515 MB to 2.359 MB, state writes from 3.30B to 1.00M, and 60s host energy from 108.38 to 16.24 J/run, measured via CMOS host telemetry rather than chip physics.

Q5 & Details Of Ethics Concerns

  • TC-CRP stands for Task-driven Competition for Constrained Redistribution of Parameters.
  • English README & provenance: Thank you for flagging the Chinese README. We replaced supplement/README.md with a concise English index covering protocols, results, data sources, verification checks, and hardware details, while noting reused official splits under frozen protocols.

We hope these multi-layer controls, write-matched comparisons against Manhattan, threshold accumulation, and S4NN, overhead analyses, and clarified scope boundaries resolve your concerns.

We have revised the main text to clarify the method, strengthen its empirical support, and better define its scope. We also added supporting experiments and analyses in Appendices J–L. The blue revisions include:

  1. Clarified the method’s positioning and contributions. In the introduction, we explicitly define TC-CRP as Task-driven Competition for Constrained Redistribution of Parameters. We distinguish the primary single-layer setting from preliminary two-layer extensions and constrained readouts on frozen features. The contribution statements now include external comparisons, constraint ablations, and analyses of scalability and implementation costs.

  2. Expanded and connected the related work. We discuss Manhattan sign updates, threshold-triggered residual accumulation, multilayer temporal learning with S4NN, and frozen-feature readout adaptation. We explain how these approaches relate to TC-CRP and connect the discussion to the new experimental comparisons.

  3. Clarified the formulation and experimental accounting. In the problem setup, we distinguish the core linear formulation from the extensions. In the experimental section, we explain how theoretical per-pass write capacity is calculated and clarify that budget multipliers represent capacity multiples, not actual training epochs. We also explain why the primary experiments use wide weight bounds and summarize additional results where bounds actively block updates.

  4. Strengthened the evidence supporting the main discussion. Appendix J adds write-matched Manhattan, threshold-accumulation, and integer S4NN comparisons; unpaired-update ablations; tight-bound residual checks; and latency-shift and observation-window robustness tests. The main text now reflects the resulting qualifications: baseline comparisons depend on the dataset and budget, while the benefit of row conservation depends on the representation and update rule.

  5. Expanded the treatment of scalability and practical use. Appendix K adds two-layer experiments with frozen-hidden controls and constrained CIFAR10-DVS readouts on frozen CNN features. The main discussion now covers both promising results and failure modes, including clipping-induced dead zones and inefficient hidden-layer write allocation. We explicitly distinguish these preliminary prototypes and hybrid models from fully constrained multilayer spiking networks.

  6. Revised the practical-cost discussion and limitations. Appendix L quantifies training memory, isolated runtime, write locality, coordinate concentration, shared-target write savings, and host CPU/GPU energy. The main text now acknowledges auxiliary history-storage costs, hotspot risks despite fewer write reversals, and the distinction between host-software measurements and dedicated neuromorphic-device performance. We also clarify the limited seeds in some extensions and the reuse of official test sets.

  7. Updated future directions, disclosures, and reproducibility. We refocused future work on deeper integer spiking models, constraint-aware selection, and device-level evaluation. We also updated the disclosure and reproducibility statements and expanded the supplement with protocols, per-seed records, input and training-order verification, and portable CPU reproduction checks.

Reviewer_AxXYOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper investigates supervised learning under integer unit updates, row-sum conservation, weight bounds, and a cumulative write budget. TC-CRP converts class-discriminative votes into paired weight transfers and compares current-vote, cumulative-memory, and execution-subtracted priorities. It establishes feasibility and conditional per-row alignment optimality, then evaluates matched-write classification and shared-vote trajectories. The results support directional selection over random updates; however, the additional classification benefits of execution subtraction appear task-dependent and uncertain.

Critical strengths
  1. The paper provides a precisely defined learning framework that enforces integer weights, row-sum conservation, weight bounds, and a cumulative write budget throughout training. The guarantees and their limitations are clearly established in Section 4 and Appendix A.
  2. The separation of directional selection, accumulation, and execution subtraction provides useful insight into constrained learning. In particular, Appendix I shows that improving demand tracking need not improve classification, clarifying a limitation of execution-based feedback.
Critical weaknesses

The empirical benefit of historical memory, particularly execution subtraction, remains limited and task-dependent. All six primary simultaneous intervals include zero (Table 4), and H2 versus CUM changes sign across the two image tasks; N-MNIST provides positive evidence, while CIFAR-10 reverses the mean advantage of memory (Appendix E.2).

Questions for authors
  1. What is the root cause of the loader hash mismatch? Does it affect sample identities, data splits, encoding, or ordering? Please point to any existing documentation or checks that establish correspondence with the historical experimental inputs.
  2. Did preliminary results on MNIST/Fashion-MNIST influence decisions regarding encoding, quota, demand scale, budget, or method selection before the current protocol was finalized? Please clarify if any design choices depended on these early results.
3 comments

Thank you for recognizing the formulation, guarantees, and separation of tracking from classification. We have traced the loader discrepancy to path sanitization and verified correspondence with the historical inputs. We also clarify which choices were informed by preliminary experiments and how the memory results qualify the paper's claims.

1. Loader Mismatch & Input Correspondence (Q1)

The discrepancy stems solely from removing two machine-specific fallback paths in the released loader, leaving sample processing and splitting identical. Direct reconstruction confirms correspondence with the historical inputs:

  1. Inputs: Rebuilding MNIST and Fashion-MNIST from raw data matched the ordered feature and label tensors for each 59,000-sample training pool and 10,000-sample test set, including their shapes, dtypes, and SHA-256 hashes.
  2. Splits & order: All splits and 410 epoch permutations across the 30 primary runs match the archived digests. These input, split, and order checks are documented in primary_input_verification.json.
  3. End-to-end reproduction: Six Windows runs across the five follow-up experiments and one cross-platform WSL repeat match archived weights and predictions, as documented in portable_verification.json.

2. Design Provenance Across Five Choices (Q2)

Yes. Preliminary MNIST and Fashion-MNIST results informed several design choices before the current protocol was frozen:

  1. Encoding: Five-seed comparison grids evaluated raw first-spike timing, frozen LIF features, and alternative pairing strategies, leading to the choice of raw first-spike timing with top-kk pairing. The signal-only alternative degraded performance on Fashion-MNIST.
  2. Quota (k=128k=128): Earlier hidden-layer experiments compared quotas of 32, 128, and 256 and selected 128 in that setting. The final single-layer study also used 128.
  3. Demand construction and scale: The centered integer demand was inherited from H1. Centering and positive scaling preserve A's coordinate ranking, whereas demand scale affects the balance between accumulation and unit execution subtraction in H2. Calibrated variants (S-CUM and S-H2) were evaluated separately, with the failed fresh-seed confirmation retained in the appendix.
  4. Budget (JJ): Three write budgets (3/6/12×3/6/12\times capacity: 3,187,2003{,}187{,}200, 6,374,4006{,}374{,}400, and 12,748,80012{,}748{,}800) were pre-specified across five seeds; the largest budget was the sole primary test endpoint, without epoch or budget selection. Later budget cuts in the longer-budget follow-up were 20-epoch reachability fixes.
  5. Methods: H1 motivated H2, and CUM was included to isolate accumulation from execution subtraction. An early H2–CUM comparison did not meet its acceptance criterion, and A remained the default rule. RAND was introduced subsequently to test directional selection against A.

The surviving records establish these development choices, though they do not fully reconstruct the initial rationale for the encoding constants and budget multipliers. The paper notes prior use of the image test sets. Five fresh seeds further test split and order stability: descriptive CUM–A intervals exclude zero on both image tasks, while H2–CUM intervals cross zero. N-MNIST evaluates an event dataset not previously used in our project, with J=14,407,680J=14{,}407{,}680 fixed via a training-only reachability probe before test evaluation.

3. Task-Dependent Memory & Scope (W1)

We agree that the consistency of memory benefits across tasks is limited. Their magnitude, however, can be substantial in an individual setting. On N-MNIST, accumulation contributes +10.17+10.17 pp, whereas execution subtraction adds +0.75+0.75 pp. This distinction matters when assessing historical memory as a whole versus H2's incremental contribution:

  1. Accumulation vs. subtraction: The N-MNIST differences have five-seed, unadjusted 95% intervals of [8.33,12.01][8.33, 12.01] and [0.06,1.45][0.06, 1.45] pp, respectively. Primary simultaneous intervals in Table 4 include zero, and CIFAR-10 reverses memory's mean advantage. These observations support a substantial accumulation benefit in a tested setting while limiting claims of consistent improvement across tasks.
  2. Tracking vs. classification: Appendix I tests whether better demand tracking is a reliable criterion for choosing a feedback rule. On fixed external vote streams, S-H2 cuts terminal tracking error versus S-CUM by factors of 44.6744.67 and 46.2846.28. In separate closed-loop confirmation, its Fashion-MNIST validation accuracy is lower by 2.24292.2429 pp than H2 and 2.58192.5819 pp than S-CUM. For this calibration, selecting the rule by tracking error alone would favor S-H2, even though S-CUM achieved higher validation accuracy in the separate closed-loop comparison.

The updated supplement includes these verification records, the portable reproduction procedure, and supporting experimental details.

We have revised the main text to clarify the method, strengthen its empirical support, and better define its scope. We also added supporting experiments and analyses in Appendices J–L. The blue revisions include:

  1. Clarified the method’s positioning and contributions. In the introduction, we explicitly define TC-CRP as Task-driven Competition for Constrained Redistribution of Parameters. We distinguish the primary single-layer setting from preliminary two-layer extensions and constrained readouts on frozen features. The contribution statements now include external comparisons, constraint ablations, and analyses of scalability and implementation costs.

  2. Expanded and connected the related work. We discuss Manhattan sign updates, threshold-triggered residual accumulation, multilayer temporal learning with S4NN, and frozen-feature readout adaptation. We explain how these approaches relate to TC-CRP and connect the discussion to the new experimental comparisons.

  3. Clarified the formulation and experimental accounting. In the problem setup, we distinguish the core linear formulation from the extensions. In the experimental section, we explain how theoretical per-pass write capacity is calculated and clarify that budget multipliers represent capacity multiples, not actual training epochs. We also explain why the primary experiments use wide weight bounds and summarize additional results where bounds actively block updates.

  4. Strengthened the evidence supporting the main discussion. Appendix J adds write-matched Manhattan, threshold-accumulation, and integer S4NN comparisons; unpaired-update ablations; tight-bound residual checks; and latency-shift and observation-window robustness tests. The main text now reflects the resulting qualifications: baseline comparisons depend on the dataset and budget, while the benefit of row conservation depends on the representation and update rule.

  5. Expanded the treatment of scalability and practical use. Appendix K adds two-layer experiments with frozen-hidden controls and constrained CIFAR10-DVS readouts on frozen CNN features. The main discussion now covers both promising results and failure modes, including clipping-induced dead zones and inefficient hidden-layer write allocation. We explicitly distinguish these preliminary prototypes and hybrid models from fully constrained multilayer spiking networks.

  6. Revised the practical-cost discussion and limitations. Appendix L quantifies training memory, isolated runtime, write locality, coordinate concentration, shared-target write savings, and host CPU/GPU energy. The main text now acknowledges auxiliary history-storage costs, hotspot risks despite fewer write reversals, and the distinction between host-software measurements and dedicated neuromorphic-device performance. We also clarify the limited seeds in some extensions and the reuse of official test sets.

  7. Updated future directions, disclosures, and reproducibility. We refocused future work on deeper integer spiking models, constraint-aware selection, and device-level evaluation. We also updated the disclosure and reproducibility statements and expanded the supplement with protocols, per-seed records, input and training-order verification, and portable CPU reproduction checks.

Thank you for the follow-up and constructive feedback. We are glad that our clarifications, verification checks, write-matched baselines, and scoped revisions resolved your reproducibility and procedural concerns and improved the paper's transparency.

Within its intended scope, TC-CRP offers a feasibility-preserving discrete optimization baseline under a hard cumulative update budget—combining a constrained linear maximization oracle (LMO) with strict invariants and performing competitively against tuned threshold and sign baselines without per-task tuning (Section 5; Appendix J). We appreciate your point on its boundaries: the task-dependent gain of error-feedback-style H2 over dual-averaging-style CUM reflects open-loop tracking versus closed-loop generalization trade-offs, and scaling beyond single-layer, inactive-bound regimes still faces practical obstacles (Appendices K–L).

Building on this baseline, we plan to explore more scalable active-set updates for deeper non-convex architectures in follow-up work. Thank you again for your time and thoughtful review, which strengthened the paper's rigor.

Reviewer_1mY2OpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes TC-CRP, a budget-constrained integer learning framework that converts supervised classification signals into zero-sum weight transfers. Each update consists of paired +1 and −1 modifications, ensuring integer weights, row-sum conservation, coordinate bounds, and explicit accounting of cumulative write operations. The authors investigate three update-selection strategies: A, which uses the current batch signal; CUM, which accumulates historical learning demands; and H2, which additionally subtracts previously executed updates. The paper provides theoretical guarantees of update feasibility and conditional optimality for the coordinate-selection procedure. Experiments on MNIST, Fashion-MNIST, N-MNIST, and DVS128 Gesture demonstrate the effectiveness of directional weight selection compared with random selection. Historical accumulation generally improves mean classification accuracy, although the additional benefit of execution subtraction varies across tasks.

Critical strengths
  1. The paper addresses a meaningful problem: learning under limited parameter-update capabilities.
  2. The paper provides formal guarantees for constraint preservation and conditional optimality of the update-selection procedure.
  3. The experimental evaluation includes multiple datasets, controlled comparisons among different memory strategies, random-selection baselines, and additional analyses of weight bounds and update budgets.
Critical weaknesses
  1. The algorithmic novelty is somewhat incremental, and stronger baseline comparisons would be valuable. The proposed approach combines established ideas from multiclass mistake-driven learning, integer updates, coordinate ranking, and error-feedback mechanisms. Although their integration into a unified budget-constrained framework is useful, the individual algorithmic components are not entirely new. In particular, the H2 memory mechanism is closely related to existing error-feedback approaches. The paper would benefit from a clearer discussion of its differences from prior methods and comparisons against stronger constrained or quantized learning baselines under matched write budgets. Nevertheless, the explicit integration of these mechanisms with feasibility guarantees provides a meaningful contribution.
  2. The experimental scope limits conclusions about practical hardware benefits. The main experiments use a single-layer linear classifier with first-spike latency features, and the weight bounds are inactive in the primary settings. Although the supplementary experiments investigate tighter bounds, the practical effectiveness of the method under realistic hardware constraints remains insufficiently established. Furthermore, the experiments measure logical write counts rather than physical programming energy or device endurance. Evaluation on more complex SNN architectures or representative hardware platforms would strengthen the practical significance of the work. The results also indicate that H2 does not consistently outperform CUM, suggesting that the benefits of execution subtraction are task-dependent. These limitations do not invalidate the proposed framework, but they restrict the generality of the conclusions.
Questions for authors
  1. Could the authors further clarify the key methodological differences between H2 and conventional error-feedback approaches? In particular, what advantages arise specifically from combining execution subtraction with zero-sum integer transfers?
  2. Could the authors provide additional comparisons against stronger budget-constrained learning baselines, such as quantized or error-feedback-based methods using the same legal update mechanism? Such comparisons would help establish the incremental benefits of the proposed priority-selection strategies.
  3. How would the method scale to multilayer SNN architectures, and how closely does the logical unit-write metric correspond to the physical programming costs of the intended hardware? Clarification of the portable-loader reproducibility issue would also be helpful.
3 comments

We thank you for recognizing our budgeted integer updates and formal guarantees. Below we report new controls, storage/energy checks, and verification.

1. H2 and error feedback (W1, Q1)

Unlike unconstrained EF-SGD with a contractive compressor, H2 couples residual feedback (Mt=Mt−1+Ut−ΔWtM_t=M_{t-1}+U_t-\Delta W_t) to a constrained integer executor that commits only bound-feasible, within-budget pairs. Because inactive-row gating or bound saturation can yield ΔWt=0\Delta W_t=0 even when Mt−1+Ut≠0M_{t-1}+U_t\neq 0, standard uniform-contraction guarantees do not apply. Props. 1–3 instead establish constraint feasibility, conditional per-row alignment (active row, unblocked, untruncated fixed quota), and exact residual accounting.

Combining zero-sum transfers with execution subtraction keeps every weight update integer, bounded, and row-sum zero (making linear scores invariant to uniform input shifts) while ensuring the residual MtM_t itself remains row-sum zero and drift-free when pairs are blocked or truncated. In fixed-demand replays, H2 kept L1L_1 accounting error at 0, versus 6 under bound blocking and 6 at mid-batch truncation when subtracting proposed pairs before filtering. This is an accounting guarantee; its accuracy effect varies by task.

2. Baselines with the same legal actions (W1, Q2)

We compared Manhattan-pair (sign step) and Threshold-pair (tuned threshold accumulation) using the same paired-update operator, inputs, zero init, batch size, seeds, and write ledger. At J=750,000J=750{,}000, five-seed test accuracy (%) (SD) was:

MethodFashion-MNISTN-MNIST
A72.75 (2.16)78.33 (1.91)
CUM77.66 (1.16)82.17 (1.68)
H278.75 (0.62)83.04 (0.71)
Manhattan-pair72.64 (3.79)71.08 (1.48)
Threshold-pair77.02 (0.63)85.92 (1.26)

H2 outperformed Manhattan-pair by 6.11 pp [1.80, 10.42] on Fashion-MNIST and 11.96 pp [9.98, 13.94] on N-MNIST (descriptive 95% paired intervals across five seeds, without multiplicity correction). On N-MNIST, Threshold-pair outscored H2 at J=750,000J=750{,}000 with a separately tuned scale/threshold and ~2.8x H2's sample exposure, and reached 2.14–2.17M total writes at its 8-epoch end point. We will report full curves, exposure counts, and historical test-set reuse.

3. Active bounds, depth, storage, and energy (W2, Q3)

Active bounds. Under tight bounds (Bw=4B_w=4, J=100,000J=100{,}000), where bounds blocked >90% of pairs proposed by CUM/H2 on both tasks, CUM/H2/Threshold-pair still substantially outperformed A (77.67%/76.53%/74.90% vs. 58.52% on Fashion-MNIST; 84.44%/83.99%/84.05% vs. 52.45% on N-MNIST) with greater sample exposure within the same write budget. H2's lower mean than CUM confirms that execution subtraction need not improve accuracy under tight bounds.

Depth. A 784→128→10784\to128\to10 H2 prototype with piecewise-linear integer activations shares one write budget across layers (J=12,748,800J=12{,}748{,}800, with 100,352 initial hidden loads separate). Updating both layers versus freezing the same random hidden layer raised seed-101 validation accuracy from 83.44% to 93.76% on MNIST and 76.00% to 77.88% on Fashion-MNIST, providing preliminary evidence for hidden-layer updates; deeper spiking models remain untested.

Storage costs and training energy. Logical writes JJ count unit parameter changes, whereas actual writeback traffic and energy also depend on write granularity, update coalescing, and schedule. In a separate Fashion-MNIST comparison at J=241,000J=241{,}000, five-seed test accuracy averaged 76.74% for single-layer H2 versus 44.23% (strict) or 57.71% (author scoring) for our 400-hidden-unit integer S4NN adaptation. At one 32-bit word per unit change, both imply 0.964 MB of training-weight writeback (excluding state traffic and initialization); buffering 32-byte blocks per update (batch for H2, sample for S4NN) yields trace estimates of 2.36 MB vs. 6.52 MB, respectively.

Repeatedly resetting and training each method to J=241,000J=241{,}000 on an i7-13620H/RTX 4050 Laptop GPU (seed 101; three 60 s windows each) consumed 16.24 J/run for H2 vs. 108.38 J/run for integer S4NN (CPU package plus discrete GPU; <2% mean shift from 30 s to 60 s). These platform measurements reflect the tested architectures and update schedules; device programming energy and endurance require separate measurement.

4. Portable-loader reproducibility (Q3)

That issue reflected only an internal snapshot check triggered when author-local paths were removed for anonymous release; it did not affect data loading, splits, or encoding. An updated portable runner now reproduces training and evaluation directly from the original datasets: end-to-end CPU runs matched archived weights and validation/test predictions exactly across six representative E1–E5 configurations on Windows (and one on WSL).

We have revised the main text to clarify the method, strengthen its empirical support, and better define its scope. We also added supporting experiments and analyses in Appendices J–L. The blue revisions include:

  1. Clarified the method’s positioning and contributions. In the introduction, we explicitly define TC-CRP as Task-driven Competition for Constrained Redistribution of Parameters. We distinguish the primary single-layer setting from preliminary two-layer extensions and constrained readouts on frozen features. The contribution statements now include external comparisons, constraint ablations, and analyses of scalability and implementation costs.

  2. Expanded and connected the related work. We discuss Manhattan sign updates, threshold-triggered residual accumulation, multilayer temporal learning with S4NN, and frozen-feature readout adaptation. We explain how these approaches relate to TC-CRP and connect the discussion to the new experimental comparisons.

  3. Clarified the formulation and experimental accounting. In the problem setup, we distinguish the core linear formulation from the extensions. In the experimental section, we explain how theoretical per-pass write capacity is calculated and clarify that budget multipliers represent capacity multiples, not actual training epochs. We also explain why the primary experiments use wide weight bounds and summarize additional results where bounds actively block updates.

  4. Strengthened the evidence supporting the main discussion. Appendix J adds write-matched Manhattan, threshold-accumulation, and integer S4NN comparisons; unpaired-update ablations; tight-bound residual checks; and latency-shift and observation-window robustness tests. The main text now reflects the resulting qualifications: baseline comparisons depend on the dataset and budget, while the benefit of row conservation depends on the representation and update rule.

  5. Expanded the treatment of scalability and practical use. Appendix K adds two-layer experiments with frozen-hidden controls and constrained CIFAR10-DVS readouts on frozen CNN features. The main discussion now covers both promising results and failure modes, including clipping-induced dead zones and inefficient hidden-layer write allocation. We explicitly distinguish these preliminary prototypes and hybrid models from fully constrained multilayer spiking networks.

  6. Revised the practical-cost discussion and limitations. Appendix L quantifies training memory, isolated runtime, write locality, coordinate concentration, shared-target write savings, and host CPU/GPU energy. The main text now acknowledges auxiliary history-storage costs, hotspot risks despite fewer write reversals, and the distinction between host-software measurements and dedicated neuromorphic-device performance. We also clarify the limited seeds in some extensions and the reuse of official test sets.

  7. Updated future directions, disclosures, and reproducibility. We refocused future work on deeper integer spiking models, constraint-aware selection, and device-level evaluation. We also updated the disclosure and reproducibility statements and expanded the supplement with protocols, per-seed records, input and training-order verification, and portable CPU reproduction checks.

Thank you for the detailed rebuttal, input-verification checks, and revisions. Your clarification on the loader mismatch and candid provenance of the preliminary design choices address my reproducibility concerns, and the added baselines and scoped-down claims improve the paper's transparency. However, the core empirical limitation remains: the consistent advantage of historical memory---particularly execution subtraction---is still task-dependent, and the newly added extensions highlight notable scalability bottlenecks . Therefore, while my procedural questions are resolved, I will maintain my score of 3.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.