acceptodds
Under review as a conference paper at ICLR 2027

Neuromodulated Continuous-Coupled Dynamics for Language Understanding

Abstract

Continuous-coupled neural networks (CCNNs) compute by repeated coupling among neighboring units rather than by stacking layers, making their computation input-conditioned and their intermediate quantities directly inspectable, two properties that language understanding also calls for. However, CCNNs have been applied only to perceptual data, and language does not admit a direct transfer, with token-level units, content-dependent neighborhoods, and input-dependent integration requirements. To address these challenges, we propose the neuromodulated continuous-coupled neural network (), a hybrid dynamic architecture in which task-corpus co-occurrence embeddings define the units and a polarity-gated semantic graph infers the neighborhood from content. A neuromodulatory system of about 1,000 parameters supplies the per-input integration through firing thresholds and coupling gain at inference time. A polarity-routed readout attributes the evidence behind each prediction to named channels, and the recurrent update admits a checkable sufficient condition for contraction. Across English and Chinese benchmarks, Ne-CCNN is competitive with matched non-pretrained baselines, with gains of up to 6.8 points on six of the seven sentence-pair tasks, using 1.5–3.0M non-embedding parameters (under 5% of the pre-trained references). Ablations show that its neuromodulatory pathway contributes to accuracy on six of seven benchmarks (neutral on the seventh). Code is available at https://anonymous.4open.science/r/Ne-CCNN/

Primary area:
neurosymbolic & hybrid AI systems (physics-informed, logic & formal reasoning, etc.)
Keywords:
Neuromodulation, Continuous-coupled dynamics, Intrinsic interpretability, Natural language understanding

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_ZjScOpenReview ↗
Rating
3(+1)
Confidence
4
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

Continuous-coupled neural networks have been applied only to perceptual data, not language. This work proposes a neuromodulated continuous-coupled neural network that is suitable for language. Experimental results show some advances; however, there is no GitHub opensource.

Critical strengths

None.

Critical weaknesses
  1. The interpretability proof is missing to show the contribution.
  2. Github: Unavailable
Questions for authors
  1. The description of EXTENDED ABLATION STUDIES is conflicted to the claimed contribution. And the analysis is not in-depth.

  2. The equation descriptions are not readable in the BIOLOGICAL AND MATHEMATICAL FOUNDATIONS section.

  3. Github: Unavailable

This content can’t be shown

Allow this repository on GitHub using the read-only access button, then retry. Organization access may require administrator approval.

  1. The interpretability proof is missing to show the contribution.
5 comments

I will maintain my original rating.

AuthorsOpenReview ↗

Response to Reviewer ZjSc [1/4]

Dear Reviewer ZjSc,

Thank you for your careful and specific review, and for identifying the issues with code access and evidence presentation. Because we were not fully familiar with the workflow and did not check the anonymous repository thoroughly enough, we apologize for the inconvenience this caused your verification. We respond in the following order: code availability, interpretability evidence, extended ablations, equation readability, and the additional experiments we have completed.

Code availability

Weakness 2. "Github: Unavailable"

https://anonymous.4open.science/r/Ne-CCNN/

The core Stage 2–5 code, README, experimental environment, and device details are public. In a controlled code-path consistency check with fixed inputs and identical weights, we observed no discrepancy in the Stage 2–5 intermediate tensors (covering adjacency, resonance, LTC dynamics, neuromodulation, BioAttention, binary/multiclass PRR, and pair comparison). This check verifies code-path consistency, not end-to-end reproduction across datasets; we will test the anonymous link and reproduction configuration again before submission so that the reviewer can use it to inspect the computation.

AuthorsOpenReview ↗

Response to Reviewer ZjSc [2/4]


On interpretability evidence

Weakness 1. "The interpretability proof is missing to show the contribution."

The concern is valid: naming intermediate variables or exposing polarity channels does not constitute a complete interpretability proof. We therefore replace "interpretability proof" with the more precise behavioral faithfulness evidence, and test whether high-attribution tokens actually affect predictions through interventions.

The table below reports prediction-flip rates after top-kk deletion/perturbation:

DatasetModelk=1k=2k=3
MRNe-CCNN routed evidence0.1200.2250.335
MRBiLSTM attention0.1050.1550.205
SST-2Ne-CCNN routed evidence0.2000.2300.310
SST-2BiLSTM attention0.2150.2900.355
SUBJNe-CCNN routed evidence0.0300.0650.120
SUBJBiLSTM attention0.0850.0750.120

At k=3k=3, the matched random-control flip rates for Ne-CCNN are 0.1213 (MR), 0.1619 (SST-2), and 0.0700 (SUBJ); the paired statistics against random are t=6.58t=6.58, 4.924.92, and 2.442.44. The selected evidence tokens have a measurable effect on predictions, but the mechanism-native evidence is not uniformly better than attention.

We will state two limitations explicitly:

  1. routed evidence is only one additive component of the final logit; auxiliary and other branches also participate, so it cannot be presented as a complete explanation;
  2. word-level valence alignment is limited proxy evidence and does not prove that the model's evidence chain agrees with human reasoning. Full sentence-level rationale alignment requires a dedicated human-annotation study.

We therefore retain the narrower claim of an intervenable, measurable evidence channel with behavioral consequences, not a formal or human-aligned explanation proof.

Additional evidence: single-instance decision trace and readout ordering

To address whether decisions are traceable and more transparent than other models, we add two analyses that require no retraining.

Table R1b. Decision trace of one MR instance

The supplementary material provides a per-token trace for one MR test instance (length 17, predicted negative), listing the signed polarity sis_i, firing rate Vˉi\bar V_i, readout weight aia_i, and the terms entering zposz_{\mathrm{pos}}/znegz_{\mathrm{neg}}. For this instance:

QuantityValue
zposz_{\mathrm{pos}}0.027
znegz_{\mathrm{neg}}0.192
Routed share of logit marginabout 5%

This shows structural decomposability for a given instance: the routed terms can be separated token by token instead of exposing only the final logit. It does not automatically constitute a faithful explanation, nor does it show greater transparency than attention; attention also provides token weights. Our additional variables are polarity, firing rate, and two evidence channels, but this is only another auditable variable set, not a claim of overall superiority.

Faithfulness itself is task-dependent: on MR, native evidence is stronger than attention for k≥2k\ge2; on SST-2, attention is stronger; on SUBJ the two tie at k=3k=3 while attention is ahead at smaller kk. Our conclusion is therefore competitive, not superior; this neither supports "interpretability is generally worse than attention" nor the opposite claim.

Table R1c. Readout ordering (full test split, 5 seeds)

DatasetStatisticResult
SST-5Graded output vs. human labels, Spearman ρ0.613 ± 0.003
SST-5Ordinal MAE0.817 ± 0.006
IMDBPolarity evidence vs. human valence, Spearman ρ0.448 ± 0.005
SST-2Same0.400 ± 0.008
MRSame0.328 ± 0.007
SUBJSame0.155 ± 0.005

These results support external valence alignment (construct validity): the routed score is moderately correlated with human five-level affect labels on SST-5 (ρ=0.613), and with human valence on IMDB/SST-2/MR; SUBJ reaches only 0.155, showing that the alignment is not universal. These correlations do not test decision faithfulness and do not prove human reasoning alignment; faithfulness is tested separately by the deletion interventions above.

Reviewer_w9UbOpenReview ↗
Rating
3(+1)
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper presents Ne-CCNN, an adaptation of continuous-coupled neural networks (CCNNs) to natural language understanding. The model constructs a polarity-gated semantic graph from corpus-based embeddings and uses three neuromodulatory signals to adjust the thresholds and coupling gain of its recurrent dynamics. Classification is performed through a polarity-routed readout, with an additional comparison branch for sentence-pair tasks. The model is evaluated on both English and Chinese benchmarks.

Critical strengths

Extending continuous-coupled dynamics to language understanding is an interesting direction. The experiments span a range of English and Chinese tasks, and the paper includes component ablations as well as an analysis of the recurrent dynamics.

Critical weaknesses
  1. The paper does not clearly establish why continuous-coupled dynamics are needed for these language tasks. The model combines a semantic graph, recurrent message passing, and input-conditioned threshold and gain control, but the resulting computation appears similar to that of existing recurrent graph architectures. I am not convinced that the paper identifies a distinct problem that requires this formulation.

  2. The architecture introduces many components, but their combined benefit is not convincingly demonstrated. In Table 1, Ne-CCNN performs largely on par with the BiLSTM baseline, with small gains on some datasets and lower scores on others. The larger gains on sentence-pair tasks are obtained with an additional comparison branch, which makes the contribution of the core dynamics harder to assess.

  3. The connection between the biological motivation and the actual computation remains rather loose. The DA, NE, and ACh pathways ultimately amount to three learned adjustments to the thresholds and coupling gain. It is therefore unclear whether these biological distinctions translate into meaningful modeling choices or serve mainly as an analogy.

  4. The claimed interpretability is also difficult to assess. The positive and negative evidence channels are intuitive for sentiment classification, but their suitability for tasks such as natural language inference is less clear. Moreover, because the auxiliary and comparison branches also contribute to the final logits, the routed evidence alone does not fully explain the prediction.

  5. The overall presentation makes the central idea difficult to follow. Figure 1 packs many modules, equations, and connections into a single diagram with little visual hierarchy. Figure 2 devotes considerable space to neurotransmitter pathways, even though the implemented modulation constitutes only a small part of the model and does not simulate these pathways.

Questions for authors
  1. What does the continuous-coupled formulation offer beyond a conventional recurrent graph encoder that uses the same semantic graph and input-conditioned modulation?

  2. The comparison branch appears to be important for the stronger sentence-pair results. How would the same branch perform on top of a simpler encoder under otherwise matched settings?

  3. In Eq. (10), why are the positive and negative threshold shifts averaged over the batch? Could this make the prediction for a given sentence depend on the other samples in its inference batch?

  4. How should the polarity-routed evidence be interpreted, given that the auxiliary and comparison branches also contribute to the final logits?

7 comments
AuthorsOpenReview ↗

Response to Reviewer w9Ub [1/3]

Dear Reviewer w9Ub,

Thank you for your detailed feedback. We are particularly grateful that you recognized the direction of extending continuous-coupled dynamics to language understanding and the paper's component ablations and dynamics analysis. We also thank you for your specific comments on mechanism necessity, experimental attribution, biological terminology, and presentation. Below we respond point by point; the matched comparison for the comparison branch and the empirical results for continuous coupling are provided in a follow-up response.

This response also reports additional evidence already available: batch invariance under per-sample thresholds, the single-instance decision trace, readout ordering, and the FiLM/SE substitution controls.


On the computational form of continuous coupling (conceptual part of Question 1)

Question 1. "What does the continuous-coupled formulation offer beyond a conventional recurrent graph encoder that uses the same semantic graph and input-conditioned modulation?"

We thank you for raising this mechanism-level question. The main distinction of continuous coupling lies in the computational object, not in generic accuracy. A conventional recurrent graph encoder learns a discrete hidden-state transition; Ne-CCNN uses a time-invariant continuous coupled update that explicitly retains the membrane state U\mathbf{U}, firing rate V\mathbf{V}, input-conditioned thresholds θ\boldsymbol{\theta}, and coupling gain β\beta. This makes the entire state trajectory inspectable, rather than only the final hidden state, and, where the condition holds, yields a checkable certificate for the fixed point, gradient decay, and input sensitivity.

The direct same-graph GRU control gives a positive result:

Task and protocolNe-CCNNSame-graph GRUΔ (CCNN − GRU)
CMNLI, 5 seeds53.16 ± 0.4550.32 ± 0.46+2.85pp (5/5)
QQP-50K, 5 seeds74.9974.62+0.37pp (4/5)
Full QQP, seed 4279.3579.47−0.12pp (near tie)

The CMNLI difference has the same sign across all five seeds (p=0.000202p=0.000202); QQP-50K is a small positive difference but not significant; full-data QQP is a single-seed near tie. The full protocol and task boundaries are given in the follow-up response. The result supports continuous coupling as a task-dependent inductive bias with an inspectable state/certificate, and we do not extrapolate it to a universal accuracy advantage.


On the biological terminology for DA, NE, and ACh

Weakness 3. "The connection between the biological motivation and the actual computation remains rather loose. The DA, NE, and ACh pathways ultimately amount to three learned adjustments to the thresholds and coupling gain. It is therefore unclear whether these biological distinctions translate into meaningful modeling choices or serve mainly as an analogy."

We thank you for pressing on the correspondence between the biological motivation and the implementation. You are right that a gap remains and needs to be stated; nevertheless, the DA/NE/ACh labels are not arbitrary analogies. Our mapping borrows the action sites and effects of real neuromodulators: DA and NE modulate the positive and negative thresholds, respectively, and ACh modulates the coupling gain. The implementation does involve an engineering simplification—three input-conditioned scalar controls replace receptor cascades and molecular pathways—but this is a deliberate abstraction, not an arbitrary naming scheme.

The revision will therefore retain a three-level distinction:

  • Biological basis: the action sites and effects correspond to DA/NE threshold modulation and ACh gain modulation;
  • Computational form: input features → scalar controls → thresholds/coupling gain;
  • Engineering boundary: receptor cascades, molecular processes, and neural circuits are not simulated, and no neural equivalence is claimed.

This factorization is also supported experimentally. Replacing the three-factor control with FiLM or squeeze-and-excite gives the following Δ accuracy relative to the full model (negative = worse):

DatasetFiLM substitutionSE substitution
ChnSenti−0.68−1.17
MR−0.37−0.82
IMDB+0.27+0.15
SUBJ−0.33+0.03
SST-2−0.82−0.27
SST-5−0.46−0.53
Waimai−0.31−0.32

FiLM is worse on 6/7 datasets and SE is worse on 5/7; IMDB is the exception. This supports a computational role for the factorization—which signal adjusts the threshold and which adjusts the gain—but it supports a functional abstraction, not a claim of biological equivalence.

AuthorsOpenReview ↗

Response to Reviewer w9Ub [2/3]


On the interpretation scope of polarity-routed evidence

Weakness 4. "The claimed interpretability is also difficult to assess. The positive and negative evidence channels are intuitive for sentiment classification, but their suitability for tasks such as natural language inference is less clear. Moreover, because the auxiliary and comparison branches also contribute to the final logits, the routed evidence alone does not fully explain the prediction."

Question 4. "How should the polarity-routed evidence be interpreted, given that the auxiliary and comparison branches also contribute to the final logits?"

We thank you for focusing on the interpretation boundary of routed evidence. Routed evidence is a local additive evidence channel, not a complete decision process; the final prediction also contains an auxiliary pooled classifier and other branches.

Full-test decomposition shows that routed evidence accounts for a limited share of the final logit margin: 8.82% on MR, 11.07% on SUBJ, and 2.13% on SST-2. It should therefore be interpreted as a partial but measurable channel, not as the full decision. For NLI, we interpret the task-specific readout actually used and do not treat sentiment-polarity buckets as complete NLI semantics.

There is also positive evidence: on CMNLI, QNLI, and MNLI, removing the mechanism or the auxiliary branch causes consistent drops, indicating that these branches are not decorative and jointly support the decision with routed evidence.

Task (5 seeds)Remove neuromodulationAux logit = 0FiLM substitutionRemove pair readout
CMNLI−4.58−6.71−3.84−7.14
QNLI−8.64−5.00−7.65−6.22
MNLI−6.42−6.19−5.16−5.81

These are Δ accuracy values relative to the respective full models; negative means removal hurts. They support the joint contribution of routed, auxiliary, and pair branches, and also delimit routed evidence as a local evidence channel.

Additional experiment: single-instance decision trace

The supplementary material includes a complete decision trace for one MR test instance (length 17, predicted negative). It exposes:

  • the signed polarity sis_i of each token;
  • the firing rate Vˉi\bar V_i;
  • the readout weight aia_i;
  • the per-token terms entering zposz_{\mathrm{pos}} and znegz_{\mathrm{neg}}.

For this instance, zpos=0.027z_{\mathrm{pos}}=0.027 and zneg=0.192z_{\mathrm{neg}}=0.192, and the routed part accounts for about 5% of the logit margin. The per-token terms below show how the score is formed:

Tokensis_iVˉi\bar V_iaia_iTo zposz_{\mathrm{pos}}To znegz_{\mathrm{neg}}
films+0.020.3190.054+0.01730
smokers−0.040.3590.0540+0.0192
the−0.020.3470.0530+0.0184
downbeat−0.060.3150.0550+0.0174
resolutely−0.010.3130.0520+0.0164

This shows that the forward computation of the routed score can be decomposed token by token, rather than being only an after-the-fact score. At the same time, the auxiliary term still accounts for about 95% of the margin, so this is not a complete explanation. "Traceable" here is structural, not equivalent to causal faithfulness; faithfulness is tested separately by deletion/perturbation. The trace therefore argues for reporting aux/pair contributions together with routed evidence, not for treating the trace as the full decision chain.

Additional experiment: readout ordering

We also checked whether routed evidence orders examples by human affective intensity on the full test split:

DatasetStatisticResult (5 seeds)
SST-5Graded output vs. human labels, Spearman ρ0.613 ± 0.003
SST-5Ordinal MAE0.817 ± 0.006
IMDBPolarity evidence vs. human valence, Spearman ρ0.448 ± 0.005
SST-2Same0.400 ± 0.008
MRSame0.328 ± 0.007
SUBJSame0.155 ± 0.005

These results support measurable affective ordering in routed evidence, but they do not establish complete semantic explanation or alignment with human reasoning. We will also report the weaker auxiliary-only evidence on SST-5 and will not present the routed readout as a universal explanation.

AuthorsOpenReview ↗

Response to Reviewer w9Ub [3/3]


On the batch dependence of Eq. (10)

Question 3. "In Eq. (10), why are the positive and negative threshold shifts averaged over the batch? Could this make the prediction for a given sentence depend on the other samples in its inference batch?"

We thank you for identifying this implementation issue. The original implementation did make a sample's prediction depend on other samples in the inference batch; this is a correctness defect, not merely an unclear formula.

In the same-checkpoint diagnostic, batch-mean thresholds produced 341/12,241 (2.8%) prediction flips between batch=1 and batch=128. After changing the threshold shifts to per-sample computation and retraining, predictions are identical in both batch settings, with 0 flips and a maximum probability difference of about 4.5×10−54.5\times10^{-5}. Accuracy did not improve, so we treat this as a correctness fix rather than a performance gain.

ConfigurationBatch=1Batch=128AgreementFlips
Old batch-mean θ53.6753.7097.21%341 / 12,241
Per-sample θ (retrained)53.6353.63100.00%0

We will rewrite Eq. (10) with per-sample θi+\theta_i^+ and θi−\theta_i^-, remove the batch mean, and retain this diagnostic in the appendix to document the original issue, the fix, and its validation.


On Figure 1, Figure 2, and equation presentation

Weakness 5. "The overall presentation makes the central idea difficult to follow. Figure 1 packs many modules, equations, and connections into a single diagram with little visual hierarchy. Figure 2 devotes considerable space to neurotransmitter pathways, even though the implemented modulation constitutes only a small part of the model and does not simulate these pathways."

We thank you for this specific comment on the readability of Figures 1 and 2. Figure 1 uses a deliberate two-level design: the upper part presents the end-to-end pipeline from input projection and semantic graph to continuous coupled dynamics and readout, while the lower part presents implementation details of the component modules. The two levels are already separated in the figure; the main issue is that the caption, labels, and visual cues do not make this structure explicit enough, so readers have to infer it.

Figure 2 presents the action sites of DA/NE/ACh and approximates receptor-level simulation at the macro level: DA/NE correspond to positive/negative threshold modulation and ACh to coupling gain. Constrained by implementation precision, release latency, release intervals, and receptor cascades are not modeled in full detail. These are engineering-precision compromises rather than arbitrary analogies. We will state in the caption and main text which parts follow real action sites and which details are simplified.

The revision will focus on three points:

  • Figure 1: retain the two-level structure (full pipeline above, module details below) but rebuild the visual hierarchy—one dark line for the main path, separate representations for s and A, equations moved out of the main figure, and a legend that explains only line types;
  • Figure 2: retain the action-site panel but reduce the biochemical pathway, marking which mappings come from real action sites and which temporal details are not implemented;
  • Equations: simplify the top-level equations, move derivations and symbol definitions to the appendix, and keep only variable names and arrow relations in the main figure.

These changes do not alter the model's computation; they improve the readability of the full pipeline and the module details.


Summary of findings

  • Same-graph GRU control: CMNLI accuracy is 53.16 for CCNN versus 50.32 for the same-graph GRU (+2.85pp, 5/5).
  • Decision trace: the single-instance trace reports each token's sis_i, Vˉi\bar V_i, and aia_i, together with its contribution to the two evidence channels.
  • Readout ordering: on SST-5, the graded output has a Spearman correlation of 0.613 with human labels.
  • Batch dependence: per-sample thresholds produce identical predictions at batch sizes 1 and 128.
  • Modulation factorization: FiLM/SE substitutions reduce accuracy on most datasets.

The follow-up response provides the empirical comparison for continuous coupling and the matched attribution control for the comparison branch.

We thank you for your careful reading and for raising these issues.

Reviewer_TKfZOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

This paper proposes Ne-CCNN -- a compact, neuroscience-inspired architecture for language understanding that replaces conventional deep layer stacking with recurrent coupling dynamics. It uses a small network to generate three signals that adjust firing thresholds and coupling strength. It shows improvements on six of seven sentence-pair benchmarks against matched non-pretrained baselines.

Critical strengths
  1. Ne-CCNN extends continuous-coupled neural networks from perception to language through content-dependent semantic graphs and recurrent interactions. And the neuromodulation module uses approximately 1,000 parameters and provides consistent, albeit generally modest, improvements in ablation studies.
  2. Explicitly represented polarity channels, thresholds, and coupling strengths enable direct inspection and intervention, offering more structured explanations than conventional hidden representations.
Critical weaknesses
  1. Ne-CCNN achieves only modest improvements over BiLSTM on most single-sentence tasks and remains substantially behind pretrained BERT on major language-understanding benchmarks. This limits the demonstrated practical benefit of the proposed architecture.
  2. Although the model exposes named variables and polarity-routed evidence, this does not necessarily establish that the explanations faithfully capture the model's decision process or align with human reasoning.
Questions for authors

Can the authors quantitatively evaluate explanation faithfulness, for example by intervening on high-attribution tokens and measuring the resulting prediction changes?

2 comments
AuthorsOpenReview ↗

Response to Reviewer TKfZ [1/2]

Dear Reviewer TKfZ,

Thank you for your detailed and constructive review. We are particularly encouraged by your positive assessment of the paper's Presentation and Soundness, and by your recognition of three aspects of the work: Ne-CCNN extends continuous-coupled neural networks from perception to language through content-dependent semantic graphs and recurrent interactions; the neuromodulation module, with about 1,000 parameters, yields consistent but generally modest improvements in ablation studies; and the explicitly represented polarity channels, thresholds, and coupling strengths make decisions directly inspectable and intervenable, offering more structured explanations than conventional hidden representations. This recognition is an important basis for continuing this mechanism-oriented line of work.

We also thank you for your careful examination of the accuracy and explanation evidence. Below we address the single-sentence accuracy, the BERT comparison, and explanation faithfulness, and report the additional intervention, calibration, and human-valence analyses we have completed.


On the practical value on single-sentence tasks

Weakness 1. "Ne-CCNN achieves only modest improvements over BiLSTM on most single-sentence tasks and remains substantially behind pretrained BERT on major language-understanding benchmarks. This limits the demonstrated practical benefit of the proposed architecture."

The single-sentence results under the original paper protocol are as follows (5 seeds, accuracy in %):

DatasetNe-CCNNBiLSTMΔ (ours − baseline)
IMDB86.4886.43+0.05
MR78.1077.13+0.97
SUBJ90.7090.44+0.26
SST-280.5982.16−1.57
SST-543.0142.40+0.61
ChnSenti89.1888.80+0.38
Waimai89.4689.84−0.38

These results show task-dependent differences: MR improves by +0.97, SST-2 and Waimai are negative, and the remaining tasks lie within ±0.6. The MR gain is the clearest positive single-sentence result; because it is sensitive to the embedding-corpus protocol, we report it under that protocol rather than extrapolating it to all single-sentence tasks. On sentence-pair settings, our model also wins and loses against pre-trained/large-model references such as SpikeLM (e.g., CMNLI, MRPC, and CoLA), so the overall picture is not one of uniform inferiority.

BERT uses external pre-training data and a much larger parameter budget. We therefore treat it as an external reference rather than a matched-budget baseline; the comparison calibrates the distance to the pre-trained regime. The matched comparison with BiLSTM shows that 5/7 single-sentence tasks are within ±1.0, with a clear +0.97 gain on MR and negative gaps on SST-2/Waimai. Our positioning is to study an inspectable and intervenable computation mechanism at 1.5–3.0M non-embedding parameters without external pre-training, while reporting both its positive results and its boundaries.

The revision will report positive and negative results task by task for both single-sentence and sentence-pair settings, retaining the conclusions the evidence supports rather than making a blanket superiority claim.

AuthorsOpenReview ↗

Response to Reviewer TKfZ [2/2]


On faithfulness and alignment with human reasoning

Weakness 2. "Although the model exposes named variables and polarity-routed evidence, this does not necessarily establish that the explanations faithfully capture the model's decision process or align with human reasoning."

Question 1. "Can the authors quantitatively evaluate explanation faithfulness, for example by intervening on high-attribution tokens and measuring the resulting prediction changes?"

We thank you for explicitly asking for a quantitative evaluation of explanation faithfulness; this helps separate "the variables are inspectable" from "the explanation is faithful." Naming variables alone does not establish faithfulness, so we added deletion/perturbation experiments on high-attribution tokens, with matched random controls, bottom-token controls, and a BiLSTM attention baseline.

The table below reports prediction-flip rates after top-kk intervention:

DatasetModelk=1k=2k=3
MRNe-CCNN routed evidence0.1200.2250.335
MRBiLSTM attention0.1050.1550.205
SST-2Ne-CCNN routed evidence0.2000.2300.310
SST-2BiLSTM attention0.2150.2900.355
SUBJNe-CCNN routed evidence0.0300.0650.120
SUBJBiLSTM attention0.0850.0750.120

At k=3k=3, the random-control flip rates for Ne-CCNN are 0.1213 (MR), 0.1619 (SST-2), and 0.0700 (SUBJ); the paired statistics against random are t=6.58t=6.58, 4.924.92, and 2.442.44. These results show that the evidence tokens selected by the model have a measurable causal effect on predictions. They do not, however, dominate attention on every dataset: MR is stronger, SST-2 is weaker for k≥2k\ge2, and SUBJ ties at k=3k=3. We report this task dependence as it is.

We also add a single-instance decision trace showing that routed evidence is only one additive component of the final logit; auxiliary representations and other branches also affect the prediction. We therefore do not describe routed evidence as a complete decision explanation.

For human alignment, we performed a limited word-level valence evaluation. On SST-2, the proportions of top-kk selected tokens that are strong-valence words are 0.352/0.321/0.299 for k=1/2/3k=1/2/3, versus 0.120/0.126/0.127 for random controls; the agreement between the channel sign of selected strong-valence words and the human valence sign is 0.776/0.740/0.719, versus 0.649/0.641/0.627 for controls. MR and SUBJ also show enrichment of high-valence words, while sign agreement is clearer mainly for k≥2k\ge2.

These results provide quantitative evidence of human-valence alignment: top-kk evidence tokens are enriched in human strong-valence words, and their channel signs are consistent with human valence. They support a correspondence between routed evidence and external human judgments, but they do not cover a complete reasoning chain; sentence-level rationale alignment requires a separate human-annotation protocol and is left as future work. The revision will retain this positive result while avoiding claims of "proof" or "alignment with human reasoning."

Additional experiment: single-instance decision trace and readout ordering

We add two analyses to make "traceability" concrete.

Table R2. Decision trace of one MR instance

QuantityValue
Instance length / prediction17 / negative
zposz_{\mathrm{pos}}0.027
znegz_{\mathrm{neg}}0.192
Routed share of logit marginabout 5%

The per-token terms (si,Vˉi,ais_i,\bar V_i,a_i) are listed in the supplementary material. This shows that the forward computation can be decomposed token by token; it is not a complete decision and is not, by itself, a faithfulness proof.

Table R3. Readout ordering (full test split, 5 seeds)

DatasetStatisticResult
SST-5Graded output vs. human labels, Spearman ρ0.613 ± 0.003
SST-5Ordinal MAE0.817 ± 0.006
IMDBPolarity evidence vs. human valence, Spearman ρ0.448 ± 0.005
SST-2Same0.400 ± 0.008
MRSame0.328 ± 0.007
SUBJSame0.155 ± 0.005

This supports external valence alignment, not decision faithfulness; the lower SUBJ correlation shows that the alignment is not universal.

Additional experiment: calibration

We also evaluated decision-layer calibration. On CMNLI, the original checkpoint has a test ECE of 0.0848, which decreases to 0.0293 after calibration; accuracy does not improve. We therefore report this as positive evidence for uncertainty estimation at compact scale, not as an accuracy contribution.

ConfigurationTest ECE
Original checkpoint0.0848
Calibrated0.0293

We thank you again for raising these issues. Your comments have helped us define more precisely what the method can support and what remains outside its scope.

Comments on the paper

3 comments
AuthorsOpenReview ↗

Response to All Reviewers [3/3]

5. Manuscript changes

Both Figure 1 and Figure 2 have been revised/redrawn; the final version will use the new figures.

Abstract / Introduction

  • [w9Ub] Align continuous coupling to a task-dependent mechanism benefit with inspectable state.
  • [w9Ub] Describe the pair branch as a standard component and separate the pair gains from the mechanism evidence.

Methodology

Figure 1

  • [w9Ub, ZjSc] Rebuild the hierarchy: the upper part shows the end-to-end pipeline from input projection through semantic graph and continuous coupling to readout, and the lower part shows module details only.
  • [w9Ub] Use one dark line for the main path; separate conditioning lines and the s/A relations; keep the legend for line types only.
  • [ZjSc] Simplify in-figure equations and move the full symbol definitions to Method/Appendix.

Figure 2

  • [w9Ub] Redraw the action-site panel, keep the action-site information, and reduce the space devoted to biochemical pathways.
  • [w9Ub] Retain the biological inspiration at the level of action sites/effects: DA/NE modulate thresholds and ACh modulates coupling gain.
  • [w9Ub] State in the caption that receptor cascades/release timing are functional abstractions; use the latest redraw in the final version.

Eq. (10)

  • [w9Ub] Use per-sample �0��0� and remove the batch mean.
  • [w9Ub] Add the Appendix diagnostic: 341/12,241 flips in the old implementation, 0 flips on the retrained CMNLI checkpoint, maximum probability difference �1��1�.
  • [ZjSc] Define �2��2� at first use; move derivations/symbol table to Appendix; add one explanation sentence per equation.

Experiments

Main Results

  • [TKfZ] Update the single-sentence protocol and boundaries; add decision trace, readout ordering, and human-valence proxy with their supported scope.
  • [w9Ub] Add the same-graph and matched pair-branch tables (strict 50K; 3 seeds unless noted; QNLI 5 seeds; mean ± std), with the protocol in the caption.
  • [w9Ub] If accepted, replace the compact-model rows of the main table with matched-setting results, keeping pre-trained references separate.
  • [ZjSc] Add the PRR routed/auxiliary/comparison decomposition and attention comparison.

Ablation Studies

  • [w9Ub] Add state ablation with no-coupling/no-neuromodulation flip rates.
  • [w9Ub] Show that components have measurable roles and the joint model is usually best, while interactions mean gains are not simply additive; use retrained ablations as primary component evidence.
  • [ZjSc] Add SST-5 Aux=0, task dependence, retrained ablations, and trace/deletion/random/attention/faithfulness protocols.

Discussion

  • [w9Ub] Discuss the mechanism gain, task variation, and the roles of the pair branch versus core dynamics; highlight the inspectable state.
  • [TKfZ] Discuss single-sentence external validity and valence coverage; list further SST-5 ordering validation as future work.
  • [ZjSc] List human-rationale alignment, end-to-end reproduction, and attention-superiority tests as future work.

Appendix

  • [w9Ub] Add the batch-dependence diagnostic.
  • [w9Ub] Add the full same-graph GRU table and seed list.
  • [w9Ub] Add the matched-table seed list and protocol note; do not repeat the main-text matched table.
  • [w9Ub] Add the full NLI logit decomposition table.
  • [w9Ub] Add the full state-ablation table.
  • [w9Ub] Add the full-data protocol and Figure 2 note.
  • [ZjSc] Add the extended-ablation tables.
  • [ZjSc] Add the deletion/random/attention/faithfulness protocols.
  • [TKfZ] Add the valence-norm source and threshold.
  • [TKfZ] Add the trace and readout-ordering protocol.

We thank the reviewers again for their patience, detailed comments, and oversight; we hope the revised evidence and claim structure make the contribution and its boundaries clearer.

AuthorsOpenReview ↗

Response to All Reviewers [2/3]

3. Sentence-pair tasks: matched pair branch

To separate the sources of the sentence-pair gain, we attached the same pair branch to all compact encoders, keeping the strict 50K protocol, h256/T3, and best-validation selection fixed while varying only the encoder. Entries are test accuracy and use three seeds unless noted; CCNN QNLI uses five.

TaskCCNNBiLSTMTextCNNLIF-SNN
QNLI68.52 ± 0.8757.70 ± 0.1063.26 ± 5.7957.45 ± 0.05
MNLI51.99 ± 0.5750.80 ± 0.3050.24 ± 0.6949.61 ± 0.39
CMNLI54.52 ± 0.4454.46 ± 1.3253.87 ± 0.2752.99 ± 0.16
QQP74.60 ± 0.1474.40 ± 0.2076.86 ± 0.0475.22 ± 0.57
RTE52.23 ± 2.6954.15 ± 1.6553.91 ± 0.7454.51 ± 0.94
MRPC69.84 ± 0.4769.24 ± 0.4269.10 ± 0.3468.99 ± 0.24

The pair branch is a major contributor on sentence-pair tasks. Under the matched pair-interaction comparison, CCNN is strongest on QNLI/MNLI, close on CMNLI/MRPC, and relatively weaker on QQP/RTE. The pair branch is a standard task component; CCNN retains independent value in its inspectable state, modulation, and intervention.

The task-level pattern is consistent with the structure of the benchmarks. QNLI and MNLI require directional sentence-level inference, where an explicit premise–hypothesis interaction gives the encoder token pairs it would otherwise have to infer indirectly; this is where the pair branch has the largest headroom. CMNLI is similar but adds Chinese-specific lexical and negation patterns, so the branch helps while the remaining gap is not resolved by the branch alone. QQP is closer to symmetric duplicate detection: pooled overlap features already capture much of the signal, and the additional token-level interaction has less to add, which is consistent with TextCNN performing best. RTE is a small-data task, and both pair variants show high seed variance; the branch does not translate into a stable gain. MRPC consists of short paraphrase pairs, where lexical overlap dominates and the branch gives a small, less stable benefit. We present these as task-level interpretations of the observed pattern, not as causal claims about each dataset.

4. State use and evidence scope

We added two inference-time diagnostics that directly address whether the state is used and how much the routed channel contributes.

State ablations:

Taskfullno couplingno neuromodulationinput only
QNLI69.8769.52 (flip 4.3%)66.70 (flip 18.9%)66.08 (flip 22.2%)
MNLI52.3452.34 (flip 0.8%)49.51 (flip 21.8%)48.93 (flip 23.1%)

Neuromodulation/state is clearly used: removing it changes 19–22% of predictions. Removing coupling changes 0.8–4.3%. Continuous coupling is therefore better described as a task-dependent inductive bias and an inspectable dynamical object, not the sole source of accuracy.

NLI logit decomposition:

Taskfullrouted-onlyaux-onlycomparison-only
QNLI69.8749.4469.8768.20
MNLI52.3445.0744.0434.62

On QNLI, the routed logit has mean L2 norm 0.015, the auxiliary term 11.34, and the comparison component 10.28; the three paths jointly determine the final logits. On MNLI, the full model is higher than any single path.

AuthorsOpenReview ↗

Response to All Reviewers [1/3]

Dear Reviewers,

We sincerely apologize for the delay in providing a complete and coordinated response. We have spent this period completing the matched experiments, checking the implementation and data protocols, and aligning the claims across the paper. The note below summarizes the shared work; the individual responses provide the detailed point-by-point answers.

1. Shared corrections

Code and reproducibility. The anonymous repository is available, with the core Stage 2–5 implementation, README, environment, and device details:
https://anonymous.4open.science/r/Ne-CCNN/
We are continuing to check the reproduction configuration and documentation.

Batch dependence. The original batch-mean thresholds made a sample's prediction depend on other samples in the same inference batch. On the retrained CMNLI checkpoint, per-sample thresholds make batch=1 and batch=128 predictions identical: 0 flips and a maximum probability difference of about �0��0�. This is a correctness fix, not an accuracy gain.

Biological grounding of neuromodulation. DA/NE/ACh are not arbitrary scalars: they borrow distinct action sites and functional effects—DA/NE modulate positive/negative channel thresholds, and ACh modulates coupling gain. The biological inspiration is retained at the level of action site and functional effect; receptor cascades and release timing are deliberately abstracted for a compact implementation. Both Figure 1 and Figure 2 have been redrawn and are being refined; the final version will use the latest figures.

Single-sentence results and component evidence. The single-sentence results are competitive with BiLSTM on several tasks and vary by task and protocol; we report the full task-dependent picture. Component ablations show that each component has a measurable role and the joint model is usually best; because of interactions, the individual gains are not simply additive.

2. Continuous coupling: direct mechanism control

To test whether continuous coupling adds anything beyond a conventional recurrent graph encoder, we used the same semantic graph, modulation, and readout, and changed only the recurrent update. Protocol: strict 50K, best-validation selection, fixed seeds; entries are test accuracy.

TaskCCNNSame-graph GRUΔ
CMNLI, 5 seeds53.16 ± 0.4550.32 ± 0.46+2.85pp (5/5)
QNLI-50K, 3 seeds64.70 ± 1.1364.48 ± 1.05+0.23pp (3/3)
QQP-50K, 5 seeds74.99 ± 0.3074.62 ± 0.12+0.37pp (4/5)
MNLI-50K, 3 seeds51.33 ± 0.3950.19 ± 1.53+1.14pp (3/3)

The direction is consistently positive across all four tasks. CMNLI is strongest and seed-consistent (+2.85pp, 5/5); QNLI, QQP-50K, and MNLI also remain positive (0.23–1.14pp). These are consistent multi-seed effects, and the effect sizes are practically meaningful. The benefit is task-dependent, but it provides a measurable mechanism gain on NLI/sentence-pair tasks and retains an inspectable state trajectory.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.