acceptodds
Under review as a conference paper at ICLR 2027

Neural Dynamics as the Composition of Quantized Units

Abstract

Deep learning is commonly interpreted at two levels: the macroscopic, through aggregate trends in loss summarized by scaling laws, and the microscopic, through neurons, features, and circuits. A central challenge is understanding how these levels connect, so that we can explain how elementary computations compose and collectively shape macroscopic behavior. To this end, we study an intermediate abstraction in which training is described as the ordered acquisition of quanta: reusable computations acquired suddenly and binary-activated across examples to reduce loss. By approximating population-gradient updates, we derive quanta's acquisition dynamics. This yields an acquisition priority governed by demand, how frequently a computation is required across examples, and conditional complexity, how difficult that computation is to acquire given those already available. In a Boolean compositional task, we derive predictions for acquisition order and show how staggered discrete acquisitions can produce smooth aggregate loss and, under certain geometries of quanta composition, give rise to scaling laws. We then train a Transformer to map numerals to English number names and recover candidate quanta from its checkpoint trajectory. From these units, we construct a model that preserves much of the Transformer's behavior while exposing interpretable latent computations and acquisition dynamics consistent with the theory. Separately, the quanta structure can serve as training targets to improve transformer generalization. Together, these results suggest the quanta abstraction can provide useful computational atoms for studying a variety of macroscopic phenomena.

Primary area:
learning theory
Keywords:
learning dynamics, compositional learning, neural scaling laws, mechanistic interpretability, science of deep learning

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_J7bqOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
1
Soundness
2
Read review
Paper summary

In this paper, the authors study the representation of neural networks using quanta, which is a description between two extreme descriptions of learning dynamics: the parameter trajectory (microscopic) and the scalar loss (macroscopic). They introduce the demand, the conditional complexity, and the acquisition priority to measure which quantum is acquired earlier. They empirically study how acquisitions compose, why their aggregate looks smooth, and how macroscopic regularities arise. They also empirically investigate a task of recovering quanta.

Critical strengths

This paper tries to clarify an important problem: how the models can be represented using reusable and composable elements. The topic is very interesting. They try to understand the phenomena that a quantum tends to be acquired earlier when it is useful to more examples and when it is simpler given the currently available computations by introducing the notion of demand and conditional complexity. The idea is interesting.

Critical weaknesses

My major concern is about the conditional complexity that the authors introduced. How to compute this value in practice is unclear for me. d(I) is not available before the training. In addition, is it reasonable to assume that there is a single direction in parameter space that builds the quantum q? Since in the overparameterized regeme, the same function can be represented even with different parameters. Thus, I am not sure if we have one single direction.

Another concern is about the presentation. Section 3 is too dense, and it was difficult for me to clarify which parts and formulas are important in this work. I think adding figures that describe the concept schematically would help readers to understand. Also, some displayed formulas should be added to show important definitions and statements.

Questions for authors

Please see the weakness section.

2 comments

Dear reviewer J7bq,

We wanted to politely follow up on whether our rebuttal has addressed your concerns. Since the time for author-reviewer engagement is soon coming to a close, we would appreciate the opportunity to engage further if any of your concerns remain unresolved or if additional work would help improve your assessment of our paper.

Otherwise, if you feel that our responses have satisfactorily addressed your concerns, we would be grateful if you would consider increasing your score.

Thank you for the clarification. The two proposed methods for estimating the complexity still seem to need separate runs, and my concern has been only partially resolved. I would like to keep my score unchanged.

Reviewer_mMj3OpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

Prior work proposed that the seemingly continuous learning behavior of deep learning may in fact consist of many overlapping discrete steps, which Michaud et al. 2023 termed quanta. In many cases (e.g. Olsson et al. 2022), these quanta appear to be associated with the emergence of computational structure in models. This paper extends that picture by asking what determines the order of acquisition. Via a heuristic calculation of the population gradient update, the authors argue that a quantum’s order of acquisition is determined by the ratio of its demand (how often it is required) to its conditional complexity (how hard it is to learn given previously learned quanta). A network’s training can then be described by an effective Q-model that tracks the acquired set and its update rule, with prerequisite relations forming a learning poset. The authors validate this model on two synthetic tasks: hierarchical sparse parity (generalizing the sparse parity task from Michaud et al. 2023), and a number naming task. They run extensive validation experiments on these tasks, including demonstrating how shifting the data distribution changes learning order, deriving scaling laws from the learning poset and demand, and attempting to explicitly recover the Q-model for number naming.

Critical strengths

The ideas of the paper are compelling and I believe novel to this literature. The quantization model seems plausible and interesting, but it has previously lacked an understanding of when these quanta are learned and how they depend on each other, which would give the theory significantly more predictive power.

The idea that which quanta are learned should depend on already learned quanta seems intuitively plausible. Moreover, the idea of a learning poset seems to capture a highly-important object to investigate, and is novel to the best of my knowledge.

The idea of generalizing sparse parity to hierarchical sparse parity is clever, and I would expect to be useful beyond this paper. Existing toy models of hierarchical feature learning are in limited supply.

Furthermore, the experiments are notably broad for a learning theory paper; the authors do not just validate one isolated prediction in a toy model, but many, across several different settings and including in Transformers. I am particularly encouraged by the results changing learning order and attempting to explicitly recover the Q-model.

Critical weaknesses

I believe the paper is insufficiently careful about distinguishing assumptions, heuristic arguments, and rigorous mathematical calculations, in a way which does not meet the standards of a typical learning theory paper. For example, in Section 3.2, the authors implicitly assume that a quantum is learned at a constant rate, with constant gradient $s_q(I_t)$, without discussing that assumption. It is in fact false for simple toy models like deep linear networks, where the analogous quanta magnitudes (singular values) accelerate rapidly, following a sigmoidal equation. Of course, this does not show that the assumption is false, deep linear networks could be unrealistic in this fashion; but clearly the assumption is nontrivial and should be explicit, or the argument should be modified to not depend on it.

It is unclear to me whether the experiments actually support the proposed explanations and theory. It seems conceivable to me at the moment that the qualitative results of the paper are correct, but the theoretical explanation for them is incorrect; the evidence presented for the theoretical model currently feels insufficient to me. Many steps in the sketched derivation, like $d_q $or $s_q$, the direction $v_q(I_t)$, etc, are not measured independently anywhere. Perhaps this feels pedantic, and the authors would like to focus on the qualitative results and did not intend for these mathematical quantities to be distracting; in this case these mathematical quantities should be removed or greatly pared down.

I would suggest to the authors that they move towards either:
(A) a phenomenological model, proposing simple empirical relationships and removing speculative reasoning steps. For example, “among quanta whose prerequisites have been learned, quanta acquisition time is approximately inversely proportional to sampling frequency” instead of “there is a direction associated to each computation; examples requiring it reinforce progress along that direction; other examples contribute negligibly; acquisition requires a certain distance; the speed is approximately constant; their ratio defines conditional complexity; therefore acquisition time is complexity divided by demand.” Or, alternatively,
(B) a theoretical model, where the claims are proven rigorously as a theorem for some (possibly toy) architecture, which would force the arguments to be separated from the assumptions unambiguously, and make explicit any hidden reasoning steps. For example, something like Barak et al. 2023, who the authors cite. Even proving results for something like a deep linear network here would be useful.

Currently, I believe the paper suffers from being somewhere in between these two contribution types: it has the level of rigor of a phenomenological model, but the level of specific claims approaching a detailed theoretical model.

Questions for authors
  • Does acquisition time scale as $1/p_q$, or only rank-correlate with it? If I understand your math correctly, the derivation should predict $T_q \propto \kappa_q / p_q$, implying a linear relationship for constant $\kappa_q$, but the paper reports only the Spearman correlation.
  • Can conditional complexity be measured independently of acquisition order? Since $p_q$ is known, $\kappa_q$ can always be inferred from the observed acquisition time, so naively it seems like any learning order is consistent with the theory. But the presence of the conditional complexity $\kappa_q$ is, if I understand correctly, the largest difference between Michaud et al. 2023 and the current paper.
  • Is Section 3.2 a derivation or merely a motivating heuristic?
  • Can you derive rigorous results in a toy model? [Future work]
2 comments

Dear reviewer mMj3,

We wanted to politely follow up on whether our rebuttal has addressed your concerns. Since the time for author-reviewer engagement is soon coming to a close, we would appreciate the opportunity to engage further if any of your concerns remain unresolved or if additional work would help improve your assessment of our paper.

Otherwise, if you feel that our responses have satisfactorily addressed your concerns, we would be grateful if you would consider increasing your score.

Thank you for the detailed response to my review, clearly this took substantial effort. The switch to a variable speed is a sensible choice, and I appreciate it. The deep linear network derivation is useful for improving rigor and reducing ambiguity in the derivation - e.g. it makes clear the implicit assumption that different quanta cannot interfere with each other. And I am especially encouraged by your results on the parent pretraining experiment, since this now directly tests how previously acquired quanta can affect acquisition order, and demonstrates the theory has novel predictive power. These alone are enough to convince me to raise my score to a weak acceptance, as I believe this to be a valuable contribution despite some of my original concerns remaining.

This being said, I remain uncertain whether your results actually depend on any of the quantities used in the derivation. The following simpler hypothesis still seems consistent with the evidence to me:

Acquisition time is roughly κq/pq\kappa_q / p_q for some constant κq\kappa_q depending on which skills are already learned. At any given time, the next skill learned will be the one with the smallest κq/pq\kappa_q / p_q.

This is, as far as I understand it, just the paper's hypothesis but without the intermediate quantities vqv_q, dqd_q, and sqs_q (or aqa_q).

The new experiments "estimating κq(I)=dq(I)/sq(I)\kappa_q(I)=d_q(I)/s_q(I) from required progress and gradient speed" do not seem convincing to me, since sqs_q is averaged over the acquisition period itself, so dq/sqd_q/s_q essentially just recovers the number of relevant updates the pilot run took to acquire qq. The 4.78% result then amounts to another test of Tq∝1/pqT_q \propto 1/p_q, rather than an estimate of complexity made independently of acquisition time. And while the deep linear network results are encouraging, they are not sufficiently discriminatory for this purpose, given that the quanta are all independent of one another and so the conditional complexity here is trivial.

It remains unclear to me what the purpose of these extra quantities like the direction vq(I)v_q(I), the required progress dq(I)d_q(I), and the gradient speed sq(I)s_q(I) are. If the authors were to share an updated draft removing these and streamlining the mathematical exposition, or present more convincing results supporting them (such as a derivation in a more sophisticated toy model where acquiring one quantum changes the conditional complexity of another), I would consider raising my score further (though I expect to have a high standard here).

Reviewer_j35iOpenReview ↗
Rating
1(−1)
Confidence
3
Informativeness
2
Presentation
2(−1)
Soundness
1(−1)
Read review
Paper summary

A neural network's training process can be described by the acquisition of reusable computation unites called quanta. Each quantum switches on abruptly. Smooth loss curves and scaling laws are aggregated effects of these steps. The priority of learning quanta is found to be determined by the demand and the conditional complexity.

Contributions: The authors give heuristic expressions for the demand and the relative complexity, and show qualitative theoretical arguments why they influence learning. The authors validate the framework on two toy models: HSP and the number naming task by transformer.

Critical strengths

The authors' attempt to summarize the quantized learning dynamics into two main driving forces, demand and conditional complexity, is genuinely insightful. The HSP model further provides a controlled and exact test bed for the phenomenological observations. Hierarchical sparse parity has known building blocks, exact Bayes loss levels, and demand set by design. That makes each theoretical prediction directly measurable, for example the acquisition-time fit T∝π−0.98T \propto \pi^{-0.98}.

Critical weaknesses

To me, one of the key novelty compared to Michaud (2023) is the dependence on conditional complexity. However, the conditional complexity is never used to make a prediction; it is only proxied or inferred after the fact. The scaling law reduces to Michaud’s when the extra cost exponents are zero, as in every experiment.

Questions for authors
  1. Can you supply some new theory or experiments to show that the conditional complexity controls things in the theory's predicted way?
  2. In more realistic models, how do you identify a set of reasonable quanta (I believe it is unlikely possible because being able to locate a good set of quanta means more than solving machine interpretability)? If there is no set of quanta that we can easily prescribe, how to determine the demand and complexity, and does the main result "priority = demand + conditional complexity" change?
2 comments

Dear reviewer j35i,

We wanted to politely follow up on whether our rebuttal has addressed your concerns. Since the time for author-reviewer engagement is soon coming to a close, we would appreciate the opportunity to engage further if any of your concerns remain unresolved or if additional work would help improve your assessment of our paper.

Otherwise, if you feel that our responses have satisfactorily addressed your concerns, we would be grateful if you would consider increasing your score.

I thank the authors for the explanation. Your response is making positive progress towards resolving my concerns, but many things remain unclear. I keep the possibility of changing my score open.

  • First of all, let me remind the authors that you are allowed to upload a revised manuscript during the rebuttal period, and to this specific case my decision crucially depends on it. I understand that a new experimental strategy is roughly described, but it needs to be much more quantitative, clear, and the experiment procedure+results clearly presented. For example, I have not yet seen a concrete definition of zqz_q, and why its change is dqd_q.

  • Assuming the revised manuscript successfully shows all the declared prediction-observation agreement, I am not completely clear whether this design answers my question. To me, the simplest design is like (1) Fix pqp_q, show Tq∝κqT_q\propto \kappa_q (2) Fix κq\kappa_q, show Tq∝1/pqT_q\propto 1/p_q. It is not clear to me how your result of (a) being able to predict S or (b) observing pc/pr≈κc/κrp_c/p_r\approx \kappa_c/\kappa_r becomes equivalent to either (1) or (2).

  • Your experiment to estimate complexity mostly makes sense to me, except, as said before, that I don't see a definition of zqz_q and why it is related to the required progress used in Section 3.2.

After reading the review of the other two reviewers, I would like to add one extra comment. To me the current form of Section 3 does not qualify as a rigorous derivation. It is phenomenological at best with an intuitive picture to help justify T∝κ/pT\propto \kappa/p at most, mainly because a lot of things are not rigorous (e.g., the conditional complexity's definition is at best empirical, and is not a complexity in the complexity theory sense; also, the authors only motivated that TqT_q should grow with κq\kappa_q with common sense, which is not a derivation). I would suggest that the presentation of Section 3 towards a pheno paper, and support T∝κ/pT\propto \kappa/p with direct experimental evidence.

I think the toy model is really interesting and worth being added to the main text to help reader understand Section 3.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.