1. The paper claims to reveal shared symbolic reasoning structures inside models, but the method can only decompose input-output behavior. There is a gap between the two that has not been bridged.
The Möbius transform holds for any set function—the proof in Appendix B never uses the fact that v is a neural network. So universal matching (Eq. 2) is a property of the mathematical tool, not a finding about the model being analyzed. Yet the abstract calls the decomposition coefficients "cognitive patterns," and Section 2.2 calls them "primitive inference patterns," giving the impression that the model internally organizes its computation along AND-OR lines. Supporting that would require some form of causal verification—for instance, locating the attention heads responsible for a shared interaction and ablating them. The paper does not attempt this.
What concerns me more is the following. The Möbius coefficients are determined by the values of v(xS) across 2n masked states. The AND-OR split does introduce a nonlinear optimization over {γT}, but if two models produce similar v(xS) across those states, running the same optimization on similar functions will naturally tend to yield similar decompositions. So "shared interactions" may simply be another way of saying "behavioral similarity." If the paper compared corr(vA(xS),vB(xS)) against κ, it could settle whether the interaction decomposition carries information beyond behavioral correlation. This experiment is cheap with the existing data, but the paper does not do it.
To be fair, behavioral consistency across all 210 masked states is stronger than just agreeing on the unmasked prediction—I acknowledge that. But given that all these models were pretrained on massive English corpora with the same next-token objective, similar responses to common collocations and syntactic structures are not surprising.
2. Even setting the methodological issues aside, the finding itself is not very surprising, and the more interesting questions are left untouched.
Most modern open-weight LLMs were pretrained on heavily overlapping English data. Sharing common inference structures is a natural consequence. The paper spends considerable effort showing that "the shared part exists," but that is not the most curious question here. What would be more interesting is: which interactions differ across models? Can those differences be traced to specific training data or training decisions? Section 5 briefly argues that training data overlap may not be the main cause, but the evidence is indirect (within-family similarity is not necessarily stronger than cross-family), and the paper never really digs into what the non-shared interactions encode. If the work examined differences rather than just commonalities, it would be considerably more valuable to the community.
3. Several experimental design issues stack up and make me uneasy about the conclusions.
Sparsity: Algorithm 1 explicitly minimizes ∥Iand∥1+∥Ior∥1, and then the paper reports that the extracted interactions are sparse. I do not think this is entirely circular—doing the same optimization on a random function would probably not yield such a sparse decomposition, so the LLM output function does have some structure that favors sparsity. But the paper does not run that control, nor does it independently verify whether the theoretical conditions in Appendix A are actually satisfied. As it reads now, it looks like the method pursues sparsity and then claims to have discovered it.
Target token selection (lines 363–370) picks the token with the lowest summed rank across both models—i.e., one both models are already confident about. I understand the methodological rationale: analyzing tokens with very low confidence gives poor signal-to-noise. But the paper's conclusion ("universal symbolic representations") goes far beyond this conditioned setting.
κ has no random baseline. With n=10 there are only 1023 candidate subsets, each model retains K=80, and low-order subsets tend to have the strongest effects and be selected by both models. The chance overlap under these conditions is not negligible. Without a null model, it is hard to tell whether 62–78% is actually high.