- Rating
- 2
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper presents the first training data reconstruction attack against federated weighted Convolutional Tsetlin Machines (wCTMs), exploiting their inherent interpretability, and the fact that non-IID federated data leaves many client classes trained on only a handful of samples. From a single uploaded client model, an honest-but-curious server identifies vulnerable client classes, estimates the unknown number of training samples via the maximum clique of a clause conflict graph, assigns clauses to canvases through an integer linear program, and paints each canvas by solving a second ILP that places clauses conflict-free while filling in class-common pixels using a cross-client pixel agreement map. Evaluated on five image-recognition benchmarks (MNIST, F-MNIST, SVHN, GTSRB, CIFAR-10) under Dirichlet non-IID partitions over 100 clients, the attack detects vulnerable classes with F1 between 0.92 and 1.00, estimates sample counts with accuracy, and reconstructs up to 95% of exposed samples when . The work concludes that the very interpretability that makes Tsetlin Machines attractive is what exposes their training data, and that principled defenses such as differential privacy or model-configuration tuning trade utility for privacy.
Critical strengths
- The paper identifies and exploits a genuine, non-obvious attack surface that existing reconstruction attacks cannot reach.
- The empirical evaluation is broad, multi-metric, and honestly reports where the attack fails. Attack papers are frequently undermined by cherry-picked settings or single metrics that conflate class-generic reconstruction with sample-specific recovery. This paper's use of Matching F1 specifically separates the two, and its willingness to report SVHN as the hard case and to show degradation curves by true sample count makes the central claim credible.
Critical weaknesses
- The paper attacks weighted Convolutional Tsetlin Machines (wCTMs), a model family that rarely appears in the main ICLR program. Although Tsetlin Machines have some activity in interpretable logical learning, edge inference, and hardware acceleration communities, they are neither a mainstream ICLR architecture (Transformers, diffusion models, GNNs, state-space models) nor a primary target of the ICLR privacy community (neural networks, language models). Consequently, the paper's central contribution, "the first reconstruction attack against federated wCTMs", lacks direct relevance for most ICLR reviewers and readers.
- The attack assumes the server obtains the complete internal state of each client's uploaded wCTM, e.g., all clauses, Tsetlin automaton states, integer weights, and valid position sets. This comes from the "single-round upload of the full local model" setting of FedTMOS (S How et al., 2025). The paper does not sufficiently justify the realism of this assumption, nor does it discuss whether the attack still holds if the protocol uploads only aggregate statistics, encrypted states, or gradient approximations.
Questions for authors
- If the paper could show that "any interpretable model based on conjunctive clauses (decision trees, rule lists, Tsetlin Machines, neuro-symbolic models) leaks under few-sample federated settings," its audience would be much broader. But all experiments are confined to wCTMs, with no cross-model validation.
- The paper explicitly acknowledges that the attack only works when each client class has 1–5 training samples. Beyond 5, clauses begin to generalize, the conflict graph loses structure, and both sample-count estimation and reconstruction quality degrade sharply. This means the attack's applicability is very narrow, and the paper does not sufficiently argue whether this narrow regime is common in real federated settings.
- Section 5 discusses defenses, identifies differential privacy (DP) as a "principled defense," and suggests using a discrete Gaussian mechanism to add noise to TA states and clause weights. But the paper performs no DP experiments, reports no figures on how the attack's F1, Jaccard, or Matching F1 degrade under DP noise, and does not report what is needed to render the attack unusable. This leaves the "defense discussion" on paper and weakens the paper's practical guidance.
- After adding DP noise, does the attack still work? If so, how large an is needed? If suffices to stop the attack, the threat is limited; if even fails to stop it, the attack is severe. The paper provides no such curve.
- The paper's core technical tools, e.g., maximum clique, integer linear programming (ILP), and SAT solving, are standard tools in combinatorial optimization and operations research, not new methods of interest to the ICLR community.
- Theorem 1 proves that the maximum clique is a lower bound on the sample count. The observation is clean, but its mathematical depth is limited, essentially a counting argument via an injective map. Does the proof go beyond what is "obvious"? To be specific, Theorem 1 is a lower bound, but it is "obvious" (each conflicting clause must come from a different sample) and not tight (the paper itself says the bound loosens when ).