- Rating
- 2
- Confidence
- 1
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
The paper proposes Infinitesimal Operator Discovery, a post-hoc diagnostic for trained continuous-depth models. It sequentially decomposes local vector-field Jacobians into diagonal, topology-local, low-rank global, sparse nonlocal, and residual coordinates. It combines reconstruction budgets with null models, matched-random intervention comparisons, boundary-sensitive diagnostics, and numerical/dictionary checks. On ten controlled task families with eight seeds each, it reports mean residual energy 0.0138 and the strongest intervention support for low-rank global coordinates. The intended contribution is a structural diagnostic coordinate system, rather than identification of architectural modules or a new predictive model.
Critical strengths
- The paper addresses a useful interpretability problem for continuous-depth models and clearly distinguishes structural coordinates from identifiable modules. It explicitly warns that low reconstruction residual is not causal evidence and discusses basis and topology dependence (pp. 2–3, §3.1; p. 7, §5.6; p. 8, §6).
- The reported audit is broader than a reconstruction study: it includes untrained and label-shuffled baselines, repeated scale-matched random intervention controls, eight seeds per task, boundary comparisons, and solver/dictionary sensitivity checks. The principal task-level and component-level means are internally consistent across the tables (pp. 3–7, 10–11; Tables 1–7).
Critical weaknesses
- I could not reconstruct how the local intervention in Eq. (7) produces the loss increase used in Eq. (8). The original predictor is defined in Eq. (1), and §6 appropriately acknowledges that component removal is approximated through local Jacobian actions. A local perturbation or tangent approximation could be sufficient for this diagnostic aim; no globally integrable component or explicit vector-field factorization is required. However, the PDF does not specify the perturbation/state update and readout used to obtain the intervened loss, or whether this is a finite perturbed-prediction loss or a linearized surrogate. This leaves the central reported behavioral quantity insufficiently defined (p. 3, §3.2; p. 8, §6; Appendices B–I).
- The PDF provides the task families, 80 task-seed design, sequential projection framework, scale-matched random comparator, support threshold, and rank/radius/solver sweeps. It does not provide the concrete task generators and model/training dimensions/settings, default projection budgets and sampling settings, intervention grid and maximum-stable selection rule, random-control sampling/repetition details, or confidence-interval construction and per-task intervals. These particular omissions prevent independent interpretation of the loss comparisons and verification of the 10/10 and 8/10 support counts. The residual sweeps are useful but do not themselves establish that the causal support counts persist across those settings (pp. 3–5, 7–12).
Questions for authors
- Could you give the actual update/readout equations or short pseudocode from Eq. (7) to ΔL_m, including the perturbation initialization/scale and where the intervention is applied? Please specify whether the reported quantity is a finite perturbed-prediction loss or a local surrogate, and interpret the task-relevance claim accordingly.
- What are the intervention grid and maximum-stable selection rule, the random-matching norm/distribution and repetition count, and the seed-level CI method/confidence level? The per-task intervals underlying the support counts would make the stated threshold rule directly checkable.
- Could you point to the exact task/model/training and Jacobian-sampling/default-projection configuration, together with the order-sensitivity results mentioned in §3.1? A precise location in the stated anonymized supplement would suffice; the PDF gives the audit outline but not these instantiated settings.