- Rating
- 3
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper studies how to estimate feature interactions without enumerating all feature subsets. It introduces Coordinate-Ray Interaction Quantification (CRIQ), which uses the Multivariate Fundamental Theorem of Calculus to express discrete derivatives as integrals of mixed partial derivatives under suitable smoothness assumptions. The authors also propose Iterated Dimensionality Reduction (IDR), which rewrites these expressions as alternating sums of one-dimensional integrals of first-order derivatives. This allows the method to handle continuous piecewise-linear networks and multidimensional embeddings without computing higher-order derivative tensors. The paper derives sample-complexity bounds and evaluates estimation accuracy and convergence on a ReLU MLP, DistilBERT, and synthetic unanimity networks.
Critical strengths
-
Efficient interaction estimation is an important problem. The number of feature subsets grows quickly, making exact computation difficult when many features are involved.
-
The paper develops the connection between discrete interactions and derivative integrals systematically, with clear definitions, explicit assumptions, and supporting proofs.
Critical weaknesses
-
Average MSE may hide errors on important interactions in the unanimity dimension-scaling experiment, where many interactions are zero. A normalized error and an all-zero baseline would make the results easier to interpret. Global Spearman correlation also does not directly show whether the strongest interactions are recovered.
-
A baseline forward pass and a CRIQ backward pass are both counted as one query, but they have different costs. Gradient evaluation generally needs more computation and activation storage. Without runtime and memory results, it is hard to assess CRIQ's practical efficiency.
-
MLP and DistilBERT are tested with at most 12 input features, while the 30-feature experiment uses a simple synthetic network. These results provide limited evidence for how the method scales to more input features in more complex networks.
Questions for authors
-
It is helpful to report normalized error and an all-zero baseline for the unanimity dimension-scaling experiment. Top-n recovery by absolute interaction magnitude would also show whether the strongest interactions are identified.
-
Please compare runtime and peak memory under the same hardware and comparable batching settings. Specify whether one CRIQ query includes the forward computation required to obtain the gradient.
2 comments
Thank the authors for the detailed replies and additional experiments. They largely address my concerns, so I will retain my positive rating. Good luck.
We would like to thank the reviewer for their positive assessment and constructive feedback. In our revised manuscript, we upgraded our core estimator from the original 2-point estimator to an amortized 1-point quadrature estimator to enable single-backward-pass gradient reuse across all features, and added the requested metrics, runtime/memory profiles, and higher-dimensional benchmarks.
W1: Average MSE may hide errors on important interactions in the unanimity dimension-scaling experiment, where many interactions are zero. A normalized error and an all-zero baseline would make the results easier to interpret. Global Spearman correlation also does not directly show whether the strongest interactions are recovered.
Q1: It is helpful to report normalized error and an all-zero baseline for the unanimity dimension-scaling experiment. Top-n recovery by absolute interaction magnitude would also show whether the strongest interactions are identified.
Thank you for these helpful suggestions. Following your recommendations, we updated all evaluation plots (Figures 2–3 and Appendix G) as follows:
W2: A baseline forward pass and a CRIQ backward pass are both counted as one query, but they have different costs. Gradient evaluation generally needs more computation and activation storage. Without runtime and memory results, it is hard to assess CRIQ's practical efficiency.
Q2: Please compare runtime and peak memory under the same hardware and comparable batching settings. Specify whether one CRIQ query includes the forward computation required to obtain the gradient.
Thank you for highlighting the asymmetry between forward and backward passes and the need to evaluate memory overhead. We confirm that one CRIQ sample/query includes both the forward pass and the backward pass, and we updated our benchmarks (Appendix A.6) to reflect this:
W3: MLP and DistilBERT are tested with at most 12 input features, while the 30-feature experiment uses a simple synthetic network. These results provide limited evidence for how the method scales to more input features in more complex networks.
To provide stronger evidence of quantitative scalability on a non-synthetic deep network with larger feature counts, we added a Long-Sentence NLP Benchmark on
DistilBERTevaluated on 20 real SST-2 sentences of length n=25 (Figures 13~15 in Appendix G for SII). CRIQ avoids the severe initial variance of black-box methods and delivers stable MSE/NMSE and steadily increasing rank-based metrics across runtime.