- Rating
- 3(+1)
- Confidence
- 4
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper proposes DASH for estimating Shapley feature attributions from multi-source data with heterogeneous feature coverage. The method derives the influence function of a constrained WLS Shapley estimator and uses it to construct control variates from partially observed auxiliary sites. It also introduces permutation-based source screening and variance-based calibration weights. Simulations and two real-data experiments show improvements over single-site and imputation-based estimators. I would be willing to reconsider my rating if the authors satisfactorily address these concerns.
Critical strengths
-
The paper addresses a relevant problem in multi-source model interpretation, where different sites observe different subsets of features.
-
The influence-function-based construction provides a reasonable way to incorporate partially observed auxiliary data without imputing missing features.
-
The experimental results are encouraging, particularly the improvements over single-site and imputation-based estimators.
Critical weaknesses
-
The method relies on several estimated quantities, including density ratios, influence functions, and control variates, but the theoretical treatment of their combined estimation errors is limited.
-
The permutation-based screening procedure is an important component of DASH, yet its validity under unequal source distributions and estimated importance weights is not established.
-
The numerical experiments mainly consider low-dimensional settings with moderate feature missingness. It is unclear how stable the proposed method remains when the number of features or missing-data patterns increases.
Questions for authors
1. In Section 3.3 (Page 5), the permutation p-value is defined using the indicator . Since larger values of indicate greater disagreement between sites, should the inequality be reversed? Please clarify whether this is a typo or also affects the implementation.
2. In Algorithm 1 (Appendix B, Step 11), the influence function is computed as , whereas Eq. (10) gives . These expressions are generally different. Which expression is used in the experiments?
3. In Appendix A, the influence-function expansion under covariate shift assumes . This seems demanding for density ratios estimated using a gradient-boosted classifier. Is this rate plausible under the proposed implementation, and how sensitive are the results to density-ratio estimation errors?
4. The calibration weights in Eqs. (17)–(18) are derived under no covariate shift, but the same weights are retained when importance weighting is introduced in Eq. (16). Are these weights still variance-optimal under covariate shift? If not, how much efficiency could be lost?
5. In the NACC application, the missing features are artificially removed from an initially complete dataset. Could the authors clarify how the method would perform with naturally occurring blockwise missingness, where missingness patterns may be associated with site characteristics or patient populations?
Dear Reviewer,
Thank you again for your constructive feedback on our submission.
We would like to kindly follow up on our detailed responses and the revised manuscript, which includes substantial theoretical clarifications and additional experimental evaluations addressing your original concerns.
As the discussion period approaches its conclusion, we would sincerely appreciate your reconsideration of our responses and revisions. If any substantive concerns remain unresolved, we would be grateful for your further feedback.
Thank you very much for your time and consideration.