- Rating
- 4(+1)
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper presents Treatment-Space Graph Selection (TSGS), a framework for improving causal effect estimation in large, sparse combinatorial treatment spaces. Comparison methods smooth outcomes indiscriminately across similar treatments while TSGS instead selects information-sharing structures adaptively, guided by the causal contrasts of interest, so that meaningful treatment interactions are preserved. The framework decouples treatment feasibility from statistical borrowing and comes with theoretical guarantees. Across simulated settings, the author claimed that TSGS consistently outperforms no borrowing.
Critical strengths
The paper addresses an important problem in causal inference with combinatorial treatments, where the exponentially growing treatment space creates substantial data sparsity. The proposed TSGS framework offers an interesting perspective by distinguishing treatment feasibility from statistical information sharing and adapting the borrowing strategy to the causal contrasts of interest.
Critical weaknesses
The empirical analysis in the main paper is somewhat limited. I encourage the authors to provide more detailed insights into why the proposed method works, particularly in comparison with alternative approaches for combining information across treatments. A deeper discussion of the factors driving the observed performance differences would help strengthen the empirical evaluation.
Questions for authors
-
For the real time evaluation, the ACTG 175 experiment provides useful evidence, but more details about the dataset, treatment combinations, and data sparsity would be helpful. Could the authors provide additional real-world evaluations or discuss the practical significance of these gains? Also, any evaluations to confirm the combination structure makes sense?
-
How does TSGS perform when neighboring treatment combinations have substantially different effects or strong interactions? Could the authors include simulations on an ablation study varying treatment-effect heterogeneity to investigate when information borrowing becomes harmful and whether TSGS can adaptively avoid it?
-
The experimental analysis could provide more insight into why competing methods underperform. Are the differences mainly driven by oversmoothing, or treatment sparsity? A more detailed comparison and component-wise ablation would help clarify the sources of improvement.
7 comments
Response to 9KVe — Part 2/7: ACTG treatment structure and clinical contrasts
Q: [Q1 — Question for Authors] What treatments and support does ACTG contain, is the combination structure meaningful, and what practical evidence supports it?
A: The ACTG 175 study supplies four randomized regimens and three directly interpretable one-drug addition comparisons, scored across 100 prespecified held-out splits. The feasibility graph follows observed intervention composition, and the reference-span projection keeps shared treatment structure distinct from borrowable contrasts. Appendix H extends practical evaluation to measured five-component laboratory combinations. The combination structure is justified at three levels: intervention semantics defines valid comparisons, the protected function space states which variation is not directly graph-penalized, and held-out target evaluation tests whether borrowing is useful.
Original — ACTG cohort and geometry (Appendix F.15). The public ACTG 175 analysis comprises 2,139 participants, with week-20 CD4 change as the outcome:
| Assigned regimen | Number of participants |
|---|---|
| AZT | 532 |
| AZT + ddI | 522 |
| AZT + ddC | 524 |
| ddI | 561 |
Figure 11(A) depicts the three valid one-drug additions: AZT→AZT+ddI, AZT→AZT+ddC, and ddI→AZT+ddI. These edges follow treatment composition and have straightforward domain interpretations; their inclusion does not assume the corresponding effects are equal. Four randomized arms do not form a complete Boolean square, so we do not report a synergy requiring unassigned combinations. F.15 documents the data version, baseline feature construction, randomization propensity, graph/penalty choices and independent scoring protocol. With approximately 500 subjects per arm, ACTG is not an extreme-sparsity benchmark; scarce direct support is tested separately by H1 and F.13.
Structural rationale in the original model. The fixed geometry encodes feasible intervention comparisons. The statistical borrowing graph is chosen separately, because a scientifically meaningful comparison need not be suitable for smoothing. The prespecified dictionary gives a transparent regularization preference, not a claim that the outcome surface is confined to that dictionary: retains all complementary variation.
ACTG specifically uses , rather than the full-cube degree-two dictionary. With and a three-dimensional orthonormal contrast basis , the fitted arm-by-feature coefficients satisfy
\operatorname{tr}(H^\top L_GH) =\operatorname{tr}\{\Gamma(U^\top L_GU)\Gamma^\top\}.$$ Since $L_Gb_0=0$, the shared covariate-dependent treatment component is unpenalized by the graph term, while the three-dimensional treatment-contrast complement is eligible for smoothing (F.15). This gives the prior function family a precise role, without stipulating that the biological effects must be smooth. **Original — held-out clinical contrasts (Table 25).** The 100 original stratified ACTG splits evaluate different borrowing rules using the same clinical outcome features and scoring protocol. Table 25 reports: | Clinical contrast family | NoGraph | PredictiveCV | TSGS | TSGS wins among splits | |---|---:|---:|---:|---:| | All local additions | 110,487 | 109,171 | 108,987 | 86% | | Add ddI to AZT | 112,764 | 112,354 | 111,961 | 70% | | Add ddC to AZT | 89,899 | 88,535 | 88,479 | 89% | | Add AZT to ddI | 128,798 | 126,626 | 126,627 | 76% | | Combination vs. monotherapy | 54,674 | 53,973 | 53,922 | 79% | These are held-out conditional-contrast scores, not clinical-effect SEs. [Figure 11(B,D)](https://anonymous.4open.science/r/treatment-figures-review-630D/figure_11.png) connects target-dependent selected rules with held-out score differences, and Figure 11(C) shows the randomized clinical contrasts. PredictiveCV closely matches TSGS on individual targets. This directly supports target-conditioned statistical borrowing on the observed regimen geometry; the scientific structure is specified independently of outcome fitting.Response to 9KVe — Part 3/7: Additional real-world evaluations
A (continued — Q1):
Added — measured combinations (Appendix H.5, Table 33). We analyzed the published in-vitro combination-growth measurements of Lozano-Huntelman et al. (2021): 56 five-component panels, 243 dose contexts per panel, three technical repetitions, and 31 measured nonempty combination nodes. The observed structure contains 75 activation edges, 70 complete squares, and five four-to-five-component terminal additions. The degree-two dictionary has rank 16 and a rank-15 complement. This additional application uses the projected-sequence/linear TSGS specialization on observed laboratory growth measurements. Train/validation/test roles retain physical-well identity to avoid duplicate-well leakage.
The prespecified terminal decision is whether to add the fifth component. The actual held-out measured-growth results are:
| Method | Terminal contrast MSE ↓ | Local growth reduction (pp) ↑ |
|---|---|---|
| Raw NoGraph | 0.117534 | 12.672 |
| Target-tuned complement ridge | 0.109780 | 12.565 |
| Predictive-tuned complement ridge | 0.110156 | 12.542 |
| LowOrder | 0.121508 | 11.755 |
| FullVal | 0.109546 | 12.578 |
| PredictiveCV | 0.109884 | 12.534 |
| TSGS | 0.108628 | 12.612 |
| Same-density RandomGraphVal | 0.109580 | 12.586 |
| TotalSurfaceVal | 0.106814 | 12.703 |
TSGS's local-gain increment is pp over FullVal (exploratory date-cluster interval ) and pp over PredictiveCV (). Raw NoGraph and TotalSurfaceVal achieve higher mean local gains. All-edge MSE is nearly tied between TSGS and PredictiveCV. The increments over tuned controls are therefore small; these are measured laboratory growth outcomes with repeated technical measurements rather than patient clinical endpoints.
Added — ACTG decisions (Appendix H.4). A separate out-of-fold policy check estimates a local CD4 gain of 43.144 cells/mm$^3$ for TSGS, against 42.861 for FullVal and 42.697 for PredictiveCV; a training-mean fixed policy obtains 46.891. The values are out-of-fold AIPW policy estimates under a separate protocol, with the fixed-policy comparator retained.
Revision. The revised §2–§3 clarifies scientific edges and the protected function space; the original F.15 remains the primary ACTG protocol. Appendix H adds the distinct measured-combination and policy checks, including stronger comparators and their limits. Scientific plausibility of a comparison and statistical usefulness of borrowing are kept separate.
Response to 9KVe — Part 1/7: External comparisons and selective borrowing
Our response is split into seven comments because of the character limit.
Q: [W1 — Critical Weakness] What explains the gains over other cross-treatment sharing methods?
A: TSGS beats all five original external baselines on both edge and square PEHE in the rough R2 benchmark. The original matched FullVal and target-alignment experiments show why: selecting where to borrow and which contrast to validate changes risk beyond simply tuning graph-penalty strength. Theorem 4.1 exposes the target-specific bias–variance cost of the chosen graph; Appendix H adds controlled edge-placement and penalty-location comparisons. The explanation is strongest where compatible and incompatible treatment relations coexist.
Original — comparisons with treatment-sharing methods (Tables 1 and 4). R1 and R2 use six binary components and 64 bundles. R1 has relatively smooth complementary variation; R2 introduces localized rough transitions, with 12 jump edges among 192 one-component geometry edges. This tests whether sharing can remain useful while preserving difficult local contrasts.
The first five baselines and TSGS were in the original submission; POUL is an Appendix H.2 adaptation added for this discussion. Results are edge/square PEHE, mean ± seed-level SE over seeds 201–210 (lower is better). We show R2 first because it tests selective borrowing across locally incompatible treatment relations:
R2 — localized rough transitions (primary advantage).
R1 — smooth complementary variation (cross-regime check).
Appendix F.2 documents the original implementations, including SIN-vector, the binary-vector adaptation of the generalized Robinson method in Causal Effect Inference for Structured Treatments.
Added — extended comparison (Appendix H.2). The POUL adaptation retains joint component/off–on embeddings, a nonlinear map after aggregation, a frozen nuisance in the orthogonalized stage and exact covariate-dependent centering under known assignment probabilities. Its architecture and training protocol were fixed before evaluation on the original generators: 3,000/3,000 training/validation observations, 5,000 evaluation covariates and seeds 201–210, followed by a separate two-stage refit.
In R2, TSGS lowers mean edge and square PEHE by 55.3% and 64.5% relative to this adaptation; the within-seed POUL-minus-TSGS edge-PEHE difference is +0.407205±0.005259, favoring TSGS in all ten seeds. POUL is better in smooth R1. The matched ablations below test the design choices directly.
Revision. Section 5.1 retains the original rough-regime and matched FullVal comparisons; Appendix H.2–H.3 supplies the POUL results and matched mechanism controls. The causal estimands and TSGS algorithm remain those of the submitted manuscript.