acceptodds
Under review as a conference paper at ICLR 2027

When More Outcomes Stop Helping: The Cost of Interference in Network Experiments

Abstract

Network experiments can assign treatment broadly while measuring only outcomes. Under interference, these measurements need not yield mutually compatible causal observations. We formulate this constraint as a budgeted experiment on a labeled exposure-conflict graph. Including the analysis support in the design record preserves the physical experiment's minimax risk. For Direct and Global effects with bounded second moments on bounded-degeneracy networks, we establish the common decomposition . It separates the cost of outcome measurement from a contrast-specific, topology-dependent risk floor. We develop Design-Supported Conflict-Graph Design (DS-CGD): an offline search builds a finite library of compatible supports, and convex mixture optimization allocates probability using pretreatment residual scores. The resulting exact inclusion probabilities yield a design-unbiased generalized-difference estimator. Across evaluated conditions, geometric-mean Direct-effect RMSE is 12% lower than the best matched baseline in each condition on synthetic graphs, and 16% and 22% lower on the two observed network datasets (CAI and ca-GrQc), respectively. The corresponding Global-effect reductions are 40%, 4%, and 11%.

Primary area:
causal reasoning
Keywords:
Causality

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_Yo3rOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper studies design-based minimax risk for network experiments under one-hop interference and a hard budget on outcome measurements. It uses an exposure-conflict graph to represent jointly feasible observations and gives matching rates on bounded-degeneracy networks, separating measurement cost from contrast-specific network difficulty. The theory distinguishes unrestricted from exactly design-unbiased Direct-effect estimation and extends to growing degeneracy. DS-CGD uses a finite library of compatible supports, an optimized mixture, and exact inclusion probabilities for design-unbiased generalized-difference estimation.

Critical strengths
  1. The theory gives a useful and precise characterization of the measurement budget. Matching bounds for the measurement and network terms explain when further measurements can improve risk only within a constant factor. The distinction between unrestricted and exactly unbiased Direct-effect estimation also reveals the statistical cost of requiring unbiasedness.

  2. The practical design connects feasibility and valid estimation clearly. Compatible supports ensure that the selected exposures can occur together. Conditional on the frozen predictions and design, positive inclusion probabilities and residual correction give design unbiasedness. The authors also explicitly distinguish the finite-library marginal variance objective from a minimax risk guarantee.

Critical weaknesses

My main reservation concerns the historical information used in the practical evaluation. Appendix C.3 uses 200 historical rounds for every node and target exposure to construct predictions and residual scores. Performance with much less history or less accurate residual scores is not established by the reported experiments. The matched methods share this information, so this concerns empirical scope rather than comparison fairness or design unbiasedness.

Questions for authors

Are results with fewer historical rounds or coarser residual scores already available? If so, a brief summary would help clarify how much historical information is needed for the reported gains.

6 comments
AuthorsOpenReview ↗

Response to Reviewer Yo3r (1/4): Historical sample size and Direct effects

Rebuttal to Reviewer Yo3r

Thank you for your careful review and for asking how the results depend on historical information.

Q1: Performance with much less history or less accurate residual scores is not established by the reported experiments.

We now report R = 5, 20, 50, 100, 200. Historical data help us predict outcomes and estimate allocation scores. Better predictions reduce residual variation. The scores guide measurement toward target exposures with larger expected squared residuals.

Let R count historical observations per node–exposure event. Fix a randomization design independently of the historical and current outcome noise. For event v, write the current outcome as Yv=μv+εvY_v=\mu_v+\varepsilon_v and historical outcome r as Yv,rhist=μv+ηv,rY^{\mathrm{hist}}_{v,r}=\mu_v+\eta_{v,r}. Assume centered noise, independent and identically distributed historical rounds, and independence between historical and current noise. Using the historical mean as the predictor gives m^v=μv+R−1∑rηv,r\widehat m_v=\mu_v+R^{-1}\sum_r\eta_{v,r}.

Let IvI_v indicate inclusion of event v, with probability pv>0p_v>0. Write sv=+1s_v=+1 for the treatment term and sv=−1s_v=-1 for the control term, and set cv=sv(Iv/pv−1)/nc_v=s_v(I_v/p_v-1)/n. The regression-adjusted estimator then satisfies

\widehat\tau_R-\tau(Y) &=\sum_v c_v(Y_v-\widehat m_v)\\\\ &=\sum_v c_v\varepsilon_v -\frac1R\sum_{r=1}^{R}\sum_v c_v\eta_{v,r}. \end{aligned}$$ Here, $\tau(Y)$ is the current finite-population treatment effect. Conditional on the randomization, the cross terms have zero expectation because the noise is centered and independent across rounds and between history and current outcomes. Thus, averaging over outcomes, history, and randomization, $$\begin{aligned} \mathbb E[(\widehat\tau_R-\tau(Y))^2] &=\mathbb E\\!\left[\left(\sum_v c_v\varepsilon_v\right)^2\right] +\frac1{R^2}\sum_{r=1}^{R} \mathbb E\\!\left[\left(\sum_v c_v\eta_{v,r}\right)^2\right]\\\\ &=V_{\mathrm{current}}+\frac{R}{R^2}V_{\mathrm{history}}\\\\ &=V_{\mathrm{current}}+\frac{V_{\mathrm{history}}}{R}. \end{aligned}$$ $V_{\mathrm{current}}$ is the first expectation: the variance contribution of current outcome noise. $V_{\mathrm{history}}$ is the corresponding contribution of one historical round. More history reduces the historical contribution at rate `1/R`. Once it is small relative to current outcome noise, further history brings little improvement. DS-CGD also updates its predictor and allocation with R, so its RMSE can fluctuate across R. ER and PA networks, n = 750 and 2,000, B = 0.2n, degree-aligned outcomes. Methods share historical inputs and current potential outcomes. Tables report mean RMSE. **Direct effects, n = 750, B = 150. Mean RMSE.** | Network | Method | R = 5 | R = 20 | R = 50 | R = 100 | R = 200 | | :--- | :--- | ---: | ---: | ---: | ---: | ---: | | **ER** | **DS-CGD** | **0.2421** | **0.2404** | **0.2467** | **0.2476** | **0.2466** | | ER | Generalized CGD | 0.3443 | 0.3037 | 0.2917 | 0.2873 | 0.2852 | | ER | Original CGD | 0.3950 | 0.3477 | 0.3315 | 0.3265 | 0.3238 | | ER | Independent set | 0.1924 | 0.1840 | 0.1816 | 0.1807 | 0.1803 | | **PA** | **DS-CGD** | **0.1758** | **0.1741** | **0.1751** | **0.1755** | **0.1751** | | PA | Generalized CGD | 0.3849 | 0.3771 | 0.3752 | 0.3743 | 0.3739 | | PA | Original CGD | 0.4313 | 0.4230 | 0.4212 | 0.4204 | 0.4199 | | PA | Independent set | 0.2992 | 0.2689 | 0.2613 | 0.2584 | 0.2567 | **Direct effects, n = 2,000, B = 400. Mean RMSE.** | Network | Method | R = 5 | R = 20 | R = 50 | R = 100 | R = 200 | | :--- | :--- | ---: | ---: | ---: | ---: | ---: | | **ER** | **DS-CGD** | **0.1045** | **0.0930** | **0.0944** | **0.0937** | **0.0940** | | ER | Generalized CGD | 0.1721 | 0.1713 | 0.1713 | 0.1712 | 0.1713 | | ER | Original CGD | 0.1963 | 0.1957 | 0.1954 | 0.1953 | 0.1953 | | ER | Independent set | 0.1092 | 0.1089 | 0.1088 | 0.1088 | 0.1088 | | **PA** | **DS-CGD** | **0.1047** | **0.1028** | **0.1007** | **0.1010** | **0.1013** | | PA | Generalized CGD | 0.2314 | 0.2306 | 0.2305 | 0.2304 | 0.2304 | | PA | Original CGD | 0.2602 | 0.2593 | 0.2591 | 0.2590 | 0.2590 | | PA | Independent set | 0.1614 | 0.1579 | 0.1570 | 0.1566 | 0.1564 |
AuthorsOpenReview ↗

Response to Reviewer Yo3r (2/4): Historical sample size and Global effects

Global effects, n = 750, B = 150. Mean RMSE.

NetworkMethodR = 5R = 20R = 50R = 100R = 200
ERDS-CGD0.23560.23010.23040.23090.2312
ERGeneralized CGD0.67970.60670.58480.57690.5731
EROriginal CGD0.98350.86750.83200.81900.8126
ERCC0.36490.34680.34120.33900.3377
ERRGCR0.57250.56090.55870.55750.5568
PADS-CGD0.14350.14130.14520.14260.1440
PAGeneralized CGD0.93020.91420.91030.91470.9169
PAOriginal CGD1.32201.29811.28841.29011.2889
PACC0.38950.36490.35800.35510.3533
PARGCR1.13921.02500.99820.98870.9836

Global effects, n = 2,000, B = 400. Mean RMSE.

NetworkMethodR = 5R = 20R = 50R = 100R = 200
ERDS-CGD0.08270.08060.08040.07990.0799
ERGeneralized CGD0.35170.35060.35050.35050.3502
EROriginal CGD0.49730.49620.49580.49580.4955
ERCC0.14040.14060.14080.14100.1411
ERRGCR0.33860.33820.33820.33810.3381
PADS-CGD0.09710.09620.09430.09340.0937
PAGeneralized CGD0.60640.60470.60460.60450.6042
PAOriginal CGD0.85430.85090.85070.85030.8505
PACC0.20530.20210.20140.20110.2010
PARGCR0.60560.62210.62940.63330.6360

Direct-effect figure · Global-effect figure · Anonymous figure page


AuthorsOpenReview ↗

Response to Reviewer Yo3r (3/4): Residual score sensitivity: rationale and protocol

Q2: Are results with fewer historical rounds or coarser residual scores already available?

Yes. Replacing individual allocation scores with four shared scores per exposure increases mean estimation RMSE by at most 1.9% in our eight settings. Incorrectly assigning scores to other nodes has a larger effect in the covariate-predictable settings. The results for fewer historical rounds are above.

The question is whether DS-CGD needs an accurate score for every node. We test this by deliberately removing detail from the scores. Within each exposure, we sort nodes by their original scores and divide them into 8, 4, 2, or 1 groups of approximately equal size. Every node in a group receives that group's mean score. With two groups, the design sees only a lower-score group and a higher-score group. With one group, it receives no score differences between nodes in that exposure. We then rerun DS-CGD and measure the RMSE of the treatment-effect estimate. This tests whether rough measurement priorities are enough.

We separately test inaccurate priorities by randomly reassigning 25%, 50%, or 100% of scores to other nodes within the same exposure. The score values stay the same, but they no longer describe the same nodes. This tests what happens when the design gives measurement priority to the wrong events.

For event v=(i,a)v=(i,a), the score estimates how much outcome variation remains after prediction. Write m^v\widehat m_v for the prediction and Yv,rhistY^{\mathrm{hist}}_{v,r} for historical outcome r. Before the numerical floor and normalization,

\widehat a_v^0&=\frac1R\sum_{r=1}^{R}(Y^{\mathrm{hist}}_{v,r}-\widehat m_v)^2,\\\\ \mathbb E[\widehat a_v^0\mid\widehat m_v] & =\underbrace{\sigma_v^2}_{\text{outcome noise}} +\underbrace{(\mu_v-\widehat m_v)^2}_{\text{prediction error}} =a_v. \end{aligned}$$ Here, $\mu_v$ and $\sigma_v^2$ are the outcome mean and variance. The expectation uses the independence of an event's historical outcomes from the data used to fit its prediction. After the design is fixed, let $Y_v$ be the current outcome, $I_v$ indicate inclusion, and $p_v$ be its probability. The residual correction for this event contributes $$\frac{1}{n^2}\mathbb E\\!\left[ \left(\frac{I_v}{p_v}-1\right)^2(Y_v-\widehat m_v)^2 \right] =\frac{a_v}{n^2}\left(\frac1{p_v}-1\right)$$ to the diagonal variance term. Thus, an inaccurate score matters when it gives too little measurement probability to an event with a large residual. A coarse score can still preserve useful priorities. Assigning the scores to the wrong nodes can lose those priorities. We evaluate the complete estimator's RMSE, which also includes covariance between events. We use ER and PA networks with $n=750$, $B=150$, and $R=200$. Predictions and current outcomes are shared across score variants. For each variant, we rerun both candidate selection and weight optimization. The tables report mean estimation RMSE.
Reviewer_xaUXOpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
1
Soundness
3
Read review
Paper summary

The paper considers experimental design for networks experiments when subject to a measurement budget B<<nB << n. That is, in a treatment assignment mechanism that minimizes risk when the the nn units in the study are subject to arbitrary neighborhood interference and only B<<nB << n outcomes can be measured.

In this setting, the paper develops a precise minimax characterization of optimal risk for different causal effects (direct effect, direct effect under restriction that estimator is unbiased, and global effect). They then develop an estimator which allocates measurement budget and treatment assignments to minimize the risk, though the estimator is not guaranteed to match the minimax bound. Experiments compare the RMSE of the developed method with baselines from the network interference literature.

Critical strengths

The paper identifies an interesting setting: the intersection of budget constraints and arbitrary neighborhood interference. The theoretical results seem substantive. The algorithm DS-CGD is interesting and the experiments suggest that in some settings the developed algorithm outperforms state-of-the-art estimators for network interference experiments.

Critical weaknesses

The most significant weakness of the paper is its poor presentation. Many terms are presented without definition (at least initially), sentences are often hard to parse, and the significance of results is not well-explained. The literature review (esp. the 2nd paragraph) does not distinguish the work from the literature (e.g., why is Zhang et al. 2025 different? Is it that Zhang et al. choose which units to reveal sequentially/adaptively?). Some concrete examples (meant to illustrate point):

  • (Lines 016-017) I am not sure what this sentence is supposed to mean: “Including the analysis support in the design record preserves the physical experiment’s minimax risk.”

  • (Line 019) the terms GG, MM, kk, and Ψc(G)\Psi_c(G) are all written but not defined. Later definitions are supplied, MM is a scale related to finite-population boundedness, kk captures the degeneracy of the graph GG, and Ψc(G)\Psi_c(G) is a functional; these should be explained in the abstract or otherwise the formula should not be displayed.

  • (Line 022) ‘Pretreatment residual score’ is referenced in the abstract. It is unclear what this means. It is also referred to in Sections 5 and 6, where again it is not clear what it means.

  • (Line 033-034) The introduction should not rely on the exposition of a related work. The weather-insurance information experiment should be explained better in the main text if it is to be referenced and motivate this work.

  • In Equation (1), M and k are again referenced without definition.

Continuing this point, the setting and results in Sections 5 and 6 are very difficult to understand. How are S1,…,SKS_1,\dots,S_K chosen? Are they given? What is the ‘frozen pretreatment score’ a^v\hat{a}_v? How is it calculated? What is the reference design that we regularize with respect to? In part because I was unable to understand Section 5, I cannot also understand the setting in which the current estimator is compared to the baselines. One immediate question: if conflict graph design and other estimators are budget-unaware, then what implementation did you use to compare fairly to a budget-aware algorithm?

With respect to the set-up and results of Sections 3 and 4, I am struggling to understand (i) the full intuition about the results and (ii) why do they differ from the minimax construction of Kandiros et al. 2026? What from among the results is substantive? What was hard to prove? Why? In particular, is it non-trivial to show that the risk characterization decomposes into budget-aware component and network-effect component? Separation in the lower bound seems to follow immediately, is separation in the upper bound hard? Furthermore, if the budget constraint component is ultimately separable from the cost of conflict pattern, how is budget constraint related to the subsequent analysis? If it is not, then why is the analysis different than the minimax characterization Kandiros et al. prove for the unrestricted outcome observation? If it is implied by the results this should be stated. Otherwise, it seems that the paper is actually saying something interesting about minimax risk beyond just the budget-aware setting. This should be highlighted.

Decision: Clear reject. Reason: Poor presentation means manuscript needs significant revisions to articulate why their results contribute significantly to our understanding of causal inference under network interference with budget constraints.

Questions for authors

See above. To highlight a few:

  • What is the intuition behind the results? Which results follow immediately from prior work, and which represent significant contributions? For example, Proposition 1 seems straightforward. Is Theorem 1 more involved? Which parts? What was the key analysis?

  • Is characterization of minimax risk reducible separable to budget constraint component and characterization of risk without budget constraint? If so, how do results of literature inform Theorem 1? Or, how does calculation of U,Φ,ΘU, \Phi, \Theta inform broader literature on no-budget network experiments?

  • What is precise setting of Section 5? How do we calculate compatible-support library? How do we calculate pretreatment score? What is the intuition behind the algorithm? Why is it better than alternatives?

  • Re. Section 6: if conflict graph design and other estimators are budget-unaware, then what implementation did you use to compare fairly to a budget-aware algorithm?

14 comments
AuthorsOpenReview ↗

Response to Reviewer xaUX (1/6): Minimax guarantees and presentation

Thank you for the careful review and for identifying where the exposition needs clarification.

Q1: though the estimator is not guaranteed to match the minimax bound.

Re: The estimators constructed for Theorem 1 match the minimax rate under its stated assumptions. However ,the practical DS-CGD estimator is not guaranteed to match that rate.

Q2: The most significant weakness of the paper is its poor presentation. Many terms are presented without definition (at least initially), sentences are often hard to parse, …

  • (Lines 016-017) I am not sure what this sentence is supposed to mean: “Including the analysis support in the design record preserves the physical experiment’s minimax risk.”
  • (Line 019) the terms GG, MM, kk, and Ψc(G)\Psi_c(G) are all written but not defined. Later definitions are supplied, MM is a scale related to finite-population boundedness, kk captures the degeneracy of the graph GG, and Ψc(G)\Psi_c(G) is a functional; these should be explained in the abstract or otherwise the formula should not be displayed.
  • (Line 022) ‘Pretreatment residual score’ is referenced in the abstract. It is unclear what this means. It is also referred to in Sections 5 and 6, where again it is not clear what it means.
  • (Line 033-034) The introduction should not rely on the exposition of a related work. The weather-insurance information experiment should be explained better in the main text if it is to be referenced and motivate this work.
  • In Equation (1), MM and kk are again referenced without definition.

Re: The revised abstract states that the graph representation and physical experiment have the same minimax risk. Proposition 1 establishes this equivalence between physical assignments and measurements and compatible target exposures.

The abstract now describes the risk decomposition in words and defines the population size and measurement budget at first use. For Equation (1), GG is the interference network. MM bounds the outcome moment in each target arm. The degeneracy bound kk means that every induced subgraph contains a vertex of degree at most kk. Finally, Ψc(G)\Psi_c(G) is the network risk term for contrast and estimator class cc.

Section 5 now introduces the score as a historical estimate of squared prediction error; Q6 gives its calculation. The Introduction explains the field motivation directly. Information interventions can affect connected households, while post-season outcome revisits are costly. Treatment assignment and the choice of households to revisit therefore require joint design.

AuthorsOpenReview ↗

Response to Reviewer xaUX (2/6): Significance, related work, and experimental setting

Q3: the significance of results is not well-explained.

Re: Our results characterize the measurement requirement for a causal estimand: how many outcomes suffice to attain the optimal precision available under full measurement, and what limits that precision. This connects the cost of outcome collection to the information needed for causal estimation.

The distinction matters under interference. Treatment assignment determines which potential outcomes are realized; measurement reveals those outcomes. More measurements provide more information about the realized assignment. Yet the causal contrast also involves potential outcomes under other exposures. Conflicting exposure requirements restrict which of these outcomes can be observed together. Thus, the value of an additional measurement depends on the target contrast and its exposure requirements.

A hard budget introduces a further difficulty. Retaining fewer target outcomes lowers their inclusion probabilities and increases inverse probability weights. This can amplify variance from rare and dependent exposures. Two sources of difficulty do not automatically imply two additive costs. Even after optimizing assignment and measurement jointly, whether the budget amplifies the network penalty remains a substantive question.

Theorem 1 resolves this question under bounded second moments and a fixed degeneracy bound kk:

RB,c(G;M)≍kM2B−1+Ψc(G).R_{B,c}(G;M)\asymp_k M^2\\{B^{-1}+\Psi_c(G)\\}.

Here cc specifies the causal contrast and estimator class; comparison constants depend on kk. A feasible design attains both terms together. The result establishes additive costs despite coupled design constraints: measurement scarcity contributes M2/BM^2/B, while exposure conflicts determine the remaining risk order.

This has a consequential implication: a smaller sufficient budget can reflect a harder causal problem. When the network term is large, fewer measurements suffice to reach its risk order, but the attainable precision is poor. Measurement sufficiency and estimation accuracy therefore answer different questions. The causal estimand and the estimator's bias requirements determine both, even on the same network.

For experimental planning, the results distinguish when more outcome collection improves the risk order from when progress requires changing the experiment or the assumptions on potential outcomes. They provide a benchmark for evaluating repeated randomization or additional outcome structure. The broader contribution is a way to ask which resource supplies the causal information needed for a specified precision target.

Q4: The literature review (esp. the 2nd paragraph) does not distinguish the work from the literature (e.g., why is Zhang et al. 2025 different? Is it that Zhang et al. choose which units to reveal sequentially/adaptively?).

Re: The distinction concerns information timing, the optimized design, and the statistical target. Zhang et al.'s CGAS uses a selected sample to fit a regression and applies CGD to a sampled complement; see Section 6 and Algorithm 3.

DS-CGD uses historical data to specify predictions and optimize a distribution over compatible exposure sets before current outcomes are collected. Exact inclusion probabilities support a regression-adjusted estimator with conditional design unbiasedness. Our theory characterizes minimax risk over all admissible designs and estimators under a hard measurement budget. Appendix A details this comparison.

Q5: Continuing this point, the setting and results in Sections 5 and 6 are very difficult to understand.

Re: Section 5 jointly designs treatment assignment and outcome measurement. Section 6 compares the resulting effect estimates at a common budget. The workflow gives the sequence: construct compatible observation plans, optimize their probabilities, draw one plan, and implement its assignment and measurements. Regression adjustment then combines pretreatment predictions with residuals weighted by inverse inclusion probabilities. Section 5 now introduces this sequence before the formulas.

AuthorsOpenReview ↗

Response to Reviewer xaUX (3/6): Algorithm specification, baseline fairness, and intuition

Q6: How are S1,…,SKS_1,\ldots,S_K chosen? Are they given? What is the ‘frozen pretreatment score’ a^v\widehat a_v? How is it calculated? What is the reference design that we regularize with respect to?

Re: SkS_k is an observation plan containing compatible unit–exposure pairs. Each pair (i,a)(i,a) requests unit ii's outcome under exposure ei,ae_{i,a}. The assignment requirements agree, and ∣Sk∣≤B|S_k|\le B satisfies the budget. Algorithm 1 constructs the library from allocation-first proposals, topology-first compatible sets, and blocks covering every target event.

Let Yˉiahist\bar Y_{ia}^{\rm hist} and Sia,hist2S_{ia,\rm hist}^2 denote the mean and variance from RR historical rounds. With cross-fitted prediction m^ia\widehat m_{ia}, the allocation score is

a^ia=max⁡10−8,(1−R−1)Sia,hist2+(Yˉiahist−m^ia)2,\widehat a_{ia}=\max\\{10^{-8},(1-R^{-1})S_{ia,\rm hist}^2+(\bar Y_{ia}^{\rm hist}-\widehat m_{ia})^2\\},

followed by division by the mean score. Historical variation and prediction error estimate residual uncertainty. All inputs are specified before current randomization.

Let S(v)S(v) be the coverage block containing vv, and Avk=1v∈SkA_{vk}=\mathbf1\\{v\in S_k\\}. The reference weights are wkref=∣v:S(v)=Sk∣/(2n)w_k^{\rm ref}=|\\{v:S(v)=S_k\\}|/(2n). Constraints Aw≥ϵAwref>0Aw\ge\epsilon Aw^{\rm ref}>0, with 0<ϵ≤10<\epsilon\le1, preserve a fraction of this coverage. Regularization stabilizes the weights around the reference. Final inclusion probabilities are p=Aw^p=A\widehat w.

Q7: In part because I was unable to understand Section 5, I cannot also understand the setting in which the current estimator is compared to the baselines. One immediate question: if conflict graph design and other estimators are budget-unaware, then what implementation did you use to compare fairly to a budget-aware algorithm?

Re. Section 6: if conflict graph design and other estimators are budget-unaware, then what implementation did you use to compare fairly to a budget-aware algorithm?

Re: Every method measures BB distinct units and uses the same regression-adjusted estimator within each matched comparison. Methods share the network, potential outcomes, estimand, covariates, and predictions. Appendix C.3 specifies these adaptations:

MethodOutcome-measurement rule
Generalized / Original CGDGenerate the method's native support. Uniformly retain BB target events if oversized; otherwise fill unused measurement slots.
Independent-set designUniformly reduce the focal independent set to BB when necessary; fill unused measurement slots.
CCRetain spectral clustering and cluster assignment. Select BB units before assignment and use their realized target events.
RGCRRetain randomized three-net clustering and cluster assignment. Sample BB units without replacement and use their realized target events.
DS-CGDDraw a compatible set from the optimized mixture; fill remaining measurement slots.

Selected target events enter residual correction; additional measurements complete the physical budget. Weights account for each adapted observation law. DS-CGD uses exact p=Aw^p=A\widehat w; baselines use their documented analytic or numerical calculations. The archive retains numerical MSE uncertainty.

Q8: With respect to the set-up and results of Sections 3 and 4, I am struggling to understand (i) the full intuition about the results …

Re: The number of measured outcomes and the information about the causal contrast are different quantities. Two target exposures may require opposite treatments for a shared neighbor. A single assignment realizes at most one of them. Measuring additional units reveals outcomes under that same assignment. Under arbitrary neighborhood interference, these observations remain compatible with different values of the unrealized target potential outcome.

This explains why network risk can persist under full measurement. A limited budget further restricts how many realized target outcomes are observed. Sections 3–4 characterize the precision achievable under both restrictions.

Reviewer_ZDHNOpenReview ↗
Rating
3
Confidence
4
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper investigates causal effect estimation in network experiments under interference when only a limited number of outcomes can be measured. The authors formulate the problem as a budget-constrained experimental design using a labeled exposure-conflict graph and establish a minimax risk decomposition that separates the effect of outcome measurement budget from a network-induced risk floor.

They establish matching minimax upper and lower bounds for Direct and Global effects, characterizing how estimation risk depends on the measurement budget and network structure. The theoretical results identify a network-induced risk floor and characterize regimes in which increasing the number of measured outcomes no longer improves estimation accuracy. The authors propose Design-Supported Conflict-Graph Design (DS-CGD), which optimizes the selection of compatible exposure sets and their assignment probabilities using pretreatment information, enabling design-unbiased estimation. The proposed method is evaluated on synthetic and real-world networks, demonstrating improvements in estimation accuracy over baseline approaches for both Direct and Global treatment effects.

Critical strengths
  1. The paper addresses an important problem in causal inference under network interference by considering experimental settings where outcome measurements are costly or constrained by a limited budget. Incorporating outcome measurement constraints into network experimental design provides a meaningful extension to existing research on causal inference under interference.
  2. The paper establishes minimax upper and lower bounds for estimating causal effects under network interference, providing theoretical insights into how estimation risk depends on the outcome measurement budget and network structure.
  3. The proposed DS-CGD method integrates treatment assignment and outcome measurement into a unified design framework. By leveraging exposure-conflict graphs and optimizing sampling probabilities using pretreatment information, the method offers a principled approach to improving estimation efficiency.
Critical weaknesses
  1. Potential selection bias induced by the joint optimization of sample selection and treatment allocation. In the proposed DS-CGD framework, the selection of outcome measurement samples is closely entangled with the optimization of treatment allocation. This raises concerns about whether the optimization procedure may introduce systematic selection bias by favoring certain units or exposure configurations.
Questions for authors
  1. What is the computational cost of the proposed DS-CGD framework, particularly for large-scale networks? Since the method involves searching for compatible exposure supports followed by convex optimization of allocation, it would be helpful for the authors to provide results or analysis for empirical runtime comparisons.

  2. How does the proposed DS-CGD framework account for high-degree nodes during the outcome sample selection procedure? Since high-degree nodes have more neighbors and potentially more exposure conflicts, they may be less likely to be included in compatible exposure supports. It would be helpful for the authors to clarify how node degree influences inclusion probabilities and whether the proposed optimization systematically favors low-degree nodes.

4 comments
AuthorsOpenReview ↗

Response to Reviewer ZDHN (1/4): Selection bias and design unbiasedness

Rebuttal to Reviewer ZDHN

Thank you for your careful review and for the questions about selection bias, computational cost, and high-degree nodes.

Q1: Potential selection bias induced by the joint optimization of sample selection and treatment allocation.

Joint optimization preserves design unbiasedness. We fix the predictions and design before observing current outcomes. The estimator uses each event's probability of having its target exposure realized and its outcome selected for analysis.

For event v=(i,a)v=(i,a), let yvy_v be node i's potential outcome under target exposure a, and m^v\widehat m_v its historical prediction. Write sv=+1s_v=+1 for the treatment term and sv=−1s_v=-1 for the control term. Then τ(y)=n−1∑vsvyv\tau(y)=n^{-1}\sum_v s_vy_v and τ(m^)=n−1∑vsvm^v\tau(\widehat m)=n^{-1}\sum_v s_v\widehat m_v. Let IvI_v indicate that event v enters the analysis, with its target exposure realized and outcome measured. Its probability under the implemented design is pv>0p_v>0. Conditional on the fixed potential outcomes and the frozen design,

\widehat\tau &=\tau(\widehat m)+\frac1n\sum_v\frac{I_v}{p_v}s_v(y_v-\widehat m_v),\\\\ \mathbb E_D[\widehat\tau] &=\tau(\widehat m)+\frac1n\sum_v\frac{\mathbb E_D[I_v]}{p_v}s_v(y_v-\widehat m_v)\\\\ &=\tau(\widehat m)+\frac1n\sum_v s_v(y_v-\widehat m_v) =\tau(y). \end{aligned}$$ Here, $\mathbb E_D$ averages over treatment and measurement randomization. Since $\mathbb E_D[I_v]=p_v$, the correction recovers each residual in expectation. This holds even when treatment and measurement are chosen jointly. Unequal selection changes precision while preserving unbiasedness for the population effect. The following checks compute the exact estimator expectation over each frozen finite mixture. The last column is the largest absolute bias across the checked conditions. | Experiment | Method | Conditions | Largest absolute bias | | :--- | :--- | ---: | ---: | | **Main experiments** | **DS-CGD** | **560** | **$2.19\times10^{-16}$** | | Main experiments | CC | 284 | $1.54\times10^{-15}$ | | **Additional experiments** | **DS-CGD** | **5,800** | **$2.06\times10^{-16}$** | | Additional experiments | CC | 5,160 | $9.21\times10^{-16}$ | | **Large networks** | **DS-CGD** | **3,456** | **$8.67\times10^{-17}$** | | Large networks | CC | 3,456 | $2.65\times10^{-16}$ |
AuthorsOpenReview ↗

Response to Reviewer ZDHN (2/4): Computational cost and large networks

Q2: What is the computational cost of the proposed DS-CGD framework, particularly for large-scale networks?

The runtime experiment uses ER and PA networks, n = 500 and 750, and B = 0.2n. Tables report mean runtime in seconds.

Direct effects. Runtime (s).

MethodER, n = 500ER, n = 750PA, n = 500PA, n = 750
DS-CGD18.85837.77112.17432.778
Generalized CGD1.6172.4251.6092.433
Original CGD2.2492.6921.8972.440
Independent set3.3164.1139.90914.465

Global effects. Runtime (s).

MethodER, n = 500ER, n = 750PA, n = 500PA, n = 750
DS-CGD5.4239.6005.9147.644
Generalized CGD2.0423.1055.1119.318
Original CGD4.7647.2327.17711.968
CC42.473192.39542.595190.844
RGCR175.362110.14767.56475.778

Direct runtime–RMSE figure · Global runtime–RMSE figure

Large networks. n = 50,000; B = 10,000. Median runtime (s).

MethodDirect: ERDirect: PAGlobal: ERGlobal: PA
DS-CGD608.220453.559267.817428.601

DS-CGD optimizes the design offline, before treatment assignment and measurement. This computation is paid once for a fixed design, so runtime is less critical than in an online procedure. The practical tradeoff is computation before the experiment versus estimation precision with a limited measurement budget.

DS-CGD takes longer than both CGD baselines for Direct effects because it searches for compatible sets and optimizes their selection probabilities. In the timed implementation, allocation updates solve linear programs and periodically refine the active weights. CC solves a semidefinite clustering problem; RGCR repeatedly samples clusterings to estimate exposure probabilities. Both take longer than DS-CGD in the Global comparisons.

With predictions and scores supplied, a conservative accounting of the implemented total design time is

Ttotal=O!(Ln[(1+Δ2)log⁡n+K])+Tsolve.T_{\mathrm{total}}=O\\!\left(Ln\bigl[(1+\Delta^2)\log n+K\bigr]\right)+T_{\mathrm{solve}}.

Here n is the number of nodes, Δ\Delta is maximum degree, K is the number of candidate sets, and L counts search attempts and objective evaluations. TsolveT_{\mathrm{solve}} includes all linear programs and corrective optimization. The Δ2\Delta^2 term accounts for identifying and checking conflicts across two steps for Global effects; it reduces to Δ\Delta for Direct effects.

AuthorsOpenReview ↗

Response to Reviewer ZDHN (3/4): Node degree and inclusion probabilities: theory

Q3: It would be helpful for the authors to clarify how node degree influences inclusion probabilities and whether the proposed optimization systematically favors low-degree nodes.

DS-CGD gives high-degree nodes lower treated-exposure inclusion probabilities in our Direct experiments, but nearly equal probabilities in our Global experiments. Does high degree itself force low inclusion? The answer depends on which exposures the hub rules out. Here inclusion means that the target exposure is realized and the outcome enters the analysis.

For Global effects, neighboring treated targets can occur together. With equal allocation scores and B dividing n, DS-CGD's optimization problem has the exact solution

pi,0=pi,1=B2n,i=1,…,n,p_{i,0}=p_{i,1}=\frac{B}{2n},\qquad i=1,\ldots,n,

regardless of node degree. To see why, every feasible design satisfies ∑vpv≤B\sum_v p_v\le B, so ∑v1/pv≥4n2/B\sum_v1/p_v\ge4n^2/B. Our reference distribution over sets of B events attains this bound and has zero regularization cost. These are therefore the optimized probabilities, not just feasible ones. With equal scores, even a hub receives the same inclusion probability.

For Direct effects, treating a hub requires all its neighbors to receive control. Nearby treated targets are then excluded. We can quantify this effect on dense ER networks G(n,ρ)G(n,\rho), where the edge probability 0<ρ<10<\rho<1 is fixed and mean degree is about ρn\rho n. Assume the ratio of the largest to smallest allocation score stays bounded. For the exact DS-CGD optimizer, with probability tending to one,

\asymp \frac1n\min\\!\left\\{B,\frac{\log n}{-\log(1-\rho)}\right\\}.$$ **Proof idea.** Write $\ell=\log n/[-\log(1-\rho)]$. Direct treated targets must form an independent set. In dense ER graphs, the largest independent set has size $O(\ell)$ with high probability. Hence $n\overline p_{\mathrm D}\le\min\\{B,O(\ell)\\}$. For the matching lower bound, our greedy coverage procedure partitions the events into $J=O(n/B+n/\ell)$ compatible sets. Its reference distribution selects each set with probability proportional to its size. Since the optimized objective is no larger than the reference value, the bounded score ratio gives $$\sum_v\frac1{p_v}\le CnJ \quad\Longrightarrow\quad \overline p_{\mathrm D} \ge\frac{n}{\sum_i1/p_{i,1}} \ge\frac1{CJ} \gtrsim\frac{\min\\{B,\ell\\}}n.$$ The first lower bound is Cauchy--Schwarz; C is a constant independent of n and B. Together, the two bounds give the stated order for DS-CGD's optimized probabilities. **Intuition and importance.** `When does a larger budget stop increasing treated inclusion?` If B is much smaller than $\ell$, average Direct inclusion has order B/n: measurement is the bottleneck. If B is much larger than $\ell$, it has order $\ell/n$: the network cannot accommodate more simultaneous Direct treated targets at a larger order. In particular, with B = 0.2n and fixed density, average Direct inclusion has order $\log n/n$, whereas the equal-score Global probability stays at 0.1. This identifies why a large measurement budget need not remove the difficulty of observing Direct exposures in dense networks. The ER result concerns average inclusion across a network; the following experiment examines high- and low-degree nodes within the same network.
AuthorsOpenReview ↗

Response to Reviewer ZDHN (4/4): Node degree and inclusion probabilities: experiments

We test n = 750 and 2,000 with B = 0.2n and equal noise variance across nodes. The high degree group contains 8 hubs at n = 750 and 20 at n = 2,000. The low degree group contains the half of the network with the lowest degrees, excluding hubs. Hub connections increase through five levels. At the highest level:

DS-CGD inclusion probabilities for treated exposures.

nEffectMean high degreeMean low degreeProbability: highProbability: low
750Direct674.79.50.015210.08395
750Global674.79.50.099890.10001
2,000Direct1801.120.10.008900.07700
2,000Global1801.120.10.099690.09998

The Global probabilities are close to the B/(2n)=0.1B/(2n)=0.1 benchmark. Direct hubs receive fewer opportunities because their exposures exclude nearby treated targets. Thus, selection favors low-degree nodes for Direct effects in these tests, with little difference for Global effects.

Does lower inclusion translate into worse estimation? With a fixed design and independent, centered prediction residuals of variance σ2\sigma^2, averaging over outcomes and randomization gives

=\frac{\sigma^2}{n^2}\sum_v\left(\frac1{p_v}-1\right).$$ Small inclusion probabilities therefore carry a precision cost. DS-CGD's allocation objective weighs this cost using the residual scores. It can give more probability to sets containing poorly observed targets, while respecting their conflicts. The tables report mean RMSE for each subgroup effect. Subgroup sizes differ, so comparisons across methods within the same subgroup isolate design performance. **Direct subgroup effects. Mean RMSE.** | Method | n = 750, low | n = 750, high | n = 2,000, low | n = 2,000, high | | :--- | ---: | ---: | ---: | ---: | | **DS-CGD** | **0.254** | **3.359** | **0.159** | **2.803** | | Generalized CGD | 1.175 | 7.656 | 1.145 | 7.732 | | Original CGD | 1.318 | 8.301 | 1.280 | 8.476 | | Independent set | 0.202 | 18.211 | 0.127 | 17.453 | **Global subgroup effects. Mean RMSE.** | Method | n = 750, low | n = 750, high | n = 2,000, low | n = 2,000, high | | :--- | ---: | ---: | ---: | ---: | | **DS-CGD** | **0.214** | **1.289** | **0.138** | **0.877** | | Generalized CGD | 3.253 | 24.181 | 3.290 | 23.088 | | Original CGD | 4.588 | 33.234 | 4.646 | 31.693 | | CC | 0.330 | 3.307 | 0.327 | 2.048 | | RGCR | 0.223 | 1.407 | 0.140 | 0.857 | DS-CGD has the lowest Direct hub RMSE at both sizes, despite the hubs' lower inclusion probabilities. The independent set design performs better for low-degree nodes. For Global effects, DS-CGD improves on CC and both CGD baselines; RGCR is slightly lower at n = 2,000. [Inclusion probabilities across all five hub levels](https://anonymous.4open.science/r/supplementary-materials-a4e31c79/03_low_and_high_degree/02_inclusion_probabilities/high_degree_inclusion.png) · [Measurement probabilities](https://anonymous.4open.science/r/supplementary-materials-a4e31c79/03_low_and_high_degree/02_inclusion_probabilities/high_degree_measurement.png) [Subgroup RMSE, n = 750](https://anonymous.4open.science/r/supplementary-materials-a4e31c79/03_low_and_high_degree/01_estimation/high_degree_750.png) · [Subgroup RMSE, n = 2,000](https://anonymous.4open.science/r/supplementary-materials-a4e31c79/03_low_and_high_degree/01_estimation/high_degree_2000.png) We also increase density throughout ER and PA networks, with n = 750 and 2,000 and B = 0.2n. Direct RMSE increases overall, while Global RMSE is relatively stable or decreases. This pattern is consistent with the distinction above: Direct treated targets compete for compatible positions; Global targets in the same arm can coexist. DS-CGD has the lowest mean Direct RMSE at the highest tested density in each comparison. [Network density: Direct effects](https://anonymous.4open.science/r/supplementary-materials-a4e31c79/03_low_and_high_degree/03_network_density/density_direct.png) · [Network density: Global effects](https://anonymous.4open.science/r/supplementary-materials-a4e31c79/03_low_and_high_degree/03_network_density/density_global.png)

Comments on the paper

1 comment

General Response to the Area Chair and Reviewers

We sincerely thank the reviewers and the Area Chair for their careful assessment. We appreciate the recognition that “The theoretical results seem substantive” [xaUX] and that the paper addresses “an important problem in causal inference under interference” [ZDHN]. We also value the suggestions on presentation and theoretical insight (xaUX–Q2, Q3, Q9–Q14), estimation validity and computational cost (ZDHN–Q1, Q2), and the role of historical information (Yo3r–Q1, Q2). We have incorporated the corresponding revisions and additional results into the revised PDF, with changes marked in blue.

  1. Clearer motivation and presentation

    • We explain the motivating field experiment before introducing the formal framework. The abstract now describes the risk decomposition in words, and the notation is introduced before Equation (1). (xaUX–Q2)
    • We explain how outcome measurement and exposure compatibility jointly limit causal information. Section 5 now presents the experimental workflow before the optimization and estimation formulas. (xaUX–Q3, Q5, Q8)
  2. Theoretical contribution and relation to prior work

    • We clarify the progression from matching minimax bounds, to separating measurement and exposure-conflict costs, to determining the measurement budget needed to attain the precision available under full measurement. (xaUX–Q3, Q10–Q12)
    • We identify the new budget reduction argument and its control of covariance. We distinguish these contributions from existing results and explain what the network risk profiles reveal about causal estimation under full measurement. (xaUX–Q9, Q11, Q13, Q14)
    • We distinguish the minimax guarantees of the theoretical constructions from the guarantees of practical DS-CGD. We also clarify the distinction between sequential active sampling and our offline joint design. (xaUX–Q1, Q4)
  3. Joint design, estimation, and fair comparisons

    • We explain the construction of compatible exposure sets, the role of historical residual scores, and the reference design. These ingredients guide the joint allocation of treatment and outcome measurement to use the available budget efficiently. (xaUX–Q6, Q15)
    • We clarify conditional design unbiasedness using exact inclusion probabilities and regression adjustment. Appendix C.3 specifies each baseline's budget adaptation and the shared estimator and inputs within each matched comparison. (xaUX–Q7, ZDHN–Q1)
  4. Additional evidence on robustness and computational cost

    • We add empirical bias checks and analyses of inclusion probabilities, estimation error among high-degree nodes, and network density. (ZDHN–Q1, Q3)
    • We report runtime–RMSE comparisons and timings on networks with 50,000 nodes. Section 5 accounts for both candidate search and numerical optimization costs. (ZDHN–Q2)
    • We examine historical sample size, coarse or perturbed allocation scores, and missing historical records. The additional experimental results are organized in Appendix C.4. (Yo3r–Q1, Q2)

To help the reviewers and AC inspect the algorithm workflow, experimental figures, and supporting tables, we created an anonymous supplementary-materials page. References such as xaUX–Q2 point to the numbered questions in our corresponding reviewer replies.

Thank you again for the time and care invested in these reviews. We welcome further questions and would be happy to discuss any aspect of the revision.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.