- Rating
- 1
- Confidence
- 2
- Informativeness
- 1
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper study causal inference in randomized network experiments when the mechanism mapping treatment assignments to physical exposure is unknown. The paper show the same assignment & outcome distribution can support exposure effects of opposite signs, and proposes Target-Aware Bridge Auditing (TAB) to acquire additional exposure measurements. The paper also develops target-dependent query complexity bounds and search/confirmation results in structured settings. Validation includes synthetic experiments, non-oracle network simulations, and public-data analyses illustrating ambiguity when receipt is unobserved.
Critical strengths
The nonidentification construction establishes why randomization alone may not resolve the scientific target. This provides a concrete motivation for acquiring exposure measurements rather than simply collecting more assignment & outcome observations.
The nested retention rule supports causal coverage under adaptive querying and stopping, conditional on the stated assumptions. Hence, I think maintaining an envelope over plausible mechanisms is a useful alternative to selecting a single exposure map and ignoring structural uncertainty.
Critical weaknesses
My main concern is whether the certified intervals are informative in the settings where the method is intended to help. In Appendix F.3, the favorable width of 0.554 uses simulation oracle band. Replacing this with analytic bands on the recorded query paths gives a terminal mean width of 35.972, compared with a known effect-support diameter of only 1.8. I recognize that this replay does not rerun query selection under the analytic bands, but the gap is substantial. The non-oracle experiments also include vacuous results: Table 6 reports width 1.7927 against support diameter 1.8, while Table 12 reports width 2.000 against support diameter 2. I, therefore, remain unconvinced that the acquisition gains demonstrated with oracle calibration translate into useful certified inference without substantially more information from the main experiment.
I suppose that target-aware query selection does not offer a sufficiently consistent advantage over simple count balancing. In Table 12, count-based querying obtains essentially the same width as target acquisition (0.160 vs 0.161), using fewer labels (14.05 versus 16.10), with comparable empirical coverage. Table 8 provides another setting where count balancing gives a narrower interval, although at higher cost. Furthermore, the main non-oracle policy falls back to count balancing when the target score has zero 1 step gain. I acknowledge that target-aware acquisition performs better in other settings, but I find it difficult to separate its contribution from the fallback and improved interval calibration.
Questions for authors
I would ask the authors to prioritize addressing the concerns raised in weakness, especially the vacuous intervals and the inconsistent gains over simple baselines.
Question: Bridge labels test the assignment-to-exposure mechanism, but cannot establish outcome exposure sufficiency. If sender identity or dose affects outcomes beyond binary receipt, what justifies interpreting the narrowed envelope as evidence about the physical-exposure effect?
5 comments
A2. Independent contribution of target-aware querying
Q: What does target-aware querying add, and when does it improve on count balancing?
A: TAB’s practical contribution is to spend bridge labels on distinctions that change the causal effect. TAB is designed for a specific gap in randomized network experiments: randomization identifies who was assigned treatment, but the physical exposure—and thus the causal effect of interest—may remain ambiguous. Bridge measurements narrow the mechanisms and effects they imply. Under the stated assumptions, Theorem 5.3 guarantees envelope validity under adaptive querying and stopping. TAB preserves uncertainty across plausible mechanisms and directs limited audit effort toward measurements that distinguish effects.
Target-aware scoring allocates queries by their effect on the causal envelope, whereas count balancing allocates by mechanism counts. Count balancing splits mechanisms into similarly sized groups; TAB scores queries by expected reduction in worst-case causal-envelope width per unit information cost. Mechanism count can misrepresent causal uncertainty: many mechanisms may imply the same effect while a few imply different effects. TAB targets those effect differences, with count balancing as a fallback when no feasible query improves the envelope.
The theory explains when targeting causal geometry is useful and when count balancing is a good approximation. This broader objective has a theoretical basis. Theorem 5.4 measures difficulty by the packing of distinct causal targets and whether feasible queries separate them. Proposition D.10 shows count balancing is a valid approximation when child interval width scales with mechanism count. TAB thus covers count-aligned settings and the broader case where counts and causal targets diverge; the search-and-confirmation results characterize the information needed to locate and verify a target.
Figure 3 shows that target-directed queries reduce audit depth when model and target geometries are misaligned. Figure 3 shows the consequence: target-directed queries resolve the population effect with fewer exact queries when geometries are misaligned, while both methods require the same number in the aligned control.
The controlled comparison in Table 8 and the end-to-end comparison in Table 12 show how the gain depends on the setting. Table 8 holds candidate bands fixed: at budget 16, TAB has nearly the same width at lower cost; at budget 256, it is cheaper and narrower. Table 12 shows comparable non-oracle performance in another network configuration. Together with Figure 3, the comparisons show that target geometry helps when mechanism counts do not represent causal uncertainty.
| Setting | Plateau-aware TAB: width / cost or labels | Count balancing: width / cost or labels |
|---|---|---|
| Table 8, budget 16 | 0.8582 / 25.046 | 0.8462 / 26.670 |
| Table 8, budget 256 | 0.1195 / 23.107 | 0.1369 / 26.670 |
| Table 12, non-oracle network | 0.161 / 16.10 labels; coverage 0.983 | 0.160 / 14.05 labels; coverage 0.987 |
Taken together, the results position TAB as a target-centered extension of count balancing. TAB generalizes count balancing by retaining count-based allocation when counts reflect causal uncertainty and redirecting measurements toward effect-relevant distinctions otherwise. It couples this allocation rule with a valid envelope under adaptive querying and stopping.
A3. Outcome exposure sufficiency
Q: If sender identity or dose affects outcomes beyond binary receipt, how can the narrowed envelope be interpreted as a physical-exposure effect?
A: A causal interpretation starts by defining exposure to include the attributes that can affect the outcome. We would define exposure at the level that captures the treatment attributes relevant to the outcome. If sender identity and dose may affect outcomes, exposure should record receipt, sender, and dose. Aligning the definition, candidate mechanisms, and bridge labels distinguishes outcome-relevant states that a binary “received” indicator would pool.
Assumption 3.2 formalizes this requirement, and Theorem 5.7 supports sensitivity analysis for bounded departures. This is the interpretation required by Assumption 3.2, which represents outcomes as . If only a coarser measure such as binary receipt is available, scientifically justified bounds on unexplained outcome variation can enter a sensitivity analysis. Theorem 5.7 translates such a bound into an adjustment to candidate intervals. Thus, plausible sender or dose effects call for a richer measured exposure or a sensitivity-adjusted envelope based on an informative bound.
Closing
These responses connect the practical value of TAB to its information requirements, query strategy, and exposure definition. We hope this response clarifies when the envelope is informative, what target-aware querying contributes, and how the exposure definition determines the causal interpretation. We appreciate the reviewer’s careful examination of these issues.
Response to Reviewer aubW
We thank Reviewer aubW for examining whether the certified intervals are informative in the intended settings, whether target-aware querying adds value beyond count balancing, and how the exposure definition supports the causal interpretation. We distinguish main-trial information, bridge measurements, query allocation, and outcome exposure sufficiency.
A1. Informativeness of non-oracle intervals
Q: Are the certified intervals informative in the network-experiment settings where the method is intended to help?
A: Interval informativeness depends on both the main experiment’s randomized information and the exposure information supplied by bridge labels. The practical question is whether bridge auditing can make the causal envelope informative under the randomization structure available in the main experiment. The main experiment estimates effects under candidate mechanisms; bridge labels distinguish which mechanisms remain plausible. Theorem 5.3 guarantees coverage under adaptive querying and stopping, while the experiments assess interval width.
The experiments show that bridge audits can narrow the envelope when the main trial provides sufficient independent randomized information. Bridge auditing narrows the envelope when the main experiment provides enough independent randomized information, but cannot replace a design with too few effective independent blocks. Table 13 isolates auditing with main-trial information fixed; Tables 6 and 12 show how precision changes with randomized information.
The results distinguish uncertainty that bridge auditing can reduce from precision governed by the main trial. When the main design has little independent randomized information, a valid envelope can remain wide. With more informative designs, bridge auditing narrows uncertainty from the unknown exposure mechanism. The experiments characterize when this reduction yields a useful interval; coverage alone does not guarantee narrowness.
Appendix F.3 isolates candidate-band calibration by replaying the recorded query paths with analytic bands. Appendix F.3 isolates candidate-band calibration by replacing the bands along recorded query paths. The mean width falls from 82.074 to 35.972, against an effect-support diameter of 1.8. Since paths are fixed, this is a calibration diagnostic, not a full adaptive rerun with analytic bands; it shows that candidate-band construction remains a precision bottleneck in this configuration. The oracle acquisition result addresses a different component.
Table 13 separates the contributions of adding labels, selecting queries, and using anytime reporting. With main-trial information fixed, Table 13 separates auditing, query selection, and reporting. The passive-union width is 0.591; 30 fixed queries with terminal reporting reduce it to 0.228. Target queries with terminal reporting give 0.194, isolating the query rule; anytime reporting on that path gives 0.161 with 0.983 path coverage.
Together, these comparisons identify the settings in which bridge labels improve the precision of the causal envelope. Bridge auditing is useful when the main experiment can estimate candidate-specific effects and much of the remaining uncertainty is structural. When effective independent blocks are too few, auditing cannot replace main-trial information. The experiments characterize this boundary: bridge measurements reduce mechanism uncertainty, while the main design governs precision.