- Rating
- 4
- Confidence
- 3
- Informativeness
- 3
- Presentation
- 3
- Soundness
- 3
Read review
Paper summary
The paper proposes JCSL, a post-hoc method for returning compact source sets with a marginal recall guarantee under exchangeability. It combines joint source-set inference from detector scores and propagation constraints, recall-weighted node ordering, and order-adaptive conformal scoring. Experiments share a frozen detector across three contact networks and 15 SI/SIR conditions, comparing set size and the frequency of meeting the recall target.
Critical strengths
The method has a clear rationale. The individual node confidence can miss how possible sources jointly explain an observed spread. JCSL uses that structure to improve the ordering and then calibrates the resulting sets. The inference, ordering and calibration steps fit together well, and the results make their scope clear. I see a useful contribution beyond calibrating the detector's original node scores. The expedriments are also comprehensive and validated the claims.
Critical weaknesses
The practical scope still needs more evidence, like the experiments use complete observations taken very soon after seeding, though it is unclear how much of the reduction would remain when investigation starts later or reports are incomplete. Incomplete reports also make it harder to ensure that the candidate set contains every true source. Moreover, the cost of joint inference is another limit on practibility, the paper does not show how the model designs affect set efficiency as graph size and complexity increase.
Questions for authors
See weaknesses.
4 comments
Response to Reviewer 95ts (Part 2/4)
1. Later investigations
[Weakness] "The practical scope still needs more evidence, like the experiments use complete observations taken very soon after seeding, though it is unclear how much of the reduction would remain when investigation starts later ..."
1.1 Results (Appendix K, Table 39). We agree that this is an important practical case. Early snapshots are also where localization is most actionable (e.g., for containing an outbreak), yet existing conformal methods rank candidates by individual detector scores without jointly using the propagation constraints among candidate sources, which can leave unnecessarily large sets when early intervention matters most. To measure how much of the reduction remains when investigation starts later, we evaluate first snapshots taken one to five steps after seeding on highSchool, the largest of the three benchmark networks, under slow, medium, and fast spreading. JCSL fits its reference distribution on the fitting split; here we fit it either globally or conditionally on the observed candidate count (Appendix K); neither choice affects the recall guarantee. For each setting, the table reports the strongest of all evaluated baselines.
Mean set sizes (global fit / conditional fit / strongest baseline), with the mean fraction of affected nodes in parentheses:
| First snapshot | Slow | Medium | Fast |
|---|---|---|---|
| step 1 | (3%) 10.3 / 10.9 / 13.6 | (5%) 10.8 / 11.2 / 16.7 | (6%) 9.9 / 8.7 / 15.7 |
| step 2 | (7%) 22.3 / 20.8 / 24.2 | (19%) 53.8 / 42.1 / 51.0 | (31%) 67.0 / 53.4 / 63.2 |
| step 3 | (17%) 58.4 / 48.8 / 50.4 | (46%) 259.3 / 167.3 / 187.0 | (67%) 368.3 / 275.6 / 294.0 |
| step 4 | (29%) 145.0 / 108.9 / 116.4 | (68%) 444.8 / 346.8 / 373.2 | (85%) 525.8 / 492.9 / 505.1 |
| step 5 | (41%) 241.7 / 178.6 / 167.9 | (81%) 550.8 / 452.1 / 460.3 | (92%) 579.7 / 586.2 / 591.0 |
With the conditional fit, JCSL remains smaller than the strongest baseline in 11 of the 12 delayed settings, by 6.6% on average and by up to 17.5% (step 2, medium spreading). For example, at step 3 under medium spreading, when nearly half of the nodes are already affected, JCSL returns 167.3 nodes versus 187.0. Over the delayed settings, the mean inclusion rates are 0.898 (global fit), 0.892 (conditional fit), and 0.886 (strongest baselines), all close to the nominal 0.90. These results show that JCSL maintains its advantage over the baselines in most delayed settings, not only with early snapshots.
Response to Reviewer 95ts (Part 3/4)
2. Incomplete reports
[Weakness] "... though it is unclear how much of the reduction would remain when investigation starts later or reports are incomplete."
2.1 Missing reports (Appendix I.2, Tables 29–30). On all three networks under slow, medium, and fast spreading, we hide each node's reports over the whole observation window at rate 2–10%. Hidden nodes are encoded as unknown, and true sources can be hidden, so the candidate set may miss sources. Perturbed inputs pass through the same detector, and fitting and calibration use data with the same perturbation. Aggregated over the nine network–speed conditions and compared with the strongest baseline (the baseline with the smallest mean set in each condition):
| Missing rate | JCSL smaller than strongest baseline | Mean size reduction vs. strongest baseline | Inclusion (JCSL / strongest baseline) |
|---|---|---|---|
| 0% (clean) | 9/9 | 22.2% | 0.895 / 0.898 |
| 2% | 9/9 | 20.7% | 0.894 / 0.902 |
| 4% | 9/9 | 18.7% | 0.905 / 0.906 |
| 6% | 9/9 | 17.0% | 0.904 / 0.908 |
| 8% | 9/9 | 14.1% | 0.905 / 0.902 |
| 10% | 8/9 | 11.0% | 0.900 / 0.901 |
The reductions decrease gradually but persist at every missing rate. At 10%, true sources are frequently excluded from the candidate set (containment 0.43–0.50), so this setting directly tests the case where the containment assumption fails. JCSL nevertheless remains smaller than the strongest baseline in 8 of 9 conditions while keeping its inclusion rates at 0.890–0.915.
2.2 Affected nodes reported as susceptible (Appendix I.2, Table 31). A related form of incomplete reporting is a false negative, which reports an infected or recovered node as susceptible. With 10% false negatives, JCSL is smaller than the strongest baseline in all nine conditions, by 11.1% on average, with an average inclusion rate of 0.893 for both JCSL and the strongest baseline.
3. Source containment under incomplete reports
[Weakness] "Incomplete reports also make it harder to ensure that the candidate set contains every true source."
3.1 When containment holds. Under complete SI/SIR observations, affected nodes never return to the susceptible state, so containment holds at any observation time, and Theorem 2 applies to the delayed snapshots in Section 1. We agree that incomplete reports can break containment; the missing-report setting in Section 2.1 tests this case directly.
3.2 What the guarantee needs (Appendix I.4, Proposition 8). Partial recall needs only enough sources in the candidate set, ("recall attainability"), not all of them. Any output restricted to the candidate set has recall at most . If unattainable calibration examples receive score and denotes the probability of unattainability given the fitting data , then Thus the guarantee of Theorem 2 holds whenever recall attainability holds almost surely, even if some sources are omitted, and degrades by at most otherwise. Inference, ordering, and calibration do not depend on whether the candidate set contains all sources; only this probability statement depends on recall attainability. Moreover, the lower bound is pessimistic. If calibration and test examples are i.i.d., unattainable test examples add to the failures only when the calibrated threshold becomes infinite, which requires about 10% or more of the calibration examples to be unattainable; otherwise, they count toward the failures that calibration already allows. With our calibration examples and , the success probability is therefore at least whenever . The unattainability rates observed with 10% missing reports or 10% false negatives (at most 0.083) are close to this level, consistent with the near-nominal inclusion observed there.
Response to Reviewer 95ts (Part 4/4)
4. Set efficiency and computational cost on larger networks
[Weakness] "Moreover, the cost of joint inference is another limit on practibility, the paper does not show how the model designs affect set efficiency as graph size and complexity increase."
4.1 Set efficiency and cost on larger networks (Appendices J.2–J.3, Tables 36–37). We join two or four copies of highSchool with sparse or dense cross-module edges, giving connected networks with 1,548 and 3,096 nodes, draw sources over the whole graph, and observe from step 3. To show how JCSL's designs affect set efficiency at these scales, we also compare two ablations that keep its reference distribution, scoring, and calibration but order the nodes without the propagation constraints or by the raw detector scores. Mean set sizes at width 12 (three seeds), with JCSL's average computation time per input:
| Network (nodes, cross edges) | JCSL | Strongest baseline | No propagation constraints | Detector order | JCSL time per input |
|---|---|---|---|---|---|
| 774, original | 61.1 | 62.1 | 66.9 | 89.6 | 0.4 s |
| 1,548, sparse | 48.4 | 58.7 | 55.0 | 84.9 | 0.9 s |
| 1,548, dense | 55.2 | 81.4 | 61.5 | 112.5 | 0.8 s |
| 3,096, sparse | 40.0 | 62.4 | 48.5 | 79.0 | 1.6 s |
| 3,096, dense | 45.7 | 84.9 | 50.2 | 111.0 | 1.8 s |
JCSL's advantage over the strongest baseline is larger on the larger networks, from 1.7% on the original graph to 17.6–32.2% at 1,548 nodes and 36.0–46.1% at 3,096 nodes, that is, 10–39 fewer candidates on the enlarged graphs at a cost of 0.8–1.8 s of computation per input. Since each additional candidate typically requires follow-up such as testing or contact tracing, this trade-off is favorable in investigation settings. Inclusion rates are 0.888–0.914 for JCSL and 0.893–0.911 for the strongest baseline. Both design components contribute on all five networks: the propagation constraints reduce the set by 8.8–17.6% relative to the ablation without them, and the refined order reduces it by 31.8–58.8% relative to the raw detector order.
4.2 Width budget (Appendices J.1–J.2, Tables 35–37). The width budget controls the cost of joint inference. Across the five networks of Section 4.1, width 4 saves 52–64% of the time relative to width 12, and JCSL with width 4 remains 16.0–45.0% smaller than the strongest baseline on the enlarged graphs. On the benchmark networks, width 4 lowers inference time by 4–23% and changes mean set sizes by at most 0.043 nodes.
4.3 Cost and practical use (Section 5.3 and Appendix J.3, Table 38). JCSL is slower than conformal baselines that only calibrate precomputed scores. Its cost is incurred once per input, is parallel across inputs, and does not depend on the calibration threshold; in the main experiments on highSchool, testing takes about 0.001–0.12 s per input (0.3–46 s for 400 inputs; Section 5.3), and our GPU implementation is 1.2–2.3 times faster than four CPU workers, with identical orders. JCSL is therefore best suited to settings in which a few seconds of computation per input are acceptable, as in investigations where each candidate requires follow-up; faster implementations are left for future work.
5. Summary
5.1 Taken together, these results extend the evidence on the practical scope of JCSL. The reductions persist with delayed first snapshots (Section 1) and under incomplete reports (Section 2), and the recall guarantee requires only that the candidate set contain enough sources. In the larger-network study, JCSL remains smaller than the strongest baseline on all five networks, and the cost of joint inference can be controlled through the width budget.
Thank you again for your positive assessment and constructive comments. We would be glad to address any further questions.
Response to Reviewer 95ts (Part 1/4)
Thank you for your positive and thoughtful review, and for recognizing the rationale of joint source-set inference, how the inference, ordering, and calibration steps fit together, and the breadth of the experiments. Your comments on later investigations, incomplete reports, and computational cost concern the conditions under which JCSL would be used in practice, and addressing them has made the paper considerably stronger.
We have uploaded a revised version that addresses your comments. The new analyses are presented in the appendices listed below, each referenced from the corresponding section of the main text:
Appendix H (propagation model mismatch) provides further robustness results. For brevity, the tables below aggregate results over conditions; the corresponding appendices give further per-condition results.