- Rating
- 2
- Confidence
- 2
- Informativeness
- 1
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper formulates tokamak disruption precursor discovery as a multiple-instance learning problem with discharge-level labels only. It extracts candidate precursors and constructs event chains through local scoring, progressive suffix masking, and a shared event vocabulary. Experiments are conducted on 1,816 J-TEXT discharges and validated across multiple truncation, retraining, and scoring-head configurations.
Critical strengths
- The paper addresses a valuable problem and systematically investigates disruption precursor discovery when only discharge-level labels are available.
- The paper introduces an intuitive progressive suffix-masking mechanism that removes previously identified evidence and all subsequent observations, encouraging later models to search for earlier candidate precursors.
- The experimental analysis is relatively comprehensive. Multiple retraining and truncation controls, scoring-head ablations, and multi-seed analyses demonstrate that progressively shortening the input can expose earlier candidate evidence.
Critical weaknesses
- Validation of the candidate events appears insufficient. Residual-based triggers can wait indefinitely for subsequent high-risk confirmation, yet there is no independent alignment with diagnostic events or expert annotation to verify whether these earlier onsets correspond to precursor activity.
- The ensemble classifier lacks sufficient independent evaluation. The evaluation set was used to fit weights and thresholds and was inspected during analysis development, while the existing cross-fitting procedure does not cover the full analysis pipeline.
- The experimental comparisons do not adequately control for confounding factors. In particular, even after length matching, the length-only classifier achieves an AUPRC of 0.453 against a positive-class prevalence of 0.214. Event coverage has also not been matched across methods, complicating the interpretation of lead-time and chain-depth comparisons.
- Some baselines do not fully correspond to their original formulations. The TimeMIL-style baseline omits wavelet positional encoding, and the way attention signals are incorporated into the unified event grammar does not adequately support a fair assessment of the models’ native localization capabilities.
- The current results primarily support the effects of progressively shortening the input and repeated fitting. Both fixed truncation and the progressive Z-head reproduce the main effects, leaving the additional empirical value of evidence-guided truncation and local scoring insufficiently established.
- The independent value of event chains requires further validation. The reference run on the evaluation set contains only five chains involving events from multiple families, and directed adjacency shows no consistent gains after controlling for geometric information. This weakens the evidence for the claimed insights into chain-like evolution.
Questions for authors
See weaknesses.
4 comments
Reply to Reviewer kGRU. W1 validation of the events and W2 independent evaluation
W1. Validation of the candidate events.
Waiting for confirmation (Appendix C.1). For the events that rise from low risk the delay between residual trigger and confirmed rise has a median of 8 ms and a 90th percentile of 88 ms, and 7.7% wait longer than 100 ms. With the delay capped at 25 ms the lead of PM-MIL is 175 ms against 55 ms for Single MIL (+120 [104, 133] ms), so the conclusions do not rest on long waits. The method sets no maximum wait because that would be a preset time window, which it avoids (Section 3.3).
Event-level comparison (new Appendix A.5). An author with expertise in tokamak physics and diagnostics describes the MHD route to a disruption on J-TEXT in three stages, a small perturbation whose Mirnov frequency comes and goes, a stable mode whose poloidal mode number is identified, and the growth of that mode with rising amplitude and falling frequency. Fixed rules mark these stages on the Mirnov features without using any event, a consistency check on the features the model also reads, not an independent identification of a tearing mode. In the reference run, D starts within 10 ms of the onset of the stable mode in 40 of the 43 disruptive discharges with C followed by D in which that onset lies inside the record (88% to 95% over rule variants), the growth follows a median 47 ms later inside D, and C lies before the stable mode in all 43. The rounds meet the stages in reverse order, the growth in round 0, the stable mode in round 1 and the perturbation in later rounds. The rules also mark a stable mode in 46% of the non-disruptive discharges, so the appearance of a mode does not tell which discharges disrupt, while its growth does more often. Like every family, C carries disruption information at its own stage. 54% of the pool discharges with C disrupt, against 81% and 84% for the late families D and E and a base rate of 22%, as fits an early sign that does not always develop.
Relation to disruption. With a nominal 100 ms removed from the end of every record PM-MIL still reaches AUPRC 0.758 out of fold at a disruption share of 0.223, against 0.647 for one model, and its chains separate disruptive from non-disruptive discharges at chain AUROC 0.736 (0.725 from event and family counts alone, Table 8). The revision also adds a reading of the six families by this author (Table 5). C is read as the onset of an early MHD instability and D as a stable mode and its growth, E as plasma-wall contact with impurity influx and F as a rapid impurity rise.
W2. Independent evaluation of the ensemble classifier.
You are right. As submitted, the form of the ensemble rule was chosen on the evaluation set, the ensemble weights and thresholds were fitted there, and the cross-fitting did not cover the whole pipeline.
No fit on the evaluation set (Section 4, Appendix B.4). The weights now come from the validation data inside each round, the threshold from ND discharges of the development pool, and the chain classifier for evaluation chains is fitted on the pool. The form of the rule is kept. Tokens, architecture, losses, masking and grammar were fixed before the reference run, though after earlier runs whose evaluation scores were known, and the M4 schedule, length matching and all controls came after evaluation results had been seen (Table 9). With a nominal 100 ms removed PM-MIL reaches AUPRC 0.804 on the evaluation set, against 0.674 for attention MIL and 0.558 for Single MIL.
End-to-end outer cross-validation (Appendix B.3). In five folds over all discharges every step (training, checkpoint selection, round admission and stopping, event thresholds, vocabulary, chain classifier, ensemble weights and threshold) uses four folds and the fifth is only scored, which gives 1,807 scored discharges with 403 disruptions instead of 39. PM-MIL then exceeds attention MIL by +0.108 [0.067, 0.151] and Single MIL by +0.111 [0.072, 0.153] in AUPRC, and its earliest-event lead is 175 against 75 ms (+100 [80, 115]). The folds contain discharges seen during development, so this removes every fit on the scored discharges but adds no unseen data (Section 6).
Reply to Reviewer kGRU. W3 confounders and W4 baselines
W3. Confounding factors.
Record length. The value of 0.453 came from our protocol and not from a remaining length cue, and thank you for pointing at it. We had averaged the predicted probabilities over the three random ND crops before computing the metric, while each disruptive record was scored once. Averaging shrinks the spread of the ND scores, which inflates the metric of a classifier whose ND scores vary between crops, most of all one that sees only the length. Computed on each crop, the length-only classifier has AUPRC 0.196 to 0.228 at a prevalence of 0.214, and out of fold 0.220 at 0.223. For the models the change was small, and no single-model value on the evaluation set moved by more than 0.007. All metrics are now computed per crop and then averaged (Appendix B.1), and the sentence that matching removes most but not all of the length cue is gone.
Event coverage (Appendix C.3). We repeat the chain comparisons on the discharges where both methods have events, and at an equal number of ND discharges with events, reached by raising the ND bag gate after training. The earlier and deeper chains of PM-MIL relative to Single MIL hold under both matchings (+120 [109, 137] and +85 [72, 101] ms in lead, +24.0 [20.4, 27.8] and +11.4 [8.8, 14.2] points in the multi-event share). Against M4 the differences stay small, with M4 at most 8 ms earlier and 2.4 points deeper by distinct families at equal ND budget (−8 [−18, −2] ms and −2.4 [−4.1, −0.7] points). The chain AUROC margin of PM-MIL over M4 is +0.104 [0.068, 0.140] at equal ND budget and shrinks to +0.028 [−0.002, 0.057] on common discharges of the pool, so part of it reflects which discharges carry events.
W4. Fidelity of the baselines and native localization.
TimeMIL. We agree and no longer present R2 as TimeMIL. It is now named a class-token Transformer aggregator (TimeMIL-like, without wavelet positional encoding). We also ran R2 with the wavelet positional encoding of TimeMIL, re-implemented from its equations with the constants of the official TimeMIL code (Appendix D). The result does not change. As a single fit its attention first exceeds an ND-calibrated level on 15% of the disruptive discharges, always within the final 100 ms. Neither the single fit nor the fixed schedule yields an event on a disruptive discharge, and with a nominal 30 ms removed their AUPRC is 0.714 and 0.869, with paired intervals from PM-MIL that span zero (+0.116 [−0.022, 0.237] and −0.038 [−0.103, 0.027]). Like the original methods, R1 and R2 are trained with a bag loss only, without the auxiliary losses and the attention regularizer of our backbone.
Native localization (Table 14). We now also read the localization directly from each model. The attention of R1 reaches an ND-calibrated level on 42% of the disruptive discharges and that of R2 on 23%, always inside the final 100 ms, and 96% and 75% of their attention mass on disruptive discharges lies in the final 100 ms (48% for a uniform signal). The terminal concentration of these baselines is therefore their own. For the models that score tokens, the share of the signal in the final 100 ms falls from 80% for Single MIL to 44% to 53% under progressive shortening. Your point also led us to correct our explanation of why attention yields few events under the grammar. At round 0 the attention peaks of R1 are higher on disruptive than on ND discharges (median 17.8 against 7.1 times uniform attention), but 90% of them are one or two tokens wide, and the grammar asks for three tokens.
A further baseline. Top-k pooling MIL in the style of weakly supervised temporal localization is added (R3). One fit gives a lead of 40 ms with 90% of its events in the final 100 ms. Under the fixed schedule it gives 220 ms and a chain AUROC of 0.71 on the pool.
Reply to Reviewer kGRU. W5 added value and W6 chains
W5. Additional value of evidence-guided truncation and local scoring. We agree that progressive shortening with retraining produces the gains in lead, concentration and depth. The revision shows what the two specific choices add (Section 5.5, Table 2).
Against the fixed schedule. M4 needs a step chosen in advance. Rerun with steps of 10, 25, 50 and 100 ms (one run per step, seed 42), its chain AUROC on the development pool is 0.562, 0.646, 0.709 and 0.707 (PM-MIL 0.714), and the 100-ms step lowers the lead to 155 ms (Appendix C.4). The limitation is physical. The step fixes where the training cutoffs fall, so two precursors closer than one step can fall between the same pair of cutoffs and no model is trained with the earlier one as its latest evidence, while precursors farther apart are found again by several successive models. That spacing is set by the plasma, differs between processes and discharges and changes with machine size. The rule-marked stages of MHD activity show this independently of any model. The onset of a stable mode precedes the onset of its growth by a median of 41 ms, with a 10th to 90th percentile range of 10 to 88 ms (Appendix A.5). With a 10-ms step the later rounds start 5.1 events per discharge inside the growth stage of the mode, against 0.5 to 1.1 for PM-MIL, and the best fixed step (50 ms) lies near the median spacing, which a schedule has to know in advance. Out of fold the chains of PM-MIL separate disruptive from non-disruptive discharges better than those of the 25-ms schedule (0.736 against 0.648, +0.088 [0.041, 0.133]) and are deeper per episode (+3.6 [0.4, 6.6] points).
Against contextual scoring. The progressive Z-head matches PM-MIL on chain AUROC (0.734) and is earlier and deeper. Its token score reads the whole record, so it says that a discharge disrupts and not which signals at which time carried the evidence. Scored by local risk, its event starts on disruptive discharges carry the risk of random times (0.73 against 0.73), while those of PM-MIL are enriched (0.83 against 0.79, Table 13). The vocabulary shows the same. Built from the same 2,356 candidate events with the same clustering, the local vocabulary gives the 512 PM-MIL chains a Markov chain AUROC of 0.707 against 0.666 for the contextual features of the same models, and leaving one encoder out changes it little (minimum adjusted Rand index 0.83) and the contextual vocabulary strongly (0.40, Table 3). A local score confines the attribution of an event to the channels of its own window (Section 5.4).
W6. Independent value of the chains.
Sample size. Out of fold PM-MIL has 636 event-bearing discharges, 109 of them (17.1%) with two or more families, against five such chains on the evaluation set (Appendix B.3).
What carries the information (Section 5.4, Appendix B.3). Out of fold, event count alone reaches AUROC 0.553 on event-bearing discharges, adding the six family counts gives 0.725 (+0.171 [0.128, 0.214]), and the Markov chain classifier reaches 0.736. With record and event geometry included, family counts still add +0.016 [0.006, 0.028] and directed adjacency adds −0.009 [−0.021, 0.002]. You are right about directed adjacency, and the larger sample confirms it. The ordered classifier is above its shuffled counterpart in every estimate (+0.009 [−0.005, 0.022] out of fold, 0.019 to 0.024 for the three seeds on the evaluation set), but the gain is small and its out-of-fold interval includes zero, so the revision places the information in event presence and family composition.
What the chain view is used for. Order matters in the description. The change from C to D raises the magnetic-fluctuation contribution at the onset by 1.26 [1.02, 1.50] reference standard deviations and the reverse change alters it by −0.40 [−0.73, −0.09], and 39 of the 50 changes from C to D join separate high-risk episodes while all 15 from D to C stay inside one. If the reading of C as the onset of an early MHD instability and of D as a stable mode and its growth holds, a pre-emptive response at C could keep a large island from forming, whereas at D the island has to be suppressed (Section 6). These are observational statements, and the paper does not claim that C causes D.
In sum, given only discharge labels, a model can be made to expose candidate precursors well before the terminal phase, each scored from the signals of its own time window, without a truncation step chosen in advance. Among the compared methods, only the combination of local scoring and evidence-guided masking does all of this at once.
Summary of the revision and of our answers, starting with early MHD activity
We compared the model's event times with the onsets an author marked while reading the diagnostic traces (new Appendix A.6). The author, who has expertise in tokamak physics and diagnostics, saw the 27 input signals but no model output, discharge number or label, and could time the onset of a persistent coherent mode in 18 of 40 evaluation discharges (30 disruptive). Over the timed discharges in which a method has events, the nearest event start lies a median 3.1 ms from the mark for PM-MIL (14 discharges), against 32.8 ms for M4 (14), 51.7 ms for the progressive Z-head (15) and 72.8 ms for Single MIL (14). The earliest MHD activity is weaker. Marking it in the 19 evaluation discharges that contain C windows, the same author gave a time in 6 without cues and in 11 with the C windows highlighted, and of these 11 marks 7 lie before the first C window, 4 inside one and none after them. Where the onset is clear, PM-MIL coincides with the expert, and where it is weak, it shows the expert where to look.
Family C is that weak stage. Weak Mirnov activity without an identified mode number is common, in 89% of the non-disruptive discharges, and hard to judge in a single discharge. PM-MIL retrieves a recurring part of it. 64% to 76% of the C events on disruptive discharges recur in independently seeded runs within 25 ms (31% to 36% by chance), and in all 43 C-to-D discharges whose stable-mode onset lies inside the record, C precedes it by a median of 140 ms. The author reads C as the onset of an early MHD instability that may contain an early, small magnetic island (Table 5, Appendix A.5).
W1. Rules for three stages of MHD activity put D within 10 ms of the onset of a stable mode in 40 of 43 C-to-D discharges, and with the confirmation wait capped at 25 ms the lead of PM-MIL stays 175 against 55 ms for Single MIL (Appendix C.1).
W2. No ensemble weight, threshold or chain classifier is fitted on the evaluation set any more, and an outer five-fold cross-validation repeats every step on four folds and scores the fifth (1,807 discharges, 403 disruptions, Appendix B.3).
W3. The value of 0.453 came from averaging the ND scores over crops before computing the metric. Computed per crop, the length-only classifier stays at the prevalence (0.196 to 0.228 at 0.214, Appendix B.1), and the gains over Single MIL hold when event coverage is matched (Appendix C.3).
W4. R2 is no longer presented as TimeMIL, and with the wavelet positional encoding of TimeMIL the result does not change. Read directly from each model, the attention of R1 and R2 lies mostly in the final 100 ms (Table 14). Top-k pooling MIL is added.
W5 (Section 5.5, Table 2). Out of fold, seven methods are checked on whether their events (1) lie beyond the terminal phase, (2) are informative about disruption, (3) are each scored from the signals of their own time window and (4) are found without a preset truncation step.
Progressive shortening with retraining, shared by M4, the progressive Z-head and PM-MIL, gives the gains in lead, concentration and depth. Without evidence-guided cutoffs, M4 needs its step in advance and its chains carry less information about disruption (+0.088 [0.041, 0.133] in chain AUROC). Without local scoring, the progressive Z-head is somewhat earlier, but its events cannot be traced to the signals of their own window. Attention MIL meets requirement 2 only because its interval is wide (chains on 21% of the disruptive discharges).
W6. Out of fold the information lies in event presence and family composition. Adding the six family counts to the event count raises AUROC from 0.553 to 0.725 on event-bearing discharges, and order adds no detectable gain (Section 5.4).