acceptodds
Under review as a conference paper at ICLR 2027

Progressive-Masking MIL Discovers Disruption Precursor Event Chains

Abstract

Tokamak disruptions are abrupt losses of plasma confinement that can damage the device, which motivates the search for early precursor events. Reliable labels in disruption prediction, however, usually record only whether a discharge ultimately disrupts, while the timing of precursor events is rarely annotated. We therefore treat each discharge as a bag of 5-ms tokens and use multiple-instance learning (MIL) to place candidate precursors in time from the discharge label alone. A bag-level classifier can reach a good score from late-stage signatures, so progressive masking removes the evidence it has already identified along with every token that follows, and retraining on what remains drives each successive model toward earlier precursors. An event grammar converts the outputs of the frozen models into candidate events, and a shared vocabulary groups them into six families that are linked into event chains ending in either disruption or normal termination. On 1,816 J-TEXT discharges, twelve rounds place the earliest disruption-associated event at a median of 185 ms before the recording endpoint, against 65 ms for a single MIL model. The fraction of disruption-associated events inside the final 100 ms falls from 70% to 32%, and the share of multi-event chains rises from 3.9% to 25.8%. Using chain content for disruption classification, AUROC increases from 0.49 to 0.71 and held-out AUPRC from 0.857 to 0.903. The chains reveal how precursors evolve and can serve as candidate training labels for real-time disruption predictors, with no manual event annotation.

Primary area:
applications to physical sciences (physics, chemistry, biology, etc.)
Keywords:
magnetic confinement fusion, disruption prediction, multiple-instance learning, progressive masking

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_kGRUOpenReview ↗
Rating
2
Confidence
2
Informativeness
1
Presentation
2
Soundness
2
Read review
Paper summary

This paper formulates tokamak disruption precursor discovery as a multiple-instance learning problem with discharge-level labels only. It extracts candidate precursors and constructs event chains through local scoring, progressive suffix masking, and a shared event vocabulary. Experiments are conducted on 1,816 J-TEXT discharges and validated across multiple truncation, retraining, and scoring-head configurations.

Critical strengths
  1. The paper addresses a valuable problem and systematically investigates disruption precursor discovery when only discharge-level labels are available.
  2. The paper introduces an intuitive progressive suffix-masking mechanism that removes previously identified evidence and all subsequent observations, encouraging later models to search for earlier candidate precursors.
  3. The experimental analysis is relatively comprehensive. Multiple retraining and truncation controls, scoring-head ablations, and multi-seed analyses demonstrate that progressively shortening the input can expose earlier candidate evidence.
Critical weaknesses
  1. Validation of the candidate events appears insufficient. Residual-based triggers can wait indefinitely for subsequent high-risk confirmation, yet there is no independent alignment with diagnostic events or expert annotation to verify whether these earlier onsets correspond to precursor activity.
  2. The ensemble classifier lacks sufficient independent evaluation. The evaluation set was used to fit weights and thresholds and was inspected during analysis development, while the existing cross-fitting procedure does not cover the full analysis pipeline.
  3. The experimental comparisons do not adequately control for confounding factors. In particular, even after length matching, the length-only classifier achieves an AUPRC of 0.453 against a positive-class prevalence of 0.214. Event coverage has also not been matched across methods, complicating the interpretation of lead-time and chain-depth comparisons.
  4. Some baselines do not fully correspond to their original formulations. The TimeMIL-style baseline omits wavelet positional encoding, and the way attention signals are incorporated into the unified event grammar does not adequately support a fair assessment of the models’ native localization capabilities.
  5. The current results primarily support the effects of progressively shortening the input and repeated fitting. Both fixed truncation and the progressive Z-head reproduce the main effects, leaving the additional empirical value of evidence-guided truncation and local scoring insufficiently established.
  6. The independent value of event chains requires further validation. The reference run on the evaluation set contains only five chains involving events from multiple families, and directed adjacency shows no consistent gains after controlling for geometric information. This weakens the evidence for the claimed insights into chain-like evolution.
Questions for authors

See weaknesses.

4 comments
AuthorsOpenReview ↗

Summary of the revision and of our answers, starting with early MHD activity

We compared the model's event times with the onsets an author marked while reading the diagnostic traces (new Appendix A.6). The author, who has expertise in tokamak physics and diagnostics, saw the 27 input signals but no model output, discharge number or label, and could time the onset of a persistent coherent mode in 18 of 40 evaluation discharges (30 disruptive). Over the timed discharges in which a method has events, the nearest event start lies a median 3.1 ms from the mark for PM-MIL (14 discharges), against 32.8 ms for M4 (14), 51.7 ms for the progressive Z-head (15) and 72.8 ms for Single MIL (14). The earliest MHD activity is weaker. Marking it in the 19 evaluation discharges that contain C windows, the same author gave a time in 6 without cues and in 11 with the C windows highlighted, and of these 11 marks 7 lie before the first C window, 4 inside one and none after them. Where the onset is clear, PM-MIL coincides with the expert, and where it is weak, it shows the expert where to look.

Family C is that weak stage. Weak Mirnov activity without an identified mode number is common, in 89% of the non-disruptive discharges, and hard to judge in a single discharge. PM-MIL retrieves a recurring part of it. 64% to 76% of the C events on disruptive discharges recur in independently seeded runs within 25 ms (31% to 36% by chance), and in all 43 C-to-D discharges whose stable-mode onset lies inside the record, C precedes it by a median of 140 ms. The author reads C as the onset of an early MHD instability that may contain an early, small magnetic island (Table 5, Appendix A.5).

W1. Rules for three stages of MHD activity put D within 10 ms of the onset of a stable mode in 40 of 43 C-to-D discharges, and with the confirmation wait capped at 25 ms the lead of PM-MIL stays 175 against 55 ms for Single MIL (Appendix C.1).

W2. No ensemble weight, threshold or chain classifier is fitted on the evaluation set any more, and an outer five-fold cross-validation repeats every step on four folds and scores the fifth (1,807 discharges, 403 disruptions, Appendix B.3).

W3. The value of 0.453 came from averaging the ND scores over crops before computing the metric. Computed per crop, the length-only classifier stays at the prevalence (0.196 to 0.228 at 0.214, Appendix B.1), and the gains over Single MIL hold when event coverage is matched (Appendix C.3).

W4. R2 is no longer presented as TimeMIL, and with the wavelet positional encoding of TimeMIL the result does not change. Read directly from each model, the attention of R1 and R2 lies mostly in the final 100 ms (Table 14). Top-k pooling MIL is added.

W5 (Section 5.5, Table 2). Out of fold, seven methods are checked on whether their events (1) lie beyond the terminal phase, (2) are informative about disruption, (3) are each scored from the signals of their own time window and (4) are found without a preset truncation step.

MethodEarliest-event lead (ms)Events in final 100 msChain AUROCAUPRC, nominal 100 ms removedRequirements met
Single MIL (M1)7559.0%0.5020.6473 only
Attention MIL (R1)2097.7%0.6590.6502 only
Z-head, single model10052.0%0.7140.6082 only
M0-repeat14540.1%0.6470.6941, 3
Fixed schedule (M4, 25 ms)17033.9%0.6480.7821, 3
Z-head + progressive masking20030.0%0.7340.7971, 2, 4
PM-MIL17533.3%0.7360.7581, 2, 3, 4

Progressive shortening with retraining, shared by M4, the progressive Z-head and PM-MIL, gives the gains in lead, concentration and depth. Without evidence-guided cutoffs, M4 needs its step in advance and its chains carry less information about disruption (+0.088 [0.041, 0.133] in chain AUROC). Without local scoring, the progressive Z-head is somewhat earlier, but its events cannot be traced to the signals of their own window. Attention MIL meets requirement 2 only because its interval is wide (chains on 21% of the disruptive discharges).

W6. Out of fold the information lies in event presence and family composition. Adding the six family counts to the event count raises AUROC from 0.553 to 0.725 on event-bearing discharges, and order adds no detectable gain (Section 5.4).

AuthorsOpenReview ↗

Reply to Reviewer kGRU. W1 validation of the events and W2 independent evaluation

W1. Validation of the candidate events.

Waiting for confirmation (Appendix C.1). For the events that rise from low risk the delay between residual trigger and confirmed rise has a median of 8 ms and a 90th percentile of 88 ms, and 7.7% wait longer than 100 ms. With the delay capped at 25 ms the lead of PM-MIL is 175 ms against 55 ms for Single MIL (+120 [104, 133] ms), so the conclusions do not rest on long waits. The method sets no maximum wait because that would be a preset time window, which it avoids (Section 3.3).

Event-level comparison (new Appendix A.5). An author with expertise in tokamak physics and diagnostics describes the MHD route to a disruption on J-TEXT in three stages, a small perturbation whose Mirnov frequency comes and goes, a stable mode whose poloidal mode number is identified, and the growth of that mode with rising amplitude and falling frequency. Fixed rules mark these stages on the Mirnov features without using any event, a consistency check on the features the model also reads, not an independent identification of a tearing mode. In the reference run, D starts within 10 ms of the onset of the stable mode in 40 of the 43 disruptive discharges with C followed by D in which that onset lies inside the record (88% to 95% over rule variants), the growth follows a median 47 ms later inside D, and C lies before the stable mode in all 43. The rounds meet the stages in reverse order, the growth in round 0, the stable mode in round 1 and the perturbation in later rounds. The rules also mark a stable mode in 46% of the non-disruptive discharges, so the appearance of a mode does not tell which discharges disrupt, while its growth does more often. Like every family, C carries disruption information at its own stage. 54% of the pool discharges with C disrupt, against 81% and 84% for the late families D and E and a base rate of 22%, as fits an early sign that does not always develop.

Relation to disruption. With a nominal 100 ms removed from the end of every record PM-MIL still reaches AUPRC 0.758 out of fold at a disruption share of 0.223, against 0.647 for one model, and its chains separate disruptive from non-disruptive discharges at chain AUROC 0.736 (0.725 from event and family counts alone, Table 8). The revision also adds a reading of the six families by this author (Table 5). C is read as the onset of an early MHD instability and D as a stable mode and its growth, E as plasma-wall contact with impurity influx and F as a rapid impurity rise.

W2. Independent evaluation of the ensemble classifier.

You are right. As submitted, the form of the ensemble rule was chosen on the evaluation set, the ensemble weights and thresholds were fitted there, and the cross-fitting did not cover the whole pipeline.

No fit on the evaluation set (Section 4, Appendix B.4). The weights now come from the validation data inside each round, the threshold from ND discharges of the development pool, and the chain classifier for evaluation chains is fitted on the pool. The form of the rule is kept. Tokens, architecture, losses, masking and grammar were fixed before the reference run, though after earlier runs whose evaluation scores were known, and the M4 schedule, length matching and all controls came after evaluation results had been seen (Table 9). With a nominal 100 ms removed PM-MIL reaches AUPRC 0.804 on the evaluation set, against 0.674 for attention MIL and 0.558 for Single MIL.

End-to-end outer cross-validation (Appendix B.3). In five folds over all discharges every step (training, checkpoint selection, round admission and stopping, event thresholds, vocabulary, chain classifier, ensemble weights and threshold) uses four folds and the fifth is only scored, which gives 1,807 scored discharges with 403 disruptions instead of 39. PM-MIL then exceeds attention MIL by +0.108 [0.067, 0.151] and Single MIL by +0.111 [0.072, 0.153] in AUPRC, and its earliest-event lead is 175 against 75 ms (+100 [80, 115]). The folds contain discharges seen during development, so this removes every fit on the scored discharges but adds no unseen data (Section 6).

AuthorsOpenReview ↗

Reply to Reviewer kGRU. W3 confounders and W4 baselines

W3. Confounding factors.

Record length. The value of 0.453 came from our protocol and not from a remaining length cue, and thank you for pointing at it. We had averaged the predicted probabilities over the three random ND crops before computing the metric, while each disruptive record was scored once. Averaging shrinks the spread of the ND scores, which inflates the metric of a classifier whose ND scores vary between crops, most of all one that sees only the length. Computed on each crop, the length-only classifier has AUPRC 0.196 to 0.228 at a prevalence of 0.214, and out of fold 0.220 at 0.223. For the models the change was small, and no single-model value on the evaluation set moved by more than 0.007. All metrics are now computed per crop and then averaged (Appendix B.1), and the sentence that matching removes most but not all of the length cue is gone.

Event coverage (Appendix C.3). We repeat the chain comparisons on the discharges where both methods have events, and at an equal number of ND discharges with events, reached by raising the ND bag gate after training. The earlier and deeper chains of PM-MIL relative to Single MIL hold under both matchings (+120 [109, 137] and +85 [72, 101] ms in lead, +24.0 [20.4, 27.8] and +11.4 [8.8, 14.2] points in the multi-event share). Against M4 the differences stay small, with M4 at most 8 ms earlier and 2.4 points deeper by distinct families at equal ND budget (−8 [−18, −2] ms and −2.4 [−4.1, −0.7] points). The chain AUROC margin of PM-MIL over M4 is +0.104 [0.068, 0.140] at equal ND budget and shrinks to +0.028 [−0.002, 0.057] on common discharges of the pool, so part of it reflects which discharges carry events.

W4. Fidelity of the baselines and native localization.

TimeMIL. We agree and no longer present R2 as TimeMIL. It is now named a class-token Transformer aggregator (TimeMIL-like, without wavelet positional encoding). We also ran R2 with the wavelet positional encoding of TimeMIL, re-implemented from its equations with the constants of the official TimeMIL code (Appendix D). The result does not change. As a single fit its attention first exceeds an ND-calibrated level on 15% of the disruptive discharges, always within the final 100 ms. Neither the single fit nor the fixed schedule yields an event on a disruptive discharge, and with a nominal 30 ms removed their AUPRC is 0.714 and 0.869, with paired intervals from PM-MIL that span zero (+0.116 [−0.022, 0.237] and −0.038 [−0.103, 0.027]). Like the original methods, R1 and R2 are trained with a bag loss only, without the auxiliary losses and the attention regularizer of our backbone.

Native localization (Table 14). We now also read the localization directly from each model. The attention of R1 reaches an ND-calibrated level on 42% of the disruptive discharges and that of R2 on 23%, always inside the final 100 ms, and 96% and 75% of their attention mass on disruptive discharges lies in the final 100 ms (48% for a uniform signal). The terminal concentration of these baselines is therefore their own. For the models that score tokens, the share of the signal in the final 100 ms falls from 80% for Single MIL to 44% to 53% under progressive shortening. Your point also led us to correct our explanation of why attention yields few events under the grammar. At round 0 the attention peaks of R1 are higher on disruptive than on ND discharges (median 17.8 against 7.1 times uniform attention), but 90% of them are one or two tokens wide, and the grammar asks for three tokens.

A further baseline. Top-k pooling MIL in the style of weakly supervised temporal localization is added (R3). One fit gives a lead of 40 ms with 90% of its events in the final 100 ms. Under the fixed schedule it gives 220 ms and a chain AUROC of 0.71 on the pool.

AuthorsOpenReview ↗

Reply to Reviewer kGRU. W5 added value and W6 chains

W5. Additional value of evidence-guided truncation and local scoring. We agree that progressive shortening with retraining produces the gains in lead, concentration and depth. The revision shows what the two specific choices add (Section 5.5, Table 2).

Against the fixed schedule. M4 needs a step chosen in advance. Rerun with steps of 10, 25, 50 and 100 ms (one run per step, seed 42), its chain AUROC on the development pool is 0.562, 0.646, 0.709 and 0.707 (PM-MIL 0.714), and the 100-ms step lowers the lead to 155 ms (Appendix C.4). The limitation is physical. The step fixes where the training cutoffs fall, so two precursors closer than one step can fall between the same pair of cutoffs and no model is trained with the earlier one as its latest evidence, while precursors farther apart are found again by several successive models. That spacing is set by the plasma, differs between processes and discharges and changes with machine size. The rule-marked stages of MHD activity show this independently of any model. The onset of a stable mode precedes the onset of its growth by a median of 41 ms, with a 10th to 90th percentile range of 10 to 88 ms (Appendix A.5). With a 10-ms step the later rounds start 5.1 events per discharge inside the growth stage of the mode, against 0.5 to 1.1 for PM-MIL, and the best fixed step (50 ms) lies near the median spacing, which a schedule has to know in advance. Out of fold the chains of PM-MIL separate disruptive from non-disruptive discharges better than those of the 25-ms schedule (0.736 against 0.648, +0.088 [0.041, 0.133]) and are deeper per episode (+3.6 [0.4, 6.6] points).

Against contextual scoring. The progressive Z-head matches PM-MIL on chain AUROC (0.734) and is earlier and deeper. Its token score reads the whole record, so it says that a discharge disrupts and not which signals at which time carried the evidence. Scored by local risk, its event starts on disruptive discharges carry the risk of random times (0.73 against 0.73), while those of PM-MIL are enriched (0.83 against 0.79, Table 13). The vocabulary shows the same. Built from the same 2,356 candidate events with the same clustering, the local vocabulary gives the 512 PM-MIL chains a Markov chain AUROC of 0.707 against 0.666 for the contextual features of the same models, and leaving one encoder out changes it little (minimum adjusted Rand index 0.83) and the contextual vocabulary strongly (0.40, Table 3). A local score confines the attribution of an event to the channels of its own window (Section 5.4).

W6. Independent value of the chains.

Sample size. Out of fold PM-MIL has 636 event-bearing discharges, 109 of them (17.1%) with two or more families, against five such chains on the evaluation set (Appendix B.3).

What carries the information (Section 5.4, Appendix B.3). Out of fold, event count alone reaches AUROC 0.553 on event-bearing discharges, adding the six family counts gives 0.725 (+0.171 [0.128, 0.214]), and the Markov chain classifier reaches 0.736. With record and event geometry included, family counts still add +0.016 [0.006, 0.028] and directed adjacency adds −0.009 [−0.021, 0.002]. You are right about directed adjacency, and the larger sample confirms it. The ordered classifier is above its shuffled counterpart in every estimate (+0.009 [−0.005, 0.022] out of fold, 0.019 to 0.024 for the three seeds on the evaluation set), but the gain is small and its out-of-fold interval includes zero, so the revision places the information in event presence and family composition.

What the chain view is used for. Order matters in the description. The change from C to D raises the magnetic-fluctuation contribution at the onset by 1.26 [1.02, 1.50] reference standard deviations and the reverse change alters it by −0.40 [−0.73, −0.09], and 39 of the 50 changes from C to D join separate high-risk episodes while all 15 from D to C stay inside one. If the reading of C as the onset of an early MHD instability and of D as a stable mode and its growth holds, a pre-emptive response at C could keep a large island from forming, whereas at D the island has to be suppressed (Section 6). These are observational statements, and the paper does not claim that C causes D.

In sum, given only discharge labels, a model can be made to expose candidate precursors well before the terminal phase, each scored from the signals of its own time window, without a truncation step chosen in advance. Among the compared methods, only the combination of local scoring and evidence-guided masking does all of this at once.

Reviewer_96XYOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper tackles the weakly supervised discovery of Tokamak plasma disruption precursor events under discharge-level labels only. The proposed Progressive-Masking MIL (PM-MIL) framework iteratively erases model-identified discriminative evidence together with its subsequent temporal segments across training rounds, steering later models toward earlier candidate events. These candidates are then organized into event families and event chains via rule-based extraction, cross-model feature representations, and clustering. Experiments on 1,816 discharges from the J-TEXT dataset assess the method across classification performance, event lead time, temporal concentration, and event-chain structure, with comparisons against several MIL aggregators and masking baselines.

Critical strengths
  1. The paper targets fine-grained disruption precursor event discovery under discharge-level labels, extending the research objective from discharge classification to candidate-event identification and analysis of their temporal relationships.
  2. The paper includes ablations across different MIL aggregators, repeated training, fixed truncation, local evidence erasure, and scoring branches, providing empirical evidence for analyzing the role of different design choices within the framework.
Critical weaknesses
  1. One of the paper's main design contributions is adaptively determining the truncation position for subsequent training rounds based on model-identified discriminative evidence. However, Table 2 shows that a fixed truncation strategy not relying on model evidence (M4) achieves comparable results. On the development set, PM-MIL and M4 yield earliest-event lead times of 185 ms and 186 ms, respectively; both report 33% of events within the last 100 ms, and multi-event chain ratios of 23.2% versus 23.5%. These results suggest that progressive truncation and retraining account for the main event-discovery gains, but do not yet demonstrate a clear advantage of adaptively selecting the truncation position over fixed truncation.
  2. The paper extracts candidate events via model risk scores, prediction residuals, and predefined rules, and characterizes their features through attribution analysis on physical diagnostic signals. However, these analyses primarily reveal the signal patterns the model relies on; they do not directly establish the correspondence between candidate events and genuine physical instabilities. Without expert annotations or known physical events as references, the paper also cannot directly assess the accuracy of event detection and temporal localization. Consequently, the existing experiments provide only limited support for the physical validity of candidate events.
  3. Event detection depends on several predefined parameters, including the 1.75σ threshold on prediction residuals, the 0.99/0.90 quantiles for risk scores, and the confirmation condition of three consecutive tokens. These parameters affect event boundaries, the number of events, and ultimately the event-chain structure. Although the paper compares different event representations and some clustering category counts, it does not systematically analyze how these event-extraction parameters influence the main results; the sensitivity of event lead time and event-chain statistics to parameter settings therefore remains unclear.
  4. The paper mainly compares Attention MIL, TimeMIL-style aggregators, and various training variants of the proposed framework. These baselines are useful for analyzing internal design choices, but do not sufficiently cover other weakly supervised temporal localization and anomaly detection strategies. Given that the core task is identifying local discriminative events under sequence-level supervision, comparisons with other relevant methods under unified inputs, supervision conditions, and evaluation protocols would allow a more comprehensive assessment of PM-MIL's relative performance.
Questions for authors
  1. Given that the fixed truncation strategy (M4) performs comparably to PM-MIL on the main event-discovery metrics, could the authors further clarify the specific advantages of evidence-guided masking over fixed truncation, and provide corresponding experimental evidence?
  2. Could the authors further validate the physical meaning of candidate events and the accuracy of their temporal localization through expert annotations or comparative analyses against known physical instability events?
  3. Could the authors analyze how variations in the prediction-residual threshold, risk-score quantiles, and event-confirmation duration affect event lead time, the number of events, and the multi-event chain ratio?
  4. Could the authors supplement comparisons with other weakly supervised temporal localization or anomaly detection methods, reporting results under consistent input features, supervision conditions, and event evaluation protocols?
5 comments
AuthorsOpenReview ↗

Summary of the revision and of our answers to your four questions

Thank you for the four concrete questions. Each led to new work, and the answers are summarized here.

W1/Q1. Evidence-guided masking against fixed truncation (Section 5.5, Table 2). In the outer five-fold cross-validation each of the 1,807 discharges (403 disruptions) is scored only by models and fits from the other folds (Appendix B.3). A method meets requirement 1 if its events lie beyond the terminal phase, 2 if they are informative about disruption, 3 if each is scored from the signals of its own window and 4 if no truncation step is preset.

MethodEarliest-event lead (ms)Events in final 100 msChain AUROCAUPRC, nominal 100 ms removedRequirements met
Single MIL (M1)7559.0%0.5020.6473 only
Attention MIL (R1)2097.7%0.6590.6502 only
Z-head, single model10052.0%0.7140.6082 only
M0-repeat14540.1%0.6470.6941, 3
Fixed schedule (M4, 25 ms)17033.9%0.6480.7821, 3
Z-head + progressive masking20030.0%0.7340.7971, 2, 4
PM-MIL17533.3%0.7360.7581, 2, 3, 4

M4 and PM-MIL differ in requirements 2 and 4, not in lead, concentration or depth, which both owe to progressive shortening with retraining. M4 needs its step chosen in advance, its chain AUROC on the pool ranges from 0.562 to 0.709 over steps of 10 to 100 ms (Appendix C.4), and at 25 ms its chains separate disruptive from non-disruptive discharges less well (+0.088 [0.041, 0.133] in chain AUROC). Attention MIL meets requirement 2 only because its interval is wide (chains on 21% of the disruptive discharges).

W3/Q3. Sensitivity to the event-extraction constants (Appendix C.2). Over 29 settings of the residual threshold, the risk quantiles, the persistence and the ND bag gate, PM-MIL stays +93 to +133 ms earlier than Single MIL with 95% intervals above zero in every setting, shows no detectable difference from M4 in lead and depth, and has a higher chain AUROC than M4 in every setting.

W2/Q2. Physical meaning and timing. An author with expertise in tokamak physics and diagnostics read all six candidate families (Table 5). Fixed rules for three stages of MHD activity put D within 10 ms of the onset of a stable mode in 40 of 43 C-to-D discharges and C before it in all (Appendix A.5). In a blind reading by this author, who saw no model output, the nearest PM-MIL event start lies a median 3.1 ms from the marked onset of a persistent mode (timed in 18 of 40 evaluation discharges), against 32.8 to 72.8 ms for the other methods (Appendix A.6).

W4/Q4. Other weakly supervised and anomaly detection methods (Appendix D). Top-k pooling MIL, R2 with the wavelet positional encoding of TimeMIL and a forecasting anomaly detector trained on ND discharges only were added. As single fits the weakly supervised baselines place their evidence at the end of the record, and the anomaly detector has events on only 47 of the 364 disruptive discharges of the pool. Under the fixed schedule top-k MIL shows that progressive shortening works under another pooling, but its token scores are contextual and its step is preset.

Evaluation. No ensemble weight, threshold or chain classifier is fitted on the evaluation set any more (Section 4, Appendix B.4).

AuthorsOpenReview ↗

Reply to Reviewer 96XY. W1/Q1, evidence-guided masking against fixed truncation

W1 / Q1. Evidence-guided masking against fixed truncation.

You are right that M4 and PM-MIL agree on lead, concentration and depth (Table 1), the gains of the progressive shortening with retraining that both share. The two differ in what has to be known in advance and in what the chains contain.

A preset step. We chose the 25-ms step of M4 using the PM-MIL run, so that twelve rounds, the number the reference run admitted, reach back 275 ms, about the median length of a disruptive record (256 ms, Table 9). We reran M4 with steps of 10, 25, 50 and 100 ms (Appendix C.4, Table 12, one run per step, seed 42). Its chain AUROC on the development pool is 0.562, 0.646, 0.709 and 0.707, against 0.714 for PM-MIL, and the 100-ms step lowers the median lead from 185 to 155 ms. Only 50 ms matches PM-MIL in lead and chain AUROC together, it is not the step that the record lengths suggest, and no rule known in advance picks it. PM-MIL reaches that level without choosing a step, because it places each cutoff at the evidence the previous round used. Small steps also find the same evidence again. With 10-ms and 25-ms steps, 81.5% and 74.9% of the events of later rounds start inside an event that an earlier round had found, against 60.1% for PM-MIL, with 9.7 and 7.2 source events per retained event against 4.1, and each candidate step costs a full training run.

Why a preset step is a physical limitation, not only a tuning issue. The step fixes where the training cutoffs fall, on a grid of one step back from the end of each record, and every model concentrates on the latest evidence its prefix leaves (one MIL model places 70% of its DIS events in the final 100 ms). Two precursors closer than one step can therefore fall between the same pair of cutoffs, so that no model is trained with the earlier one as its latest evidence, and precursors farther apart than one step are found again by several successive models. The spacing that decides this is set by the plasma. It differs between processes. In our data the families read as MHD activity (C and D) start a median 215 and 160 ms before the disruption and those read as impurity events (E and F) 85 and 80 ms (Table 4). It also changes with machine size and operating regime, since the growth of tearing modes is tied to the resistive diffusion time, which grows with the square of the minor radius. The rule-marked stages of MHD activity show it directly. The onset of a stable mode precedes the onset of its growth by a median of 41 ms, with a 10th to 90th percentile range of 10 to 88 ms (Appendix A.5), and the cutoffs of PM-MIL show the same spread, with steps of 17 to 145 ms (median 46 ms), 42% of them outside a factor of two of 50 ms and a median ratio of 3.7 between the largest and the smallest step of one discharge (Appendix C.4). With a 10-ms step the later rounds start 5.1 events per discharge inside the growth stage of the mode, against 0.5 to 1.1 for PM-MIL. The best fixed step on J-TEXT (50 ms) works because it lies near the median spacing, a property of the data that PM-MIL measures as it goes and a schedule must know beforehand. On a new device that number is unknown, and finding it costs a sweep of full training runs judged against the disruption labels.

Information in the chains. Out of fold (403 disruptions, every discharge scored only by models and fits that were not trained on it) the chains of PM-MIL separate disruptive from non-disruptive discharges with AUROC 0.736 against 0.648 for M4 (+0.088 [0.041, 0.133]) and are deeper per episode (+3.6 [0.4, 6.6] points, Appendix B.3). On the pool the margin is +0.098 [0.066, 0.131] with each method's own discharges and shrinks to +0.028 [−0.002, 0.057] on common discharges, so part of it reflects which discharges carry events (Appendix C.3).

AuthorsOpenReview ↗

Reply to Reviewer 96XY. W3/Q3, sensitivity to the event-extraction parameters

W3 / Q3. Sensitivity to the event-extraction parameters. We extracted the events of the frozen models again, without retraining, over a grid of the residual threshold (1.5, 1.75, 2 sigma), the risk quantile pair (0.98/0.85, 0.99/0.90, 0.995/0.95) and the persistence (2, 3, 4 tokens), which gives 27 combinations, plus two settings of the ND bag gate, 2.5% and 10% (Appendix C.2, Table 11). Every new event receives its family from the frozen vocabulary. PM-MIL on the development pool, three-seed means.

SettingEventsLead (ms)Multi-event chains
Default (1.75 sigma, 0.99/0.90, 3 tokens)2,83018523.2%
Threshold 1.5 / 2 sigma3,226 / 2,488189 / 18224.0% / 22.4%
Quantiles 0.98/0.85 and 0.995/0.953,582 / 1,961195 / 16824.2% / 21.7%
Persistence 2 / 4 tokens3,048 / 2,522187 / 18025.4% / 19.4%
Range over the 27 combinations1,510 to 4,260158 to 19816.5% to 27.5%

The absolute values move with the constants, as you expected, and the comparisons of lead and of depth by segments do not. Over the grid PM-MIL minus Single MIL lies between +93 and +133 ms in lead and between +14.4 and +22.2 points in the multi-event share, with both 95% intervals above zero in all 29 settings. PM-MIL minus M4 lies between −8 and +4 ms and between −2.0 and +0.9 points, with intervals that contain zero in all settings, while by distinct families M4 is 1.2 to 2.3 points deeper. The chain AUROC of PM-MIL stays between 0.671 and 0.728 over the 29 settings and is above that of M4 in each of them (by 0.082 to 0.146, point estimates). The grid changes what is extracted from fixed models. The masking rule that places the cutoffs during training uses the same residual threshold (Appendix A.2) and was not varied, since that needs retraining. The token length and the persistence describe the measurements, the time over which a pattern can be resolved on this device, and can be set from its diagnostics, while the risk quantiles and the bag gate are calibrated for each model on ND discharges. A truncation step would instead require the spacing of precursors, which is not known in advance (Appendix C.2).

Reviewer_zneYOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes PM-MIL to find possible tokamak disruption precursors from past records, using only the label of each discharge. It combines local scoring with contextual attention. After each round, it removes the evidence and trains models by processing the full records. Events are grouped into six families and arranged into chains.

Experiments use 1,816 J-TEXT discharges. Compared with one MIL model, PM-MIL finds earlier events and chains with more events. The median lead of the earliest event increases from 60 to 165 ms on the evaluation set. Classification is also less affected when the last observations are removed. The controls suggest that shortening the records and retraining explain the main gains in lead and depth. Event family composition is useful, but the extra benefit of directed adjacency is still unclear.

Critical strengths

S1. The paper studies precursor discovery without labels for event timing. Its shared vocabulary makes it easier to compare events found by separately trained models.

S2. The experiments compare progressive shortening with repeated training and local erasing. These controls help explain where the gains come from.

Critical weaknesses

W1. The benefit over simpler methods is not clear.

The fixed truncation schedule and progressive contextual scoring give similar gains in lead and depth. It is not clear that choosing cutoffs from model evidence or using local scoring improves these measures. Table 5 does show benefits from local representations for the vocabulary. But when the methods are compared on the same evaluation discharges, PM-MIL does not show a clear gain over the fixed schedule in chain classification.

W2. The detected events need more validation.

Finding earlier events does not necessarily mean that their locations are more accurate. There is no limit on the wait for high risk confirmation. The method can also split a period that is already at high risk into several events. These rules may affect the reported effect. The paper needs stronger evidence that the event families match physical events. Also, the evaluation set was checked while the analysis was developed, so it is not an untouched test set.

Questions for authors

Q1. What does PM-MIL add beyond simply shortening the records and retraining several models? Apart from Table 5, is there more evidence that the proposed choices give better events or a better vocabulary?

Q2. What is the distribution of the delay from a residual trigger to high risk confirmation? Do the lead and depth results still hold when this delay is limited? Please also check events that rise from low risk separately from events found within an existing period of high risk.

Q3. Which method and analysis choices were fixed before checking the evaluation set? Is there a separate dataset that was never used during development? Alternatively, can the whole procedure be tested with an outer split, keeping all training, selection and calibration within the training part?

6 comments
AuthorsOpenReview ↗

Summary of the revision and of our answers, starting with your Q2

Thank you for naming the analysis that would change your assessment. We answer Q2 first and then summarize W1, Q1, Q3 and W2.

Q2. Confirmation delay and subdivision. The earlier and deeper chains come neither from long confirmation waits nor from cutting one high-risk stretch into pieces. With the wait from residual trigger to confirmed rise capped at 25 ms, PM-MIL keeps a median earliest lead of 175 ms against 55 ms for Single MIL (+120 [104, 133] ms), and counting only events that rise from low risk gives 153 against 75 ms. Per episode, which first joins segments that touch, 15.1% of the PM-MIL chains hold more than one episode, against 0.8% for Single MIL (Appendix C.1).

W1 and Q1. What the two choices add (Section 5.5, Table 2). You were right for the submitted version. M4 and the contextual Z-head reproduce the gains in lead, concentration and depth, because like PM-MIL both shorten the records progressively and retrain. To measure what the two choices add, seven methods are checked out of fold (1,807 discharges, 403 disruptions, Appendix B.3) against four requirements, events beyond the terminal phase (1), informative about disruption (2), each scored from the signals of its own window (3) and found without a preset truncation step (4).

MethodEarliest-event lead (ms)Events in final 100 msChain AUROCAUPRC, nominal 100 ms removedRequirements met
Single MIL (M1)7559.0%0.5020.6473 only
Attention MIL (R1)2097.7%0.6590.6502 only
Z-head, single model10052.0%0.7140.6082 only
M0-repeat14540.1%0.6470.6941, 3
Fixed schedule (M4, 25 ms)17033.9%0.6480.7821, 3
Z-head + progressive masking20030.0%0.7340.7971, 2, 4
PM-MIL17533.3%0.7360.7581, 2, 3, 4

M4 keeps the local score, but its result depends on the step (chain AUROC 0.562 to 0.709 on the pool over steps of 10 to 100 ms, Appendix C.4), and at 25 ms its chains separate disruptive from non-disruptive discharges less well (+0.088 [0.041, 0.133]). The progressive Z-head needs no step, but its token score reads the whole record, so a high score says that the discharge disrupts, not which signals at that time carried the evidence. Attention MIL meets requirement 2 only because its interval is wide (chains on 21% of the disruptive discharges).

Q3. Evaluation independence. No ensemble weight, threshold or chain classifier is fitted on the evaluation set any more (Section 4), the revision states when each design choice was fixed and on which data (Table 9), and the outer cross-validation of the table above repeats every step on four folds and scores the fifth.

W2. What supports the events. The chains carry information about disruption on discharges that no scoring model or fit was trained on (chain AUROC 0.736 out of fold). An author with expertise in tokamak physics and diagnostics read all six candidate families (Table 5). Rules for the stages of MHD activity put D within 10 ms of the onset of a stable mode in 40 of 43 C-to-D discharges and C before it in all (Appendix A.5), and in a blind reading by this author the nearest PM-MIL event start lies a median 3.1 ms from the marked onset of a persistent mode (timed in 18 of 40 evaluation discharges), against 32.8 to 72.8 ms for the other methods (Appendix A.6).

AuthorsOpenReview ↗

Reply to Reviewer zneY. W1 and Q1, the preset step

Your summary of the paper is accurate.

Evidence-guided cutoff against a preset step. M4 needs a step and, with a cap on rounds, a horizon. We chose 25 ms using the PM-MIL run, so that twelve rounds, the number the reference run admitted, reach back 275 ms, about the median length of a disruptive record (Table 9). We reran M4 with four steps (Appendix C.4, Table 12, development pool, one run per step, seed 42).

10 ms25 ms50 ms100 msPM-MIL
Median earliest lead (ms)180185175155185
Events in final 100 ms32.7%32.8%34.1%40.7%32.2%
Chain AUROC0.5620.6460.7090.7070.714
Source events per retained event9.77.24.03.14.1

On the pool PM-MIL reaches the level of the best step without choosing one. The step that the record lengths suggest, 25 ms, gives 0.646 (0.616 as three-seed mean). The best step on the pool is 50 ms, and no rule known in advance picks it. The 100-ms step loses lead, and small steps find the same evidence again (81.5% and 74.9% of later-round events start inside an event of an earlier round, against 60.1% for PM-MIL).

Why a preset step is a physical limitation, not only a tuning issue. The step fixes where the training cutoffs fall, on a grid of one step back from the end of each record, and every model concentrates on the latest evidence its prefix leaves (one MIL model places 70% of its DIS events in the final 100 ms). Two precursors closer than one step can therefore fall between the same pair of cutoffs, so that no model is trained with the earlier one as its latest evidence, and precursors farther apart than one step are found again by several successive models. The spacing that decides this is set by the plasma. It differs between processes. In our data the families read as MHD activity (C and D) start a median 215 and 160 ms before the disruption and those read as impurity events (E and F) 85 and 80 ms (Table 4). It also changes with machine size and operating regime, since the growth of tearing modes is tied to the resistive diffusion time, which grows with the square of the minor radius. The rule-marked stages of MHD activity show it directly. The onset of a stable mode precedes the onset of its growth by a median of 41 ms, with a 10th to 90th percentile range of 10 to 88 ms (Appendix A.5), and the cutoffs of PM-MIL show the same spread, with steps of 17 to 145 ms (median 46 ms), 42% of them outside a factor of two of 50 ms and a median ratio of 3.7 between the largest and the smallest step of one discharge (Appendix C.4). With a 10-ms step the later rounds start 5.1 events per discharge inside the growth stage of the mode, against 0.5 to 1.1 for PM-MIL. The best fixed step on J-TEXT (50 ms) works because it lies near the median spacing, a property of the data that PM-MIL measures as it goes and a schedule must know beforehand. On a new device that number is unknown, and finding it costs a sweep of full training runs judged against the disruption labels.

AuthorsOpenReview ↗

Reply to Reviewer zneY. W1 and Q1, chain information and local scoring

Matched evaluation cohorts. On the matched evaluation cohort of the reference pair the difference is unchanged at −0.011 [−0.078, 0.061] (54 discharges, Appendix B.2). Averaged over the three seed pairs and fitted in the same way it is +0.063 [0.005, 0.131] (Appendix C.3), and +0.065 [0.009, 0.130] with each classifier fitted on all pool chains of its method (Appendix B.1). The evaluation-set chain AUROC reported in the submission (Table 1) changed in two ways. It had been cross-fitted inside the evaluation discharges of each method and is now fitted on the pool, and the PM-MIL entry is now a seed mean like the others (0.79 against 0.68, and 0.71 against 0.67 for the runs of seed 42, where the submitted reference-run value was 0.63). With 39 disruptions the evaluation set cannot settle this comparison, so it rests on the outer cross-validation, 0.736 against 0.648, +0.088 [0.041, 0.133]. On the pool the margin is +0.098 [0.066, 0.131] with each method's own discharges and +0.104 [0.068, 0.140] at an equal number of ND discharges with events, and it shrinks to +0.028 [−0.002, 0.057] on common discharges (Appendix C.3).

Local against contextual scoring. Under progressive masking the Z-head gives earlier and deeper chains with the same chain AUROC (0.734 against 0.736 out of fold). Its token score reads the whole record. On disruptive discharges its contextual risk is above the upper level in 95% of random windows, as at its event starts, so the score does not single out a time. Scored by local risk, its event starts on disruptive discharges carry the risk of random times (0.73 against 0.73), while those of PM-MIL are enriched (0.83 against 0.79). On ND discharges both are enriched, the Z-head events less (0.30 against 0.25) than those of PM-MIL (0.57 against 0.44, Table 13). A local score confines the attribution of an event to the channels of its own window, so that the attribution at event onsets can be compared across discharges (Section 5.4). Under either head the start of an event comes from the contextual residual (Section 6).

Vocabulary utility (Q1). Built from the same 2,356 candidate events and scored on the 512 PM-MIL chains, vocabularies from raw features (chain AUROC 0.612 and 0.622), attribution profiles (0.632) and the contextual features of the same models (0.666) all stay below the local vocabulary (0.707). It is also the stable one. Leaving one encoder out changes it little (minimum adjusted Rand index 0.83) and the contextual vocabulary strongly (0.40, Table 3).

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.