acceptodds
Under review as a conference paper at ICLR 2027

GALACTIC: Global and Local Counterfactuals for Time-Series Classification

Abstract

Time-series classifiers inform decisions from clinical diagnosis to industrial monitoring, yet the usual explanations, saliency over timesteps and discriminative subsequences, indicate what supported a prediction rather than what would change it. Counterfactual explanations address this, but existing methods for time-series classifiers describe individual predictions and leave open how recurring temporal changes connect classes across a population. Summarizing many local counterfactuals is the natural route to a population-level answer, and we show why it falls short: an objective asking only for the smallest change to one series drives that change toward the decision boundary, leaving no margin for similar series, and selecting among such changes cannot add a margin the objective never asked for. We introduce GALACTIC, the first unified local-to-global counterfactual framework for differentiable multiclass time-series classifiers. At the local level, GALACTIC-L asks each counterfactual to be as small as possible while crossing the decision boundary by a margin shared across the structural subgroup of its series. At the global level, GALACTIC-G selects rather than generates: it accepts the perturbations of any local generator and keeps a small reusable set for each directed class transition. It expresses coverage, sparsity, and perturbation magnitude in one unit, bits, under a Minimum Description Length criterion, so no weights between them are set, and greedy selection carries the guarantee under stated conditions. Across ten UCR datasets, eight local generators, and six adapted global methods, the perturbations of GALACTIC-L cover on average more of each class transition, at a shorter description length, than those of any competing generator.

Primary area:
interpretability and explainable AI
Keywords:
Counterfactual Explanations, Time-Series Classification, Explainability, Global Explanations

Reviews

Among:
Shape:
1234this paper, Top 15%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.80 over 5 reviews, above 85% of the others.
Reviewer_ciSROpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper proposes GALACTIC, a method that explains time series classifiers with counterfactuals at two levels. GALACTIC-L creates a counterfactual for each series and pushes it clearly past the decision boundary, so that the same change can also work for similar series. GALACTIC-G then picks a few changes per class pair that flip as many series as possible, using an MDL score. On 10 UCR datasets and 3 classifiers, GALACTIC covers more series with shorter explanations than 8 local and 6 global baselines.

Critical strengths
  1. Clear motivation. The paper explains why a small counterfactual change for one series may not work for other series. Its margin requirement directly addresses this issue and improves the results of all tested generators (Table 1).
  2. Thorough evaluation. The experiments include 14 baselines, 3 classifiers, 3 seeds, and sensitivity studies. A held-out test (Table 8) shows that the method still performs better when selection and scoring use different series.
  3. Flexible global selection. GALACTIC-G can select explanations from any local generator, making it useful beyond GALACTIC-L. Its MDL formulation balances coverage and explanation complexity without manually choosing weights between them.
Critical weaknesses
  1. The main metric is the method's own objective. GALACTIC optimizes MDL and is also judged mainly by MDL. This MDL score charges very little for large changes, and GALACTIC makes larger changes than the closest competitor, CELS (16.7% vs 9.8% of timesteps changed). The paper should show that the gain is not just from making bigger changes.
  2. The changes may be unrealistic. The paper does not measure plausibility. In Figures 5 and 6, the counterfactuals contain sharp spikes, and some changes (about +4 and -6) are larger than the series itself (about -2 to 3). Such changes may exploit the classifier rather than explain it. Also, most global baselines are tabular methods that barely work on time series, so the global comparison is weak. The theory in the appendix is correct but simple, and the main guarantee relies on an assumption (Assumption 1) that is never checked.
Questions for authors
  1. If the baselines' changes are scaled to the same size, does GALACTIC still win?
  2. Can you report a plausibility measure for all methods?
  3. How do the results change with a different MDL design (e.g., a linear instead of log cost for change size)?
  4. Why do the changes in Figures 5 and 6 contain spikes, and how large are the changes compared to the series?
  5. Does Assumption 1 hold in your experiments?
  6. How does the method work when the target is not the second most likely class?
2 comments

Thank you for the positive assessment.

W1 - Q1: The original comparison controlled action count, not edit size. We now match the ℓ2\ell_2 budget per transition across all nine local pools and rerun selection (Table 12). Galactic-L remains first (Eff 0.8640.864 vs. 0.7830.783), with the second-smallest original median ℓ2\ell_2 norm. It edits more timesteps than CELS (16.7%16.7\% vs. 9.8%9.8\%), but has higher Eff (0.9240.924 vs. 0.7930.793). Both are non-dominated on Eff, flipping cost and edit extent (Table 17). Sparsity is a trade-off.

W2 - Q2, Q4: Tables 15-16 now measure nearest-target distance, warped-distance shape and edit amplitude. Galactic-G has distance 0.1690.169 of the dataset range vs. 0.1830.183 for GLANCE and 0.2090.209 for GLOBE-CE. GLANCE has better warped-distance shape (0.8530.853 vs. 0.9050.905), but lower Eff (0.6570.657 vs. 0.9240.924). FACEGroup is nearer by construction, returning observed target-class series, but has Eff 0.7570.757 and 0.6020.602. Figure 6 shows edits with frequent sign changes on SyntheticControl. Galactic-L has the smallest local peak change (3.0%3.0\% of the value range vs. up to 7.7%7.7\%, Table 16). Figure 7 also illustrates smoother edits on Coffee and transition-focused edits on Trace. The revised Ethics Statement clarifies that these diagnostics do not guarantee physical plausibility.

W2 - Q5: We revised the proof (Appendix A): MDL reduction is unconditionally submodular, while the (1−1/e)(1-1/e) bound holds under the stated monotonicity condition for ∣S∣<2μ|S|<2\mu (Proposition 4). Of 115,918115{,}918 examined candidates, 52,25052{,}250 covered nothing new and would violate this condition, yet all 2,0842{,}084 accepted additions satisfied it. Greedy accepts only MDL-reducing additions and yields comparable MDL to Flat Optimal across 8686 pools (p=0.59p=0.59, Sec.5.2). Baseline comparisons use exact Flat Optimal MILP selection.

W2, Global baselines: Since existing global methods target tabular data, we added two controls. Applying the same global selector to eight published time-series generators yields Galactic-L Eff 0.9240.924 at 2.292.29 bits vs. 0.7930.793 at 3.953.95 for CELS (Table 19). A gradient-based universal perturbation per transition, under a matched ℓ2\ell_2 budget but without support selection or a reference margin, achieves 0.8940.894 at 3.313.31 bits and edits all timesteps, vs. our 0.9240.924 at 2.292.29 bits with 16.7%16.7\% edited (Appendix C.6). Table 18 and Figure 11 additionally assess global methods independently of MDL.

Q3: We reran selection under the alternative linear penalty cδ=∥δ∥0+∥δ∥1c_\delta=\|\delta\|_0+\|\delta\|_1, rather than merely rescoring solutions (Table 14). CELS then has lower cost (8.0118.011 vs. 10.69210.692), but our Eff is higher (0.9270.927 vs. 0.8330.833). The other seven sources cost 23.223.2-182.5182.5. We clarify in the Abstract that coding choices define an exchange rate even without tuned coefficients. The linear penalty is not a description length in bits, unlike the logarithmic code-length surrogate (Appendix A).

Q6: Runner-up targeting is a default, not a restriction on t≠yt\neq y. Across 6,4216{,}421 instances per arm, random alternative targets succeed in 82.9%82.9\% of attempts vs. 89.0%89.0\% for runner-up targets, with TwoPatterns and SyntheticControl hardest (33.4%33.4\% and 33.9%33.9\%, Appendix C.1.6). Mean ℓ2\ell_2 among successes is 0.5650.565 vs. 0.6460.646, but these are different successful subsets, so this does not establish that alternative targets require smaller edits.

Thank you for the detailed comments, which address most of my concerns. I will keep my score.

Reviewer_1TiCOpenReview ↗
Rating
3(+1)
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes GALACTIC, a unified local-to-global counterfactual framework for differentiable multiclass time-series classifiers. At the local level, GALACTIC-L generates one counterfactual per series through a two-phase procedure, first finding a valid counterfactual and then refining the solution to satisfy a reference margin beyond the decision boundary, so that the perturbation is more likely to transfer. At the global level, GALACTIC-G selects, for each directed class transition group, a small set of perturbations from a candidate pool produced by local generators, using an MDL-inspired objective that expresses coverage, sparsity, and magnitude in bits. GALACTIC is evaluated on 10 UCR datasets against eight local generators and six adapted global baselines, and across three classifier architectures. Results show that GALACTIC-L provides the best candidate pool for global selection in coverage and description length and GALACTIC-G achieves the highest effectiveness and shortest description.

Critical strengths

S1: This paper addresses a real problem and introduces a reasonable modular design. The authors identify a real gap that existing global counterfactual methods are designed for tabular features and binary outcomes, and none targets multiclass time-series classification. The framework separates a local generator from a global selector, and the selector is generator-agnostic which can be reused with any local methods. This work sets a useful foundation for future research on global explanations for time series classifiers.

S2: The four RQs and the experimental design in Section 5 provide a well-controlled evaluation protocol. RQ1 tests ablations of internal design, RQ2 benchmarks local generators under a fixed global selector, RQ3 benchmarks GALACTIC-G against adapted global baselines, and RQ4 evaluates robustness across classifier architectures. Table 3 further shows an interesting result that while GALACTIC-L mostly draws as a local method, it wins most in the local-under-global setting, which justifies the reusability and also isolates the contribution of local generator's candidates to global selection.

Critical weaknesses

W1: The global benchmarking results in Table 2 use MDL as a metric, which favours GALACTIC-G, and all other metrics barely show an advantage of GALACTIC-G. MDL is the optimization objective of GALACTIC-G and is also used as an evaluation metric, while none of the global baselines optimizes it, so GALACTIC-G's MDL wins are largely expected. On all other metrics presented in Table2, the advantage is limited: the Win/Draw/Lose record is 4/2/4 for effectiveness, and GALACTIC-G loses on most datasets for the remaining metrics. Thus, Table 2 fails to justify that GALACTIC-G outperforms the global baselines. This is a major flaw because GALACTIC-G is one of the two main methodological contributions.

W2: A few key components of the pipeline that determine the reusability results are unspecified or not analyzed in the main text. (i) The structural subgrouping procedure in Section 4.1 is unclear to me (e.g., how the segment layouts and gradients are turned into clustering features and how K is chosen). It's unclear how subgrouping contributes to the results. (ii) $ \mu=4$ is used throughout, but the relative magnitude of μ\mu and ∣Xy→t∣|X_{y\rightarrow t}| is unspecified, even though it directly affects the difficulty of coverage. (iii) The 10 UCR datasets listed in Table 4 vary significantly in data size and number of classes, which leads to very different number of series per directed class transition, and thus very different coverage difficulty. How the method behaves across such heterogeneous datasets is unclear, since the results are mostly reported as averages over datasets.

Questions for authors

Major How is the clustering done exactly to form the subgroups? How many subgroups are typically formed and how large is each subgroup (especially on small datasets)? Is the method robust to different subgrouping strategies? Why is a fixed μ=4\mu = 4 used for all cases, rather than a budget that scales with ∣Xy→t∣|X_{y\rightarrow t}|? How does the relative magnitude of the two numbers affect coverage results?

Suggestions Since reusability is the central claim, the headline results in the main text should present held-out performances, rather than leaving that in Appendix. (Future work) The authors may extend the current method to multivariate time series setting, and also consider cases where the target class is not only the runner-up.

3 comments

Thank you for the comments.

W1: The 4/2/44/2/4 record compares against the best competitor per dataset, not one baseline. Independently of MDL, Table 18 and Figure 11 compare all six global baselines on Eff, flipping cost and edit extent: Galactic-G is undominated and dominates three. GLANCE edits fewer timesteps but has lower Eff (0.6570.657 vs. 0.9240.924). On jointly covered instances, our flipping cost is lower (0.1260.126 vs. 0.1430.143, Table 20). At matched coverage, our local model code is 4.814.81 bits shorter (Table 13). Table 5 examines three architectures and three seeds.

W2(i) - Q: KK-medoids groups each source class using equally weighted standardized segment-gap and temporally pooled gradient features, with KK chosen by silhouette over {2,…,6}\{2,\ldots,6\} (Sec.4.1, Appendix B.6). Among 4444 source classes, 3434 choose K=2K=2. Subgroup sizes span 11-13631363 (median 1212), with six singletons among 105105 subgroups. Singletons remain valid because each series is optimized individually. We clarify that subgrouping is an internal support-sharing device, not an independent Eff gain: alternative groupings yield Eff 0.9230.923-0.9280.928 (Table 11). Phase 1 alone reaches 0.6940.694 (Table 1).

W2(ii) - Q: μ=4\mu=4 is a cap, not a requirement: the exact selector uses 1.851.85 actions on average and all four in only 24%24\% of transitions. Appendix C.1.2 tests μ=1\mu=1-88. Transition sizes span 11-63966396, but coverage increases with size (Spearman ρ=0.61\rho=0.61), averaging 0.9470.947 in the largest quartile vs. 0.2010.201 in the smallest. Small transitions have fewer reusable candidates and use just 0.290.29 actions on average, so the cap is not binding there.

W2(iii) - Suggestion: Table 3 now reports held-out transfer with disjoint selection and evaluation halves (Eff 0.8930.893 vs. 0.7580.758 for CELS). The appendix reports per-dataset results, and datasets are equally weighted. Multivariate inputs remain future work. Although runner-up targets are the default, any t≠yt\neq y is supported: alternative targets succeed in 82.9%82.9\% of attempts vs. 89.0%89.0\% for runner-up targets under matched conditions (Appendix C.1.6). This does not imply equal target difficulty.

Reviewer_1TiCOpenReview ↗

Concerns were mostly addressed. Raising score to weak accept.

Thank you for the response and the revised manuscript. My concerns about the subgrouping and the action budget are resolved by the clarifications. Regarding W1, the additional evidence (Table 18 and Figure 11) shows that GALACTIC-G achieves competitive trade-offs among Avg Eff, Afc, and Act. Given that the central contribution is the improved transferability of GALACTIC-L's counterfactuals, which RQ2 and the held-out results support well, I consider the current RQ3 results sufficient supporting evidence for the proposed framework. I would still encourage a minor wording adjustment of Contribution 5 and Conclusion. The current phrasing gives the impression of overall superiority. I suggest making these claims more precise and acknowledging the competitive trade-offs. I am raising my score to 3 (weak acceptance).

Thank you for the positive assessment. We have updated Contribution 5 and the Conclusion as suggested.

Reviewer_ccJZOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper introduce GALACTIC, a novel way to explain how a time-series classifier make its choice. GALACTIC-L use the group information and the shared target margins to make local counterfactuals, so they can works better on other samples.

At the same time, GALACTIC-G use a Minimum Description Length goal to pick a small and reusable set of changes for each classes change.

Critical strengths
  1. This paper address the important challenge of building counterfactual explanations for time-series classifiers, and it put local generation and global selection together into one framework.
  2. The experiments include many local and global baseline methods, different classifier types, and transfer tests on the independent test sets, which gives strong proofs that the new approach work well.
Critical weaknesses

W1. GALACTIC-L combine subgroup clustering, importance-guided masking, shared target margins, and two-phase optimization. But the experiments does not clearly separate if the better cross-instance transferability come from the subgroup-aware optimization or just from using a more larger target margins. A margin-only baseline without the subgroup information would helps show that the proposed structural mechanism is need.

W2. The method could generate some synthetic time series and they could not exist in the real world.

Questions for authors
  1. Can the authors compare GALACTIC-L against a margin-only baseline without subgroup clustering?

  2. Could authors clarify the meaning of synthetic time series?

1 comment

Thank you for the comments.

W1 - Q1: Under identical global selection, with the same reference margin γt\gamma_t throughout, Galactic-L reaches Eff 0.9240.924 with default subgrouping vs.\ 0.9260.926 without it, the margin-only baseline requested. Alternative groupings reach 0.9230.923-0.9280.928 (Table 11). Phase 1 alone reaches 0.6940.694 (Table 1), indicating that refinement, not subgrouping, drives transfer. This is not a fixed-support ablation, since Phase 2 can expand and prune supports. Subgroups organize shared candidate supports and provide an intermediate explanation level (Figure 6, Sec.3), rather than independently improving average Eff or MDL.

W2 - Q2: A synthetic series is a generated counterfactual x⃗cf=x⃗+δ⃗\vec x^{cf}=\vec x+\vec\delta that changes the classifier's prediction, not a validated real-world intervention. Masks, total variation, and pruning encourage localized edits but cannot guarantee physical plausibility. Table 15 now reports nearest-target distance alongside Eff: Galactic-G achieves 0.1690.169 of the dataset range vs. 0.1830.183 for GLANCE and 0.2090.209 for GLOBE-CE. FACEGroup is nearer by construction but has lower coverage. We revised the Ethics Statement to distinguish classifier validity, localization, and physical realism.

Reviewer_poTEOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper introduces GALACTIC, the first unified local-to-global counterfactual framework for differentiable multiclass time-series classifiers. At the local level, GALACTICL asks each counterfactual to be as small as possible while crossing the decision boundary by a margin shared across the structural subgroup of its series. At the global level, GALACTIC-G selects rather than generates: it accepts the perturbations of any local generator and keeps a small reusable set for each directed class transition. It expresses coverage, sparsity, and perturbation magnitude in one unit, bits, under a Minimum Description Length criterion, so no weights between them are set, and greedy selection carries the (1 − 1/e) guarantee under stated conditions. Across ten UCR datasets, eight local generators, and six adapted global methods, the perturbations of GALACTIC-L cover on average more of each class transition, at a shorter description length, than those of any competing generator.

Critical strengths

The paper addresses an important and relatively underexplored problem: understanding cluster transitions in time-series data through both instance-level and cluster-level counterfactual explanations. Integrating temporal segmentation, subgroup-aware importance estimation, constrained perturbation generation, and representative global explanation selection is a meaningful methodological contribution. This is consequential because existing approaches primarily emphasize supervised time-series classification or isolated explanations, whereas GALACTIC attempts to provide consistent explanations across multiple levels of granularity.

The MDL-based formulation provides an interesting mechanism for balancing explanation coverage and complexity without manually selecting trade-off weights. The accompanying analysis of supermodularity and greedy approximation, together with the empirical comparison of optimal, greedy, and hierarchical strategies, strengthens the methodological contribution. The experimental evaluations across 10 UCR datasets also provide encouraging evidence that structurally constrained perturbations can yield more localized explanations than several adapted baselines.

Critical weaknesses

A major limitation is that the experimental evaluation treats the ground-truth class labels of UCR datasets as cluster assignments rather than applying actual unsupervised clustering algorithms. Consequently, the experiments primarily evaluate counterfactual explanations of predefined class partitions through a supervised surrogate classifier, rather than explanations of clusters discovered by unsupervised algorithms. Furthermore, counterfactual validity is measured using surrogate predictions, without sufficiently establishing whether the perturbations correspond to meaningful transitions under the original clustering mechanism. This substantially weakens the central claim of a model-agnostic framework for unsupervised time-series clustering. Validation using multiple clustering algorithms, different cluster structures, and surrogate-fidelity assessments is essential to establish the proposed framework's applicability.

The claimed greedy approximation guarantee depends on assumptions about admissible additions and the monotonicity of the MDL reduction. However, the theoretical discussion does not sufficiently clarify whether these conditions hold for arbitrary candidate sets and throughout the actual greedy selection procedure. In particular, the admissible-addition condition does not establish unconditional monotonicity over the entire power set, making the scope of the reported approximation guarantee important to clarify. Additionally, the proposed structural constraints restrict perturbations to important temporal regions but do not directly guarantee temporal plausibility or consistency with the data distribution. The evaluation emphasizes effectiveness, sparsity, and proximity without adequately assessing whether the generated counterfactuals are realistic, physically meaningful, or supported by the target-cluster distribution. These issues affect both theoretical reliability and practical interpretability.

Questions for authors

The following questions are intended to clarify issues that could materially change the assessment.

  1. Validation under actual unsupervised clustering: Why are UCR ground-truth class labels used directly as cluster partitions instead of applying unsupervised clustering algorithms? Can the authors provide experiments using representative methods such as k-means, k-medoids, DTW-based clustering, or spectral clustering to demonstrate that GALACTIC generalizes to genuinely discovered cluster structures?

  2. Surrogate fidelity and counterfactual validity: How sensitive are the generated counterfactuals to the accuracy and calibration of the surrogate classifier, particularly near cluster boundaries? Can the authors quantify whether counterfactual transitions predicted by the surrogate are consistent with the original clustering model or an appropriate out-of-sample cluster-assignment mechanism?

  3. Theoretical approximation guarantees: Can the authors clarify whether the admissible-addition and monotonicity assumptions required for the (1−1/e)(1-1/e) approximation guarantee are satisfied throughout the greedy optimization process? How does the guarantee change when candidate perturbations have overlapping coverage or when some additions fail to cover previously uncovered instances?

  4. Counterfactual plausibility and structural preservation: Beyond L2 proximity and the number of modified timesteps or segments, how do the authors evaluate whether generated counterfactuals preserve realistic temporal dependencies and correspond to plausible target-cluster patterns? Can additional quantitative evaluation, such as nearest-target-manifold distance or temporal-shape similarity, support the claimed structural preservation?

1 comment

Thank you for the positive assessment.

W1 - Q1: Galactic explains supervised time-series classifiers, not clustering (Sec. 3). Effectiveness is evaluated against the explained classifier, not cluster assignments. We clarified that KK-medoids is an internal grouping device for shared segmentation and candidate supports (Sec.~4.1, Appendix B.6). γt\gamma_t is transition-specific. A classifier trained on cluster assignments would instead introduce a surrogate with a potentially different boundary. Replacing KK-medoids with spectral clustering or DTW kk-means yields no significant differences (p≥0.49p\geq0.49, Appendix C.2).

Q2: No surrogate is used: counterfactuals are checked against the explained classifier. We do not claim invariance to probability recalibration, since Phase 1 uses a probability threshold and Phase 2 a logit margin. Refinement reduces perturbations within 10%10\% of the reference margin from 90.7%90.7\% to 19.1%19.1\% (Table 2). Table 5 examines three architectures and three seeds.

W2 - Q3: We corrected the theoretical conditions (Appendix A). MDL reduction is unconditionally submodular, while the (1−1/e)(1-1/e) guarantee requires monotonicity for ∣S∣<2μ|S|<2\mu, including unions with an optimal set (Proposition 4). Of 115,918115{,}918 examined candidates, 52,25052{,}250 covered nothing new and would violate this condition, yet all 2,0842{,}084 accepted additions satisfied it (Remark 1). Empirically, greedy matches the exact optimum on 4141 of 8686 pools, with no significant MDL difference (p=0.59p=0.59, Sec.5.2). Baseline comparisons use exact Flat Optimal MILP selection.

Q4: Temporal masking does not guarantee realistic dynamics. We now make this limitation explicit (Ethics Statement). Table 15 reports nearest-target distance: Galactic-G achieves 0.1690.169 vs.\ 0.1830.183 for GLANCE and 0.2090.209 for GLOBE-CE. FACEGroup is nearer by construction. GLANCE has better warped-distance shape (0.8530.853 vs.\ 0.9050.905), but lower Eff (0.6570.657 vs.\ 0.9240.924). These are empirical proxies, not physical-plausibility guarantees.

Reviewer_TkkdOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

GALACTIC explains univariate time-series classifiers at two levels. GALACTIC-L computes a counterfactual towards the runner-up class with a masked gradient search (Phase 1), then drives it to a reference margin γt\gamma_t, the median margin of the target class (Phase 2). GALACTIC-G selects at most μ\mu perturbations per directed class transition from any candidate pool by minimizing an MDL code, exactly with a MILP by default. On ten UCR datasets, the paper reports higher average coverage and a shorter description length than eight local generators and six adapted tabular global methods.

Critical strengths
  1. A clear problem and a modular design. The motivation that per-instance minimal counterfactuals leave no margin for reuse is well explained (Proposition 1), and separating generation (GALACTIC-L) from a generator-agnostic selector (GALACTIC-G) is practical: both the selector and Phase 2 apply to any candidate pool, and Phase 2 improves both effectiveness and MDL for all eight other generators (Table 1). The resulting explanation can be read at four levels, from one counterfactual to the coverage of every directed transition (Fig. 6).
  2. The main design choices are ablated. RQ1 compares the default doubling-support mask with four single-signal masks and an unconstrained control (Fig. 2): among the masks, only the default reaches full effectiveness, and it edits the smallest fraction of the series and runs about an order of magnitude faster. Fig. 3 crosses five selectors with all nine candidate pools, and the selectors keep the same order on every pool, which separates the contribution of the pool from that of the selector.
  3. An extensive and transparent evaluation. Eight local baselines and six adapted global baselines start from the authors' released code and run under the same wall-clock budget; the conclusions are repeated for three architectures and three seeds (Table 3); the coverage lead survives a held-out protocol that selects on one half of the population and scores on the other (Table 8); sensitivity analyses cover μ\mu, the pointer width, the choice of perturbed series and the κ\kappa/second-attempt settings (Figs. 7–8, Tables 6–7); and the paper reports the axes on which GALACTIC loses (Tables 2, 9, 10) and checks the win/draw/loss rows under several effectiveness floors (Table 5). Code, checkpoints and per-table scripts are released.
  4. Selection is exact, and both stages are efficient. Flat Optimal solves the MDL problem as a MILP to proven optimality at the reported scales (App. B.8), so the main selection results do not depend on the greedy heuristic or its approximation guarantee; Flat Greedy shows no significant difference (p=0.59p=0.59) and is exactly optimal on 41 of the 86 pools (Sec. 5.2). Generating and selecting the GALACTIC-L pool takes 229 s on average, against 434–6443 s for the other sources (Table 10), and the runtime of every variant is reported (Fig. 7, App. C.1.1).
Critical weaknesses
  1. MDL mostly restates coverage. The code charges psz=1p_{sz}=1 bit per covered series and M=max⁡δcδ+2pszM=\max_\delta c_\delta+2p_{sz} per uncovered one, while a perturbation's code grows only logarithmically with its norms. Across the 86 rows of Table 12, MDL≈15.8−14.8⋅Eff\mathrm{MDL}\approx 15.8-14.8\cdot\mathrm{Eff} (R2=0.96R^2=0.96, my fit), and the MDL ranking of the nine sources in Table 10 equals their Eff ranking. The headline result that GALACTIC-L covers more of each transition at a shorter description length (Abstract) is therefore one result, not two. The claim that no trade-off weights are set is also overstated: App. C.2 itself says the objective "must supply an exchange rate", and pszp_{sz}, MM and the logarithmic code fix that rate in favor of coverage.
  2. The central claim, that a shared margin makes perturbations reusable, is not tested in isolation. The paper credits the margin for the lead (Sec. 5.3, App. C.2), but it reports no margin statistics for the pools of the other eight generators, although App. A says Sec. 5.2 measures where their counterfactuals land. It also reports no global results for the Phase-1-only pool of GALACTIC-L (Table 1 shows "–"). Phase 2 does not close the gap for other generators (CELS reaches 0.841 vs. 0.924), so the margin alone does not appear to explain the lead. The lead comes mostly from the five multiclass datasets (Eff 0.851 vs. 0.610 for CELS); on the five binary datasets it is small (0.997 vs. 0.991 for MASCOTS; my calculation from Table 12).
  3. Coverage is bought with larger edits whose plausibility is not measured. When the best single perturbation covers less than 90% of a transition's series, Phase 2 raises γt\gamma_t up to the 99th percentile (App. B.6), so the generator already uses the evaluation criterion. GALACTIC-L loses to the best other source on Afc and Act on 8 and 9 of 10 datasets (Table 10). The examples look like spikes, not coherent changes: the perturbations selected in Fig. 6(c) jump in both directions by about 2–6 on z-normalized series, and Phase 2 adds a sharp spike in Fig. 5. This conflicts with the requirement of small, localized changes (Sec. 1). No plausibility measure is used (Ethics Statement), and Afc averages over covered series only, so it cannot compare sources with different coverage directly. The summaries may therefore be larger, universal-perturbation-like edits rather than more informative ones, yet no gradient-based universal perturbation per transition is tested (the closest baseline, GLOBE-CE⋆, samples random directions per source class).
Questions for authors
  1. What margins do the pools of all nine sources reach, and what are the global results for the Phase-1-only pool, without the direction term gˉ\bar{g}, and without raising γt\gamma_t? Were the fixed settings in Sec. 4.1 and App. B.6 (λi\lambda_i, κ\kappa, ρ\rho, the total-variation weight, the 80% pruning level) tuned on the ten evaluation datasets?
  2. How does coverage compare at matched edit budgets, e.g., with every source scaled to the same ℓ2\ell_2 norm? How does it compare with a per-transition universal perturbation, and under a plausibility measure such as the distance to the nearest target-class series?
  3. How is MM computed in Tables 12 and 13? The same Flat Optimal selection has different MDL values (mean 2.29 in Table 10 vs. 2.5 in Table 2; SwedishLeaf: 5.965 vs. 7.1).

I would raise my rating to 3 if the lead holds under Questions 1 and 2.

3 comments

Thank you for the comments.

W1 - Q3: Eff and MDL are related through the outlier penalty but remain distinct. At matched coverage, Galactic-L reduces model cost by 4.814.81 bits on average (paired p=2×10−5p=2\times10^{-5}, Table 13), outperforming all eight local baselines in mean cost. Rankings remain unchanged for pointer widths of 11, 22, and 6464 bits (Table 9). We clarify in the Abstract that MDL avoids tuned trade-off coefficients, not coding-dependent trade-offs (Table 14). The MDL discrepancy comes from the pool, not the selection: the local pool and the global pool differ, so the outlier cost MM, the max cost over the pool, differs between them (Appendix B.9).

W2 - Q1: Phase 2 raises transfer to other series from 0.1950.195 to 0.5310.531 and reduces near-boundary perturbations from 0.9070.907 to 0.1910.191 (Table 2, all nine pools). Under global selection, Eff rises from 0.6940.694 to 0.9240.924, while MDL falls from 5.715.71 to 2.292.29 bits (Table 1). We clarify in Sec.~5.2 that this isolates refinement as a whole, not fixed-support margin optimization, since Phase 2 can expand and prune supports (Algorithm 1). Refining CELS improves its Eff to only 0.8410.841. Removing the direction term slightly raises Eff from 0.9210.921 to 0.9260.926 (Appendix C.1.1). The advantage is larger on multiclass datasets (0.8510.851 vs.\ 0.6100.610 for CELS) than on binary ones (0.9970.997 vs.\ 0.9910.991 for MASCOTS, Appendix C.5).

W3 - Q2: To address escalation bias, Table 3 now evaluates held-out transitions, retaining the lead (0.8930.893 vs.\ 0.7580.758 for CELS). Disabling escalation reduces Eff from 0.9270.927 to 0.9150.915 under Full and from 0.8840.884 to 0.8730.873 under a separate held-out protocol (Appendix C.2). Escalation helps but does not explain the lead.

Our edits affect 16.7%16.7\% of timesteps vs.\ 9.8%9.8\% for CELS, but achieve higher Eff (0.9240.924 vs.\ 0.7930.793, Table 1). At matched ℓ2\ell_2 budgets, Galactic-L remains ahead (0.8640.864 vs.\ 0.7830.783, Table 12). On Eff, flipping cost and edit extent, only Galactic-L and CELS are non-dominated (Table 17). Afc does average over covered series only, which is why coverage is matched before comparing cost (Table 13) and the cost columns are recomputed on jointly covered instances (Table 20). We now separate classifier validity and localization from physical plausibility (Ethics Statement, Tables 15-16). For the spikes in Figures 5-6, Galactic-L has the smallest peak amplitude among local generators (3.0%3.0\% of the value range, Table 16). Figure 7 illustrates other patterns. GLOBE-CE's budget, applied per transition, moves Avg. Eff from 0.5460.546 to 0.5470.547, up to 5.67×5.67\times more directions on SwedishLeaf (Appendix B.3). A per-transition universal perturbation, gradient-fitted at an ℓ2\ell_2 budget matched to our median edit size with no support selection or reference margin, reaches 0.8940.894 at 3.313.31 bits editing every timestep, against 0.9240.924 at 2.292.29 bits at 16.7%16.7\% (Appendix C.6).

Q1, tuning protocol: Appendix C.1.1 now documents screening 1111 (ρ,κ)(\rho,\kappa) combinations on three development datasets and ten additional settings on two. The other seven were excluded from tuning. Separately, varying the Phase-2 total-variation weight τ∈{0,0.25,1}\tau\in\{0,0.25,1\} against the default 0.50.5 across all ten datasets yields Eff 0.9240.924-0.9290.929. The full 5×55\times5 (ρ,κ)(\rho,\kappa) grid yields 0.9170.917-0.9320.932 (Table 8). The original setting scores 0.9210.921, just 0.0110.011 below the best, and was retained without reselection.

Reviewer_TkkdOpenReview ↗

Response to the authors' rebuttal

Thank you for the response. The new experiments address my main concern that the coverage lead might come from larger edits, escalation or tuning, so I am raising my rating to 3. Two issues remain and should be stated in the main text. First, the evidence supports the Phase-2 refinement as a whole, not the shared margin specifically: margins do not rank the generators (Table 2) and removing the subgroups leaves effectiveness unchanged (Table 11), yet Sec. 5.3 still credits margin-consistent candidates. Second, at matched coverage, the description length of GALACTIC-L is significantly shorter than that of only three of the eight sources, not including CELS, the strongest baseline (Table 13), so the MDL advantage over CELS follows from the higher coverage rather than being a separate result.

Thank you for the positive assessment and for the two notes. Sec. 5.3 now attributes the global lead to Phase-2 refinement. Sec. 5.2 and Appendix C.3 now state that, at matched coverage, our model code is significantly shorter than three of the eight sources and not against CELS, so the MDL lead over CELS follows from coverage.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.