acceptodds
Under review as a conference paper at ICLR 2027

RealFlux: A Counterfactual Explanation Framework for Molecular Graphs Inspired by Local Optimization in Medicinal Chemistry

Abstract

To address out-of-distribution (OOD) bias and topological leakage in graph masking, together with the decoupling of localization and generation in the evaluated Counterfactual Masking (CM) pipeline, we propose RealFlux, a unified molecular graph counterfactual explanation framework inspired by local optimization in medicinal chemistry. RealFlux introduces native directional utility (Native-\(U\)) for forward-utility-based localization without an external gradient explainer and combines Context-Locked Enforcement (CLE) with In-Distribution Gating (IDG) for constrained local intervention and predictor-representation validation. Across three pharmacological classification tasks, RealFlux achieves 97.7%–98.3% chemical validity and significantly higher classification flip rate than the evaluated CM baselines in all six pairwise comparisons after Holm correction (BSR, 0.286–0.680). On the common-substructure masking benchmark, the combined prediction discrepancy reaches \(\lvert\Deltay\rvert=0.583\), with the same relative advantage across GIN, GraphSAGE, and GAT. Median per-molecule inference latency is 0.86 s, more than an order of magnitude lower than that of the evaluated 3D diffusion baseline. Overall, RealFlux provides a directionally controllable and computationally efficient framework for model-guided molecular counterfactual analysis.

Primary area:
interpretability and explainable AI
Keywords:
Explainable AI (XAI), Counterfactual Explanations, Graph Neural Networks (GNNs), AI for Drug Discovery

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_bDSgOpenReview ↗
Rating
2(+1)
Confidence
2
Informativeness
1
Presentation
2
Soundness
2(+1)
Read review
Paper summary

This paper introduces RealFlux, a molecular counterfactual explanation framework that performs local chemical editing toward a desired prediction direction. It combines (i) Native-U for forward-utility-based site localization, (ii) CLE for preserving the non-edited molecular context, and (iii) IDG for filtering candidates that are distant from the predictor's training distribution. Localization and subsequent editing share the same directional utility.

Critical strengths

None

Critical weaknesses
  1. Eq. (3) averages utility over each region's feasible edits, making rankings sensitive to action-set composition and potentially obscuring rare but highly effective edits. The paper does not justify why mean utility is an appropriate criterion for locating successful counterfactual edits.

  2. Native-U scores regions independently using first-step edits. Hence, it likely fails to capture cases where individually weak edits become effective only when multiple sites are modified jointly.

  3. The novelty appears limited. CLE resembles existing scaffold-constrained/local editing mechanisms, while IDG is a relatively standard nearest-neighbor representation-space OOD filter.

  4. IDG's benefit is lower distribution distance (W) in the same representation space used for gating. w/o IDG yields higher mean targeted success and validity and lower leakage in Table 4, while worsening W. This does not establish that IDG improves counterfactual quality.

  5. Scalability with respect to action-space size is unclear. Native-U evaluates first-step candidates across all regions, but the paper reports only latency, without candidate counts, predictor-evaluation counts, or scaling with molecular or action-space size. Therefore, efficiency beyond the tested setup remains unestablished.

Questions for authors
  1. What is the definition of Context Retention? CLE is described as a hard invariant that preserves the non-edited molecular context, yet the reported CR is only 0.654 for RealFlux (Full). If CR measures preservation of the locked context, this value seems difficult to reconcile with the hard-invariant statement.

  2. The paper gives the high-level definitions of Native-U, CLE, and IDG, but many details remain unclear to me. For example, how is the action library constructed, how many candidates are generated per region, how does multi-step local refinement proceed, what are the stopping criteria, and how exactly is the “unified selector” implemented?

8 comments

Response to Reviewer bDSg

Thank you for your rigorous and insightful review. Below, we address all of your technical concerns and questions in four structured themes, supported by new held-out edit-split experiments, 3-seed per-seed ablation breakdowns, formal metric reconciliations, and scalability statistics.


Theme 1: Justification of Mean Utility in Eq. (3) and Handling Multi-Site Synergistic Edits (Weaknesses 3.2 & 3.3)

  1. Why Mean Utility Outperforms Max Utility for Region Localization (Response to Weakness 3.2):

    Thank you for raising this fundamental question. In discrete medicinal chemistry optimization, a region's suitability as an intervention site depends on whether its local chemical neighborhood supports generalizable directional editability across subsequent refinement and feasibility filters (CLE, IDG, SA≤4.5\mathrm{SA}\le 4.5), rather than a single fragile outlier edit. Ranking regions by max⁡c∈Cfirst(r)u(c)\max_{c\in\mathcal{C}_{\mathrm{first}}(r)} u(c) suffers from the winner's curse: a region with one extreme mutation but mostly deleterious edits will be selected, only to fail when that single mutation violates downstream manifold/SA gates or requires second-step refinement.

    To test this rigorously without in-sample bias, we randomly split each candidate region's first-step edits Cfirst(r)\mathcal{C}_{\mathrm{first}}(r) into a selection half AA and a held-out validation half BB (55 molecules ×\times 3 seeds, 20 random 50/50 splits per molecule). We selected the top region r∗r^{\ast} on AA using Mean (Eq. 3) vs. Max (max⁡c∈Au(c)\max_{c\in A} u(c)) and evaluated the chosen region on the unseen edits in BB:

Region Ranking RuleHeld-Out Utility on BB ↑\uparrowHeld-Out Flip Rate on BB ↑\uparrowTop-1 Region Stability Across Splits ↑\uparrow
Mean Utility (Eq. 3, Native-UU)0.7710.2130.846
Max Utility (max⁡cu(c)\max_{c} u(c))0.6930.2000.812
Δ\Delta (Mean −- Max)+0.078$^\dagger$+0.013$^\ddagger$+0.034
Per-Seed Δ\Delta (Seed 5 / 15 / 25)+0.082 / +0.056 / +0.094+0.007 / +0.017 / +0.015—

(Notes: †^\daggerAll three per-seed 95% bootstrap CIs exclude zero: Seed 5 +0.082[0.007,0.188]+0.082\\ [0.007, 0.188], Seed 15 +0.056[0.012,0.103]+0.056\\ [0.012, 0.103], Seed 25 +0.094[0.023,0.184]+0.094\\ [0.023, 0.184]. ‡^\ddaggerAll three per-seed 95% CIs include zero: +0.007[−0.008,0.022],+0.017[−0.001,0.037],+0.015[−0.001,0.036]+0.007\\ [-0.008, 0.022], +0.017\\ [-0.001, 0.037], +0.015\\ [-0.001, 0.036].)

As shown above, Mean utility achieves significantly higher held-out utility (0.7710.771 vs. 0.6930.693, 95% CIs >0>0 in 3/3 seeds) and higher Top-1 region stability (0.8460.846 vs. 0.8120.812) while matching or slightly exceeding Max on held-out flip rate (0.2130.213 vs. 0.2000.200). Furthermore, in E3 (Table 6), Native-UU achieves 0.7740.774 sign consistency and ρ=0.458\rho=0.458 rank correlation with observed CReM fragment interventions (vs. 0.6420.642 and 0.0640.064 for Grad-CAM).

  1. First-Step Regional Probing vs. Multi-Step Synergistic Refinement (Response to Weakness 3.3):

    Enumerating joint multi-site combinations during initial site localization would incur O(∣A∣k)\mathcal{O}(|\mathcal{A}|^k) combinatorial explosion. Instead, RealFlux decouples O(1)\mathcal{O}(1)-batch regional localization from multi-step / multi-anchor synergistic intervention:

    • Multi-Step Coupled Refinement (H≤2H\le 2): After Native-UU selects r∗r^{\ast}, Step-2 local refinement applies coupled secondary modifications (e.g., combining a step-1 functional transformation with a step-2 calibrated substituent attachment or terminal pruning), retaining candidates where the combined edit achieves u(c)>0u(c)>0.
    • Multi-Anchor & Multi-Site Empirical Validation: Across E2 Multiple Anchors (667 pairs), E4 L2 Multi-Anchor linking (60 tasks), and Cases B/C (Fig. 2), RealFlux effectively executes coordinated multi-point modifications:
Multi-Site / Multi-Anchor BenchmarkMetricCM (CReM)CM (DiffLinker)RealFlux (Ours)
E2 Multiple Anchors (667 pairs, Tab. 3)Prediction Discrepancy ∣Δy^∣\lvert\Delta\hat{y}\rvert ↓\downarrow / Validity ↑\uparrow1.519 / 0.5381.101 / 0.5910.168 / 0.900 (600 pairs)
E4 L2 Multi-Anchor Stress (Tab. 7)Targeted Success ↑\uparrow / Context Retention ↑\uparrow0.233 / 0.8000.333 / 0.7420.583 / 0.833
Case B (CID 3233055, Fig. 2)Coordinated carbonyl reduction + pyrazine chlorinationy^=0.652\hat{y}=0.652 (Fail)y^=0.638\hat{y}=0.638 (Fail)y^=0.249\hat{y}=0.249, QED=0.831\mathrm{QED}=0.831
Case C (CID 3240659, Fig. 2)Synergistic dual-carbonyl reduction on diketopiperaziney^=0.528\hat{y}=0.528 (Fail)y^=0.597\hat{y}=0.597 (Fail)y^=0.286\hat{y}=0.286, QED=0.594\mathrm{QED}=0.594

Theme 2: Methodological Novelty of Closed-Loop Integration and IDG Manifold Guardrail Trade-offs (Weaknesses 3.4 & 3.5)

  1. Methodological Novelty of the Unified Closed-Loop Framework (Response to Weakness 3.4): Existing molecular counterfactual pipelines decouple post-hoc attribution (e.g., Grad-CAM) from predictor-agnostic candidate generation (e.g., SELFIES string mutation in MMACE, database lookup in CReM, or 3D diffusion in DiffLinker). RealFlux's core methodological innovation is reformulating molecular counterfactual explanation as a training-free, closed-loop constrained search that unifies site localization (Native-U), topological invariance (CLE), and representation-manifold gating (IDG) over the same executable action space and directional objective. Moreover, CLE and IDG introduce critical domain-specific advances beyond standard heuristics:

    • CLE vs. Standard Scaffold Protection: Standard tools typically freeze a static Bemis–Murcko ring core or require manual atom masks. CLE enforces exact subgraph isomorphism and aromaticity invariance (G′[VC]≅C(G,r)∧Arom⁡(G′[VC])=Arom⁡(C)\mathcal{G}'[\mathcal{V}_{\mathcal{C}}]\cong\mathcal{C}(\mathcal{G},r) \wedge \operatorname{Arom}(\mathcal{G}'[\mathcal{V}_{\mathcal{C}}])=\operatorname{Arom}(\mathcal{C})) dynamically across arbitrary single-anchor (L1), multi-anchor linker (L2), and full-ring replacement (L3) complements, preventing topological leakage through message passing.
    • IDG as an Active Generation-Time Constraint: Whereas prior XAI benchmarks (e.g., CM, Janisiów et al. 2026) compute embedding distances only as a post-hoc diagnostic, IDG enforces predictor-manifold proximity during candidate generation using the same forward pass as Native-U, preventing the search from drifting into out-of-distribution (OOD) space.
  2. Why IDG is Essential & Non-Circular Despite Nominal Flip-Rate Trade-offs (Response to Weakness 3.5):

    • Preventing Adversarial OOD Shortcuts: GNN classifiers are highly vulnerable to OOD structural distortions that cross the decision boundary in extrapolated regions of latent space. As shown in the 3-seed per-seed breakdown below, removing IDG (w/o IDG) expands the unfiltered pool from 8,978 to 9,425 candidates, allowing the optimizer to exploit OOD adversarial shortcuts that slightly inflate nominal Targeted Success (0.622$\to$ 0.644) and Validity (0.979$\to$ 0.991) while catastrophically degrading latent manifold distance W by +90.3% (2.529$\to$ 4.812, worsening by +1.87, +2.31, and +2.66 across Seeds 5, 15, and 25). In XAI, a counterfactual that flips the model only by leaving the training manifold is an adversarial artifact rather than a valid explanation; IDG serves as the essential guardrail that prevents this failure mode.
    • No Circularity Between IDG Gating and W Evaluation: IDG applies a local pointwise 90$\text{th}−percentilenearest−neighborthreshold(-percentile nearest-neighbor threshold (\tau_{\mathrm{IDG}})calibratedonthe∗validation∗splitrelativeto∗training∗embeddings,whereasWmeasuresthe∗global1−Wasserstein(EarthMover′s)distance∗betweenthegeneratedpopulationandtheunseen∗test∗splitdistribution() calibrated on the *validation* split relative to *training* embeddings, whereas W measures the *global 1-Wasserstein (Earth Mover's) distance* between the generated population and the unseen *test* split distribution (n_{\min}=1,032$).
Ablation BranchSeedE4 Context Retention ↑\uparrowE4 Targeted Success (TS) ↑\uparrowE2 Prediction Leakage ∣Δy^∣↓\lvert\Delta\hat{y}\rvert\downarrowE2 Validity ↑\uparrowE2 Representation Distance W ↓\downarrow
RealFlux (Full)5 / 15 / 250.743 / 0.627 / 0.5930.717 / 0.550 / 0.6000.253 / 0.242 / 0.2740.975 / 0.977 / 0.9732.442 / 2.388 / 2.756
Mean ±\pm SD0.654 ±\pm 0.0640.622 ±\pm 0.0700.256 ±\pm 0.0130.975 ±\pm 0.0022.529 ±\pm 0.162
w/o CLE5 / 15 / 250.340 / 0.168 / 0.1920.350 / 0.167 / 0.2330.196 / 0.170 / 0.1860.959 / 0.975 / 0.9403.299 / 3.287 / 2.789
Mean ±\pm SD0.233 ±\pm 0.076 (3/3 ↓\downarrow)0.250 ±\pm 0.076 (3/3 ↓\downarrow)0.184 ±\pm 0.0110.958 ±\pm 0.0143.125 ±\pm 0.238 (3/3 ↑\uparrow)
w/o IDG5 / 15 / 250.747 / 0.640 / 0.6070.733 / 0.583 / 0.6170.203 / 0.186 / 0.2040.981 / 0.983 / 0.9864.315 / 4.700 / 5.421
Mean ±\pm SD0.664 ±\pm 0.0600.644 ±\pm 0.0640.197 ±\pm 0.0080.983 ±\pm 0.0024.812 ±\pm 0.458 (+90.3% ↑\uparrow)

Theme 3: Formal Definition of Context Retention (CR) and Reconciliation with the CLE Hard Invariant (Question 3.7)

We appreciate the opportunity to clarify this metric. The apparent tension between CR=0.654\mathrm{CR}=0.654 and the CLE hard invariant (Eq. 5) stems from the distinction between a candidate-level generation filter (CLE) and a task-level full-molecule evaluation metric (CR\mathrm{CR}). We have added the explicit definition of CR\mathrm{CR} to Section 4 and Appendix A.3:

CR(G,G′)=∣VC∣∣V∣⋅I![G′[VC]≅C(G,r);∧;Arom⁡!(G′[VC])=Arom⁡!(C(G,r))]\mathrm{CR}(\mathcal{G}, \mathcal{G}') = \frac{|\mathcal{V}_{\mathcal{C}}|}{|\mathcal{V}|} \cdot \mathbb{I}\\!\left[\mathcal{G}'[\mathcal{V}_{\mathcal{C}}] \cong \mathcal{C}(\mathcal{G}, r) \\;\wedge\\; \operatorname{Arom}\\!\big(\mathcal{G}'[\mathcal{V}_{\mathcal{C}}]\big) = \operatorname{Arom}\\!\big(\mathcal{C}(\mathcal{G}, r)\big)\right]

where G=(V,E)\mathcal{G}=(\mathcal{V}, \mathcal{E}) is the parent molecule, rr is the target region, C(G,r)=G∖r\mathcal{C}(\mathcal{G}, r)=\mathcal{G}\setminus r is the locked context with heavy-atom node set VC=V∖Vr\mathcal{V}_{\mathcal{C}} = \mathcal{V}\setminus\mathcal{V}_r, and I[⋅]\mathbb{I}[\cdot] is the exact subgraph-isomorphism and aromaticity-invariance indicator. If a task yields no valid candidate within the budget, CR=0.0\mathrm{CR}=0.0 is recorded in the task denominator.

Why CR=0.654±0.064\mathrm{CR}=0.654\pm 0.064 is completely consistent with 100% CLE enforcement:

  1. Candidate-Level Hard Invariant (I[⋅]=1\mathbb{I}[\cdot]=1) vs. Whole-Molecule Atom Ratio (∣VC∣/∣V∣<1|\mathcal{V}_{\mathcal{C}}|/|\mathcal{V}|<1): During generation, any candidate violating G′[VC]≅C\mathcal{G}'[\mathcal{V}_{\mathcal{C}}]\cong\mathcal{C} is immediately rejected by CLE; hence 100% of generated RealFlux candidates satisfy I[⋅]=1\mathbb{I}[\cdot]=1. However, because the target region rr is replaced or edited (∣Vr∣>0|\mathcal{V}_r|>0), the preserved context accounts for ∣VC∣/∣V∣=1−∣Vr∣/∣V∣<1.0|\mathcal{V}_{\mathcal{C}}|/|\mathcal{V}| = 1 - |\mathcal{V}_r|/|\mathcal{V}| < 1.0 of the parent molecule's heavy atoms.

  2. Stratification Across Topological Levels (Table 7): In L1 (single substituent, ∣Vr∣/∣V∣≈17|\mathcal{V}_r|/|\mathcal{V}|\approx 17\\%) and L2 (multi-anchor linker), RealFlux achieves CR=0.825\mathrm{CR}=0.825 and 0.833. In L3 (full-ring replacement, where the replaced ring system comprises ≈41\approx 41\\% of parent heavy atoms), the remaining context fraction is naturally CR=0.588\mathrm{CR}=0.588.

  3. Macro-Averaging & Ablation Proof (Table 4): Macro-averaging across all 60 tasks (20 each in L1, L2, L3, including 3–83\text{--}8 no-output tasks per seed counted as 0.0) across Seeds 5, 15, and 25 yields 0.743, 0.627, and 0.593, giving 0.654±0.064\mathbf{0.654 \pm 0.064}. Removing CLE (w/o CLE) allows edits to invade the locked scaffold (I[⋅]=0\mathbb{I}[\cdot]=0), causing CR\mathrm{CR} to drop by -0.421 to 0.233±0.076\mathbf{0.233 \pm 0.076}:

Breakdown DimensionSetting / SeedRealFlux (Full) CR ↑\uparroww/o CLE CR ↑\uparrowCM (CReM) CR ↑\uparrowCM (DiffLinker) CR ↑\uparrowRealFlux (Full) TS ↑\uparroww/o CLE TS ↑\uparrow
By Topological LevelL1 (Single-anchor, 20 tasks)0.825—0.6000.7420.700—
(Table 7, 3-seed mean)L2 (Multi-anchor, 20 tasks)0.833—0.8000.7420.583—
L3 (Full-ring, 20 tasks)0.588—0.3500.6580.583—
By Random SeedSeed 5 (60 tasks, L1–L3)0.7430.340——0.7170.350
(Table 4, 60 tasks/seed)Seed 15 (60 tasks, L1–L3)0.6270.168——0.5500.167
Seed 25 (60 tasks, L1–L3)0.5930.192——0.6000.233
Macro SummaryMean ±\pm SD (3 seeds)0.654 ±\pm 0.0640.233 ±\pm 0.076——0.622 ±\pm 0.0700.250 ±\pm 0.076
Reviewer_ramZOpenReview ↗
Rating
3(+1)
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper introduces a framework for generating molecular counterfactual explanations through constrained, predictor-guided chemical modifications. Its main contributions are directional utility-based localization, preservation of molecular scaffolds and in-distribution filtering of generated molecules. The paper is generally well written, clearly motivated, and supported by extensive experiments and ablation studies. Validation covers three pharmacological classification datasets and an additional molecular masking benchmark, with comparisons against established methods such as CReM and DiffLinker.

However, comparisons against simpler predictor-guided optimization baselines are missing, making it difficult to isolate the methodological contribution. Furthermore, no experimental biological validation or independent assessment of the generated molecules' biological activity is provided, limiting the practical significance of the reported improvements.

Critical strengths
  1. The proposed methodology is conceptually straightforward and does not rely on unnecessarily complex architectures or specialized generative models. By combining directional optimization with chemically constrained molecular modifications, the authors provide an interpretable and computationally efficient approach that can be applied to different GNN predictors.

  2. The authors evaluate their approach on three pharmacological classification datasets and an additional molecular masking benchmark, comparing against established counterfactual generation methods based on CReM and DiffLinker. They also include ablation studies and experiments across different GNN architectures, providing useful evidence of the method's effectiveness and robustness within the evaluated settings.

Critical weaknesses
  1. The proposed method evaluates counterfactual quality primarily through changes in the predictions of the same GNN used to guide molecular modifications. However, these changes do not necessarily correspond to meaningful differences in the molecules' actual biological properties. While the authors provide qualitative examples, there is no experimental validation or systematic independent assessment of the generated counterfactuals. This limits the extent to which the proposed framework can be considered a biologically meaningful explanation method rather than a model-guided optimization procedure.

  2. Although RealFlux achieves substantial improvements over the evaluated baselines, these comparisons do not fully isolate the contribution of the proposed framework. In particular, RealFlux incorporates directional predictor feedback during generation, whereas the evaluated CReM and DiffLinker pipelines do not use equivalent guidance. It would be valuable to compare against simpler chemically constrained optimization strategies using the same directional objective and computational budget. Without such comparisons, it remains unclear whether the reported improvements justify the additional methodological components.

Questions for authors
  1. Have the authors considered evaluating whether the generated counterfactuals correspond to experimentally observed changes in molecular activity? For instance, public bioactivity datasets such as ChEMBL or experimentally characterized matched molecular pairs could provide an opportunity to assess whether the predicted directional changes agree with measured activity differences, rather than relying exclusively on the optimized GNN's predictions.

  2. Have the authors considered extending the experimental comparison to other molecular counterfactual explanation methods, such as MMACE (Wellawatte et al., 2022) or MEG (Numeroso & Bacciu, 2021)? Additionally, a simpler baseline based on local molecular edits guided by the same directional utility, with comparable chemical constraints and computational budgets, could help clarify whether the reported improvements arise from the specific RealFlux framework or from predictor-guided optimization more generally.

6 comments

Response to Reviewer ramZ

Dear Reviewer ramZ,

We sincerely appreciate your constructive assessment of our manuscript. We are grateful that you highlighted our framework's conceptual clarity, interpretability, and computational efficiency without requiring specialized generative retraining (Strength 2.2), as well as the breadth of our evaluation across three pharmacological classification benchmarks, the common-substructure masking benchmark, ablations, and three GNN architectures (Strength 2.3). Below, we address your two core questions regarding independent biological/experimental validation (Summary 2.1a, Weakness 2.4 & Question 2.6) and controlled baseline comparisons (Summary 2.1b, Weakness 2.5 & Question 2.7).


Theme 1: Biological Relevance, Cross-Architecture Independence, and Alignment with Experimental ChEMBL/MMP Data (Summary 2.1, Weakness 2.4 & Question 2.6)

  1. Fidelity in Counterfactual XAI vs. Independent Assessment: In post-hoc explainability, the primary objective is explanation fidelity—faithfully characterizing what the target GNN has learned rather than substituting for wet-lab synthesis. Optimizing directly against an external oracle would detach the explanation from the target model's decision mechanism. Nevertheless, to verify that RealFlux captures genuine biochemical structure–activity relationships (SAR) rather than single-GNN adversarial artifacts, we evaluate our counterfactuals across three independent validation axes:
    • Axis 1 — Orthogonal Cross-Architecture Transfer (Table 5): We evaluate counterfactuals across independently trained GIN, GraphSAGE, and GAT models. If RealFlux merely exploited adversarial quirks of a single GIN, prediction discrepancies ∣Δy^∣|\Delta\hat{y}| and latent manifold distances W would degrade on other architectures. Instead, RealFlux achieves the lowest combined ∣Δy^∣|\Delta\hat{y}| across all three architectures (0.583, 0.524, and 0.557; all Holm-corrected p<0.0001 with zero rank reversals across the 18-test family) and consistently outperforms DiffLinker in representation distance W.
    • Axis 2 — Quantitative Alignment with Experimental AqSolDB/PubChem MMPs and ChEMBL Mutations (Tables 3, 5 & 6):
      • Experimental Superstructure MMPs (E2): The 881 pairs in E2 are experimentally characterized matched molecular pairs (MMPs) extracted from AqSolDB (experimental aqueous solubility) and PubChem sharing an identical core. RealFlux reduces paired prediction discrepancy to ∣Δy^∣=0.253|\Delta\hat{y}|=0.253 (single-anchor) and 0.168 (multi-anchor), while keeping generated candidates at W=2.333—nearly indistinguishable from the experimental test set (W=2.245).
      • ChEMBL Fragment Interventions (E3): Using property shifts across ChEMBL-attested CReM fragment replacements as reference, Native-U achieves sign consistency of 0.774 (GIN), 0.596 (GraphSAGE), and 0.593 (GAT), and marginal rank correlation ρ=0.458–0.510\rho=\mathbf{0.458\text{--}0.510} (vs. 0.064–0.1570.064\text{--}0.157 for Grad-CAM).
ArchitectureMethodE2 Single Anchor ∣Δy^∣↓\lvert\Delta\hat{y}\rvert\downarrowE2 Multi Anchor ∣Δy^∣↓\lvert\Delta\hat{y}\rvert\downarrowE2 Combined ∣Δy^∣↓\lvert\Delta\hat{y}\rvert\downarrowE2 Single Anchor W↓W\downarrowE3 Mean Consistency [Lower CI] ↑\uparrowE3 Marginal Rank ρ\rho [Lower CI] ↑\uparrow
GINFeature Zeroing / Grad-CAM*2.9192.1472.91446.8380.6420.064
CM (CReM)1.7351.5192.4802.414——
CM (DiffLinker)1.1841.1011.8806.620——
RealFlux (Native-U)0.2530.1680.5832.3330.774 [0.737]0.458 [0.308]
GraphSAGEFeature Zeroing / Grad-CAM*0.8870.9961.1924.4120.5510.130
CM (CReM)1.6811.4612.4253.515——
CM (DiffLinker)1.1400.8891.5148.113——
RealFlux (Native-U)0.2230.1450.5243.6460.596 [0.551]0.480 [0.328]
GATFeature Zeroing / Grad-CAM*1.1311.0371.1996.1610.5510.157
CM (CReM)1.7441.4902.4884.623——
CM (DiffLinker)1.1850.8751.4408.953——
RealFlux (Native-U)0.2300.1570.5574.8010.593 [0.547]0.510 [0.361]

(Note: In row 1 of each architecture, E2 columns report Feature Zeroing and E3 columns report Grad-CAM.)

  • Axis 3 — Alignment with Experimentally Confirmed CYP3A4 Bioactivity & Wet-Lab SAR (Section 5.4, Fig. 2): All three case-study leads are experimentally confirmed CYP3A4 inhibitors from public PubChem/ChEMBL bioassays. RealFlux's edits directly mirror wet-lab medicinal chemistry de-risking rules while preserving QED:
Case & PubChem CIDExperimentally Documented Liability & Literature SAROriginal (y^\hat{y} / QED)RealFlux Edit (y^\hat{y} ↓\downarrow / QED ↑\uparrow)CM (CReM) (y^\hat{y} / QED)CM (DiffLinker) (y^\hat{y} / QED)
Case A (CID 667700)3,4-Dimethoxyphenyl oxidative O-demethylation soft spot →\to phenolic hydroxyl conversion blocks CYP metabolic activation0.817 / 0.8620.402 / 0.859 (Scaffold preserved)0.924 / — (Fails to flip)0.886 / 0.481 (Breaks benzene ring)
Case B (CID 3233055)Electron-rich pyrazine ring oxidation →\to carbonyl reduction + electron-withdrawing chlorination passivates ring0.809 / 0.8350.249 / 0.831 (Spiro core preserved)0.652 / — (Distal edit fails)0.638 / 0.786 (Opens spiro ring)
Case C (CID 3240659)Rigid 2,5-diketopiperazine + heme-coordinating imidazole →\to carbonyl reduction modulates steric/electronic heme binding0.874 / 0.5680.286 / 0.594 (-67.367.3\\% y^\hat{y} drop)0.528 / — (Alkyl chain only)0.597 / 0.291 (Cleaves core ring)

Theme 2: Isolating Methodological Contributions, Controlled Local-Edit Baselines, and Comparison with MMACE/MEG (Summary 2.1, Weakness 2.5 & Question 2.7)

  1. Isolating RealFlux Components Against Simpler Predictor-Guided Local-Edit Baselines (Table 4): To verify whether our gains stem from the specific RealFlux design rather than generic predictor guidance, we evaluated four controlled predictor-guided baselines sharing the exact same discrete edit space, directional objective u(c), and 2-batch query budget (across Seeds 5, 15, and 25):
    • w/o Native-U (Simpler Gradient-Guided Local Optimization + Identical CLE/IDG & u(c) Refinement): Uses the exact same local edit library, CLE/IDG constraints, and directional refinement (u(c)>0), replacing only Native-U site probing with Grad-CAM. Targeted Success (TS) drops by -14.4 pp\text{pp} (0.622 →\to 0.478), action consistency drops from 0.748 to 0.672, and P50 latency increases 3.0$\times$ (0.862 s\text{s} →\to 2.557 s\text{s}).
    • Max-Utility Local Search (Greedy Single-Edit Ranking max⁡cu(c)\max_c u(c) in the Same Edit Space): Across 20 random 50/50 held-out edit splits per molecule (55 molecules ×\times 3 seeds), ranking sites by max⁡cu(c)\max_c u(c) reduces held-out utility from 0.771 to 0.693 (Δ=+0.078\Delta = +0.078, 95% CIs >0 on all 3 seeds) and reduces Top-1 stability from 0.846 to 0.812.
    • w/o CLE (Unconstrained Directional Local Search): Without CLE, Context Retention collapses from 0.654 to 0.233 and TS drops to 0.250, proving that naive predictor-guided search destroys core scaffolds.
    • w/o IDG (Directional Local Search Without Manifold Gating): Without IDG, latent Wasserstein distance W degrades by +90.3% (2.529 →\to 4.812), showing that ungated optimization exploits out-of-distribution (OOD) shortcuts.
Controlled Predictor-Guided Variant (3 Seeds)Context Retention ↑\uparrowTargeted Success (TS) ↑\uparrowPrediction Leakage ∣Δy^∣↓\lvert\Delta\hat{y}\rvert\downarrowManifold Distance W ↓\downarrowAction Consistency ↑\uparrowValidity ↑\uparrowP50 Latency (s) ↓\downarrow
RealFlux (Full Pipeline)0.654 ±\pm 0.0640.622 ±\pm 0.0700.256 ±\pm 0.0132.529 ±\pm 0.1620.748 ±\pm 0.0680.979 ±\pm 0.0030.862 ±\pm 0.005
w/o Native-U (Grad-CAM + u(c) Search)0.658 ±\pm 0.0570.478 ±\pm 0.1130.238 ±\pm 0.0142.148 ±\pm 0.1860.672 ±\pm 0.0490.977 ±\pm 0.0042.557 ±\pm 0.069
w/o CLE (Unconstrained u(c) Search)0.233 ±\pm 0.0760.250 ±\pm 0.0760.184 ±\pm 0.0113.125 ±\pm 0.2380.727 ±\pm 0.0650.979 ±\pm 0.0030.950 ±\pm 0.184
w/o IDG (Ungated u(c) Search)0.664 ±\pm 0.0600.644 ±\pm 0.0640.197 ±\pm 0.0084.812 ±\pm 0.4580.748 ±\pm 0.0680.991 ±\pm 0.0050.813 ±\pm 0.139
  1. Comparison with MMACE (Wellawatte et al., 2022), MEG (Numeroso & Bacciu, 2021), and GNNExplainer:
    • Mechanistic Comparison: MMACE enumerates STONED mutations over SELFIES strings without a hard subgraph-isomorphism scaffold lock (CLE) or predictor-manifold gate (IDG); as shown by Janisiów et al. (AAAI 2026, Fig. 3), unconstrained SELFIES edits frequently introduce chemically strained or implausible motifs (e.g., triple bonds inside rings). MEG applies multi-step reinforcement learning over raw atom/bond perturbations without fragment-level valence libraries, CLE subgraph locking, or IDG gating, requiring extensive per-molecule RL rollouts.
    • Quantitative Benchmark Comparison: In the foundational CM study (Janisiów et al., AAAI 2026, Table 2), MMACE and GNNExplainer (subgraph perturbation) were benchmarked on the exact same three TDC datasets (CYP3A4, CYP2D6, hERG) and identical 3-layer GIN architecture (hidden 512, dropout 0.3). We summarize the unified comparison below (added to Appendix A.2):
Reviewer_RsmYOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

RealFlux generates molecular counterfactuals through local chemical edits. It selects an intervention region using the mean directional prediction change of feasible first-step edits (Native-U), preserves the complementary molecular context through CLE, and rejects candidates far from training molecules in the predictor's embedding space through IDG. Experiments cover three ADMET classification tasks, an AqSolDB common-substructure masking benchmark, three GNN architectures, and component/topology ablations. The reported results show substantially higher valid-candidate flip rates than the evaluated CM pipelines, alongside trade-offs in similarity, diversity, and synthetic accessibility.

Critical strengths
  • The empirical improvements over the evaluated pipelines are substantial, especially on CYP3A4/CYP2D6/hERG versus CM (DiffLinker).
  • The ablation supports the usefulness of actionable site selection: replacing Native-U with Grad-CAM reduces targeted success substantially.
  • Cross-architecture masking results and the explicit partial-ring failure analysis help establish the method's practical scope.
Critical weaknesses
  • The search procedure is insufficiently specified. The paper describes replacement, deletion, and addition actions, but leaves the edit rules/library, candidate enumeration limits, refinement strategy, and termination beyond the stated time cap unclear. These choices affect the results directly: Native-U averages over the available edits, and the headline flip rate is computed over generated candidates. Changing the candidate pool can therefore change both the selected intervention region and the reported success rate.
  • Context Retention is not defined, making its reported value of 0.654 difficult to reconcile with the hard context-preservation constraint. The authors should explain what this metric counts and how it relates to the constraint. This is an ambiguity in interpreting the results, not evidence that the constraint is implemented incorrectly.
  • The methodological advance over existing predictor-guided molecular search is not clearly isolated. CReM already supports protecting atoms, editing selected positions, and optimizing candidates using model scores. Native-U introduces a distinct region-selection score, but the main comparisons use generators whose native sampling is separate from the target predictor. The directional advantage could therefore partly reflect the use of property guidance, rather than the particular Native-U strategy.
  • The Grad-CAM ablation supports the usefulness of Native-U's site selection, but a conventional predictor-guided search using the same edit space, constraints, and predictor-query budget would more directly test its added value. This comparison matters because it would distinguish the proposed methodological contribution from the established benefit of using the predictor to guide molecular edits.
Questions for authors
  1. Please specify the edit rules/library, enumeration and refinement budgets, stopping rule, and exact native-pool/unified-selector procedures. How is Context Retention defined, and why is it 0.654 under the hard invariant in Eq. (5)? These clarifications are necessary to interpret the reported method and ablations.

  2. Can you quantify the gain over a conventional predictor-guided search using the same edit space, CLE/IDG constraints, and predictor-query budget? This would determine whether Native-U's site-level mean utility provides a consequential advantage beyond introducing property guidance into the comparison pipeline.

13 comments

Response to Reviewer RsmY

Dear Reviewer RsmY,

We sincerely thank you for your thorough review and constructive feedback. We are encouraged that you recognized: (1) our substantial empirical improvements over the evaluated Counterfactual Masking (CM) pipelines across CYP3A4, CYP2D6, and hERG (Strength 1.1); (2) our ablation study confirming the effectiveness of actionable site selection via Native-UU over Grad-CAM (Strength 1.2); and (3) our cross-architecture masking analyses (GIN, GraphSAGE, GAT) and explicit L4 partial-ring failure boundary evaluation that establish the practical scope of RealFlux (Strength 1.3). Below, we address your technical concerns and questions in three consolidated themes with complete experimental data.

Theme 1: Search Procedure, Candidate Budgets, Deterministic Termination, and Pool Sensitivity (Weakness 1.4 & Question 1.8)

RealFlux operates over a strictly bounded, deterministic search protocol rather than an open-ended heuristic search. We clarify the exact algorithmic specifications and candidate-pool sensitivity below (added to Section 3.2 and Appendix A.2):

  1. Action Library Construction: Each input graph G\mathcal{G} is decomposed via BRICS and standard functional-group rules into m≈4–8m \approx 4\text{--}8 candidate regions R={r1,…,rm}\mathcal{R}=\{r_1,\ldots,r_m\}. For each region rr, executable replacement, deletion, and addition actions A(r)\mathcal{A}(r) are drawn from three frozen sources: (i) base BRICS and functional-group edits; (ii) an in-distribution low-SA fragment library (ILF: ChEMBL fragments with SA≤2.5\mathrm{SA}\le 2.5 and training frequency ≥2\ge 2, gated by intermediate product SA≤3.5\mathrm{SA}\le 3.5); and (iii) directionally calibrated fragments whose training-fold 95% lower confidence bound of property shift satisfies LCI95>0\mathrm{LCI}_{95}>0.
  2. Enumeration Budgets & Multi-Step Local Refinement:
    • Step 1 (Site Probing): Across all regions, first-step feasible edits Cfirst(r)\mathcal{C}_{\mathrm{first}}(r) passing RDKit sanitization, single-component checks, CLE (Eq. 5), and SA≤4.5\mathrm{SA}\le 4.5 are canonicalized, deduplicated, and capped at Ncand≤60N_{\mathrm{cand}}\le 60 per molecule. A single batched GNN forward pass (Batch Call #1) computes penultimate embeddings for IDG (Eq. 8) and directional utilities u(c)u(c) (Eqs. 1–2) to select r∗=arg⁡min⁡rscore⁡(r)r^*=\arg\min_r \operatorname{score}(r) (Eq. 3).
    • Step 2 (Local Refinement): Starting from top-utility endpoints in r∗r^*, refinement expands up to path depth H≤2H\le 2 via single-step directional edits (attaching calibrated fragments for τ=+1\tau=+1 or pruning terminal groups for τ=−1\tau=-1). Refined candidates must satisfy CLE, IDG, SA(G′)≤4.5\mathrm{SA}(\mathcal{G}')\le 4.5, Tanimoto Sim(G,G′)≥0.6\mathrm{Sim}(\mathcal{G},\mathcal{G}')\ge 0.6, and u(c)>0u(c)>0 (preventing pool dilution), evaluated in Batch Call #2.
  3. Stopping Rule & Unified Selector: The native pool is capped at Kpool≤10K_{\mathrm{pool}}\le 10 via deterministic SHA-256 hash truncation prior to evaluation-stage scoring. Search terminates deterministically upon completing the 2-step, 2-batch pass (≤70\le 70 forward graph evaluations total; median wall-clock latency 0.86 s0.86\text{ s}), with the 300 s300\text{ s} cap serving purely as an outer safeguard. From valid flipped candidates (u(c)>0u(c)>0 crossing y^=0.5\hat{y}=0.5), the unified selector (aligned with CM) greedily picks Kout≤3K_{\mathrm{out}}\le 3 molecules: the first maximizes target utility/similarity to G\mathcal{G}, and subsequent picks balance similarity to G\mathcal{G} with internal Morgan-fingerprint diversity.
  4. Robustness to Candidate-Pool Boundaries:
    • Region Selection Stability: Under 20 random 50/50 splits of Cfirst(r)\mathcal{C}_{\mathrm{first}}(r) per molecule (55 molecules ×\times 3 seeds), Native-UU achieves 0.846 Top-1 region stability, confirming that site selection is robust to action-pool subsampling.
    • Flip-Rate & Success Stability: Truncating the candidate pool from the full generated pool to Kpool=10K_{\mathrm{pool}}=10 changes Success Rate by at most 1.3–2.3 pp1.3\text{--}2.3\text{ pp} (Table 1). Moreover, under the frozen native-pool protocol with zero post-generation reranking (Table 2, N=100N=100), RealFlux achieves Direction Consistency Rates (DCR) of 0.890–0.990.

Theme 2: Formal Definition of Context Retention (CR) and Reconciliation with Eq. (5) (Weakness 1.5 & Question 1.8)

We thank the reviewer for highlighting this terminology ambiguity. The apparent discrepancy arises because Eq. (5) (CLE) is a candidate-level hard generation constraint enforcing exact non-target subgraph isomorphism, whereas Context Retention (CR\mathrm{CR}) is a task-level macro evaluation metric measuring the heavy-atom fraction of the preserved context relative to the entire parent molecule. We have added the explicit mathematical definition of CR\mathrm{CR} to Section 4 and Appendix A.3:

CR(G,G′)=∣VC∣∣V∣⋅I![G′[VC]≅C(G,r);∧;Arom⁡!(G′[VC])=Arom⁡!(C(G,r))]\mathrm{CR}(\mathcal{G}, \mathcal{G}') = \frac{|\mathcal{V}_{\mathcal{C}}|}{|\mathcal{V}|} \cdot \mathbb{I}\\!\left[\mathcal{G}'[\mathcal{V}_{\mathcal{C}}] \cong \mathcal{C}(\mathcal{G}, r) \\;\wedge\\; \operatorname{Arom}\\!\big(\mathcal{G}'[\mathcal{V}_{\mathcal{C}}]\big) = \operatorname{Arom}\\!\big(\mathcal{C}(\mathcal{G}, r)\big)\right]

where G=(V,E)\mathcal{G}=(\mathcal{V}, \mathcal{E}) is the parent molecule, rr is the target intervention region, C(G,r)=G∖r\mathcal{C}(\mathcal{G}, r)=\mathcal{G}\setminus r is the locked context with heavy-atom node set VC=V∖Vr\mathcal{V}_{\mathcal{C}} = \mathcal{V}\setminus\mathcal{V}_r, and I[⋅]\mathbb{I}[\cdot] is the exact subgraph-isomorphism and aromaticity-invariance indicator. Under our strict evaluation protocol, if a task produces no valid candidate within the budget, CR=0.0\mathrm{CR}=0.0 is included in the task denominator.

This formulation fully reconciles CR=0.654±0.064\mathrm{CR}=0.654\pm 0.064 with the hard invariant in Eq. (5):

  1. Theoretical Upper Bound < 1.0: Every accepted RealFlux candidate satisfies I[⋅]=1\mathbb{I}[\cdot]=1 (100% context preservation). However, because the target region rr is actively modified (∣Vr∣>0|\mathcal{V}_r|>0), the retained context accounts for ∣VC∣/∣V∣=1−∣Vr∣/∣V∣<1.0|\mathcal{V}_{\mathcal{C}}|/|\mathcal{V}| = 1 - |\mathcal{V}_r|/|\mathcal{V}| < 1.0 of the parent molecule's heavy atoms.

  2. Stratification Across Topological Levels (Table 7): As shown below, CR\mathrm{CR} scales directly with the relative size of the non-intervened scaffold ∣VC∣/∣V∣|\mathcal{V}_{\mathcal{C}}|/|\mathcal{V}|. In L1 (single substituent, ∣Vr∣/∣V∣≈17|\mathcal{V}_r|/|\mathcal{V}|\approx 17\\%) and L2 (multi-anchor linker), RealFlux achieves CR=0.825\mathrm{CR}=0.825 and 0.833. In L3 (full-ring replacement, where the replaced ring system comprises ≈41\approx 41\\% of parent heavy atoms), the non-target context fraction is naturally CR=0.588\mathrm{CR}=0.588.

  3. Macro-Averaging & Ablation Verification (Table 4): Averaging across the 60 stress-test tasks (20 each in L1, L2, and L3, including 3–83\text{--}8 no-output tasks per seed scored as 0.0) across Seeds 5, 15, and 25 yields 0.743, 0.627, and 0.593, giving the reported mean of 0.654±0.064\mathbf{0.654 \pm 0.064}. When the hard invariant is disabled (w/o CLE), edits corrupt the non-target scaffold (I[⋅]=0\mathbb{I}[\cdot]=0), causing CR\mathrm{CR} to collapse by -0.421 to 0.233±0.076\mathbf{0.233 \pm 0.076}.

Comments on the paper

1 comment

Dear Reviewers,

As we enter the final stage of the author-reviewer discussion, I would like to encourage everyone to actively engage with the authors’ responses and any additional evidence they have provided.

Please take this opportunity to reconsider your initial assessments with an open mind, particularly where the authors have offered substantive clarifications, detailed arguments, or additional experimental results. The goal of the discussion is not to defend initial positions, but to reach the fairest and most informed evaluation of the submission.

I encourage you to engage directly with the authors on any remaining concerns, clearly distinguishing between issues that have been satisfactorily addressed and those that remain unresolved. Where appropriate, please update your reviews and scores accordingly, or explain why your original assessment still stands.

Your active and constructive participation in these final exchanges is essential to reaching a fair, evidence-based decision.

Your AC

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.