- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper proposes SMASH for circRNA-miRNA interaction prediction. The model combines multi-scale hypergraph representation learning with adaptive masked reconstruction. RNA representations are constructed from training association profiles together with sequence, structural, and composition similarities, and different propagation depths are adaptively combined at the node level. The reconstruction branch adjusts node masking probabilities according to historical reconstruction errors, while the learned RNA representations are used by a CatBoost classifier for final prediction. Experiments on three CMI datasets show competitive AUC and AUPR results, together with ablation, cold-start, negative-sampling, parameter, and efficiency analyses.
Critical strengths
S1. The method combines multi-scale hypergraph learning and masked reconstruction in a coherent framework, and the adaptive masking mechanism is reasonably motivated by heterogeneous reconstruction difficulty across RNA nodes.
S2. The experimental study is relatively comprehensive. The paper reports three datasets, component ablations, parameter analyses, cold-start experiments, negative-sampling controls, and computational cost.
Critical weaknesses
W1. The baseline comparison is not fully controlled, as several results are taken from original papers using different splits, negative sampling, and preprocessing. This makes the reported performance gains difficult to attribute solely to SMASH.
W2. The benefit of adaptive masking is not sufficiently isolated. The ablation removes the whole reconstruction branch rather than directly comparing adaptive and uniform masking under the same architecture.
W3. The cold-start evaluation is transductive rather than fully inductive, since held-out RNAs remain in the graph and their similarity features are available during representation learning.
W4. The dense adaptive propagation has time and memory complexity, which may limit scalability to substantially larger RNA datasets.
Questions for authors
Q1. Could the authors rerun the strongest competing methods using exactly the same folds, negative samples, and preprocessing as SMASH? This would make the performance comparison more convincing.
Q2. Could the authors provide a direct adaptive-masking versus uniform-masking comparison using the same reconstruction architecture and masking ratio?
Q3. How does SMASH perform in a fully inductive setting where test circRNAs or miRNAs are completely absent during representation learning and inserted only at inference time?
Q4. How sensitive are the results to the IntaRNA-based negative sampling strategy? Would the ranking of SMASH and competing methods remain similar under alternative negative-sampling schemes?
Q5. The final predictor uses CatBoost on top of learned embeddings. Could the authors report the performance of the neural prediction head alone, or a simpler classifier, to clarify how much of the final gain comes from representation learning versus the downstream classifier?
8 comments
Response to W2 and Q2: Adaptive vs. Uniform Masking under a Controlled Architecture
Thank you for emphasizing the need to isolate the contribution of adaptive masking. We agree that removing the entire GraphMAE branch evaluates the contribution of the reconstruction branch as a whole. A direct comparison between adaptive and uniform masking is needed to evaluate the masking policy within that branch. We have therefore supplemented the ablation study with a uniform-masking variant of SMASH that retains the same reconstruction architecture.
In this comparison, both configurations retain the reconstruction encoder, decoder, and masking-token mechanism. The reconstruction objective and its loss weight remain unchanged. The remaining SMASH components, downstream CatBoost classifier, and training settings are also held fixed. Both configurations use the same positive and negative samples and exactly the same five-fold assignments on CMI-9905. The experimental change is confined to the rule used to select masked nodes.
Both configurations use the same average masking probability, . In the uniform variant, each node is independently selected for masking with probability 0.5 at each masking step. In the adaptive variant, node-selection probabilities are updated using historical reconstruction errors while retaining the same configured average masking probability. This matches the expected masking budget across the two configurations. Under Bernoulli sampling, the realized number of masked nodes can vary between iterations. We will clarify this distinction in the revised description of the masking procedure.
Table R4. Direct comparison of adaptive and uniform masking within the same SMASH reconstruction architecture on CMI-9905 at .
| Configuration | AUC | AUPR | Acc (%) | Sen (%) | Spe (%) | Pre (%) | MCC (%) |
|---|---|---|---|---|---|---|---|
| Uniform masking | 0.9146±0.0065 | 0.9126±0.0084 | 84.72±0.76 | 82.18±1.16 | 87.27±0.66 | 86.59±0.68 | 69.54±1.49 |
| Adaptive masking | 0.9469±0.0032 | 0.9487±0.0025 | 88.47±0.58 | 88.47±0.76 | 88.48±0.92 | 89.20±0.77 | 76.93±1.16 |
| Difference | 0.0323 | 0.0361 | 3.75 | 6.29 | 1.21 | 2.61 | 7.39 |
With this controlled comparison, adaptive masking achieves an AUC of 0.9469 ± 0.0032, compared with 0.9146 ± 0.0065 for uniform masking. AUPR increases from 0.9126 ± 0.0084 to 0.9487 ± 0.0025. These correspond to absolute improvements of 0.0323 AUC and 0.0361 AUPR. Accuracy increases by 3.75 percentage points, and MCC increases by 7.39 percentage points. Adaptive masking also achieves higher mean sensitivity, specificity, and precision under the same evaluation conditions.
The uniform-masking variant retains the reconstruction objective and the capacity of the reconstruction network. Its comparison with the original SMASH configuration therefore directly evaluates the effect of the masking policy within the same model. The observed differences support the empirical benefit of error-feedback masking on CMI-9905. We will state this conclusion within the scope of the evaluated dataset and configuration.
For completeness, Table R5 provides the five-fold results of the newly added uniform-masking experiment.
Table R5. Five-fold performance of SMASH with uniform Bernoulli masking at on CMI-9905.
| Fold | AUC | AUPR | Acc (%) | Sen (%) | Spe (%) | Pre (%) | MCC (%) |
|---|---|---|---|---|---|---|---|
| Fold 1 | 0.9234 | 0.9243 | 85.46 | 82.69 | 88.24 | 87.55 | 71.03 |
| Fold 2 | 0.9078 | 0.9033 | 83.57 | 80.31 | 86.82 | 85.91 | 67.28 |
| Fold 3 | 0.9103 | 0.9061 | 84.40 | 82.18 | 86.62 | 86.00 | 68.87 |
| Fold 4 | 0.9191 | 0.9166 | 85.23 | 83.44 | 87.03 | 86.54 | 70.51 |
| Fold 5 | 0.9126 | 0.9129 | 84.96 | 82.28 | 87.63 | 86.93 | 70.01 |
| Mean ± SD | 0.9146±0.0065 | 0.9126±0.0084 | 84.72±0.76 | 82.18±1.16 | 87.27±0.66 | 86.59±0.68 | 69.54±1.49 |
We will add the uniform-masking configuration to the main ablation table and revise the accompanying discussion to distinguish the questions answered by the two experiments. The branch-removal ablation measures the contribution of the reconstruction branch. The adaptive-versus-uniform comparison measures the contribution of the masking policy while preserving that branch. The revised text will report the direct AUC and AUPR differences and use this comparison as the evidence supporting the benefit of adaptive masking.
We will also expand the appendix to include the complete fold-level results for both masking configurations and the corresponding mean and standard deviation. The experimental description will specify the shared reconstruction architecture, the masking probability, and the settings held fixed across the comparison. It will explicitly identify the common positive and negative samples and fold assignments.
Response to W3 and Q3: Fully Inductive Cold-Start Evaluation
Thank you for identifying the distinction between our original cold-start evaluation and fully inductive prediction. We agree that the original RNA-disjoint setting is transductive: held-out RNAs remained in the graph, and their similarity features were available during representation learning, although their interaction labels were withheld. This evaluates prediction for a fixed node set rather than entirely unseen RNAs. We will label these results transductive RNA-disjoint evaluation in the main text.
To address this request, we conducted supplementary fully inductive experiments for unseen circRNAs and miRNAs on CMI-9905, CMI-9589, and CMI-20208. These evaluate an inductive SMASH extension using training RNAs as fixed feature references and uniform masking. This differs from the main method's error-feedback masking.
For unseen-circRNA evaluation, circRNAs are partitioned into five outer folds. In each fold, held-out circRNAs are excluded from the representation-learning graph and input feature coordinates, as well as training similarity-fusion statistics and the reconstruction objective. All positive and negative pairs involving them are excluded from encoder training and CatBoost fitting, including internal validation. Training uses the remaining circRNAs and available miRNAs. The unseen-miRNA experiment applies the same restrictions with RNA types exchanged.
Because similarity and association profiles depend on the node set, each outer fold uses fixed reference coordinates defined by training RNAs. Training and test nodes are represented relative to these same references. Similarity-fusion coefficients are calculated using training-node submatrices and kept fixed at inference. Association-derived inputs and the initial graph use only training associations. Unavailable association entries for new RNAs are zero; their true labels are reserved for evaluation.
After training, held-out RNAs are inserted in a batch, and their features become available to the frozen encoder. Feature-derived connections among newly inserted nodes are permitted during inference. Encoder and CatBoost parameters remain frozen, without gradient updates or classifier refitting. This protocol evaluates batch insertion of previously unseen RNAs. Fixed reference coordinates preserve the feature dimensions required by the trained model.
Table R6. Fully inductive prediction for unseen circRNAs.
| Dataset | AUC | AUPR | Acc (%) | Sen (%) | Spe (%) | Pre (%) | MCC (%) |
|---|---|---|---|---|---|---|---|
| CMI-9905 | 0.8199±0.0245 | 0.8333±0.0263 | 75.46±1.87 | 77.03±3.85 | 73.77±2.97 | 75.99±1.89 | 50.91±3.71 |
| CMI-9589 | 0.8168±0.0236 | 0.8324±0.0220 | 74.93±2.38 | 77.41±2.85 | 72.26±4.87 | 75.13±3.25 | 49.83±4.83 |
| CMI-20208 | 0.8482±0.0150 | 0.8548±0.0166 | 76.25±0.93 | 84.40±3.68 | 67.48±4.81 | 73.73±2.06 | 52.94±1.72 |
Table R7. Fully inductive prediction for unseen miRNAs.
| Dataset | AUC | AUPR | Acc (%) | Sen (%) | Spe (%) | Pre (%) | MCC (%) |
|---|---|---|---|---|---|---|---|
| CMI-9905 | 0.6870±0.0392 | 0.7309±0.0323 | 64.19±1.77 | 48.36±4.23 | 81.21±4.08 | 73.67±3.52 | 31.27±3.76 |
| CMI-9589 | 0.7011±0.0300 | 0.7269±0.0230 | 64.95±2.40 | 49.15±6.96 | 81.94±3.06 | 74.57±1.54 | 32.82±3.74 |
| CMI-20208 | 0.7079±0.0156 | 0.7310±0.0172 | 64.84±0.92 | 48.38±3.42 | 82.53±3.52 | 75.05±2.74 | 32.82±2.19 |
For unseen circRNAs, mean AUC ranges from 0.8168 to 0.8482 across the three datasets, and mean AUPR from 0.8324 to 0.8548. For unseen miRNAs, the corresponding AUC range is 0.6870–0.7079, with AUPR values of 0.7269–0.7310. These estimates apply when evaluated RNA identities and their features are unavailable during representation learning and become available only at inference.
The unseen-miRNA setting shows lower predictive performance. At the fixed threshold of 0.5, mean sensitivity ranges from 48.36% to 49.15%, versus specificity of 81.21% to 82.53%. This indicates that many positive interactions involving unseen miRNAs remain undetected at this operating point. We will acknowledge this limitation in the discussion.
The original transductive and supplementary inductive results will be reported separately. Their comparison also involves changes in node availability, input representation, masking configuration, and candidate-negative construction. Therefore, score differences cannot be attributed solely to removing test RNAs from representation learning.
We will revise the main experimental section to distinguish transductive RNA-disjoint evaluation from inference on newly introduced RNAs. The main text will summarize supplementary AUC and AUPR results and acknowledge weaker performance for unseen miRNAs. Appendix G will include complete fold-level metrics and, for each fold, counts of held-out RNAs and evaluated positive and negative pairs. It will document fixed training-reference construction, batch insertion, and frozen inference conditions. Claims will remain within this experimental scope and distinguish these results from the original adaptive-masking configuration.
We sincerely thank you for your thoughtful review and for recognizing SMASH's coherent framework and comprehensive evaluation. We addressed your concerns through matched baseline reruns, adaptive-versus-uniform masking controls, fully inductive cold-start tests, alternative negative-sampling and classifier comparisons, and scalability measurements. We hope these new results address your questions, and would be grateful if you would consider revisiting your rating. Thank you again for your valuable feedback.
Response to W1 and Q1: Fair Baseline Comparison under a Unified Evaluation Protocol
Thank you for highlighting the limitations of the original baseline comparison. We agree that differences in data partitions, negative-sample construction, and preprocessing can affect the interpretation of reported performance gains. To address this concern, we have rerun MRHRL, BEROLECMI, and BioDGW-CMI on CMI-9905, CMI-9589, and CMI-20208 under the evaluation protocol used for SMASH. The resulting comparisons are presented in Tables R1-R3.
For each dataset, all four methods use exactly the same positive pairs, IntaRNA-screened negative pairs, and five-fold assignments. Thus, training and held-out pairs are identical across methods in each fold, controlling the sample composition and partitions that differed among the original literature-reported results.
Shared preprocessing includes data cleaning, RNA index correspondence, and preparation of common base inputs. Each baseline retains its own representation construction and prediction mechanism, preserving its model definition while using a common processed dataset.
Table R1. Five-fold comparison on CMI-9905 using identical samples, fold assignments, and shared data preprocessing.
Table R2. Five-fold comparison on CMI-9589 using identical samples, fold assignments, and shared data preprocessing.
Table R3. Five-fold comparison on CMI-20208 using identical samples, fold assignments, and shared data preprocessing.
Under this matched protocol, SMASH achieves the highest mean AUC and AUPR among the evaluated methods on all three datasets. On CMI-9905, MRHRL is the strongest competing method on these two metrics. SMASH increases AUC from 0.9309 to 0.9469 and AUPR from 0.9342 to 0.9487, corresponding to absolute differences of 0.0160 and 0.0145. On CMI-9589, BioDGW-CMI has the highest baseline AUC and AUPR. The corresponding SMASH advantages are 0.0145 AUC and 0.0177 AUPR. On CMI-20208, SMASH exceeds MRHRL by 0.0289 AUC and 0.0329 AUPR. These results provide direct evidence for the observed ranking-performance advantages under a common evaluation protocol.
The threshold-based metrics provide additional context. On CMI-9905, MRHRL has slightly higher mean sensitivity than SMASH, at 88.57% and 88.47%, respectively. SMASH has higher mean accuracy, specificity, precision, and MCC on that dataset. On CMI-9589 and CMI-20208, SMASH has the highest reported mean for each metric shown in the tables.
For CMI-20208, we evaluated the same three competing methods used in the other two dataset comparisons. We could not obtain public implementations of WSCD and CA-CMA, so these two methods were excluded from the controlled reruns. We will present the matched results for MRHRL, BEROLECMI, and BioDGW-CMI and explain the resulting baseline coverage in its caption and accompanying text.
We will update the main experimental section and Tables 2 and 17-19 with these matched comparisons. Captions will identify rerun results and the shared samples, fold assignments, and preprocessing.