acceptodds
Withdrawn from ICLR 2027

Perturbation Modeling via Signed Propagation and Localized Conditioning

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Single-cell perturbation modeling is formulated as conditional generation of a perturbed cell population. Recent models use gene co-expression graphs to pretrain static gene embeddings or to mask dense attention backbones, which score every gene pair. The response direction among gene pairs is an important factor in perturbation modeling, and graphs built from absolute correlation values discard the sign that sets response direction. We present SPARC, an attention-free flow matching velocity field in which a signed graph represents the gene-mixing operator. Genes exchange information through separate positive and negative edges, normalized independently and gated at every layer by a cell state. A shared recurrent block extends propagation depth at the parameter cost of one block. Every perturbed genetic target gets a conditioning vector and is kept in every training batch. Evaluated on genetic, drug, and cytokine benchmarks, SPARC improves Spearman correlation of log-fold changes over the strongest baseline in all settings by up to 0.31, and leads in direction match by up to 0.039. Generated cells also recover more known transcription factor-target pairs and perturbed pathways than the strongest baseline.

Primary area:
applications to physical sciences (physics, chemistry, biology, etc.)
Keywords:
Perturbation modelling, Conditional flow matching, Signed graph neural networks

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_4GQgOpenReview ↗
Rating
2
Confidence
3
Informativeness
1
Presentation
2
Soundness
2
Read review
Paper summary

This paper introduces SPARC, a gene-level single-cell perturbation prediction model based on conditional flow matching. Instead of attention, the model propagates information across genes through co-expression adjacency graphs split into positive and negative components, complemented by a global pooling channel and weight-shared recurrent blocks. To model genetic perturbations, an action vector is injected into each target gene node, and a sampling strategy ensures that target genes appear in every training batch. The training objective combines a flow-matching loss with a Sinkhorn endpoint loss weighted by differentially expressed genes (DEGs).

Critical strengths
  • The approach is simple and computationally efficient. It uses the graph directly as the mixing operator between genes, without attention, and normalizes the positive and negative channels separately, which prevents negative neighbors from being diluted by positive ones.

  • The study reports two notable findings: (i) performance drops substantially when the source distribution is too close to the control distribution; and (ii) the Sinkhorn endpoint generates fewer off-support cells than MMD.

Critical weaknesses
  • The ablation studies suggest that the main performance gains likely stem from the DEG-weighted endpoint loss. This loss concentrates the objective on genes with strong responses during training, which are precisely the genes on which Spearman LFC and direction match are computed.

  • The ablation analysis is incomplete. The unsigned variant simultaneously changes the edge signs, edge weights, and sampling order, so the contribution of each factor cannot be isolated.

  • The model applies max⁡(xt,0)\max(x_t, 0) to its input both during training and at every solver step. By the authors’ own derivation, for genes with x1=0x_1 = 0, xtx_t is negative with probability 1/2 for all t<1t < 1. After rectification, the network cannot see the magnitude of these values, so the velocity along these coordinates is not identifiable. This design choice is neither motivated nor ablated. Notably, the public scDFM code clips only the final generated samples, not the network inputs.

  • The paper does not report the number of positive versus negative edges or the proportion of empty rows in the negative channel. It also provides no analysis of the gating mechanism or of the relative contributions of the positive and negative channels. Moreover, the TRRUST evaluation ranks candidate pairs by the absolute correlation ∣R∣\lvert R\rvert computed on generated cells and ignores the annotated activation/repression direction. The claim that signs convey the direction of the response therefore remains empirically unverified.

Questions for authors

See my Critical Weakness.

2 comments
AuthorsOpenReview ↗

Response to Reviewer 4GQg

Thank you for the precise technical review. We ran every experiment you proposed (Appendix E, blue). The results support the design.

W1. DEG-weighted endpoint loss. The weights are computed from training conditions only and enter only the loss; held-out conditions have no weights. Two experiments separate the loss from the architecture.

First, we removed DEG weighting entirely (Table 9, ComboSciPlex):

ModelEndpointSpearman LFCDirectionPearson deltaMSE (×103\times 10^3)
scDFMMMD0.8290.8990.8932.8
SPARCUnweighted MMD0.8940.8960.8632.87
SPARCUnweighted Sinkhorn0.8960.9250.9222.24
SPARCDEG-weighted Sinkhorn (original benchmark)0.9080.9380.9352.0

Without DEG weighting, SPARC with unweighted Sinkhorn achieves 0.896 Spearman LFC, 0.925 direction match, 0.922 Pearson delta, and 2.24×10−32.24 \times 10^{-3} MSE, outperforming the published scDFM benchmark scores on all four metrics (Table 9). These are separate runs from the single-seed curves in Fig. 3A. The original DEG-weighted benchmark configuration achieves 0.908 Spearman LFC; this benchmark reference is from a separate training cohort and is not a within-seed matched loss-only comparison. With an unweighted MMD endpoint, SPARC retains a Spearman LFC of 0.894, compared with scDFM's published 0.829. The MMD implementations retain their original bandwidth conventions, as detailed in Appendix E.3.

Second, we fixed the DEG-weighted loss and changed only the routing (next point).

W2. Confounded unsigned ablation. Agreed, so we supplemented the original compound ablation with a capacity-matched control. Both arms keep two learned branches and gates, the same graph support, absolute edge magnitudes, neighbor ordering, sampled genes, parameter count and objective. The unsigned arm feeds the same magnitude-weighted adjacency to both branches (Table 8, seven conditions, 300k steps):

SeedSpearman LFC: unsignedSpearman LFC: signedDirection: unsignedDirection: signed
420.8940.9110.9210.933
430.9170.9190.9380.951
440.8910.9240.9110.955
Mean0.9010.9180.9230.946

Sign is the only factor that differs, and signed routing wins on both metrics in every seed.

W3. Input rectification. We formalized your argument (Eq. 3) and ran the single-change experiment: the lower clamp is removed in training and at every solver evaluation; the upper cap and final-output clipping are unchanged (Table 10, ten Norman additive conditions, 300k steps):

SeedRectifiedLower-unclamped
420.8420.864
430.8410.868
440.8610.881

Preserving negative intermediate states improves Spearman LFC in all three matched training pairs, by 0.023 on average. Table 10 uses independently retrained matched pairs, whose rectified arm averages 0.848, whereas the original benchmark checkpoints average 0.884 in Table 1. As clarified in Appendix E.4, these absolute scores are not interchangeable, and the controlled conclusion is the consistent within-pair improvement from removing the lower clamp.

W4a. Graph statistics, gating and channel contributions. Table 12 reports edge counts and empty rows for every dataset. Norman additive has 16,229 positive and 1,613 negative directed entries with 611 empty negative rows; ComboSciPlex has 17,832 and 2,186 with 496. Both channels are available for 38.7% and 50.4% of evaluated genes. In the trained Norman model at t=0.75, mean positive/negative gates are 0.852/0.767 with gated message norms 14.98/5.03. Masking one channel at inference, with weights, noise, controls and solver fixed, lowers Spearman LFC from 0.825 to 0.813 (negative masked) and 0.735 (positive masked). Both gates are open and both channels are used.

W4b. TRRUST and the meaning of sign. The signs encode co-variation type and route messages through separate learned transforms; we do not equate negative correlation with repression. TRRUST precision ranks by absolute correlation and is now stated as TF-target association recovery. The claim that sign informs response direction is tested directly: direction match rises from 0.923 to 0.946 under the capacity-matched control (Table 8), down-DEG direction accuracy is 0.870 against 0.783 and 0.640 for the simple predictors (Table 11), and top-20 direction-correct candidate precision is 16.5% for signed against 12.0% for unsigned routing (E.5).

Please let us know if any further experiments or clarification are required during the discussion period or before the camera ready version. We will be happy to discuss and perform further experiments.

Dear Reviewer 4GQg,

We posted our detailed rebuttal and revised manuscript, including a capacity-matched signed-versus-unsigned comparison, independently retrained input-rectification controls, and additional channel-masking and gating diagnostics.

These experiments were specifically designed to address your concerns about the independent contribution of signed propagation, potential confounding factors, and the interpretation of the model components.

With the discussion period nearing its end, we would appreciate your feedback on whether these controlled experiments resolve your principal concerns. If any technical objections remain, please let us know so that we can address them explicitly.

We ask that these new results and the corresponding manuscript revisions be considered in your final assessment.

Best regards,

The Authors

Reviewer_hp6dOpenReview ↗
Rating
2
Confidence
3
Informativeness
1
Presentation
2
Soundness
2
Read review
Paper summary

The paper presented SPARC, a perturbation modeling framework. It is built with an attention free backbone that uses continuous flow matching with a signed graph as the gene-mixing operator. It is evaluated in four different settings (three datasets) demonstrating consistent performance in metrics capturing ranking and sign of gene-level responses.

Critical strengths

The use of established ideas from the machine learning community (e.g., recurrent weight sharing; signed GNNs) to target the limitations they identify in perturbation prediction frameworks is a nice approach, and the transparency of presenting the practical gains of the method (namely its inherent tradeoffs) is appreciated.

Critical weaknesses

The authors motivate the approach as a more “lightweight” deep learning framework that is best at prediction of perturbation direction; however, both of these objectives are not grounded / aligned with recent literature, which is accurately cited (e.g., Ahlmann-Eltze et al., 2025; Viñas Torné et al., 2025).

  1. Missing linear baselines; Ahlmann-Eltze et al (and others) showed that linear models often perform as well as deep learning methods for perturbation prediction tasks. Thus, when suggesting a “simpler” (less-parametrized) architecture it is valuable to compare to such baselines.

  2. Parametrization vs accuracy tradeoff; In practical applications (e.g. translational research), train compute time (especially when considering 4 hrs vs 18 hours and the limited availability of complex, multi-perturbation, huge datasets) will likely be secondary to accurate predictions. The authors here prioritize Spearman LFC, (captures gene ordering), over Pearson-Δ\Delta (captures expression shift magnitudes).

Questions for authors
  1. Can the authors relate to the weaknesses:
    1.a the lack of naive linear baselines;
    1.b the value/superiority of perturbation directionality in practical use cases.
  2. Can the authors clarify what they see as the contribution of this work ? it will be valuable to address both:
    2.a the importance of the work to the single-cell perturbation modeling community?
    2.b the relevance for the broader ML community and fit to ICLR.
3 comments
AuthorsOpenReview ↗

Response to Reviewer hp6d

Thank you for the review. We ran the requested linear baselines and a direct test of the practical value of direction; both are in Appendix E (blue).

W1 / Q1a. Linear baselines. SPARC beats every simple predictor in every setting (Table 7). Against training-only PCA/ridge, Spearman LFC is 0.884 vs 0.755 on Norman additive and 0.772 vs 0.534 on Norman holdout. On the three conditions where the additive predictor is defined, it is 0.900 vs 0.702. Against matching mean on ComboSciPlex it is 0.908 vs 0.820, and against the cytokine-conditional mean on Parse-PBMC 0.732 vs 0.619. PCA/ridge on Norman additive (0.755) is above every published deep baseline in Table 1, which reproduces the Ahlmann-Eltze et al. finding; SPARC exceeds it by 0.129.

W2. Accuracy versus compute. SPARC does not trade accuracy for compute. It has the lowest MSE in all four settings, leads Pearson delta on ComboSciPlex (0.935 vs 0.893), and is within 0.016 of the best Pearson delta in the other three. It is best on 15 of the 20 primary metric-setting cells in Tables 1-4. It does this with about 10.9M parameters on Norman and about 4 hours of training against 18 for scDFM. The efficiency comes on top of the accuracy. The simple-baseline comparisons further support improvements beyond Spearman LFC. On Norman additive, SPARC outperforms both PCA/ridge and matching mean across all five additional response metrics under the shared evaluation protocol. In particular, Pearson delta improves by 0.138 and 0.078, respectively, while MSE decreases by 3.04×10−33.04 \times 10^{-3} and 2.93×10−32.93 \times 10^{-3} (Appendix E.1). Thus, the ranking advantage is accompanied by improvements in response magnitude and direction prediction.

Q1b. Why direction matters in practice. Predicting suppression versus activation decides the follow-up experiment. On observed downregulated DE genes across all ten Norman additive conditions (Table 11):

PredictorDown-DEG direction accuracy
PCA/ridge0.640
Matching mean0.783
SPARC0.870

For candidate prioritization, we rank genes by predicted absolute log-fold change alone and score against observed DE genes with the correct direction. Signed routing gives 16.5% top-20 precision against 12.0% for the capacity-matched unsigned model (Appendix E.5). At the pathway level, every SPARC variant exceeds scDFM in top-20 GSEA pathway recall (0.500 to 0.850 across variants, 0.725 to 0.750 for the full configuration, against 0.447; Fig. 4D).

Q2a. Contribution to perturbation modeling. Prior models keep the gene graph outside the generative process, as static embeddings (GEARS) or as a binary attention mask (scDFM), and build it from absolute correlation. SPARC makes the signed graph the gene-mixing operator of the velocity field and places actions on the perturbed genes themselves. The capacity-matched comparison shows the sign carries usable signal: +0.017 Spearman LFC and +0.023 direction match, positive in all three seeds (Table 8).

Q2b. Contribution to ML and fit to ICLR. SPARC is a structured velocity parameterization for conditional generation over thousands of unordered tokens with a noisy relational prior. Sparse typed propagation, a pooled global state and weight-shared recurrence replace dense pairwise attention; localized conditioning replaces broadcast conditioning. We evaluate the design choices through component ablations, with additional matched comparisons specifically targeting signed routing and input-state rectification. The input-state analysis (Eq. 3, Table 10) applies to any flow-matching model that clips its inputs.

Please let us know if any further experiments or clarification are required during the discussion period or before the camera ready version. We will be happy to discuss and perform further experiments.

Dear Reviewer hp6d,

We submitted our detailed point-by-point response and revised manuscript, including new linear and additive baselines, controlled experiments isolating signed propagation, and additional evidence for the practical value of perturbation directionality.

These additions directly address the central concerns in your review, including the contribution to perturbation modeling and the broader ML community.

As the discussion period is approaching its conclusion, we would appreciate your assessment of these new results. If any other concerns remain, please identify them so that we can address them directly before the discussion closes.

We hope the final evaluation will take into account the revised manuscript and the additional experimental evidence.

Best regards,

The Authors

Reviewer_hp6dOpenReview ↗

Rebuttal response

Dear Authors,

I truly appreciate your rebuttal efforts, additional experiments, text revisions and responses here.

With that, I am afraid my concerns remain unresolved regarding the overall added value of the method, both in terms of (1) performance improvements (/ emphasis on directionality), and (2) broader novelty and conceptual contribution, which I still find limited and insufficient to justify acceptance.

Reviewer_8siBOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The authors introduce SPARC a flow matching (FM) model for single cell perturbation modelling. In contrast to previous single cell FM models (e.g. scDFM) the velocity network does not use attention and instead transcriptomics based information is propagated via a signed gene level co-expression graph. SPARC is a combination of existing approaches, integrating methods of graph propagation from other fields into an existing FM framework for perturbation modelling. The model is validated using multiple perturbation datasets Norman CRISPRa, ComboSciPlex and Parse-PBMC, generally showing strong performance. Further biological evaluation is also carried out e.g. on GSEA pathways. There is also a set of multiple ablation studies.

Critical strengths
  • The use of their graph propagation / conditioning method seems well motivated biologically, addressing what looks like a real gap in existing methods. I think this means that the paper will be useful to researchers in the area as a different approach that could be built on in the future.

  • The work presented is broadly easy to follow with thoughtful figures that help with understanding.

Critical weaknesses

Performance improvements are generally modest. Furthermore, ablations of key components of the model, (specifically signed_off) shows similar performance to the non ablated model. The above coupled with the fact that existing methods share much of the flow matching framework for perturbation prediction means that performance gains may not be pronounced enough over existing models to reflect impactful improvements.

Simple baselines are missing e.g. additive baseline [1], average baselines [2].

[1] Deep-learning-based gene perturbation effect prediction does not yet outperform simple linear baselines
[2] Systema: a framework for evaluating genetic perturbation response prediction beyond systematic variation

Questions for authors
  • How does your method compare with related work? E.g. ‘Beyond Independent Genes: Learning Module-Inductive Representations for Single-Cell Gene Perturbation Prediction’. Further the contributions section could be after the related work section and more directly reference how your approach uses or differs from the work in this section.

  • Are there more insights / explanation / discussion of the results? Currently the main part of the results section mostly states what can be read in the figures/tables. And the discussion at the end is very short.

  • What data was used to find the number of training iterations and model hyperparameters? I found this unclear in the paper current state. What are variations in performance on a per treatment / perturbation basis?

  • Is MMD in the text Maximum Mean Discrepancy from figure 2? This term should be defined in the main text to improve clarity. Clarity can be further improved by referencing the appropriate appendices in the main text.

3 comments
AuthorsOpenReview ↗

Response to Reviewer 8siB

Thank you for the review and for recognizing the biological motivation of signed propagation. All requested experiments are now in the paper (Appendix E, marked in blue), and each one strengthens the result.

W1a. Size of the gains. Over the strongest published baseline, SPARC improves Spearman LFC by +0.313 on Norman additive (0.571 to 0.884), +0.076 on Norman holdout, +0.079 on ComboSciPlex and +0.125 on Parse-PBMC, with seed s.d. at most 0.018. SPARC also has the lowest MSE in all four settings and is best on 15 of the 20 primary metric-setting cells in Tables 1-4, at about 4 hours of training against 18 for scDFM.

W1b. What signed propagation contributes on its own. The original signed_off ablation also changed edge weights and sampler order, so we supplemented it with a capacity-matched control that isolates the routing change. Both arms keep two learned branches and gates, the same graph support, absolute edge magnitudes, neighbor ordering, sampled genes, parameter count and DEG-weighted Sinkhorn objective. Only the routing differs (Table 8, all seven ComboSciPlex conditions, 300k steps):

SeedSpearman LFC: unsignedSpearman LFC: signedDirection: unsignedDirection: signed
420.8940.9110.9210.933
430.9170.9190.9380.951
440.8910.9240.9110.955
Mean0.9010.9180.9230.946

Signed routing wins on both metrics in every seed, with average improvements of +0.017 Spearman LFC and +0.023 direction match. This supports a consistent contribution from signed routing under a fixed objective.

W2. Simple baselines. We added training-only PCA/ridge, additive, matching-mean and cytokine-conditional-mean predictors (Table 7). On Spearman LFC, SPARC outperforms each predictor on its evaluated conditions:

SettingBaselineConditionsBaseline Spearman LFCSPARC
Norman additivePCA/ridge100.7550.884
Norman holdoutPCA/ridge100.5340.772
Norman additiveMatching mean100.7050.884
Norman holdoutMatching mean100.5670.772
Norman additiveAdditive30.7020.900
ComboSciPlexMatching mean70.8200.908
Parse-PBMCCytokine mean100.6190.732

PCA/ridge uses ten training-derived components with five-fold validation over training conditions; the additive predictor is evaluated on the three conditions whose two single responses are both in training. Consistent with Ahlmann-Eltze et al., PCA/ridge is a strong reference on Norman additive (0.755, above every published deep baseline in Table 1). SPARC clears it by 0.129.

Q1. Related work. We added a direct comparison with scBIG and moved Contributions after Related Work. scBIG induces gene modules from pretrained representations and protein interactions, mixes them through latent bottleneck attention, and learns a control-to-perturbed latent flow. SPARC operates at gene resolution in expression space, propagates along signed edges inside the velocity field, and places a local action on each genetic target. To our knowledge no prior perturbation model propagates hidden states along signed edges in a generative velocity field.

Q2. Insight beyond the tables. The revision ties each design choice to a measured effect: signed routing improves ranking and direction under a fixed loss (Table 8); removing DEG weighting leaves SPARC ahead of scDFM on all four response metrics (Table 9); preserving negative intermediate states improves ranking in three of three seeds (Table 10); both channels carry signal in the trained model (E.6). The Discussion is rewritten around these findings.

Q3. Hyperparameters, training length and per-treatment variation. The benchmark splits provide no validation set, so, as is standard for these benchmarks, we did not tune per dataset: one configuration (Tables 5-6) is shared unchanged across all four settings, with widths, depth and optimizer set to common defaults for models of this size. Training length is fixed at 300k steps and all numbers use the final checkpoint; the held-out curves in Fig. 3 were not used to select checkpoints (Appendix A). Per-perturbation results are in Fig. 4B, and Appendix E.2 reports per-seed results and a per-condition example (SRT2104+Alvespimycin: 0.730 signed vs 0.636 unsigned at seed 42).

Q4. MMD. Yes, maximum mean discrepancy. It is now defined at first use, the estimator is specified in Appendix A, and the main text points to the relevant appendices.

Please let us know if any further experiments or clarification are required during the discussion period or before the camera ready version. We will be happy to discuss and perform further experiments.

Dear Reviewer 8siB,

We have provided a detailed response to your questions review regarding the numerical revisions in Tables 1 and 3.

We traced the provenance of both changes, restored the original benchmark results, and explicitly separated the independently retrained signed-routing experiment from the original benchmark evaluation. We also provided the complete six-metric comparison between the two training cohorts and clarified why the other benchmark results remain unaffected.

As the discussion period is approaching its conclusion, we would appreciate your feedback on whether this explanation resolves your concerns about numerical consistency and evaluation provenance.

If any specific questions remain, please let us know. We want to ensure that the corrected manuscript and the clarification are fully considered before the final assessment.

Best regards,

The Authors

I thank the authors for their rebuttal - they have mostly addressed my concerns by adding new baselines and text revisions. I did find the changes to Tables 1 and 3 during the discussion confusing, but appreciate that they have been clarified.

My remaining concern is that one of the papers main claim is that signed propagation drives performance. The capacity-matched comparison shows that contribution is approx. +0.02 on Spearman LFC and direction. It seems to me that every other source of performance tested - Gaussian source, Sinkhorn endpoint, removing the clamp - has an effect as large or larger. Furthermore, the retraining shifts reported for identical configurations are of a similar magnitude, which would indicate the signed-routing effect is of a similar magnitude to run to run variation. It therefore seems difficult to make any mechanistic claims that this contribution is central, whereas it is the one the title and the first contribution of the paper rest on. Given this central tension, I would find it difficult to increase my score.

Comments on the paper

3 comments
AuthorsOpenReview ↗

## Meta Response to the Area Chair and Reviewers

We thank the Area Chair and Reviewers 8siB, hp6d and 4GQg for their careful reading. Every point raised has been addressed with a new experiment or a revision to the text. Changes are marked in blue, and the new experiments are collected in Appendix E.

Simple baselines. We added linear, additive and conditional-mean predictors trained only on training conditions. SPARC outperforms the reported simple predictors on Spearman LFC across their respective evaluation settings.

Signed propagation contributes on its own. We added a capacity-matched control whose architecture and objective differ only in routing. Signed routing improves both direction match and response ranking in every training seed, most clearly direction match (+0.023 on average).

Strong performance persists without DEG weighting. On ComboSciPlex, SPARC trained with unweighted Sinkhorn outperforms the published scDFM benchmark scores on all four reported response metrics. Separately, the capacity-matched comparison isolates a positive contribution from signed routing under the same DEG-weighted objective.

Input rectification. We formalized Reviewer 4GQg's argument and ran the proposed experiment. Preserving negative intermediate states improves Spearman LFC in all three independently retrained matched pairs.

Direction is practically useful. SPARC predicts suppressed genes more accurately than the simple predictors, and signed routing yields better direction-correct candidates when genes are selected from predictions alone.

Both channels are used. We report edge statistics for all datasets, and gate and masking diagnostics confirming that the positive and negative channels each contribute to the trained model's predictions.

Text revisions. We added a direct comparison with scBIG, moved Contributions after Related Work, rewrote the Discussion around the new findings, clarified TRRUST as association recovery, and defined MMD and the evaluation protocol explicitly.

The three shared concerns, namely missing simple baselines, the independent contribution of signed propagation, and the practical value of direction, are each answered by a controlled experiment, and each result favors SPARC. Details and numbers are in the individual responses.

Thank you for the rebuttal. Could you explain why the SPARC Spearman LFC in Table 1 went down from 0.884 to 0.858, while the rest of the metrics are unchanged? Similarly, in Table 3 SPARC's Spearman LFC went up from 0.908 to 0.918 and the direction match from 0.938 to .946 - again with everything else unchanged. I can see this was also changed in the abstract (the improvement magnitude), but none of these changes are tracked in blue and I couldn't find the explanation in the paper or the other comments. Could you explain what was changed in the pipeline and why it doesn't affect Table2-3?

Thank you for raising this question. We have reviewed the numerical revisions and clarified the provenance of the results in Tables 1 and 3. The changes arose during table editing and integration of the new controlled experiments, not from changes to the model, data splits, or evaluation pipeline.

1. Table 1, Norman additive (0.884 to 0.858). The revised Spearman LFC entry of 0.858 was a transcription error. This value matches a DE-recall mean from a separate experimental summary, while the original Spearman standard deviation (0.018) was retained. We have restored the original benchmark result of 0.884±0.0180.884 \pm 0.018. The other Table 1 metrics remain unchanged.

2. Table 3, ComboSciPlex (0.908 to 0.918; 0.938 to 0.946). During integration of the new capacity-matched signed-routing experiment, the Spearman LFC and direction-match results from its independently retrained three-seed signed cohort were incorporated into Table 3, while the other four metrics retained their original benchmark values. This resulted in a table row combining two different training cohorts. For clarity, the complete results of both cohorts are:

MetricOriginal benchmarkNew signed cohort
Spearman LFC0.908±0.0060.908 \pm 0.0060.918±0.0070.918 \pm 0.007
Direction match0.938±0.0110.938 \pm 0.0110.946±0.0120.946 \pm 0.012
Pearson delta0.935±0.0100.935 \pm 0.0100.929±0.0180.929 \pm 0.018
MSE (×103\times 10^3)2.0±0.22.0 \pm 0.21.957±0.2981.957 \pm 0.298
Cosine discrimination0.905±0.0140.905 \pm 0.0140.837±0.0350.837 \pm 0.035
DE recall0.973±0.0010.973 \pm 0.0010.953±0.0060.953 \pm 0.006

We have restored the complete original benchmark row in Table 3 and retained the new capacity-matched comparison separately in Table 8.

Importantly, the original benchmark and the capacity-matched experiment use independently trained checkpoint sets under the same evaluation protocol. Their absolute scores therefore need not coincide. Both signed and unsigned arms in Table 8 were newly trained, and the contribution of signed routing is evaluated exclusively through their within-cohort comparison: Spearman LFC improves from 0.901 to 0.918 and direction match from 0.923 to 0.946 across three matched training seeds.

3. Consistency across the remaining results. Tables 2 and 4 were unaffected, as neither was involved in these revisions. Likewise, the remaining Table 1 metrics and the four original Table 3 metrics were unchanged because there was no change to their underlying evaluations. The abstract, main-text comparisons, Tables 1, 3, 7, and 9, and the corresponding reviewer responses have now been synchronized with the original benchmark results. The restored entries in Tables 1 and 3 are highlighted in blue.

These corrections preserve the distinction between the original benchmark evaluation and the newly added controlled experiments, without changing the conclusions drawn from either.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.