- Rating
- 2(+1)
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2(+1)
Read review
Paper summary
Background. The paper studies prediction of neuronal projection targets from gene-expression profiles. Its central challenge is that projection assays have one-sided detection errors: a negative readout may represent either no projection or a missed projection. Gene and target panels may also vary across animals and assays, producing incomplete and uneven supervision.
Method. After splitting the data into training, validation, and test sets, Gene2Wire reserves an outcome-blind 20% paired subset of development cells to estimate detection sensitivity from reference-positive pairs. Holding the resulting fixed, it then estimates the latent projection probability from incomplete readouts through the calibrated likelihood . The coefficient matrix is represented as , where captures effects shared across projection targets and captures target-specific effects. Gene expression is normalized, log-transformed, and standardized using training data; unmeasured genes are represented with zero-filled values and observation indicators. The method compares direct PU, shared-only PU-MIRT, and joint shared-plus-target-specific candidates, enumerates a bounded set of ranks and regularization values for each structure, selects the configuration with the lowest validation readout loss, and refits it on the development data.
Theory. Proposition 1 analyzes the population risk under a supplied sensitivity estimate and decomposes projection error into readout-fitting and sensitivity-calibration terms. The paper also derives a posterior score for ranking missed projections. Proposition 2 characterizes the regularization induced by the shared-factor plus residual parameterization and explains how it allocates shrinkage across coefficient directions without forcing the full predictor to be low rank.
Experiments. Simulations vary several supervision and sharing conditions, including cross-target sharing, positive-label loss, gene coverage, and target coverage. An internal ablation compares direct PU, shared-only PU-MIRT, and Gene2Wire under common detection calibration. The main comparisons evaluate Gene2Wire against GenEML, SAR-PU, and Bilinear on five projection-transcriptomic assay panels, using ranking, probability-estimation, and hidden-positive-retrieval metrics.
Critical strengths
- Statistical modeling of the practical measurement problem.
The paper provides a coherent statistical treatment of neuronal projection data with missing and noisy readouts. It explicitly models imperfect detection through a paired calibration subset and separates sensitivity estimation from latent projection estimation. Its coefficient representation also captures both effects shared across projection targets and target-specific effects, allowing the model to reflect the structure of the biological problem more flexibly than a purely target-independent or globally low-rank formulation.
- Comprehensive simulation study with consistent gains over competitors.
The simulation study covers a broad range of conditions, including different levels of cross-target sharing, positive-label loss, gene coverage, and target coverage. These controlled experiments isolate the effects of missing supervision and target-relationship structure, making the source of the performance differences easier to assess. Across the reported simulation settings, Gene2Wire achieves the strongest mean AUPRC and consistently improves probability-estimation metrics over the main competing methods, including GenEML, SAR-PU, and Bilinear.
Critical weaknesses
- The methodological novelty is unclear.
The method first estimates detection sensitivity from paired reference labels and then uses the estimated sensitivity to estimate . Compared with standard PU methods, this procedure relies on an additional paired calibration subset and introduces a two-stage estimation process. The use of calibration information may be an important contribution in this application, but it is not by itself a methodological innovation under the standard PU setting. The paper should clarify what is new beyond incorporating this additional supervision into a calibrated PU likelihood, and should distinguish the benefit of calibration from the benefit of the proposed predictor structure.
The core model for estimating is where represents structure shared across projection targets and represents target-specific effects. This idea is closely related to existing reduced-rank generalized linear models and multitask models with shared and individual effects. Yee and Hastie extended reduced-rank regression to vector generalized linear models, including logistic and categorical responses [3]. Jalali et al.'s ``A Dirty Model for Multi-task Learning'' also studied the joint modeling of shared and task-specific structures [4]. Although the paper cites these related works, it only briefly states that they motivate the proposed model and does not clearly explain how Gene2Wire differs from these approaches or why this particular parameterization is needed for the calibrated PU setting.
Thus, the main distinction of the paper appears to be placing existing structures into a missing-label problem with partial calibration by reference labels, rather than proposing a new low-rank logistic model or a new combination with PU learning. The paper should clarify what methodological difficulty in the calibrated PU setting is not addressed by the existing reduced-rank GLM and multitask-learning formulations, and why the proposed combination is preferable to those alternatives.
- The theoretical contribution is insufficient and incomplete.
Regarding novelty, She's work already gives a detailed treatment of rank penalties, singular-value penalties, and optimization methods for fitting reduced-rank generalized linear models [5]. The algorithm first estimates sensitivity from the calibration sample and then estimates the projection probability using . However, the theory is primarily conditional on : it gives no finite-sample error bound for estimating the sensitivity itself and does not analyze the error arising from using in the main model. Proposition 2 is also mainly based on existing regularization identities and spectral arguments [6,7]. Although the paper briefly cites Mazumder et al. [7] in its related-work discussion, the proof does not clearly distinguish the established steps from the result specific to this factor-plus-residual parameterization. It helps explain the regularizer, but currently appears closer to an interpretive result than to a new low-rank statistical theory.
The theoretical development is not complete. Proposition 1 analyzes the population risk and bias after minimizing a risk with fixed , but it does not provide a complete finite-sample deviation bound from empirical risk to population risk. It also does not analyze approximation error, regularization bias, optimization error, or model-selection error. Therefore, the proposition is a population-level identifiability and error-decomposition result, rather than a finite-sample guarantee for the actual estimator produced by the algorithm. Nor does the paper provide an optimality result or a lower bound corresponding to the complete estimation procedure.
The discussion of existing results is also insufficient. Several papers have already developed partial theoretical analyses. For example, Kiryo et al., which the paper cites, study bias, consistency, and estimation error for a non-negative PU risk estimator [8]; Coudray et al. study risk bounds for PU learning under the selected-at-random assumption [9]; and low-rank matrix-variate logistic regression has existing minimax lower-bound results [10]. The paper should explain precisely what it adds relative to these results, rather than providing only an error decomposition conditional on .
References
[1] T. W. Yee and T. J. Hastie. Reduced-rank vector generalized linear models. Statistical Modelling, 3(1):15--41, 2003.
[2] A. Jalali, S. Sanghavi, C. Ruan, and P. K. Ravikumar. A dirty model for multi-task learning. In Advances in Neural Information Processing Systems 23, 2010.
[3] Y. She. Reduced rank vector generalized linear models for feature extraction. Statistics and Its Interface, 6:197--209, 2013.
[4] N. Srebro, J. Rennie, and T. Jaakkola. Maximum-margin matrix factorization. In Advances in Neural Information Processing Systems 17, 2005.
[5] R. Mazumder, T. Hastie, and R. Tibshirani. Spectral regularization algorithms for learning large incomplete matrices. Journal of Machine Learning Research, 11:2287--2322, 2010.
[6] R. Kiryo, G. Niu, M. C. du Plessis, and M. Sugiyama. Positive-unlabeled learning with non-negative risk estimator. In Advances in Neural Information Processing Systems 30, 2017.
[7] O. Coudray, C. Keribin, P. Massart, and P. Pamphile. Risk bounds for positive-unlabeled learning under the selected at random assumption. Journal of Machine Learning Research, 24(107):1--31, 2023.
[8] B. Taki, M. Ghassemi, A. D. Sarwate, and W. U. Bajwa. A minimax lower bound for low-rank matrix-variate logistic regression. arXiv preprint arXiv:2105.14673, 2021.
Questions for authors
In addition to the two major weaknesses discussed above, I have the following questions for the authors. Clarifications on these points would help me better assess the scope and validity of the paper.
- Relationship to dropout-aware single-cell methods.
The paper models unmeasured genes through observation indicators, but does not discuss established methods for technical dropout in single-cell expression data, such as scImpute [1] and SAVER [2]. How is the proposed treatment of missingness related to these methods, and what aspects of the missing-data problem are shared or distinct? More importantly, can the authors evaluate Gene2Wire with and without dropout-aware correction or imputation and report whether the network conclusions and comparative performance are robust to this choice?
- Use of calibration information by competing methods.
My reading is that the paper gives the external baselines their stated inputs and procedures, but does not provide GenEML, SAR-PU, or Bilinear with the additional paired calibration subset used by Gene2Wire and by the internal PU/PU-MIRT ablation. Is this interpretation correct? If so, does the main comparison place Gene2Wire and the external baselines under different information conditions? Please clarify exactly what calibration information each method receives and discuss whether a comparison with unequal access to paired calibration data is a fair basis for the reported superiority.
- Generalizability of rank, structure, and regularization selection.
The current procedure appears to enumerate different candidate model families, ranks, and penalty values for each dataset and then select the best configuration by validation loss. Is there a simple, generalizable selection rule that can be used across datasets, or does the procedure require a dataset-specific search over these choices? Please report how the candidate sets and search ranges are determined, and whether the selected configuration is stable when the search budget or validation split changes.
- Stability of the selected structure.
How stable are the selected model family, rank, and penalty values across outer folds, animals, assay panels, or random seeds? In the real datasets, does the selected rank support the claimed shared structure, and how much of the performance comes from the shared component versus the target-specific residual? A real-data comparison among Direct PU, shared-only PU-MIRT, and Gene2Wire across different rank choices would help clarify this point.
- Biological interpretation and validation.
Do the shared directions and target-specific residuals yield reproducible gene modules or biological signatures? Can the authors provide downstream enrichment analysis, comparison with known neuronal types or projection-related genes, or independent validation of high-confidence predictions? This would clarify whether the method provides biological insight beyond improved predictive metrics.
References
[1] W. V. Li and J. J. Li. An accurate and robust imputation method scImpute for single-cell RNA-seq data. Nature Communications, 9:997, 2018.
[2] Huang, M., Wang, J., Torre, E. et al. SAVER: gene expression recovery for single-cell RNA sequencing. Nat Methods 15, 539–542 (2018). https://doi.org/10.1038/s41592-018-0033-z
4 comments
W2. Finite-sample analysis of calibration, selection and refitting.
We bound calibration error and its propagation through predictor fitting, selection and refitting.
Sensitivity estimation. For S represented target–assay strata, let n_s count paired, assayed reference-positive entries. Under conditionally independent Bernoulli detections within each stratum and a correctly specified categorical logistic detector, define
Here represents the true detector logits; is the centered nonnegative penalty in the summed calibration objective, and its optimization defect relative to the global infimum. With probability at least ,
Thus calibration-weighted sensitivity RMSE is at most . The proof combines binomial concentration, the penalized-objective comparison with the true detector, and Pinsker's inequality.
From empirical fitting to population prediction. Center the readout loss at p=1/2:
For logits bounded by M, this loss is bounded by M in absolute value, 1-Lipschitz in the logit, and Lipschitz in . The subtracted term is common to all predictors, so fitting and selection are unchanged. A product covering argument gives simultaneous concentration over predictor coefficients and detector probabilities. For independent blocks with nonnegative weights summing to one, ; allowing arbitrary dependence within each block gives a deviation of order for fixed radii. D is the full encoded predictor dimension. This uniformity accommodates fitted calibration and adaptive candidate refinement.
Let be the evaluated candidates, the development refit, and . Define and , where and is the true conditional readout probability. Then
is approximation error within candidate 's bounded class; is its population regularization cost; is its global empirical optimization defect. includes sampling deviation and any fitting-to-evaluation distribution discrepancy. is the penalty of the validation-selected predictor transported to development standardization. The refit term is directly evaluable:
Here is the penalized development objective. The independent blocks must match the target: animal-level generalization uses independent animals, with valid sampling weights and bounded features/coefficients.
Writing the right-hand side as , the calibrated probability-error bound is
Here is the evaluation distribution, its stratum masses, all evaluated strata have , and . Calibration and predictor events can be combined by a union bound; independence between the two fits is unnecessary.
Relation to the cited theory. Kiryo et al. [1] analyze non-negative PU risk; Coudray et al. [2] give classification-risk bounds under SAR; Taki et al. [3] derive a minimax lower bound for low-rank logistic coefficient estimation. Our bound propagates paired-calibration error through predictor selection and refitting. In Proposition 2, the nuclear-norm factorization identity is established [4, Lemma 6]. Minimizing the nuclear penalty plus the quadratic residual penalty with a rank-K shared component yields our truncated spectral discount.
A two-point construction gives the necessary paired-information scale: for p in [.4,.6] and e=.1/p, m independent complete paired rows imply worst-case probability MSE at least 1/(400m), regardless of the number of unpaired detections. All unpaired readouts have the same Bernoulli(.1) law in this construction. This lower bound establishes the calibration bottleneck; it does not establish minimax optimality of the complete selected estimator.
References
[1] R. Kiryo et al. Positive-Unlabeled Learning with Non-Negative Risk Estimator. NeurIPS 30, 2017.
[2] O. Coudray et al. Risk Bounds for Positive-Unlabeled Learning Under the Selected At Random Assumption. JMLR 24(107):1–31, 2023.
[3] B. Taki et al. A Minimax Lower Bound for Low-Rank Matrix-Variate Logistic Regression. Asilomar Conference on Signals, Systems, and Computers, 2021.
[4] R. Mazumder, T. Hastie, and R. Tibshirani. Spectral Regularization Algorithms for Learning Large Incomplete Matrices. JMLR 11:2287–2322, 2010.
Reviewer 76iF — 2. Expression robustness, model selection and biological signal
Q1: Does RNA-dropout correction change the result?
Gene-observation indicators identify unmeasured panels; SAVER [1] addresses noise in measured RNA expression. Our inductive SAVER adaptation fits the Poisson-lasso mean and empirical-Bayes prior on training cells, then applies the fixed transform to held-out cells. Gene panels, splits, calibration and selection are fixed. Mean reference AP over assayed pairs rises from .1593 to .1772, with an increase in each of the six held-out animals. We average targets within each animal, then animals equally. At the same budget of 11,955 selected cell–target pairs, confirmed-positive recoveries increase from 1,001 to 1,177. The endpoint-only selector recovers 1,178 after correction.
Prediction correlations before and after correction are .937–.970, and gene-coefficient correlations in common units are .859–.906; selected ranks and leading edges change. In MERGE, correction improves AP at full retention (.4397 to .4509) and reduces it under severe label loss (.3952 to .3892).
Q3–Q4: Is model selection consistent, and what does the selected rank mean?
We use the same selection procedure for every dataset: validation readout log loss selects the model family, width and penalties. We screen widths {1,2,4,8,16,min(d,T)}, refine the two best converged widths over penalties {10^-4,10^-3,10^-2,10^-1,1,10}, and retain independently tuned Direct and shared-only candidates. Test labels are withheld throughout selection. Doubling the joint search budget from 32 to 64 leaves predictions unchanged in 11 of 12 tested A1/M1 cases. Alternative grouped validation splits yield prediction correlations .967–.997; the selected model family stays the same in one to three of the three outer folds, depending on the dataset; rank and penalty also vary.
At 20% positive retention, shared-only models outperform Direct in M1 at every tested width; small widths underfit MERGE and SPIDER. The amount of sharing adapts to the dataset. Each component has at least 5% of the total coefficient norm in 52 of 87 selected M1 joint fits; 73 of 89 MERGE joint fits have negligible shared coefficients. The selected width therefore describes the capacity for pooling, while the total coefficient matrix determines gene–target predictions. We use that total matrix for biological analysis because it is invariant to rotations of the factors.
Q5: Does the inferred molecular signature reproduce in independent biological data?
We tested the 32-gene projection signature defined by the total coefficients B = LA^T + R, which are invariant to factor rotations. We transferred this frozen TAGs score to independent retrograde-tracing data from Golan et al. [2] (GEO GSE293184). Genes, coefficients and score direction were fixed before testing associations with the external tracing labels. Across 12 libraries grouped into six cohorts, depth-stratified concordance is .616 [95% interval .560,.671], compared with .532 for Cdh13 alone and .487 for depth. Concordance measures how often the score ranks a tracing-positive cell above a tracing-negative cell of similar depth; .5 denotes chance. Removing Cdh13 leaves concordance at .611. Matching additionally on detection of Bcl11b, Crym and Fezf2 gives .566 [.523,.611]. The total-coefficient signature transfers to an independent tracing dataset. We have not independently validated separate shared and residual gene modules. Dataset choice was retrospective, and tracing labels remain partially confounded with library identity.
A complementary control tests depth confounding within TAGs. We compared two models trained on the same paired references and incomplete readouts. Both use the same depth spline, assay covariates, cell-dependent detector and selection rule; one additionally receives the 32 gene-expression values. Within five sequencing-depth groups defined from training cells, adding the 32 genes increases AP for missed reference-positive projections from .0645 to .1084, with improvement in all six held-out animals (difference .0439; animal-bootstrap 95% interval [.0281,.0619]). AP averages target–depth groups within each animal and then animals equally. These follow-ups reuse the six previously examined animals, and the intervals condition on fitted predictions. The gene-expression measurements help distinguish missed projections among cells with similar sequencing depth.
References
[1] M. Huang et al. SAVER: gene expression recovery for single-cell RNA sequencing. Nature Methods 15:539–542, 2018.
[2] N. Golan et al. Anatomical Diversity of the Adult Corticospinal Tract Revealed by Single-Cell Transcriptional Profiling. Journal of Neuroscience 43(47):7929–7945, 2023. Data: GEO GSE293184.
I thank the authors for the extensive experiments and clarifications in their response, and I recognize the importance of the problem the paper raises: projection readouts can miss true projections, so an assay zero does not distinguish absence from missed detection.
My assessment of the methodological and theoretical novelty, however, remains negative. The authors' own ablation confirms that the main gains come from the better use of the detection rate estimated from the paired measurements. The published forms of the external baselines offer no natural way to consume such paired reference labels: GenEML always treats exposure as a latent variable inferred by EM, and SAR-PU estimates its propensity model from the unlabeled data itself. Their original comparison therefore reflects the calibration information available to each method rather than a new model structure. On the theory side, even with the added results, I do not see a new statistical question or new proof techniques. If the theory is meant to justify the method, I would ask the authors to show, on their actual data, how large the sample size needs to be before the probability-error bound becomes meaningful (that is, below 1).
Given the real problem, the responsive rebuttal, and the extensive experiments, I am raising my score to 2, though I still lean toward rejection. I would also encourage the authors to polish the writing: some passages read as AI-drafted and are difficult to follow.
Gene2Wire addresses a central problem in predicting neuronal projections from gene expression: a negative assay readout can be either a true absence or a missed projection. We formulate prediction under this observation process, use paired measurements to calibrate detection, and establish a common evaluation across assays with different gene and target coverage. Under matched access to paired data, Gene2Wire improves predictive performance over adapted baselines across the majority of tested settings.
W1 and Q2. Contribution and calibration access.
Gene2Wire estimates projection probabilities from assays that miss true projections and measure different sets of genes and targets. It combines a calibrated likelihood with shared and target-specific predictors. Our finite-sample analysis bounds how calibration error affects the predictor selected and refitted by the algorithm; the full result is in our W2 response.
Matched-information comparison.
In the original comparison, Gene2Wire and the PU/PU-MIRT controls used paired references to estimate detection sensitivity; GenEML, SAR-PU and Bilinear did not. We have now repeated the comparison giving every method access to the same 20% paired development subset and remaining incomplete readouts.
On MERGE-seq, averaging the nine reported measurement settings and available repetitions, Gene2Wire achieves AP .4183 versus .3851 for SAR-PU, .3825 for Qiao-logit and .3482 for GenEML. Reference log loss is .3945 versus .5145 for SAR-PU and .6380 for GenEML.
We also compared methods on the same test folds, retaining repetitions with all three folds available for every method. Among the 42 dataset–setting combinations with such results, Gene2Wire has higher AP in 40 against GenEML, 41 against SAR-PU and 32 against Qiao-logit. Its Brier score and reference log loss are lower in all 42 against GenEML and SAR-PU. GenEML and SAR-PU use adapted implementations. Qiao-logit is a logistic target-identity adaptation of Qiao's bilinear model [1]; the original Bilinear comparison used squared loss.
Which part of the method contributes?
We evaluated calibrated likelihood and cross-target sharing separately. Holding the Direct predictor, sensitivities, solver and penalty grid fixed, forward likelihood reduces reference log loss relative to signed-IPW by .235 in A1, .152 in M1 and .162 in MERGE at 20% positive retention, averaged over the tested coverage settings and folds; changes in SPIDER and TAGs are much smaller.
An assayed zero has probability 1-e p(X); an unmeasured target contributes no loss. The coefficient decomposition determines how projection predictors share information across targets. Fitting ordinary logistic or reduced-rank regression directly to assay calls instead estimates the detection probability ep(X).
For B=LA^T+R, the induced penalty is
ΩK(B)=μ∣B∣F2/2−μ∑j≤K(σj(B)−λ/μ)+2/2.
This yields linear penalization of strong shared directions and quadratic penalization of smaller departures. K controls how many directions receive the shared discount; the residual permits a full-rank total predictor. Reduced-rank GLMs [2] restrict the coefficient rank, while the Dirty model [3] pools through row/entry sparsity. Gene2Wire pools through a shared spectral component with dense target-specific departures. The factorization identity and spectral alignment use established results; the rank-K truncated discount follows from minimizing the shared nuclear penalty jointly with the quadratic residual. We also tested the Dirty penalty with matched inputs, calibration, likelihood and splits. On MERGE at 20% retention, Gene2Wire's AP is .395 versus .380 for Dirty across three folds. At full retention, Dirty has higher mean AP (.442 versus .440).
The additional joint candidates were tested against selection among independently tuned Direct and shared-only models, with calibration, validation and test entries held fixed. Across 45 combinations of gene coverage, target coverage and positive retention per dataset, their largest average AP increment is .0029 in M1, where log loss increases by .0012; average AP changes in the other datasets are below .001 in magnitude.
References
[1] M. Qiao. Deciphering the genetic code of neuronal type connectivity through bilinear modeling. eLife 12:RP91532, 2024.
[2] T. W. Yee and T. J. Hastie. Reduced-rank vector generalized linear models. Statistical Modelling 3(1):15–41, 2003.
[3] A. Jalali et al. A Dirty Model for Multi-task Learning. NeurIPS 23, 2010.