acceptodds
Under review as a conference paper at ICLR 2027

BRIDGE: A Multimodal Dataset for Biological Representation Learning across Imaging and Gene Expression

Abstract

Cellular responses to perturbation include molecular and phenotypic changes that cannot be captured in a single assay. Multimodal approaches use two or more assays to capture complementary data on these changes from the same experimental setting. For example, single-cell RNA sequencing (scRNA-seq) provides detailed but destructive measurements of transcriptomic state, while live-cell imaging non-destructively measures phenotype changes. A multimodal dataset that uses both assays captures more features of the system. Here, we introduce a multimodal dataset combining time-series live-cell imaging at 20-min resolution with endpoint long-read scRNA-seq comprising 16,296 cells from experimentally matched populations of human fibroblasts. We transiently perturb cell-cycle synchronized fibroblasts using modified mRNA (mmRNA)-mediated MYOD1 overexpression, RNA interference (RNAi)-mediated PRRX1 suppression, and their combination. Since the imaging and sequencing measurements were obtained from different cells, direct cell-to-cell supervision is unavailable. We therefore formulate the integration as a cross-modal prediction problem, using cell-cycle state as a shared biological anchor across modalities. The resulting model predicts population-level gene programs from imaging-derived features, an improvement over prediction based on cell-cycle phase alone. Together, this dataset enables the development and benchmarking of methods that integrate time-resolved cellular phenotypes with transcriptomic states, while our results demonstrate the potential of non-destructive imaging for inferring transcriptional responses to cellular perturbations.

Primary area:
datasets and benchmarks
Keywords:
multimodal learning, single-cell transcriptomics, live-cell imaging, cellular perturbation

Reviews

Among:
Shape:
1234this paper, Top 15%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.00 over 3 reviews, above 85% of the others.
Reviewer_CH1dOpenReview ↗
Rating
4(+1)
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

authors present a dataset of 20 min live FUCCI imaging with 47k tracks and endpoint long read scRNA seq with 16k cells from BJ fibroblasts under MYODI mmRNA, siPRRX1 and their combination. Modalities are aligned by condition and the paper benchmarks 10 RNA embeddings and 6 perturbation predictions.

Critical strengths

the resource is completely novel. No public equivalent method combines time-resolved FUCCI tracking paired with long-read scRNA-seq under single and combinatorial TF perturbations.

authors include useful controls for example causal temporal features, a phase/progress-matched shuffle, with acknowledgment of the condition and library confound.

Critical weaknesses

Main issue is that inferring transcriptional response from imaging is not tested.

There is no within-condition, per-program, or leave one condition out evaluation.

RNA control is a separate unsynchronized library so control-subtracted metrics partly reflect batch.

Questions for authors

How many wells are there per condition?

Were the 16 features ranked on training wells only?

What is the condition classification accuracy?

Which imaging frames are matched to the 48h atlas?

Does performance fall to chance for frames before 24h when MYOD1 has not yet been added?

Please report per-program R2 and a within-condition evaluation. Also report the additive and no change baselines for the held out double, and say how overexpression and siRNA were encoded for GEARS and scGPT.

This is a good dataset and valuable resource in perturbation modelling. The paper is well written. If all my concerns are addressed i will raise score by 1 point. If the concerns arent addressed or not justified properly i will reduce the score by 1 point.

5 comments
AuthorsOpenReview ↗

We clarify the benchmark's population level scope report within condition results and assess control batch confounding.

Dear Reviewer,

We thank you for your detailed and thoughtful comments and questions.

 

W1: Main issue is that inferring transcriptional response from imaging is not tested

The imaging and sequencing data were measured from different cells and the dataset does not support single-cell expression inference. Our held out well transcriptional program inference benchmark provides a quantitative cross-modal benchmark at a population level only.

 

W2: There is no within-condition, per-program, or leave one condition out evaluation.

With the current model architecture, the model cannot generalize to unseen conditions and cannot perform direct RNA or LOCO(Leave-One-Condition-Out) prediction. We restrict our claims to population level state recovery and report program specific and within condition results in the held-out well state recovery task below.

Table 1: Transcriptional Program Prediction:

ModelPooled RNA-atlas R2
Manual features0.576
MorphEm0.569
DINOv1 ViT-S/160.559
DINOv2 ViT-S/140.549
OpenPhenom0.547
ChAda-ViT0.52
SubCell0.458

Table 2: Condition balanced accuracy:

ModelCondition balanced accuracy95% CI lower95% CI upper
Manual features0.4370.4230.448
MorphEm0.4710.4580.483
DINOv1 ViT-S/160.3970.3880.407
DINOv2 ViT-S/140.3320.3240.341
OpenPhenom0.3270.3170.336
ChAda-ViT0.30.2910.31
SubCell0.3380.3260.35

Table 3: Per program R2

Transcriptional programManual featuresMorphEmDINOv1 ViT-S/16DINOv2 ViT-S/14OpenPhenomChAda-ViTSubCell
P01: Early myogenic regulatory program0.150.1010.0740.0670.0840.032-0.055
P02: Late skeletal-muscle differentiation0.290.3430.30.2710.2720.2680.335
P03: Sarcomere and myofibril organization0.028-0.0660.0010.0770.02-0.008-0.149
P04: Myoblast fusion competence0.2660.2680.2460.2440.2490.2310.199
P05: Fibroblast and extracellular-matrix identity0.0630.0660.0380.0270.025-0.005-0.109
P06: PRRX1-associated myofibroblast / TGF-β state0.0970.1410.1080.0670.0420.022-0.036
P07: Cell-cycle exit0.190.2110.1980.1910.1830.1970.24
P08: S-phase0.8760.8730.8650.8560.8650.8520.853
P09: G2/M0.980.9820.9810.9790.9790.9790.979
P10: Actin–cytoskeletal remodeling / cell shape0.3170.3210.3060.3030.3060.2930.226
P11: Focal adhesion / integrin mechanotransduction0.3690.3590.3310.340.3430.3070.194
P12: Type-I interferon / innate RNA sensing0.1780.1160.1230.0890.065-0.062-0.333
P13: Oxidative / mitochondrial maturation0.6180.590.5730.5720.5760.5410.501
P14: Senescence / maladaptive growth arrest0.2170.2480.1570.1180.1410.1290.203

 

W3: RNA control is a separate unsynchronized library so control-subtracted metrics partly reflect batch.

The control RNA-seq data was acquired as a separate library and is perfectly confounded by the control condition. We therefore retrained the imaging models with Control excluded from preprocessing, training and evaluation. Perturbation only R2R^2 for all imaging representations, demonstrating that the perturbation association is independent of learning the control class. This analysis does not resolve the batch confounding in perturbation vs control comparisons.

Table 4: Model retraining results without control

ModelOriginal model, perturbation-only scoringRetrained without controlGain after retraining
Manual features0.7190.8030.084
MorphEm0.7180.7550.036
DINOv1 ViT-S/160.6940.7510.056
DINOv2 ViT-S/140.6730.7440.071
OpenPhenom0.6760.7550.079
ChAda-ViT0.620.7060.086
SubCell0.5520.6850.134

 

Our next comment addresses the questions raised.

AuthorsOpenReview ↗

Here, we clarify questions on replication, temporal performance and perturbation encoding

Q1: How many wells are there per condition?

The imaging data was captured within 3/3/2/2 wells respectively for Hybrid/Negative Controls/mMYOD1/siPRRX1 Each RNA-seq experimental condition was captured as a single biological sample.

 

Q2: Were the 16 features ranked on training wells only?

Yes, the 16 features were ranked on the training wells only. The selected features have been updated in the manuscript

 

Q3: What is the condition classification accuracy?

Table 5: Condition classification accuracy
ModelCondition classification accuracy
Manual features0.437
MorphEm0.471
DINOv1 ViT-S/160.397
DINOv2 ViT-S/140.332
OpenPhenom0.327
ChAda-ViT0.3
SubCell0.338

 

Q4: Which imaging frames are matched to the 48h atlas?

No imaging frame is directly matched to a single RNA cell. All imaging observations that pass the quality metrics are evaluated against the 48-hour RNA atlas based on the condition, phase and trajectory.

 

Q5: Does performance fall to chance for frames before 24h when MYOD1 has not yet been added?

To determine the performance of MYOD1 frames before and after the 24h point, we analyzed the probability and accuracy on these as two separate batches. The pre 24h MYOD1 class recall of 0.455 increased to 0.733 after the induction mark, while the MYOD1 probability increased from 0.335 to 0.503. However, MYOD probability probability in control also increased from 0.29 to 0.502. We interpret this as well dependent drift rather than MYOD1 specific induction. (Rebuttal Table 6)

Table 6: Pre and post MYOD induction accuracy
Program scoringTrue conditionAccuracy, pre-24 hAccuracy, post-24 hMean P(MYOD1), pre-24 hMean P(MYOD1), post-24 h
Standardized meanControl0.4950.0650.2900.502
Standardized meanMYOD10.4550.7330.3350.503
Standardized meansiPRRX10.6970.3550.0640.177
Standardized meansiPRRX1/MYOD10.2320.2970.3270.400
UCellControl0.5030.0510.2860.499
UCellMYOD10.4470.7500.3290.503
UCellsiPRRX10.6890.3680.0670.178
UCellsiPRRX1/MYOD10.2330.2690.3490.417

 

Q6: Please report per-program R2 and a within-condition evaluation. Also report the additive and no change baselines for the held out double, and say how overexpression and siRNA were encoded for GEARS and scGPT

For the heldout double perturbation, the no change baseline produced (R^2=-0.017) and RMSE (=0.193), whereas the strict additive baseline produced (R^2=0.308) and RMSE (=0.159).

For GEARS and scGPT, the perturbation identity was encoded only as perturbed gene identities. The perturbation direction was not explicitly encoded for PRRX1 knockdown and MYOD1 overexpression. Instead the models learnt the target specific response associated with each perturbed gene identity from the observed single perturbation profiles.

All relevant tables and the figures have been updated in the manuscript supplement

Reviewer_CH1dOpenReview ↗

Acknowledging rebuttal

Thank you authors for detailed rebuttal. these answered my questions and I appreciate the candid scoping to population-level recovery.

some concerns remain.

(1) within-condition evaluation is missing. please report per program R2 for each condition and gain over a phase matched baseline if possible. this is because pooled result is dominated by the cell cycle programs P08 & P09.

(2) Please state whether GEARS and scGPT beat the additive baseline on the same metric.

i will raise the score to 4 if these are addressed or justified in some way.

Reviewer_RLAfOpenReview ↗
Rating
3(+2)
Confidence
3
Informativeness
2
Presentation
2
Soundness
2(+1)
Read review
Paper summary

The paper introduces BRIDGE dataset which is a multimodal dataset across live-cell imaging and single-cell RNA sequencing data. The imaging and sequencing come from different cells but the authors link the two modalities at the population level, using cell-cycle phase. They have evaluated how well single-cell RNAseq foundation models perform on these tasks. They also create a BRIDGE model, an MLP on imaging features to predict population-level scores.

Critical strengths
  1. The paper provides a novel resource where they pair live-cell imaging, with FUCCI cell-cycle reporters and long-read scRNA-seq under the same perturbations.
  2. The paper has extensive details on how the dataset was obtained.
  3. The dataset has been benchmarked against many scRNA-seq representation models.
Critical weaknesses
  1. Datasets and benchmarks papers need to showcase how they will further help advance machine learning research by providing new open problems for the community to solve which the paper fails to address. The BRIDGE dataset has interesting data with live-cell imaging + scRNA-seq but the paper fails to create a new task around it. All the three tasks reported in Section 5 focus only on RNAseq. The paper has not used its multi-modal dataset to its full potential. It attempts to do this in Section 6 but the model learns to predict perturbations only from imaging features. The gene-program predictions are probability-weighted averages of fixed RNA reference means, so the learned component maps imaging to condition probabilities. The task that they propose here is not clear, and looks underdeveloped. All the three tasks reported in the paper focus on RNA only present in Section 5. The one cross modal task in Section 6 has been described as their own modeling strategy instead of being a benchmark. What are the new benchmarking tasks that this paper is proposing?
  2. The paper mainly focuses on benchmarking scRNAseq foundation models but there is no analysis on how cellular morphology foundation models can be applied here such as SubCell, MorphEm, and DINOv3. The BRIDGE dataset could also potentially be used to see if morphology representations and scRNAseq representations align or not at some level and to what extent.
  3. The out of distribution split available in the benchmark is limited. They only showcase one OOD split in Table 3. The BRIDGE model is evaluated only on held out wells from conditions seen in training. Additional splits will make the benchmark more informative. The experimental scope provided in the data is limited as it only has one cell line, and 3 perturbation conditions which limits the scope how complex you can make the benchmarks to be.
Questions for authors
  1. Which 16 features were selected? Section 6 points to “supplement ??”. The paper describes no separate validation split.
  2. If the authors can address my weaknesses thoroughly, it could affect my assessment.
8 comments
AuthorsOpenReview ↗

We have clarified the BRIDGE benchmark and the model, added comparisons with six morphology foundation models and address the limits of the dataset.

Dear Reviewer,

We thank you for your questions and comments.

 

W1: What are the new benchmarking tasks that this paper is proposing?

We agree that the cross-modal benchmark was not clear. As the imaging and sequencing data were measured from different cells, the dataset does not support single-cell expression reconstruction. Instead, we propose the prediction of transcriptional program summaries from tracked-cell imaging histories as an unpaired population level task. The BRIDGE model acts as an initial baseline that tests whether imaging contributes information to this population level state without assuming imaging - RNA pair existence. The dataset also enables future work on temporal morphology models, direct imaging to program regression, and imaging to endpoint RNA forecasting and distribution level prediction.

 

W2: The paper mainly focuses on benchmarking scRNAseq foundation models

Based on the reviewer’s suggestion, we have extended this benchmark by evaluating representations from six additional cellular morphology foundation models MorphEm, DINOv1 ViT-S/16, DINOv2 ViT-S/14 , OpenPhenom, ChAda-ViT, and SubCell. Using identical data splits, we tested how each representation recovered the RNA-program and have added the results to the manuscript.

 

W3: The out of distribution split available in the benchmark is limited

We acknowledge that the current split of training and testing does not contribute to out of distribution generalization and are currently limited to the three perturbation conditions. Instead, our benchmark emphasizes the time-series morphology within the tested conditions enabling questions about dynamics that cannot be studied using static imaging or sequencing alone.

 

Q1: Which 16 features were selected? Section 6 points to “supplement ??”. The paper describes no separate validation split.

The 16 selected features are listed below. These features were selected based on the training wells only.

rankfeature
1std4:oblique_masked_glcm_homogeneity_mean
2std4:nearest_neighbor_dist_px
3mean4:haralick_05
4std4:oblique_zernike_41
5std4:oblique_zernike_29
6std4:oblique_zernike_48
7std4:oblique_masked_glcm_energy_mean
8mean4:glcm_contrast_std
9mean4:oblique_zernike_21
10std4:oblique_masked_glcm_dissimilarity_mean
11std4:oblique_zernike_05
12mean4:glcm_energy_mean
13mean4:local_density_100px
14mean4:neighbors_within_100px
15std4:oblique_zernike_19
16std4:area_um2

We have fixed all proofreading errors and updated the manuscript.

We used a well level training / test split rather than having a third validation set for the final evaluation. All preprocessing, feature ranking and modeling were performed on the six training wells. We have clarified the design and acknowledge that the number of independent wells limits a three way split of the dataset into train/test/validation.

Reviewer_RLAfOpenReview ↗

Not enough details in the "cross-modal" benchmark.

Thank you for updating your manuscript.

Some suggestions on content placement for readability: Section 1.1 (Biological Model System) and Sections 4.1–4.2 (data processing) could be moved to the supplement. The space this frees up could then go to the benchmarking aspects of the paper. ICLR allows up to 10 pages in the rebuttal phase, so the authors can reprioritize content to better showcase the third contribution listed in lines 81–87.

I am still not convinced by the cross-modal prediction benchmark task. Section 6 gives very few details. Explaining it more fully in the main text, using the space freed up above, would help clarify the paper’s specific contribution there. In particular, how do the authors connect the transcriptional programs to the morphology features? The details are spread across Appendices A.10–A.16 and are difficult to follow.

Some recommendations for improving the explanation:

  1. Add a schematic of the pipeline, from the imaging features to the predicted transcriptional programs.
  2. Define a fixed benchmark protocol (splits, metrics, baselines) that future methods can be compared against.
  3. State explicitly what the task measures and what it cannot measure.
  4. Explain how this task is useful to the community and how it enables other researchers to make progress in what specific domains/questions.

I think the dataset is novel and will be useful to the community, but it needs a proper benchmarking harness to be used well. The benchmarking contribution is still unclear, and clarifying it would better help the work.

AuthorsOpenReview ↗

We specify the cross-modal benchmark with additional sections and schematic

Dear Reviewer,

We thank you for the thoughtful suggestions.

We have updated the manuscript as follows

  • Shortened the biological introduction and moved the data processing sections to the supplement.
  • Added Figure 3, which provides a schematic of the complete training and evaluation pipeline from the inputs to the predictions.
  • Added Sections 7.1 - 7.3 and Table 4 detailing the benchmarking protocol and model construction, including data splits, evaluation metrics and baseline definitions.
  • Added Section 8 (Discussion) to outline the potential tasks and clarify scope of the benchmark including what it evaluates and what falls out of scope.

We will also release a complete benchmark harness for users to run the benchmarks and evaluate their own models.

Reviewer_gvkKOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
1
Soundness
2
Read review
Paper summary

The paper presents a multiomic data set that includes time series single cell imaging data as well as endpoint scRNA-seq data. The data comes from two different populations of cells (FACS sorted proliferating fibroblasts and synchronized fibroblasts). Cell cycle labels were assigned using the Fucci reporter system, as well as marker gene inference from scRNA-seq. The dataset is used for benchmarking a variety of existing models.

Critical strengths

This type of data set is potentially useful and interesting, if well presented.

Critical weaknesses

The paper is poorly organized and hard to follow. It seems like the paper is about a dataset, but then it also includes benchmarking of existing methods on that dataset, and then at the very end we learn about a model (with the same name as the data set). The introduction needs to do a better job of clarifying your contribution and explaining the organization of the paper. I didn't even know you were going to do modeling until I got to that section of the paper. The paper doesn't refer to Figures 1 and 2 at all in the text. Figure 1C, which gives an overview of the dataset, seems like it doesn't convey the top-level split of the data ("The experiment involves two data subsets.")

I couldn't make sense of the labeling approach. The cell cycle labels from Fucci are extremely limited in resolution, and no details were given on how you actually called cell cycle phases from the Fuccie markers. Similarly, the cell cycle labels from RNA-seq marker genes are generally not super trustworthy. But then you also do some kind of trajectory inference (lines 286-289 and Section A.7).

Questions for authors

What is the rationale for the trajectory inference labels? How do these trajectories supposedly relate to cell cycle labels?

2 comments
AuthorsOpenReview ↗

We clarify the rationale behind cell cycle and trajectory labels and perform analysis on phase assignments for both RNA-seq and Imaging

Dear Reviewer,

We thank you for your feedback and constructive suggestions. We agree that our manuscript did not clearly describe the dataset, the unimodal benchmark, and the cross-modal baseline. We have updated the structure and text of the manuscript to clearly define each element and how it is used.

 

  1. Labeling approach: RNA and Imaging were independently annotated for cell-cycle phase and progress
  • RNA-seq data:
    • Cell-cycle phase: The scanpy function score_genes_cell_cycle was used to assign cell-cycle phase to each cell, with gene sets based on published work [1] used as input to the function.
    • Trajectory: Control cells were embedded in a 15-component PCA and a TrajectoryNet model was trained on the velocity-matched controls. A 100 position control trajectory was constructed. Every cell was assigned to its nearest trajectory position and subsequently grouped into 10 progress bins.
  • Imaging data:
    • Cell-cycle phase: A custom phase calling pipeline was built on smoothed, track-aware FUCCI reporter dynamics using TagGFP and mKate, with transition logic and confidence filtering. We have added details about the pipeline as part of the manuscript.
    • Trajectory: Each observation was assigned a phase associated progress value and subsequently mapped to the corresponding 10 bin coordinate used for the RNA data. This was introduced to provide a shared stratification of cross modal comparison.

The rationale for using trajectory is to aid cross-modal data alignment, provide a more refined ordering at the level of the individual cells rather than only the broad category of cell cycle phase, and to enhance detection of more subtle biological signals than the cell cycle.

 

  1. In the RNA-seq data, control cells in G1, S, and G2/M cell cycle phases were isolated from a population of proliferating fibroblasts. Isolation was performed by staining with Hoechst to label cells prop by amount of genomic DNA, selecting an appropriate gate for cells in each cell cycle phase, and physical isolation by fluorescence-based cell sorting before sequencing. These experimentally derived cell-cycle sorting labels were used to validate the RNA based phase assignment applied to the full dataset. The assignment accuracy is reported below

Table 1. RNA-assigned versus sorting-derived phase

Assigned G1Assigned SAssigned G2M
Sorted G12,437 (55.1%)1,045 (23.6%)937 (21.2%)
Sorted S271 (21.9%)520 (42.0%)446 (36.1%)
Sorted G2M77 (6.4%)232 (19.4%)888 (74.2%)

 

  1. We additionally performed sensitivity analysis on the image phase calling pipeline to determine the phase-label agreement and valid observation sensitivity.

Table 2. Sensitivity analysis on phase-label agreement.

MeasureDefaultMedianRange
Phase agreement with default (%)10099.6997.1–100.0
Valid observations (%)25.5325.4321.0–30.9
Valid observations retained (%)10095.0366.3–100.0

The phase calling pipeline and the sensitivity analysis results have been added to the updated manuscript.

Dear Reviewer,

Thank you again for your feedback.

We have substantially updated and reorganized the manuscript and have added analysis and clarifications addressing your questions. We would welcome any feedback on whether the revisions have addressed your concerns

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.