acceptodds
Under review as a conference paper at ICLR 2027

FACET: Factorized Asymmetric Conditioning for Efficient Transport in High-Fidelity Fluorescence Microscopy Synthesis

Abstract

Fluorescence microscopy reveals where proteins localize, but only a limited number of proteins can be imaged in the same cell; generating these images from amino-acid sequence and the cell's morphological context enables in silico localization of unimaged proteins. The two conditions, however, play asymmetric roles: morphological context is spatially aligned with the target, whereas sequence is non-spatial and must specify protein-dependent localization within it, with recurring coarse patterns shared across proteins and finer protein-specific variation. Existing generators condition on both jointly, without separating what each explains. We introduce FACET (Factorized Asymmetric Conditioning for Efficient Transport), a probabilistic generative framework that encodes this structure as an explicit inductive bias: sequence semantics are learned from what context leaves unexplained, coarse localization regularities are shared across proteins through a semantic memory, and protein-specific variation is a bounded residual around them. A variance-preserving state projection further lets FACET perform continuous stochastic transport through a pretrained diffusion predictor with minimal parameter overhead. On held-out proteins, FACET improves spatial overlap by 34.3% on the Human Protein Atlas and 14.0% on OpenCell over a backbone-matched baseline, and reduces FID by 27.2% and 46.5%, respectively, with 75% fewer network evaluations. It also substantially improves protein-association structure recovery and yields better-calibrated predictions, while detailed ablations show complementary contributions from its design choices. These results identify factorized asymmetric conditioning, rather than generator capacity alone, as a key lever for high-fidelity, efficient, and biologically meaningful cellular image synthesis.

Primary area:
applications to physical sciences (physics, chemistry, biology, etc.)
Keywords:
Conditional generative modeling, Stochastic flow matching, Protein subcellular localization, Fluorescence microscopy synthesis

Reviews

Among:
Shape:
1234this paper, Bottom 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.20 over 5 reviews, above 49% of the others.
Reviewer_CZoMOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper addresses in silico protein localization: generating fluorescence-microscopy images of a protein from its amino-acid sequence and a morphological context (nucleus/ER/microtubule channels). Its premise is that the two conditions are asymmetric. Thee context is spatially aligned with the target but the sequence is not. That sequence semantics are hierarchical, mixing coarse localization patterns shared across proteins with finer protein-specific variation.

FACET encodes this as an explicit inductive bias: a semantic memory of K = 9 prototypes built from context-residualized target features a sequence-conditioned soft readout over that memory, and a bounded continuous residual around it. A variance-preserving state projection lets a frozen, pretrained latent-diffusion ε-predictor drive a continuous stochastic transport, with a LoRA adaptation, an ε-adapter, ordered context-then-sequence FiLM, and a gated velocity-correction head.

The Validation is perfomed on protein-held-out HPA and the OpenCell database, against CELL-E2 and a backbone-matched CELL-Diff, across IoU, FID, MS-SSIM, Haralick, PSNR, MI, plus STRING-based association recovery, calibration, ablations, cross-dataset transfer, and a sampler/NFE study. Reported gains: IoU +34.3% (HPA) and +14.0% (OpenCell), FID −27.2% and −46.5%, at 25 vs 100 NFE.

Critical strengths

1.) All systems share the same pretrained ε-predictor, VAE, and ESM2 encoder, which is the right control surface for a conditioning study. The evaluation spans two datasets, four context compositions, several metric families, transfer, and a sampler/NFE sweep, and the paper reports its own negative results: FACET loses to CELL-Diff on MI, MS-SSIM, and PSNR under nucleus-only context, and the HPA-transferred model has lower MS-SSIM than refit on OpenCell.

This transparency and extensiveness of tests and results comment is a strength and makes the positive results very credible

The Proposition 2's variance-preserving normalization is exact, covers the interpolant's latent-noise term, and lets the model add ~0.6% trainable parameters while cutting sampling from 100 to 25 NFE with better FID. If the attribution issues were resolved,
Critical weaknesses

) The conclusion states that gains are "attributable to FACET's factorized probabilistic formulation rather than generator capacity," but not all evidence support that sentence.

FACET differs from the matched baseline in at least five coupled ways: 9.83M extra trainable parameters including LoRA on the backbone, staged training, 3 auxiliary losses that the baseline does not have, the annotation-derived semantic pathway, and a different sampler at a different NFE. The pathway ablation (Tab. 5: 0.2581 → 0.2826 → 0.3328) is cumulative rather than leave-one-out, and its largest single jump comes from backbone LoRA, it is a component that is about capacity and not factorization.

No control matches parameter count, losses, and training schedule while using joint (non-factorized) conditioning, so the paper cannot distinguish "factorized asymmetry effects" from "any added conditioning capacity with these losses dynmaics."

) Most reported numbers are single point estimate s on the 97 held-out proteins (696 HPA / 454 OpenCell images): no seeds, no protein-level bootstrap, no confidence intervals. At this scale FID with ImageNet-Inception features may be biased and unstable. SOme l claimed orderings such as HPA-adapted vs OpenCell-adapted or the memory ablation gaps may lie within noise.

In addition, protein-held-out splits are not homology-controlled: ESM2 embeddings of test proteins share family-level signal with training homologs, so "unseen protein" is weaker than it sounds, and no sequence-identity filtering or paralog analysis are reported.

Questions for authors

Have you trained a control matched in trainable parameters, losses, and training schedule, but with joint conditioning (concatenation or cross-attention) and no semantic memory? Is Tab. 5 cumulative or leave-one-out, and what does a LoRA-only variant (no semantic pathway) achieve?

How much of the gain survives when the semantic memory is built without UniProt annotations, for example with k-means or VQ prototypes in the same residual space? Is there a K sweep (3–32) and a stability analysis of the prototypes?

I feel the asymmetry is asserted but not really tested: what happens with a reversed-order FiLM (sequence first, then context), or parallel injection? FiLM is a global channel-wise modulation, so what evidence is there that the spatial alignment of context is actually being exploited?

Can you provide protein-level bootstrap intervals and multi-seed runs for the headline metrics, and KID or precision/recall alongside FID at n = 696?

Are the splits homology-filtered? What is the maximum sequence identity between any test protein and the training set, and how do results change on a ≤30% identity split?

Fig. 3 shows CELL-Diff only at 100 NFE: what is CELL-Diff at 25 NFE, and does the efficiency claim survive when total training cost is counted?

The ELBO of Prop. 1 is optimized blockwise with heuristic regularizers and a fixed isotropic residual scale; how close is the staged objective to the actual bound, and what does Proposition 2 add beyond the standard stochastic-interpolant/diffusion correspondence (Albergo et al.)?

Reviewer_WKPpOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper addresses the generation of a protein fluorescence image in a cell, conditioned on the protein's amino-acid sequence and on landmark morphology channels (nucleus, ER, microtubules). The authors' starting point is that the two conditions play different roles: the morphology is spatially aligned with the target, while the sequence is non-spatial and has to specify where the protein sits within that context. They observe that a simple linear map from context features to target features already explains a large part of the target variance (R² = 0.377 on held-out data, App. B.2), and build the model around the idea that sequence semantics should be learned from what the context does not explain.

FACET implements this as a factorized latent model (Eqs. 2–4). Coarse localization is shared across proteins through a frozen memory of K = 9 prototypes, built from context-residualized image features and a UniProt annotation rule; protein-specific variation is a small residual with a fixed, bounded covariance (Eq. 5, App. D.5). Image generation itself runs on the pretrained variance-preserving ε-predictor of the CELL-Diff checkpoint (frozen except LoRA rank-16 adapters), driven by stochastic transport in a reparameterized state space (Prop. 2, App. D.6), with a memory-conditioned adapter and a context-first hierarchical correction head. The whole system adds 9.83M trainable parameters, 0.588% of the backbone (Table 11). On held-out proteins, FACET improves IoUcont by 34.3% on HPA (696 test images, 97 proteins) and 14.0% on OpenCell (454 images, 97 proteins) relative to the same backbone at 100 NFE, using only 25 NFE, with corresponding FID reductions (Tables 1, 3). Additional results cover STRING association separation (Table 4), calibration (Table 9), and ablations (Tables 5–8).

Critical strengths
  1. The central motivation is concrete, not just rhetorical. The R² = 0.377 figure (App. B.2) is a real measurement, and the memory construction follows from it directly: prototypes are computed from the residuals of the context map, so the sequence pathway is not asked to re-encode what the morphology already determines. I find this the most useful idea in the paper.

  2. The experimental controls are careful. Every method uses the same CELL-Diff checkpoint (App. E.3), so the comparison isolates the conditioning design from backbone capacity. The splits are protein-disjoint, and the calibration threshold and temperature are fit on disjoint partitions (App. E.1, App. G.1), so I do not see test leakage.

  3. The theory is complete, and the authors are unusually honest about its limits. I checked the Appendix D derivations, including the conditional-velocity step at Eqs. 40–41, and they are correct. App. H.1 states plainly that the Jacobian sensitivity analysis does not establish the benefit of the FiLM ordering, and App. H.2 correctly bounds the usable step count at 67 NFE given the inherited schedule.

  4. The efficiency is real and well documented: 25 NFE versus 100, 0.588% added parameters, and the wall-clock numbers (15 vs 78 minutes for the HPA set, App. G.2, on a single L40S per App. E.4).

Critical weaknesses
  1. The scope of the claim outruns the comparisons. All quantitative results are against the CELL-Diff checkpoint, but the abstract says that factorized asymmetric conditioning, "rather than generator capacity alone", is "a key lever", which is a general statement about how this kind of model should be built. A single backbone does not support that. Two 2026 models are relevant here. ProtiCelli (bioRxiv 2026.03.31.715748) is fairly excluded: the paper notes in App. E.3 that it conditions on protein identities from a fixed vocabulary, so held-out-protein evaluation is not defined for it. Vermeer (bioRxiv 2026.06.01.729395) is different. It is sequence-conditioned (ESM-C), supports held-out proteins, and has code and checkpoints released, so a comparison on the 696/454 splits is feasible. The paper does not attempt it, and I think this matters for the abstract-level conclusion.

  2. The efficiency headline needs a matched-NFE counterfactual. FACET reports curves at 10/25/50/67 NFE (Table 8, Fig. 3), but the CELL-Diff reference appears only at 100 NFE. There is also a structural asymmetry the authors themselves point out in App. H.2: the inherited 200-level noise schedule means FACET evaluations above 67 NFE fall outside the supported range, so the two systems cannot be pushed to the same budget. Until I see CELL-Diff at 10/25/50 NFE, I cannot tell whether the 25-versus-100 advantage comes from the conditioning design or from the baseline degrading at low NFE. This is the cheapest fix in the paper and the most important for the efficiency story.

  3. The "sequence semantics" channel depends on UniProt localization annotations more than the narrative suggests. The 9 prototypes are defined by an annotation rule, and both the prototype prior p_η and posterior q_ω are trained with annotation labels (Eq. 14, App. B.1–B.3). At inference only the sequence is used, so there is no leakage, and the paper is correct about that. But the coarse channel still carries a strong prior over localization categories that comes from the annotations, and no control separates this from the learned residual. A memory built from the same residualized features without the annotation rule (for instance, by clustering protein-averaged residuals) would show how much of the gains in Tables 4 and 7 is annotation prior versus learned structure.

  4. Statistical reporting on the main tables is thin. The test splits are 97 proteins (696 images on HPA, 454 on OpenCell). All image metrics are single-run values with no error bars, no confidence intervals, and no statement about seeds. FID at this sample size has non-negligible sampling variance, so the 27.2% and 46.5% FID reductions should at least carry uncertainty. Two related observations: in the nucleus-only rows of Table 2, FACET is worse than CELL-Diff on MI, MS-SSIM, and PSNR, which is worth discussing directly since OpenCell is a nucleus-only context and the ridge residualization uses the three-channel features (App. B.2). And the ECE improvement in Table 9 (0.21501 to 0.05442) is measured after post-hoc temperature scaling (App. G.1), so part of it belongs to a standard post-hoc procedure, with 16 samples per condition making the ECE estimate itself noisy.

Questions for authors
  1. Could you report the CELL-Diff FID and IoUcont at 10/25/50 NFE? That would settle whether the 25-versus-100 advantage holds at matched budgets.
  2. Is it possible to run at least one 2026 sequence-conditioned baseline (Vermeer has released code and checkpoints) on your 696/454 splits? If not, would you scope the abstract's "key lever" statement to this backbone?
  3. What happens to the main results if the 9 prototypes are built without UniProt annotations, e.g., by clustering protein-averaged residuals?
  4. Could the main-table metrics be reported with multiple seeds or bootstrap confidence intervals?
  5. On Table 2: what do you attribute the nucleus-only regression to, and is the three-channel ridge residualization the cause?
Reviewer_VkyxOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes FACET, a method for generating protein fluorescence microscopy images based on the pretrained CELL-Diff model. Its main idea is to use factorized asymmetric conditioning to model the different roles of protein sequences and cell morphology.

The method introduces a semantic memory and a variational residual to model protein localization. It also combines hierarchical conditioning and stochastic transport for image generation.

Experiments show that FACET performs better than the baselines on several image quality metrics while using fewer sampling steps. The paper also reports component ablations, cross-dataset transfer, and prediction calibration experiments.

Critical strengths

S1. Good generation quality and sampling efficiency.

FACET achieves good generation results across different experimental settings. It improves several metrics over the baselines while using fewer sampling steps. This shows some advantages in both performance and efficiency.

S2. A concrete approach to asymmetric conditioning.

The paper explores how to model protein sequence and cell morphology through different conditioning paths. It combines semantic memory, fine-grained residuals, and hierarchical conditioning in a pretrained generative model. This provides a useful technical approach to studying asymmetric conditioning.

Critical weaknesses

W1. Insufficient motivation and experimental support for the main Factorized Asymmetric Conditioning idea.

The paper argues that existing methods suffer from redundant information and mixed semantic features. It therefore proposes factorized asymmetric conditioning. However, previous work has already recognized the different roles of protein sequences and cell morphology. The paper does not clearly explain what specific problems remain in existing joint conditioning methods. It also lacks direct evidence that these problems affect generation quality.

The paper mainly uses linear regression to show that cell morphology can explain part of the target image features. It then subtracts the predicted features from the target features to obtain a residual. However, this only shows that some target features can be predicted from cell morphology. It does not prove that existing models have redundant semantic information or that the residual mainly contains protein-specific information.

In addition, the method section introduces many probabilistic variables and equations without enough background, an overall model overview, or a clear explanation of how the modules are connected. This makes it difficult to understand how each module relates to the main research problem.

W2. Additional supervision, model adaptation, and sampling changes make it difficult to identify the source of the improvements.

FACET adds several modules for semantic modeling, parameter adaptation, and stochastic transport on top of CELL-Diff. It also uses multi-stage training and several loss functions, making the overall method rather complex.

Although the paper provides some ablation studies, the added UniProt localization supervision, further model training, and different sampling methods may all affect the final results. The current experiments do not fully separate these effects through fair comparisons. Therefore, the improvements of the full model do not clearly show that Factorized Asymmetric Conditioning is the main reason for the gains.

Questions for authors

Q1. Evidence for information redundancy and mixed semantic features.

Using protein sequences for localization information and cell morphology for spatial information is already common in existing methods. What specific problems does FACET identify beyond this setting? Is there direct evidence that information redundancy and mixed semantic features exist in current models and affect generation quality?

Q2. Effectiveness of Factorization and Context Residualization.

The linear regression analysis shows that cell morphology can explain part of the target features. However, how can the authors show that the residual mainly contains protein-specific information? Are there direct experiments showing that Factorization and Context Residualization work better than standard conditioning methods?

Q3. Motivation for Asymmetric Conditioning.

Why do the authors choose context-first hierarchical conditioning instead of standard joint conditioning or a different order? Are there direct comparisons showing the benefits of asymmetric conditioning and the chosen order?

Q4. Roles of the modules and auxiliary losses.

FACET introduces several modules for semantic modeling, parameter adaptation, and stochastic transport. It also uses multi-stage training and multiple auxiliary losses. Could the authors explain how these components relate to the main research problem? Are there key loss ablations or simpler baselines that show why these designs are needed?

Q5. How much of the improvement comes from additional supervision and training?

FACET uses additional localization labels and further trains several modules on top of pretrained CELL-Diff. How do the authors separate the effects of extra supervision, additional training, and the proposed model design? Could standard fine-tuning or LoRA achieve similar results with the same supervision and training budget?

Q6. Evaluation metrics and robustness to different conditions.

Could the authors also report the IoU and MSFR metrics used in the original CELL-Diff paper, to make the comparison more complete? Also, FACET performs worse than CELL-Diff on several metrics in the nucleus-only setting. Does this suggest a limitation of the proposed conditioning method, or could it be related to the conditions used during training?

Q7. Definition of Semantic Memory.

What are the nine localization categories used to build the semantic memory? How are proteins with multiple localization annotations assigned to these categories? What is the reason for choosing nine prototypes?

1 comment

Thank you for the detailed rebuttal and additional experiments, which address several of my concerns. However, I still have some reservations about the central motivation. The linear analysis shows that some target features are predictable from morphological context, but it remains unclear how this establishes the claimed limitations of existing joint-conditioning methods. While the new experiments demonstrate the benefits of factorization, the performance degradation on several nucleus-only metrics also remains insufficiently explained. Therefore, I will maintain my original rating.

Reviewer_XLS5OpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

FACET generates protein fluorescence images from amino-acid sequence and cellular morphology. It models the asymmetric roles of these conditions through a context-residualized semantic memory, a sequence-conditioned prototype readout, a bounded continuous sequence residual, and context-first hierarchical modulation. A variance-preserving state projection adapts a pretrained CELL-Diff noise predictor to continuous stochastic transport, with additional modules and LoRA updates trained in stages. On protein-held-out HPA and OpenCell benchmarks, FACET improves continuous image overlap and FID over the corresponding CELL-Diff checkpoints while using 25 rather than 100 network evaluations. The paper additionally examines context ablations, transfer of HPA-trained modules to an OpenCell-trained backbone, STRING association separation, sampler variants, and foreground-probability calibration.

Critical strengths
  1. The design targets a meaningful asymmetry in the application and yields substantial gains on two datasets.

Morphological context supplies spatial organization while sequence contributes localization information. The results are not confined to one image metric: on full-context HPA, overlap increases from 0.2478 to 0.3328 and FID decreases from 45.60 to 33.20; on OpenCell, overlap rises from 0.3722 to 0.4244 and FID drops from 68.90 to 36.83 (Tables 1 and 3). The measured HPA sampling time also decreases, making the efficiency claim more useful than NFE alone.

  1. There is meaningful internal evaluation of the added pathways.

Table 5 separates the memory-conditioned pathway, LoRA, and hierarchical head, while Table 7 tests inference-time memory replacement. Sampler comparisons at matched NFE and a held-out temperature-fitting partition strengthen the efficiency and calibration analyses. The manuscript also explicitly reports worse HPA nucleus-only results for several metrics and clarifies that “HPA-adapted” transfer uses an OpenCell-trained backbone; these qualifications are important for interpreting the scope of the gains.

Critical weaknesses
  • The main comparison does not fully isolate factorization from additional adaptation and annotation supervision.

FACET uses extra training, LoRA, and a memory built from nine annotation-derived categories, whereas the principal CELL-Diff baseline is the unmodified checkpoint (Appendices B.2 and E.3). Table 5 has useful pathway comparisons, but no equally trained LoRA/adaptation baseline without semantic memory, no retrained raw-feature versus context-residualized memory comparison, and no direct comparison of context-first versus joint/reversed conditioning. Inference-time memory replacement changes the input distribution of an already trained model and cannot substitute for these controls. The reported gains support the complete adapted system more strongly than the claim that asymmetric factorization itself is the key cause.

  • The association analysis does not establish a statistically reliable improvement over the baseline.

Table 4 and Appendix F.2 test associated versus null protein pairs separately within each method. A significant contrast for FACET and a non-significant contrast for CELL-Diff is not a direct significance test of the difference between methods. Moreover, many pairs share proteins and the same eight contexts, so treating pair similarities as independent in a Mann–Whitney test can understate uncertainty. This does not negate the descriptive separation gains, but it weakens the biological-recovery claim. The principal image gains also need uncertainty estimates at the protein level to show their stability beyond the reported point estimates.

Questions for authors
  1. Matched adaptation: Can you add a CELL-Diff control with the same training examples, LoRA capacity, auxiliary losses, and sampling budget but no factorized memory, and retrained ablations for context residualization and modulation order? These would determine whether the central inductive bias adds value beyond extra adaptation and localization supervision.

  2. Biological statistics: Can you directly test the paired difference in association separation between FACET and CELL-Diff using protein-aware resampling or an appropriate dependency-preserving permutation? Please provide protein-level confidence intervals for overlap and FID, and clarify when the null switches from localization-matched to unmatched in Appendix F.2.

  3. Generative interpretation: Which parts of the implementation optimize the stated latent-variable model exactly, and which are surrogates or approximations, including the soft prototype mean, flow-matching loss, finite noise-index lookup, and final clean-latent decode? Can you also report train–test sequence similarities so that “held-out protein” generalization is distinguished from homolog-disjoint generalization?

Reviewer_ZRiHOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

FACET generates protein fluorescence images conditioned on amino-acid sequences and cellular morphology. It constructs annotation-supervised localization prototypes from context-residualized image features, refines their sequence-conditioned readout with a variational residual, and applies context-first hierarchical modulation to a pretrained diffusion predictor. Variance-preserving state normalization enables reuse of this predictor within stochastic transport, with lightweight adaptation. Evaluation covers protein-held-out HPA and OpenCell benchmarks, context and component ablations, cross-dataset module transfer, protein-association analysis, and predictive calibration. The reported results show improved spatial overlap and distributional fidelity while reducing sampling evaluations from 100 to 25.

Critical strengths
  1. The performance and sampling gains are substantial. On HPA, IoU improves from 0.2478 to 0.3328 and FID drops from 45.60 to 33.20. On OpenCell, IoU improves from 0.3722 to 0.4244 and FID drops from 68.90 to 36.83. FACET uses 25 network evaluations compared with the baseline’s 100, and the reported runtime measurements support the efficiency benefit. These results make the method practically interesting.

  2. The conditioning design addresses a meaningful distinction in the task. Cellular morphology supplies spatial structure, while sequence supplies protein-dependent localization information. Giving these inputs different roles is well motivated. The experiments across context combinations, datasets, and biological evaluations provide a useful picture of where the method works.

Critical weaknesses
  1. The experiments leave the central design claim unresolved. FACET adds UniProt localization supervision, LoRA, adapters, auxiliary losses, and further training to the original CELL-Diff checkpoint. The comparison therefore does not establish how much improvement comes from the proposed factorization. Table 5 needs a simpler joint-conditioning baseline with the same supervision and training budget. The paper also lacks direct comparisons of residualized versus raw-feature memory and context-first versus reversed or parallel FiLM. These comparisons matter because residualization and conditioning order are central to the paper’s contribution.

  2. The ELBO formulation does not clearly describe the implemented model. Equation (4) treats RR as a categorical latent variable, whereas both training and inference replace it with the deterministic mean mS=E[R∣S]m_S=\mathbb{E}[R\mid S]. Passing this mean through nonlinear residual and transport networks is generally different from marginalizing over RR. The paper calls mean readout an inference approximation, but also uses it during training. This leaves the claim of blockwise ELBO optimization unsupported and weakens the theoretical contribution.

Questions for authors
  1. How does FACET compare with a conventional joint-conditioning model trained with the same localization annotations, adaptation capacity, and training budget?

  2. What happens when the memory uses raw target features, or when FiLM conditioning is reversed or applied in parallel? These comparisons would help establish whether the proposed structure explains the gains.

  3. How does mean-readout training correspond to Equation (4)? If it does not optimize that ELBO, how should the probabilistic formulation and categorical regularizer be interpreted?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.