- Rating
- 3
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper proposes MixORNet, which combines representations of the molecules in a mixture with ESM-2 features of olfactory receptor proteins through cross-attention to predict perceptual attributes of mixtures and perceptual distances between them. It achieves competitive performance against closely related methods on two DREAM benchmarks and reports higher mean performance on the unseen-molecule splits. Its main contribution is the integration of receptor information into mixture representations and an empirical evaluation under multiple conditions.
Critical strengths
S1. A coherent approach to an important and timely problem Predicting perceptual attributes and distances at the mixture level is an important problem, given that real-world odors commonly consist of multiple molecules. Extending the use of pretrained molecular and protein representations from single molecules to mixtures is a timely direction for olfactory modeling, with potential applications in fragrance design and olfactory sensing. The architecture provides a concrete approach by connecting permutation-invariant mixture representations with receptor-conditioned representations.
S2. Relevant comparisons and competitive empirical results The paper evaluates strong, relevant methods, including POMMix and MixScentNet, on the same data splits and considers both standard and unseen-molecule settings. MixORNet is competitive with leading methods on the standard splits and reports higher mean performance on both tasks under the unseen-molecule protocol. For example, on the unseen DREAM2024 split, PLCC is 0.6724 for MixScentNet and 0.7045 for MixORNet. These results, together with the component analyses, support the practical usefulness of the approach and are the main grounds for my positive assessment.
Critical weaknesses
W1. It is unclear whether the unseen-molecule split excludes all test constituents from training. Section 4.2 states that no SMILES appearing in a test sample occurs in a training sample. However, assigning all samples containing a selected molecule to the test set does not by itself establish whether the other constituents of those test mixtures are also excluded from training. Holding out selected molecules and excluding every test constituent from training support different interpretations of generalization. Since the results in Table 2 are a central part of the empirical contribution, the handling of co-occurring constituents and the actual train/test molecule-overlap statistics are important for assessing these results.
W2. The current experiments do not isolate the contribution of receptor-sequence information to the generalization gains. Section 4.2 attributes the gains on unseen molecules to the OR fingerprint and receptor-based regularization. However, the comparisons between complete models in Tables 1 and 2 do not distinguish the effect of receptor-sequence content from that of the additional conditional representation architecture. The ablation in Table A1 that removes cross-attention still retains the protein features and is reported only on the standard split. A control that changes only the receptor input features while preserving the architecture would help address this question, particularly on the unseen-molecule splits where the main gains occur. The existing results support the predictive performance of the complete model more directly than the proposed explanation for its improvement.
Questions for authors
Q1. Does the unseen-molecule construction guarantee that every constituent of a test mixture is absent from training, or is this guarantee limited to the molecules selected for holdout? Please explain how the other constituents that co-occur with a selected molecule are handled, and report the train/test sample counts and the size of the intersection between the constituent-molecule sets for each fold. For DREAM2024, please clarify whether constituents from both mixtures in each pair are included when checking overlap, and whether the five test folds are mutually exclusive partitions or independently sampled holdouts. Fold-wise paired performance differences against MixScentNet would also help assess the consistency of the reported gains.
Q2. How much of the improvement depends on the sequence-derived receptor features? Could the authors provide a control that preserves the receptor panel and architecture but replaces the receptor features with random features drawn once for each receptor and held fixed, while retraining the remaining trainable modules? Matching input dimensions, residue counts, and feature magnitudes, and using the same splits and training configuration, would help assess the additional benefit of the actual receptor features. This comparison would be particularly informative on the unseen-molecule splits. If such results are unavailable, please clarify how strongly the current evidence supports attributing the improvement to receptor features.
Q3. For each fold in Tables 1 and 2, were model-selection decisions made without using that fold's test results? Section 3.4 motivates by performance and efficiency. Please specify which data and criteria were used to select the panel size, the stopping epoch or checkpoint, and . Please clarify whether these settings were fixed in advance, selected using a separate validation procedure within each fold, or chosen based on the reported cross-validation results. Describing the validation and model-selection procedures used for the baselines would also help assess the comparability of the results.