- Rating
- 3
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper uses crystallographic priors for long-tailed space-group recognition. The results look promising, and the ablations support the main design choices. I have a few questions below.
Critical strengths
see Questions
Critical weaknesses
see Questions
Questions for authors
-
Could you compare RSCL with a same-class-only contrastive loss and a similarity based only on crystal systems? Keep the backbone and other losses the same, so we can see what the symmetry-operation relations actually add.
-
The expert allocation is based on the CCDC distribution. Pls try initializing B to zero, with the same eight experts and losses. Does the chosen allocation help, or can the gate learn equally well without it?
-
Table 17 has just 42 CCDC extreme-tail test samples from 42 classes. Is this also the set used for the main results? If yes, pls report the mean and std over the five training seeds for this bucket. With only one sample per class, resampling within each class won’t capture the uncertainty in the test set.
-
How are the symmetry-operation sets standardized before computing their overlap? The setting and origin choice matter here. Also, I couldn’t find the value of δ or whether the features in Eq. (11) are L2-normalized.
-
I found the two evaluation settings a bit confusing. Table 1 trains a separate model on each dataset, while Table 12 tests frozen CCDC-trained models.
2 comments
The additional experiments are sufficient and address my concerns.
Thank you for suggesting the contrastive and initialization controls and for pointing out where the evaluation protocols needed clarification.
Q1. Same-class, crystal-system, and operation-set contrastive objectives
We evaluated all three configurations on the CCDC test set with the same backbone, HR, RGE, other losses, data splits, training seeds, and folds. The same-class control uses positives from the same space group. The crystal-system control also uses positives from other space groups in the same crystal system. Both controls assign a weight of 1 to each non-self term in the denominator. Full RSCL uses operation-set similarity to select cross-class positives and weights the denominator terms by 1−S.
Full RSCL improves Extreme-Tail Top-1 by 2.47 percentage points over the same-class control and 2.68 points over the crystal-system control.
Q2. Zero initialization of B
We varied the initial value of B while keeping eight experts, the losses, data splits, training seeds, and folds fixed. In both variants, B is updated during training. The table reports five-seed means on the CCDC test set.
Rule-based initialization improves Overall, Tail, and Extreme-Tail Top-1 by 0.92, 1.88, and 2.38 percentage points, respectively.
Q3. Extreme-Tail test examples and variation across seeds
The Phys-MoE row in submitted Table 2 used validation scores. The updated test evaluation uses the same 42 CCDC Extreme-Tail examples as Table 17. These examples cover 37 space groups, so we will correct the Classes entry from 42 to 37 in both CCDC rows. The CCDC benchmark contains examples for 225 of the 230 space groups.
The original per-seed validation records were unavailable. For the test-set update, we retrained Phys-MoE with five seeds and evaluated each run on these 42 examples.
Within-group resampling gives a fixed contribution from the 35 space groups with one example each. The other two groups have three and four examples, respectively, and their draws can vary across replicates.
Q4. Operation-set standardization, threshold, and feature normalization
We use the fractional-coordinate operations listed for each space group in Section 2.3 of International Tables for Crystallography, Volume A (2016), in the setting and origin of the corresponding entry. We compare the tabulated representatives by matching their linear and fractional translation parts. The default threshold is δ=0.65. We L2-normalize the projected features before computing the inner products in Equation (11). IUCr space-group tables
Q5. Evaluation protocols in Tables 1 and 12
For Table 1, we train and evaluate a separate model on each dataset. For Table 12, we train each mask configuration on CCDC and freeze its parameters before evaluating the inference routes on CCDC, CoREMOF, and Inorganic. The Extreme-Tail bucket in Table 12 is defined by the CCDC training frequencies.