- Rating
- 1(−1)
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 1
Read review
Paper summary
The authors use stellar astronomy data to train FAME, a foundation model for stellar astrophysics. The authors combine seven modalities from six surveys, crossmatched via Gaia DR3 source IDs.
They pretrain six encoders (one for each survey). Each encoder is trained both via a self-supervised objective alongside a supervised head. The encoders are then aligned to create the final model.
The paper represents substantial work, however I think it has some concerning flaws with evaluation and comparison with other astronomical foundation models. I go into these in my "weakness" section.
Critical strengths
This is a substantial piece of work. Training a ~1B parameter model is on the larger side of astronomy, and compiling the dataset to do this is a significant undertaking.
Critical weaknesses
I am concerned that astronomical labels are used at pre-training time. Particularly as the encoders are pre-trained with splits independent of the evaluation split. This seems to me to be a major oversight as it affects every downstream claimed result. The authors need to address this robustly.
No parameter scaling tests are carried out. A central expectation of a foundation model is that more data and more compute lead to better performance. This was shown for AION-1 with released variants from 300M to 3B params. FAME does not show this. Furthermore, your Figure 4 halts at 20k labels for downstream probe scaling, this hides the regime where purely supervised scaling would win out.
Linear probing is used for headline and external comparisons, whereas in AION-1 they use learned cross-attention as their probing method. An experiment using both non-linear probes and attentive methods is needed for a fair comparison.
AstroM3 and SpecCLIP are not compared (and SpecCLIP is not even cited!). Why not? They are highly relevant. The claim to be the first model combining time series, spectra and astrometry should be narrowed accordingly.
Questions for authors
Please see weaknesses.
Hi reviewers,
Thank you for your reviewing service. The authors have provided rebuttals with new results. Please confirm that you have read them, state which of your concerns are or are not addressed, and whether that changes your recommendations.