Towards Biological Intelligence:Evaluating and Aligning MLLMs for Taxonomic Perception and Knowledge Reasoning
Abstract
Biological understanding demands the integration of visual perception with structured knowledge reasoning.Although Multimodal Large Language Models (MLLMs)have made substantial progress,their capabilities in perceiving and reasoning about biological species remain largely unexplored. To address this gap,we introduce BioBench,a comprehensive benchmark comprising 5,919 images and 8,821 questions across 3,611 species,spanning five taxonomic ranks and five task types:morphological attribute recognition,taxonomic hierarchy classification,fine-grained species identification,biological knowledge reasoning,and humanistic knowledge reasoning.Evaluations of 23 MLLMs reveal significant difficulties in fine-grained discrimination and biologically grounded reasoning.To mitigate these limitations,we develop BioInstruct,a large-scale instruction-tuning dataset of 160K multimodal samples covering over 38K species,designed with few-shot species presentation to align visual traits with taxonomic hierarchies and promote generalization.Using standard Supervised Fine-Tuning on BioInstruct,a 3B-parameter model achieves substantial performance improvements,even surpassing larger frontier models.Our work establishes a foundation for biologically grounded AI,highlighting that bridging fine-grained perception with structured knowledge is critical for specialized intelligence.
Then back it, or bet against it.
Related papers
Open the market on this paper to see 7 more related papers.