acceptodds
Under review as a conference paper at ICLR 2027

Towards Biological Intelligence:Evaluating and Aligning MLLMs for Taxonomic Perception and Knowledge Reasoning

Abstract

Biological understanding demands the integration of visual perception with structured knowledge reasoning.Although Multimodal Large Language Models (MLLMs)have made substantial progress,their capabilities in perceiving and reasoning about biological species remain largely unexplored. To address this gap,we introduce BioBench,a comprehensive benchmark comprising 5,919 images and 8,821 questions across 3,611 species,spanning five taxonomic ranks and five task types:morphological attribute recognition,taxonomic hierarchy classification,fine-grained species identification,biological knowledge reasoning,and humanistic knowledge reasoning.Evaluations of 23 MLLMs reveal significant difficulties in fine-grained discrimination and biologically grounded reasoning.To mitigate these limitations,we develop BioInstruct,a large-scale instruction-tuning dataset of 160K multimodal samples covering over 38K species,designed with few-shot species presentation to align visual traits with taxonomic hierarchies and promote generalization.Using standard Supervised Fine-Tuning on BioInstruct,a 3B-parameter model achieves substantial performance improvements,even surpassing larger frontier models.Our work establishes a foundation for biologically grounded AI,highlighting that bridging fine-grained perception with structured knowledge is critical for specialized intelligence.

Primary area:
datasets and benchmarks
Keywords:
MLLM, Benchmark, Biological understanding

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_uVHKOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper introduces BioBench, a benchmark that tests how well multimodal LLMs understand species. It contains 5,919 images of 3,611 species and 8,821 multiple-choice questions, grouped into five tasks that range from recognizing body features, taxonomic rank and species to answering biological and human-related knowledge questions. The data mainly comes from BioCLIP data matched against the IUCN Red List, and most descriptions and questions are written by Qwen models and GPT-4o and then filtered. The authors also build a 160K instruction dataset called BioInstruct and fine-tune Qwen2.5-VL-3B on it. The authors evaluate 25 existing models. According to the paper the best of them is Qwen2.5-VL-7B with an average of 59.48%. Its accuracy falls from 62.68 on easy species identification to 34.85 on hard, and R1-style reasoning models do not help much. Fine-tuning on BioInstruct raises the 3B model from 57.02 to 61.20.

Critical strengths
  • The species coverage is broad. The benchmark spans 3,611 species of animals, plants and fungi, which goes well beyond earlier biology benchmarks such as VLM4Bio that only cover fishes, birds and butterflies.
  • The difficulty design is thoughtful. Taxonomic classification is split by level and species identification is graded by taxonomic distance with BioCLIP-ranked distractors, so the benchmark shows where models break down instead of giving a single number. The drop from 62.68 on easy species identification to 34.85 on hard is a good example of this.
  • Masking the species name and removing questions that can be answered without the image deals with a known weakness of multimodal benchmarks, where many questions can be solved from text alone. The training setup and the prompts used for generation are also described in enough detail to follow.
Critical weaknesses
  • Unreliable Leaderboard Metrics: The reported leaderboard scores appear statistically inconsistent with the dataset size. For example, Figure 2 lists 1,779 questions for hard species identification and 145 questions for the discriminative MAR split. However, for most models, the reported percentages cannot be back-computed to any whole number of correct answers based on these denominators, strongly suggesting that different models were scored on different subsets of questions. Furthermore, the InternVL models exhibit oddly weak and clustered performance (e.g., the 38B InternVL3 scores 40.89%, falling behind the 7B LLaVA-v1.5 at 42.72%), which raises concerns about whether appropriate inference templates were used. This scoring methodology must be audited and standardized before the rankings can be trusted.

  • Opaque Question Construction and Filtering: The pipeline for building and filtering questions lacks necessary detail. For the MAR task, the paper simply states that the base setting has "coarse differences" and the discriminative setting has "fine-grained ones," without explaining how the correct and distractor descriptions are systematically generated. Moreover, one filtering step relies on several "unnamed MLLMs"; if these include models later evaluated in the paper, the filtering process inherently biases their scores. Finally, the manual verification step is described in a single sentence without reporting the sample size or error rate, making it difficult to verify the claim that the question set is highly reliable.

  • Contradictions Between Claims and Data: Several central claims contradict the paper's own tables. The text asserts that Qwen2.5-VL-7B is the best model at 59.48%, yet Appendix Table 5 shows Qwen2.5-VL-72B achieving 66.55%. The abstract claims BioQwenVL surpasses "larger frontier models", but no closed-source models are tested, and its 61.20% score falls below both the 32B (64.62%) and 72B Qwen models. Furthermore, the full BioQwenVL model averages 59.54% in Table 4 but 61.20% in Table 3. Lastly, it is unclear whether BioInstruct and BioBench share species or images, making it impossible to determine how much of the 4.18% performance gain is true generalization.

  • Knowledge Tasks Conflated with Visual Recognition: The knowledge tasks likely measure species recognition rather than reasoning capability. A question is retained only if text-only LLMs answer it correctly when provided the true species name, and it is removed if it can be answered without the image once the name is masked. Consequently, the surviving questions are fundamentally reliant on the model recognizing the species visually. As the authors themselves note, knowledge reasoning depends heavily on accurate species identification. Without providing ablation results where the model is explicitly given the true species name, a low score could simply indicate a visual misclassification rather than a lack of biological knowledge. Therefore, the conclusion that reasoning models struggle to apply biological knowledge is insufficiently supported.

Questions for authors
  • Do BioBench and BioInstruct share species or images? Reporting the performance gain separately for species seen and unseen during training would help clarify the true generalization capability.

  • How exactly are the MAR options generated, which specific MLLMs were used for filtering, and what were the sample size and error rate for the manual verification step?

  • How do the evaluated models score on the knowledge tasks when they are explicitly provided with the true species name in the prompt?

Reviewer_skqoOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

This paper introduces BioBench and BioInstruct, a biological reasoning benchmark and instruction-tuning dataset designed to evaluate multimodal large language models (MLLMs) on tasks requiring precise, fine-grained taxonomic classification and biological reasoning. The authors evaluate 23 MLLMs and demonstrate their limitations on the proposed benchmark. They subsequently develop BioQwenVL through supervised fine-tuning (SFT) on BioInstruct and show that it consistently outperforms the evaluated baselines across the considered tasks.

Critical strengths
  1. Novel benchmark: To the best of my knowledge, this is among the first multimodal benchmarks explicitly designed to connect fine-grained taxonomic identification with biological reasoning.

  2. Relevance to scientific reasoning: The proposed benchmark and instruction dataset could be valuable resources for future research on scientific reasoning with MLLMs, particularly in biological domains where fine-grained visual understanding is essential.

  3. Extensive experimental evaluation: The authors evaluate a broad range of 23 state-of-the-art MLLMs, providing a useful assessment of their current limitations in fine-grained biological understanding and reasoning.

Critical weaknesses
  1. Lack of comparison with closed-source MLLMs: I did not find comparisons with leading closed-source models such as GPT or Claude, either in their standard configurations or with agentic capabilities. Such comparisons would help establish the difficulty of the proposed benchmark and provide a more comprehensive picture of current MLLM capabilities.

  2. Missing classification-based baseline: An important baseline would be to train a dedicated classifier on top of a vision encoder for fine-grained taxonomic identification and then provide its predicted class directly to an LLM for downstream reasoning. This would help determine whether the improvements from BioQwenVL primarily arise from better fine-grained visual recognition or from improved multimodal reasoning through instruction tuning.

  3. Distinguishing taxonomic recognition from biological reasoning: This is related to the previous point. If the answers to many benchmark questions are largely determined by correctly identifying the fine-grained taxonomic class, then a strong classification model combined with a general-purpose LLM might already achieve competitive performance. In my view, the more interesting and challenging setting would involve questions requiring multi-step biological reasoning beyond species identification. Such tasks could better demonstrate the benefits of instruction tuning and multimodal alignment. It would be useful to understand how much of the reported improvement comes from improved recognition versus reasoning.

  4. Choice of vision encoder: It appears that BioQwenVL uses the standard Qwen-VL vision encoder. I wonder whether a biology-specialized vision encoder, such as BioCLIP, could provide stronger fine-grained biological representations. A comparison or discussion would help clarify whether the performance gains are primarily attributable to the instruction-tuning strategy or the underlying visual representations.

  5. Clarification of Figure 5: In the third column corresponding to fungi, A4 and A5 are correctly predicted, whereas A1–A3 are incorrect, unlike the patterns observed in the other two panels. I initially expected species-level identification to be more challenging than higher-level taxonomic identification, making this result somewhat counterintuitive. It would be helpful if the authors could explain this behavior and whether it reflects differences in question difficulty, taxonomic hierarchy, or the model's reasoning capabilities.

Questions for authors
  1. Recognition versus reasoning: Have the authors considered a baseline where a dedicated fine-grained taxonomic classifier predicts the species or taxonomic class, which is then provided to an LLM for answering the biological reasoning questions? This comparison would help clarify the contribution of multimodal instruction tuning beyond improved visual recognition.

  2. Reasoning complexity: To what extent can the benchmark questions be answered directly from the correct taxonomic identity, as opposed to requiring additional multi-step biological reasoning? Is there an analysis separating these two types of questions?

  3. Closed-source baselines: Could the authors provide results from representative closed-source MLLMs, such as GPT or Claude, to better contextualize the difficulty of the benchmark and the performance of BioQwenVL?

  4. Figure 5: Could the authors explain why, in the fungi example, A4 and A5 are correctly predicted while A1–A3 are incorrect? How does this relate to the expected difficulty across taxonomic levels?

Reviewer_ThmZOpenReview ↗
Rating
1
Confidence
4
Informativeness
1
Presentation
2
Soundness
1
Read review
Paper summary

In this paper, the authors introduce BioBench for a taxonomy-centered multimodal benchmark comprising 5,919 images and 8,821 multiple-choice questions across 3,611 species to evaluate MLLMs on multiple tasks. Evaluating 23 open-source MLLMs reveals significant performance drops in fine-grained species discrimination, open-ended classification, and cross-level taxonomic consistency. To address these limitations, the authors construct BioInstruct, a 160K-sample instruction-tuning dataset, and use it to fine-tuning Qwen2.5-VL-3B on to develop BioQwenVL. The new model gains improvements on average accuracy.

Critical strengths
  1. BioBench is well curated. BioBench goes beyond simple species classification to evaluate 5,919 images and 8,821 questions across five taxonomic ranks (from class to species) and five progressively challenging tasks. It also uses sibling-taxa sampling and BioCLIP similarity scores provides a principled way to control distractor difficulty across easy, medium, and hard settings.
  2. Good Empirical Evaluation of 23 Models. The authors evaluate 23 models, which reveals key challenges, such as difficulty in discriminating closely related species, and suboptimaloptimization of current reasoning models.
Critical weaknesses
  1. The construction of BioInstruct is totally missed. I cannot find the pipeline of how BioInstruct was compiled.
  2. The first weakness leads to a more serious problem. What are the reltionship between BioBench and BioInstruct? It seems that both were sampled from the exact same filtered pool of 81,308 species (Section 3.2.1), and both use the exact same Qwen2.5-VL-72B / Qwen2.5-72B enrichment pipeline. The paper never tells us whether the 3,611 benchmark species (or their images and generated descriptions) were held out from BioInstruct. If BioQwenVL was trained on the profiles and dialogues of the same species tested in BioBench, the reported improvements are largely due to data leakage.
  3. There are many incosistencies in the paper. E.g., the main text says there are 38,892 species, but the A6.1 says "covering approximately 50,000 species." These inconsitencies are very questionable.
Questions for authors
  1. How BioInstruct was constructed?
  2. Is there any possible Train–Test Contamination?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.