Sub-Planner: A Generalist Agent Framework for Multi-Domain Reasoning via Adaptive Sub-problem Planning

Domain-specific downstream tasks often require professional knowledge and multi-step reasoning to derive reliable solutions in specialized scenarios. Multimodal agents offer a promising solution by enabling MLLMs to perform multi-step reasoning and leverage external tools. However, most existing agents are typically built on general models and may lack sufficient domain knowledge. Domain-specific agents often rely on reasoning and tool-use pipelines tailored to specialized domains, limiting their transferability across domains. To address these limitations, we propose a general training-free agent framework that generalizes across diverse specialized domains. Our framework combines a general MLLM for planning and verification with a domain-specific MLLM for answering specialized questions. In the planning stage, the general model adaptively generates sub-problems based on the original task and selects appropriate domain-specific tools to collect supporting evidence. In the answering stage, the domain-specific model then performs reasoning on these sub-problems based on the collected evidence. In the verification stage, the general model further verifies the consistency between tool outputs and sub-problem answers, retaining only validated evidence for final reasoning. Our framework provides a unified training-free solution that requires no domain-specific adaptation and can flexibly integrate various general and domain-specific MLLMs for diverse downstream tasks. Experimental results show that our method achieves strong performance across 17 diverse domains.