TableLoop: Tabularizing Multimodal Data via Rubric Evolution

Many prediction tasks depend on information distributed across text, images, and structured measurements. We introduce TableLoop, a flexible framework that connects language and vision-language models with tabular predictors through learned natural-language rubrics, closing the loop between what a model measures and how useful those measurements are for downstream prediction. A frozen scoring model applies each rubric to produce explicit numeric features, while a tabular predictor's validation performance and errors guide iterative revisions to the rubric. The resulting interpretable features are combined with structured measurements for final prediction. TableLoop is modular: it requires no language-model tuning and supports interchangeable scoring models, tabular predictors, and evolution methods. Across nine tasks spanning clinical risk, persuasion outcomes, essay quality, peer-review decisions, skin-lesion malignancy, and fungal species identity, TableLoop outperforms both structured-data-only prediction and calibrated direct prediction by a larger language or vision-language model in 26 of 27 evaluated configurations. Notably, TableLoop using a smaller open-weight scorer (Qwen3-32B/Qwen3-VL-32B) can outperform direct prediction with a frontier model (Claude Sonnet 4.5), showing that a smaller model can support stronger predictions when used to extract task-specific features rather than predict the target directly. Beyond prediction, TableLoop produces reusable, interpretable feature tables grounded in real observations, offering a potential source of training and adaptation data for tabular foundation models and an inspectable interface for multimodal-to-table learning.