Mechanisms Before Methods: Measuring How AI Designs AI Research

Can an AI system do more than propose a plausible research idea—can it choose a principle worth pursuing and turn it into a workable method? Existing evaluations either score open-ended proposals, where reliable negative examples are hard to obtain, or end-to-end execution, where scientific judgment is mixed with engineering. We introduce Mechanisms Before Methods (MBM), a benchmark that scores research design before any experiment is run, built from 789 papers. MBM represents each method by its mechanism: the operating principle that explains how it works without revealing its paper-specific implementation. Each paper supports two paired tasks. Mechanism Allocation chooses the documented route among plausible alternatives; Mechanism Realization works out how that route should operate on the target problem. Across ten systems the two abilities rise together (r = 0.857), while within a single paper a correct choice adds under one point of realization quality. Asking a model to draft its own realization before choosing lowers its accuracy, while prewritten realizations raise it by 4.70 points in a matched diagnostic: a plausible plan is not evidence for a choice. Reinforcement learning improves Realization F1 by 3.37 points, and the ability transfers: a short Realization phase improves later Allocation itself. MBM makes pre-experimental research decisions separately measurable, and suggests that learning how a principle works helps decide whether it is worth pursuing.