- Rating
- 1
- Confidence
- 3
- Informativeness
- 1
- Presentation
- 1
- Soundness
- 1
Read review
Paper summary
The authors study whether LLM-generated explanations of planned ML experiments help an LLM forecaster predict the experiments’ outcomes. For each experiment, the forecaster (DeepSeek V4) predicts the outcome under three contexts, twice each: from the description alone, with the matching explanation, and with an explanation borrowed from another experiment. The explanations are generated by single, tool-free DeepSeek V4 calls, mostly for pre-assigned hyperparameter or feature changes.
The evaluation covers 336 small experiments across synthetic tasks, 12 Tox21 assays, and 24 OpenML datasets with LightGBM. It adds several post-hoc follow-ups and two positive controls.
None of the three main tests passes under the decision rules the authors fixed before seeing the outcomes. Explanations do not measurably improve accuracy. In Tox21 and OpenML, matched and borrowed explanations give similar results; only in v5 do matched explanations beat borrowed ones, on log-loss error. Repeated forecasts become closer to each other in Tox21, but not on OpenML. The authors present the paired comparison design itself as the main contribution: a protocol for evaluating research-agent explanations.
Critical strengths
The study fixes its decision rules before seeing outcomes, keeps failed calls in the analysis, treats tasks as the unit of inference, and includes positive controls showing the forecaster can use real information. The negative results are reported. These are good practices, but they do not offset the weaknesses below.
Critical weaknesses
-
The paper is framed around research agents, but no agent is evaluated. Each explanation is a single, tool-free DeepSeek V4 call that sees only the public state and the intervention, with no code execution and no interaction with the experiment. The interventions themselves are assigned by rule (Tox21, OpenML) or chosen from a fixed catalog (v5). The results therefore concern single-call LLM rationales for pre-specified configuration changes; applicability to research-agent benchmarks (§8–9) is asserted, not tested.
-
The drift reduction comes from the number on the card, not from the explanation. In the prospective studies, the explanation and the forecast come from the same model with the same inputs (lines 146-151, 265-267), so the card adds no information, only a predicted number (lines 182-183). Forecasts that see the same number agree with each other without getting closer to the truth. The authors note this (lines 190-192), yet the abstract and Figure 1 highlight the 64.5%/59.1% drift reductions.
-
Numbers alone cut drift more than mechanism text (Table 25: 0.00246 vs 0.00378). With the mechanism text held fixed, numbers from the borrowed explanation cut drift at least as much as the matched ones (0.00256 vs 0.00317). The paper labels this contrast exploratory, but it is consistent with the number, not the explanation, driving the effect.
-
Point MAE does not meaningfully improve (Table 2) and worsens in the Flash replay, from 0.01823 to 0.02020.
-
Real information raises drift while lowering error (Table 28: drift goes from 0.768 to 1.275).
-
On OpenML, where most cards have no number, drift does not fall (0.01004 under description, 0.01300 with cards, line 322).
-
The reduction is unstable across identical Flash runs: 0.00426 in one, 0.00108 in the other (lines 314-316).
-
The forecaster is no better than always guessing "no change". The simplest forecast is to predict that the score will not change at all. The authors use this as a baseline (lines 199-200), but their forecaster does not beat it. On OpenML, its error is the same as the baseline (0.08368 vs 0.08341, lines 435-437), and it does better on only 13 of 24 tasks (Table 22). It only wins after one dataset, brazilian_houses, is removed (lines 436-437). For Tox21 and v5, the comparison is never shown. Without evidence that the forecaster extracts signal from the description alone, it is unclear whether these tasks contain enough predictable signal for an explanation to help, which limits what the null results can show.
-
The tests cannot detect small differences. In Tox21 and OpenML, the borrowed explanation comes from the same experiment run with a different random seed (lines 142-143), so its true outcome is usually close to the matched one. The two seeds usually move in the same direction (27 of 36 and 54 of 72 pairs), and in Tox21 the gap between them (0.0136) is smaller than the forecaster’s own error (about 0.02) (lines 143-144). A test like this cannot tell whether an explanation is specific to its own experiment. On OpenML there is almost nothing to compare. Only 13 of 72 pairs had content in both explanations, and only 1 pair had numbers in both (lines 423-424). The sample is small. Each task counts as one unit (lines 288-289), so Tox21 rests on 12 tasks and OpenML on 24. The thresholds are not justified. Two results are called equivalent if they differ by less than $\pm$0.002, $\pm$0.005 or $\pm$0.01 (Table 26; lines 1056, 1061), but the authors never say where these values come from.
Questions for authors
-
Other models: would the results hold with a model family other than DeepSeek, or with higher reasoning effort? Most calls used low effort (line 268).
-
78 of 144 cards were empty (line 423), and extraction failed your own .95 quality threshold (lines 1150-1151). What can the OpenML comparison tell us when most forecasts receive no explanation?
-
The positive control uses a researcher-written note on synthetic tasks. Would explanations written by domain experts help the forecaster on Tox21 or OpenML? That would show whether the problem is the explanations or the method.