- Rating
- 2
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
The paper argues that AI research should scale insights, not only compute. As they claim, a lot of advances in the field come from running computational experiments to see what works best, but do not necessarily give insight into why this is the case or how these findings might or might not lead to future gains in different settings (e.g., changes in architecture, training data, or method). They provide three contributions: the nanoGPT Observable Library, a dataset of observable trajectories recorded during Transformer training; OPHIS (a framework for observation-guided research); and ComfyResearch (a graphical environment for composing and sharing experiments). The paper concludes with directions for future work (how one could share insights between humans and models, identify observables effectively, etc.).
Critical strengths
- I believe the idea of using automated (LLM-based) research techniques to help consolidate insights, tricks, and general practices in the community and speed up model training is relevant and timely. They also have a well-motivated story and a clear argument for why such a framework is needed. The overall framework idea seems like a good starting point for doing this, and the paper indicates good future directions in which this work could be expanded.
Critical weaknesses
- It is hard to understand what OPHIS is actually doing in the reported experiments from the provided text. Section 3.3 describes the loop, and Appendix A reports experiments that show some improvements, but one cannot see how these improvements are motivated or what interventions actually lead to them (and whether one can learn anything from them). Also, I find the comparisons with other methods limited, which makes the results hard to evaluate. On nanochat, OPHIS starts from Recursive’s optimized solution and is compared only with that starting point (Appendix A.1, Figure 3). On grokking, the only other method is an LLM that receives the task and training setup but not the OPHIS measurements (the LLM-only baseline, Appendix A.2, Table 1).
Questions for authors
(Clarification) Figure 5 in Appendix A.3 illustrates what you mention as “overfitting early in training under certain architectural conditions”. I don't see enough information to understand exactly what is happening here. This brings me to my broader point: could you provide a complete OPHIS example from the reported study (measured observation, suggested hypothesis, interventions and their results, and final insights and the scope of these results)? I am failing to see the “whole picture” of the pipeline working to discover some “tricks” or insights, as mentioned in the paper. In Section 3.3, you write that OPHIS “outperformed the evaluated LLM-based AutoResearch baselines”. In Appendix A, I can only find Recursive’s optimized solution, which is also the starting point of OPHIS on nanochat (A.1), and the LLM-only baseline on grokking (A.2). Which baselines does this sentence refer to, and are there any other methods included in this comparison? Also, OPHIS tested many interventions, but I don't see how these lead to any insight aside from the improvement; I cannot see enough information on this.
(Clarification) I cannot follow Table 1. Appendix A.2 defines a substantial improvement as reaching the target accuracy “clearly earlier” than the unmodified baseline, which reached 95% test accuracy at about 1,250 steps. What is the numerical cutoff for this, and what counts as a failure? I cannot get a clear idea of these criteria. Also, what does “outside the comparison setting” mean in this case?
(Suggestion) Ideally, the paper would also benefit from a further explanation of how OPHIS works (even if this is relegated to the appendix), so one can understand more of the design.
2 comments
Thank you for the answers and for expanding Appendix A. The Table 1 definitions and the clarification on the baselines answer my questions, and the forking example (A.3) is the kind of example I was asking for.
But even reading it as a position paper, I agree with Reviewer CLQo that the OPHIS experiments are still too preliminary to support the main message. On nanochat, OPHIS is only compared with its own starting point, and on grokking, where there is a reference, OPHIS and the OPHIS-like LLM subset have almost the same substantial-improvement rate (72.9% vs 70.0% in Table 1). So I still don’t see what the observations add, and I will keep my score.
As a small note for next time, please highlight the changes in the revision (e.g., in red). It would have made them much easier to check.
Response to Reviewer H96h
Thank you for recognizing the idea as relevant and timely, and confirming that this framework is a well-motivated starting point, and for the specific questions and suggestion to explain OPHIS more fully. We have expanded Appendix A accordingly and answer each point below. Please also see General Response.
1. A complete OPHIS example: forking (Figure 5) (W1, Q1, Q3). This study arose during early, human-guided inspection of observable and loss curves, and is separate from Figure 3.
2. Which baselines were compared? (W1, Q1) No other methods are included in Figure 3 or Table 1. On nanochat, Recursive's optimized solution is both the re-evaluated baseline and OPHIS starting point under our 2,000-step protocol — further improvement of an already optimized solution. The 7.43 figure expresses the mean BPB difference in baseline standard-deviation units. On grokking, the 30 OPHIS-like candidates are a subset of the 76 tested LLM proposals, using similar intervention types (activation dropout, parameter-norm constraints, entropy regularization) but proposed without OPHIS measurements. These comparisons examine observation-guided intervention; they do not rank complete AutoResearch systems. We have specified these comparison units in Section 3.3.
3. Table 1 definitions. (Q2) Target attainment is the first recorded step with test accuracy of at least 95%, evaluated every 10 steps within 2,000 steps; the baseline reaches it at step 1,250.
4. "Outside the comparison setting." (Q2) We evaluate general training interventions within the implemented pipeline. Of 100 LLM proposals, 24 fell outside this scope: 12 encoded modular-arithmetic structure (e.g., circular embeddings), 9 required unsupported operations (e.g., EMA or SAM), and 3 added label-conditioned auxiliary objectives (e.g., center loss by target residue). These restrictions keep the study focused on general training interventions; they do not imply that excluded methods are invalid. Percentages use tested candidates as denominators (350, 76, 30); the 24 untested proposals are disclosed separately. We have added these definitions to Appendix A.2.