This paper presents FARL, an automated reinforcement learning (AutoRL) framework that, concretely, uses a large language model to propose and revise training configurations for a supplied task. It searches over observation features, reward programs, learning algorithms and their settings, exploration procedures, and training duration. Numerical hyperparameters are searched using Optuna within domains appropriate to the selected learner. Candidate configurations are used to train policies, which are assessed using a fixed task evaluator. The LLM examines training statistics, recorded trajectories, and previous experiments to propose changes; the search controller decides which candidates to retain and train further. The authors motivate this joint search by the interaction between training choices: a reward or observation feature that fails with one learner may work with another. FARL can test proposed explanations through controlled comparisons and record the conditions under which a change helped or hurt; other search steps may change several components together. I understand the underlying problem as automated configuration search, including program-valued choices as well as numerical hyperparameters; I view the main methodological contribution as how the system directs that search. The task interface, goal, and evaluator are supplied before search.
FARL keeps observation features, reward programs, the learning algorithm, and training settings open to revision throughout search. It reserves candidates needed for controlled comparisons and preserves promising alternatives instead of selecting solely by their latest score. Existing automated reward design, interface search, and learner selection are foundations; the addition is a system that can revisit these choices using recorded experiments. For diagnostic comparisons, the system specifies a prediction, the changed and fixed components, and the training conditions, then records the outcome. Changing the learner can require different valid hyperparameter settings; not every comparison holds every other choice identical. This can prevent treating a failure under particular settings as a universal verdict on a component. The contribution is an implemented workflow and its evidence, not a new reinforcement-learning update or a guarantee of reliable automated scientific reasoning. Optional policy relay lets an existing policy reach an intermediate state before a new learner takes over; final evaluation uses the new policy alone from normal task starts.
Evaluation uses MiniGrid DoorKey, a small grid-world key-and-door task; MuJoCo Ant locomotion; Meta-World door opening (Door); and Adroit hand-object relocation (Hand). Some settings withhold task-provided reward and engineered observation designs; others permit them, while goals and public interfaces remain supplied. Comparisons include Eureka and LiMEN, automated design baselines that keep the learning algorithm fixed. Additional training runs of selected DoorKey and Ant designs support recipe repeatability, not repeatability of the whole search. The reported selected DoorKey policies succeed in all their evaluation episodes and Ant travels farther than the tested automated baselines. Selected Hand policies successfully move an object to its target and hold it there, but their evaluations also guided selection. Ant and Hand use more total search training than the baselines. In Hand, FARL predicts that removing a particular set of added hand/object/target relational features, while retaining other generated distance features and the original inputs, will improve moving the object after lifting it. With reward, learner, training settings, budget, and seeds held fixed, transport instead worsens. FARL records the failed prediction, retains both representations, and later finds settings under which both improve. This demonstrates experimental operation, not that memory caused later success. Simplified searches either fix the learner and its hyperparameters and exploration procedure (A1), remove detailed diagnostic evidence and memory (A2), or remove diversity and deliberate revival of discarded designs (A3). A2 and A3 retain access to all design choices. Full FARL solves Door where these restricted searches fail, but uncertainty about the resource limits and stopping decisions governing the comparative ablation searches restricts attribution of the broader gains. The experiments establish effective discovered designs and a functioning experimental loop more firmly than a repeatable advantage of that loop over simpler joint search.