acceptodds
Under review as a conference paper at ICLR 2027

DAPF: A Dynamic Agent-Driven Automatic Post-Training Framework

Abstract

Domain-specific fine-tuning of large language models requires repeated decisions about training configurations, data schedules, and recovery from unsuccessful experiments. Automating these decisions requires connecting training feedback to actionable interventions while accounting for limited computational resources. We introduce DAPF, an agent-driven framework for closed-loop post-training of language and vision-language models. Given a target model, task domain, and validation performance target, DAPF iteratively generates hypotheses, designs and executes experiments, diagnoses training outcomes, and selects subsequent interventions. Its attribution module combines loss-pattern analysis with agent-generated diagnoses to propose targeted adjustments and update estimates of intervention effectiveness from experimental feedback. A budget-aware selector prioritizes expected validation improvement per unit cost and chooses the least-cost action predicted to reach the target when available. An append-only evidence graph records experimental decisions and supports recovery of model checkpoints and agent state. We evaluate DAPF on seven benchmarks using Qwen3-1.7B for text tasks and Qwen2.5-VL-3B for multimodal reasoning. Across three independent runs, DAPF achieves higher mean test accuracy than FT-Agent on every benchmark, with an average absolute gain of 1.74 percentage points. Reported average GPU runtime decreases from 1.5 to 1.3 hours, while the average number of effective optimization rounds decreases from 4.9 to 3.8. These results suggest that feedback-guided intervention selection can improve fine-tuning performance and computational efficiency within a bounded search space.

Primary area:
foundation or frontier models, including LLMs
Keywords:
auto reasearch Fine-tuning

Reviews

Among:
Shape:
1234this paper, Bottom 1%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.00 over 3 reviews, above 0% of the others.
Reviewer_6ePHOpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
2
Soundness
1
Read review
Paper summary

The paper presents an agent-driven framework for closed-loop post-training, called DAPF, which is motivated by connecting training feedback to actionable interventions. Specifically, DAPF contains 6 modules in a closed-loop workflow from hypothesis generation, design, execution to diagnosis and intervention selection. DAPF is evalauted under seven benchmarks using Qwen3-1.7B for text tasks and Qwen2.5-VL-3B for multimodal reasoning, outperforming baselines.

Critical strengths
  • The experiments cover 7 benchmarks
  • The reported tested scores contain standard deviations
  • The idea of this paper is clear and straightforward
Critical weaknesses
  • The introduction is too concise. The motivation and the final results are not well-connected. A solid description for this manuscript can lead the readers to have a better understanding of how and what the authors would like to solve, especially the current length of the introduction is almost similar to the abstract. Not saying that authors are required to use the whole page limit for the main manuscript. However, at least should be informative enough. The writing and paper organization seems not ready and requires polish.
  • Since the goal of this work aims to demonstrate a more complicated workflow integration can improve the performance against the baselines, it is necessary to demonstrate that those baselines are well-defined and setup, e.g., how do the authors select final hyper-parameters and what are the final hyper-parameters. Also, it is not clear what FT-agent means, where the authors have not mentioned any abbreviations.
  • In FT-agent and DAPF, do authors conduct training without SFT, if yes, a clear ablation studies or reasons why not doing so should be provided.
  • It is not too surprised that integrating a more informative workflow can outperform those vanilla baselines. However, more advanced baselines or recent similar works' performance can make the conclusion stronger, especially that the current tested scores' margins are limited
Questions for authors
  • One of the motivations in the introduction is that automation alone does not explain why a training run improves. Can authors show the results how DAPF can help explanation?
  • In equation 3, the training is limited by the budget. what will be the performance of DAPF with different budget limits?
  • In 4.2 BASELINES AND ATTRIBUTION ABLATION, the authors mention that they use the same maximum compute budgets. Could authors describe more on what the compute budgets refer to here given the fact that the 4.5 EFFICIENCY ANALYSIS shows the GPU runtime comparison?
Reviewer_7cWWOpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
2
Soundness
1
Read review
Paper summary

The paper proposes DAPF, an agent-driven framework for automating domain-specific post-training under a computational budget and a target validation performance. Its six-stage workflow connects requirement analysis, hypothesis generation, experiment design, training, critique, and subsequent decisions. The main contribution includes loss-pattern diagnosis, persistent estimates of intervention utility, a selector based on predicted improvement per unit cost, an append-only evidence graph. Experiments use Qwen3-1.7B for six text benchmarks and Qwen2.5-VL-3B for ScienceQA. Across three runs, DAPF reports higher mean scores than baselines on all seven benchmarks, while reducing reported average GPU runtime. An ablation show the effectiveness of about attribution-driven reasoning in auto training.

Critical strengths

Strength 1: A useful and thoughtful approach in AutoML Framework

The attribution-driven intervention plays an important role in DAPF, the reason analysis before acting would lead to the lower budget cost, which is also considered during AutoML framework. The appropriate orchestration of agent workflow in DAPF show the potential of AutoML in LLM Fine-tuning. The experiments across various benchmarks show the better performance of DAPF and the ablation show the effectiveness of attribution intervention in DAPF.

Critical weaknesses

Weakness 1: It may be better to give a detailed description and figure

DAPF uses a six stage workflow in AutoML framework which is clearly described in Sec3.1, and Figure 1 show the workflow. However, in the main Figure there may have 7 submodules (Requirement Analysis -> Hypothesis Generation -> Experiment Design -> Training Execution -> Evaluation &Attrbution -> Bayesian Belief Update -> Budget-aware Strategy Selection) which are misaligned to the six stages.

  • It is better to polish the graph to give a clear description about 6 stages and main contributions of DAPF.

Also, we notice that in the paper the model used in DAPF and FT-Agent baseline is not reported, but in most AutoML Framework, the backbone LLM need to be reported.

  • For fair comparison, the LLM used in workflow need to be reported. Moreover, if different LLM backbones are tested, it is better to report the performance of them (like Table 4 in paper FT-Agent).

And we note that the budget used in DAPF is unclear. Is it the GPU walllock time? And for effeciency comparison, average token comsumption is reported but the definition of average token consumption remain unclear. Is it average on each run? Is it average on each experiment?

  • It is better to state real definition in experiments of some objects like Budgets, averate token consumption. Moreover, for Table 2 and Figure 2 it is better to show the standard deviation like in Table 1 . Figure 2 mention the values are averaged over 3 independent repetitions, it is better to plot std in the figure.

Case study may be better to state the contribution of DAPF, the clear evidence of exact attritbuion invention may support the effect of DAPF. Also, the longly appendices are seeking their roles in the main text.

  • The final results and efficiency comparison could not fully support the contribution of DAPF. It is better to show the clear evidence of exact attribution during training. Case study is a good choice.
  • Moreover, it is better to refer contents of appendicies in main text.

Weakness 2: The agent orchestration, attribution analysis, budget-aware considering and fail-retry are important, but DAPF lack incremental analysis in the work.

The introduction part states the problem of "**Automation alone does not explain why a training run improves". ** We agree that the reason attribution analysis is very important in AutoML, hyper-parameter setting, data cleaning may be explored separately for better training performance.

Firstly, DAPF applies a six-stage workflow to separate training in clear stages. We admire these clear work separation, but the workflow idea is widely adopted like LamDAgent, FT-Agent.

  • It may be better to give a incremental analysis about why the six-stage definision is better than other workflow setting.

Based on agent workflow, recently strong modern agent harness (codex, claude code) must be included in the baseline for comparison. In MARS, it uses MCTS while also consider the budget. CoMind use multi agent system (MAS) for AutoML. It seems properly to assign the attribution to one subagent.

  • So the selection of workflow rather than multi-agent system in DAPF need to be discussed. Moreover, modern harness systems (codex, claude code) may be included as strong baselines.

In loss pattern part (section 3.6) there are some priori value settings like $ \ell_T>5\cdot \ell_1 $, are they based on human experiences? The settings may lack generation for other training works.

  • It is better to showw the robustness of the loss-pattern attribution design. Does other training works or other base models can be applied for the same pattern?

In other papers there are memory systems consoliding the training experiences for future training. Like MARS, LaMDAgent saving training experimences in one experiment for the latter iteration. ML-Master 2.0 design the cross-task long-term memory for future training jobs.

  • What's the novelty of evidence chain flow of DAPF? It is the episodic memory recording at each node, lacking of knowledge consolidation in one iteration. For more iterations, the redundant evidence flow may cause confusion.

Ref.

FT-Dojo/FT-Agent: Towards Autonomous LLM Fine-Tuning with Language Agents, 2026.

LaMDAgent: An Autonomous Framework for Post-Training Pipeline Optimization via LLM Agents, 2025.

MARS: Modular Agent with Reflective Search for Automated AI Research, 2026.

CoMind: Towards Community-Driven Agents for Machine Learning Engineering, 2026.

ML-Master 2.0: Toward Ultra-Long-Horizon Agentic Science: Cognitive Accumulation for Machine Learning Engineering, 2026.

Questions for authors

See Weakness part.

Reviewer_XtTLOpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
1
Soundness
1
Read review
Paper summary

This paper proposes DAPF, an LLM-agent-driven closed-loop post-training framework. Given a base model, a target domain, and a validation accuracy target, six agent stages (requirement analysis, falsifiable hypothesis generation, experiment design, training execution under a watchdog, critic-based evaluation, and budget-aware decision) iterate over LoRA fine-tuning runs. The framework maintains an append-only evidence graph with cryptographic hashes, attributes training outcomes to factors (learning rate, data quality, method choice, system faults) with three confidence levels, updates a persistent belief table over candidate interventions, and selects the next action by expected-per-cost gain (or the least-cost sufficient action when the target gap is predicted to be closed). Experiments on seven benchmarks with Qwen3-1.7B (text) and Qwen2.5-VL-3B (multimodal) compare DAPF against manual SFT, manual curriculum SFT, FT-Agent, and an attribution-ablated variant, reporting +1.74pp average test accuracy over FT-Agent with fewer GPU hours, iterations, and tokens.

Critical strengths

The framework maintains an append-only evidence graph with cryptographic hashes, attributes training outcomes to factors (learning rate, data quality, method choice, system faults) with three confidence levels, updates a persistent belief table over candidate interventions, and selects the next action by expected-per-cost gain (or the least-cost sufficient action when the target gap is predicted to be closed).

Critical weaknesses
  1. No per-module ablation; mechanism-level claims are unsupported, and the described method does not match the implementation. Only the attribution module is ablated. The evidence graph (§3.2), belief table and update rule (§3.3), budget-aware selector (Eqs. 8–9), fault-tolerance subsystem (§3.5), and the six-stage decomposition itself are never ablated, analyzed, or even illustrated with a real trace — the authors concede that "additional component ablations … are needed" (§6). More problematically, there is a method–implementation gap: §3.3–3.4 describe an intervention space (filter_dataset, switch_to_curriculum, run_probe) and an explicit budget-aware selector, but the actual adjuster code (Appendix C) only implements lr_multiplier, dropout_delta, extra_steps, and freeze_embedding, and the real decision rules (Appendix F) are threshold heuristics (plateau → halve LR; budget/max-rounds stop). The formal objective (Eqs. 1–3) is never instantiated — no value of B₀, δ, or per-benchmark r_tar is reported anywhere. It is therefore impossible to know which mechanism, if any, produced the reported numbers.

  2. The comparisons are not on the same dimension and are compute-confounded. All training in this paper is LoRA SFT (r=16); DAPF contributes no new training method, only a decision loop that launches multiple LoRA-SFT rounds. Comparing a multi-round adaptive search against a single fixed SFT run ("Manual SFT") conflates "more search compute" with "better mechanism." The obvious control — any hyperparameter search under the same GPU budget (random search, grid, Bayesian HPO, or best-of-N SFT) — is absent. Likewise, the efficiency claims (GPU 1.5→1.3h, rounds 4.9→3.8) are confounded by DAPF's target-met early stopping (Eq. 9 / Appendix F goal_met), a rule the baselines do not have: "fewer GPU hours" may simply mean "stopped earlier at a user-set threshold," not "better interventions."

  3. The system is an assembly of existing components without stated design principles. The pipeline stitches together known pieces: multi-role LLM agents (R&D-Agent, LaMDAgent), process-group watchdogs, append-only provenance logs, heuristic loss-pattern classifiers, and a Bayesian-style belief table. None of the components is individually new, and the paper provides no analysis or justification for why this particular composition is necessary or superior — the belief-update rule is unspecified, the gain truncation G_t = min(Δ̂_t(a), g_t), the LLM-confidence threshold 0.6, and the loss-pattern thresholds (5×, 0.7×, 1.1×, 10⁻³, 0.5μ) are unexplained magic constants.

  4. The experiments cannot validate the claimed effectiveness; observed gains are more consistent with the added search loop than with any proposed mechanism. With only 3 seeds, the gain over FT-Agent is ≤1.51pp on 5/7 benchmarks with overlapping ±std (e.g., GSM8K .6571±.011 vs .6514±.016) and no significance testing; the headline 1.74pp average is dominated by a single benchmark (BioProBench +4.65pp; excluding it the mean gain falls to ~1.26pp). The "attribution ablation" does not isolate a mechanism — it swaps the entire decision procedure for direct LLM calls — and the ablated variant still beats FT-Agent on several benchmarks, confirming that much of the gain comes from the loop (extra rounds) rather than attribution. Finally, the evaluation protocol itself undermines comparability: generation tasks are scored with ROUGE-L F1 at a permissive 0.3 threshold (admitted "permissive" in Appendix I), so the HumanEval number is not pass@k and is incomparable to any published result.

  5. Key experimental details are missing, making the results unverifiable and irreproducible. Table 3 lists Train/Valid/Test as "–" for 4 of 7 benchmarks (HumanEval, OpenBookQA, SciQ, BioProBench); since the whole protocol hinges on validation-guided selection and one-shot held-out testing, unknown splits — and unstated training data — make the central claims uncheckable, and BioProBench has no citation at all. The principal baseline, FT-Agent, has no reference and no implementation details (reimplementation or official? which LLM, which prompts?). The agent backbone LLM is only identified as "claude-haiku (configurable)" (Appendix D); per-benchmark accuracy targets r_tar and compute budgets B₀ are never reported; no per-round results, loss traces, evidence-graph instances, or probe-experiment case studies are shown; Figure 2's efficiency numbers are given only as a bar plot without a table.

Questions for authors
  1. Comparisons are compute-confounded, not mechanism-level. DAPF is a decision wrapper issuing multiple LoRA-SFT rounds vs. a single fixed SFT run — the missing control is any same-budget hyperparameter search. The "efficiency" gain is an artifact of target-met early stopping, which baselines lack.

  2. No per-module ablation. Evidence graph, belief update, budget selector, watchdog, six stages — none is ablated; only attribution is, and it swaps the whole decision procedure. Worse, the released code (Appendix C/F) implements only LR/dropout tweaks and threshold stop rules — the selector of Eqs. 8–9 and the intervention space of §3.3 appear not to exist in the system that produced Table 1.

  3. Stitched components, no design rationale. Known pieces (multi-role agents, watchdog, provenance log, heuristic loss rules) assembled with unexplained magic constants (min(Δ̂,g), 0.6, 5×, 0.7×); no principle says why this composition should work.

  4. Setup unverifiable. 4/7 benchmarks have no train/valid/test splits or training data listed (Table 3 "–"); BioProBench uncited; the core baseline FT-Agent has no reference; r_tar, B₀, and the agent LLM are undisclosed.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.