acceptodds
Under review as a conference paper at ICLR 2027

AdaptWorld: Learning to Act and Adapt over Long-horizon Tasks in Dynamic Environments

Abstract

Long-horizon tasks in dynamic environments require agents to revise decisions as evidence arrives while managing earlier commitments. AdaptWorld constructs executable training environments that make this adaptation consequential. Public-information reference comparisons guide bounded recipe revision; independent assessment admits task families, and execution-and-outcome filtering selects model demonstrations. Across finance, grid storage, inventory, and network recovery, revision raises recipe acceptance from 34.4% to 72.7%. We fine-tune two Qwen backbones on 5,120 episodes from 4,226 tasks. Four-domain training improves both backbones on CEO, Merchant, and local Vending. For 27B, it exceeds the strongest single-domain source by 11.3%, 18.2%, and 69.4%, respectively, at a matched supervision target; 35B retains stronger task-specific specialization. Broader task coverage helps, while the best difficulty mixture depends on the target. At four times the reference horizon, SFT increases completion by 23–31 percentage points across all six model–benchmark pairs. These results link decision-oriented environment construction to transfer and sustained execution. Code is available at https://anonymous.4open.science/r/anonymous_code3-1025.

Primary area:
foundation or frontier models, including LLMs
Keywords:
Long-Horizon Agents, Dynamic Environment Scaling, Synthetic training data, Agent Post-Training

Reviews

Among:
Shape:
1234this paper, Bottom 2%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.33 over 3 reviews, above 2% of the others.
Reviewer_yHhMOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

AdaptWorld constructs executable training environments in four domains. Reference-controller comparisons guide bounded recipe revision, followed by independent assessment on fresh worlds. Filtered LLM trajectories are used for supervised fine-tuning. The paper reports improved external performance for two Qwen backbones and higher completion rates at extended physical horizons.

Critical strengths
  1. Comparing reference controllers with restricted variants provides a concrete test of whether specified behaviors improve outcomes. Assessing frozen recipes on fresh worlds is a sound design choice.
  2. Under matched supervised-token budgets, the 27B mixed-domain model outperforms every single-domain model on all three external evaluations (Table 1). This supports the usefulness of the resulting training data.
Critical weaknesses
  1. The experiments do not isolate the training benefit of the distinctive construction procedure. The ablations vary source composition, coverage, and difficulty, and Table 2 shows increased recipe acceptance. However, these results do not establish the added downstream value of decision-gap-based construction over diverse, outcome-filtered trajectories. As a result, it remains unclear how much the construction procedure itself contributes to the downstream gains.
  2. The external evaluation protocol is unclear. Section 6.1 says evaluation follows the original benchmark settings, but Section 6.5 sets the reference horizons to 365 days for CEO and 30 days for Merchant, shorter than the original 500 and 365 days (CEO-Bench; MerchantBench). The paper does not explain this change, which narrows the scope of the long-horizon claims.
Questions for authors
  1. Could you compare the full pipeline with multi-domain trajectory collection that skips decision-gap-based recipe selection, using the same teachers, trajectory filtering, and supervised-token exposure?
  2. Please explain how the reported horizons differ from the original benchmark settings, and specify the modifications and execution budgets used for Table 1.
Reviewer_S8Y1OpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
2
Soundness
1
Read review
Paper summary

The paper proposes AdaptWorld a method for improving agent performance in long-horizon tasks in dynamic environments. The problem setting is a typical RL setting for MDPs/POMDPs.

Adaptworld uses domain simulators as input and uses this information to generate training instances that can be solved/rolled out to collect high-quality training data for supervised fine-tuning. Given a domain simulator with public information about constraints etc, the first step is to create training environments using a language-model author. This model generates tasks to use for generating training data. Next, an agent is used to solve the task and high quality episodes etc are extracted for supervised fine tuning (SFT). This data is then used in SFT to train a domain-adapted agent. The SFT process can allow for multiple domains to be adapted by the same agent if the training data contains a mixture of trajectories from each domain/environment.

The atuhors then conduct an empircal evaluation where they use 4 domains and then evaluate on 3 benchmarks: CEO, Merchant and local Vending. The authors also conduct some ablation studies with different hyperparameters like horizon length etc.

Critical strengths

The problem of domain adaptation is relevant and of interest to the broader AI community. The empirical results show great promise although it is not clear what the novelty is.

Critical weaknesses

The key weakness of the paper is novelty. I am not able to distill the core contribution since it appears to be using a language model to generate tasks. The prompts etc are not included and it is difficult to assess the significance of this work. The rest of the pipeline is pretty straightforward for normal post-training SFT on downstream tasks.

Fig. 2 seems oversell the claims. I am not sure how active sensing etc is involved or how the instances generate facilitate that. Also it is not clear how episodes are solved. Once instances are generated, there is no guarantee that the tasks can actually be solved. Information on this is missing.

Also, i think the approach needs a lot more than just simulators. It needs reference policies to select high-quality episodes which is unclear how that relates to novel instances and how the metric in Eqn. 2 is used to steer instance generation.

It is also not clear why no baselines from related work are used.

Overall, the paper writing is also not that easy to read although the language is simpler.

Questions for authors

Please address my weaknesses. Happy to discuss and revise my score if i have misunderstood the work.

Reviewer_V2NVOpenReview ↗
Rating
1
Confidence
2
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

The paper proposes AdaptWorld, a pipeline that builds training environments and training data for long-horizon decision tasks in dynamic environments. Its core idea is that training tasks should actually reward revising decisions based on new information. Construction: executable simulators are written for four domains (finance, grid storage, inventory, and network recovery); an LLM picks one of several program-generated candidate configurations; a reference controller is compared with versions that have one capability removed, and the configuration is edited within limits if the gap is too small; after freezing, a one-time statistical test on new instances decides admission. Training: Qwen3.6-27B and Qwen3.6-35B-A3B are run in these instances, and the runs are filtered by outcome. The resulting data are then used for ordinary SFT of the two models. (There is no RL training in the experiments.) Evaluation: three business benchmarks (CEO, Merchant, and Vending).

Critical strengths
  1. The data construction pipeline of this paper is carefully designed, and its engineering logic is detailed and rigorous. Data or RL environments built following this pipeline can be expected to yield very promising results.
  2. There is cross-domain transfer: training only on grid or network data still improves the Vending score.
  3. Task coverage and difficulty mix are compared at the same total number of training tokens, which separates the effect of "more distinct tasks" from that of "more training".
Critical weaknesses

1. Lack of engineering details in the paper (both main text and appendix)

In my opinion, this paper is an engineering-oriented dataset/environment work, so the engineering details are very important. However, some important information is missing; in particular, the construction process and the key parameters of the dataset are not reported:

  • Simulators: the paper does not state who wrote the simulators for the four domains, whether open-source projects were used, or whether their dynamics were checked against real data.
  • Generating and selecting candidate configurations: the paper does not state how many candidates are generated per template (4 in the code), which LLM is used (Kimi K3 is mentioned only in the AI use statement), what the prompt is, or what the sampling temperature is (0.7 in the code), nor does it explain why an LLM makes the choice.
  • Rewriting the task description: the paper states neither the scope nor the purpose of the rewriting.
  • Reference controllers: the paper only states that they consist of parameter estimation, state filtering, finite-step planning, and information acquisition; however, it gives no concrete decision rule for each domain and does not state who wrote them (a human or an LLM).
  • Admission thresholds and test results: the paper does not give the thresholds ρ and δ or the confidence level α.
  • The paper does not state the agent harness used for data collection; it gives neither the inference parameters (temperature, top-p, whether thinking mode is on) nor the collection budget, and it does not state how many times each instance was run.
  • Filtering pass rate: Eq. 4 gives the threshold 0.2, but the values of the normalizing scale gψ are not given.
  • (Optional) Release of example data: if some examples of the final SFT data and of the collected model trajectories were provided, they would give readers a solid and straightforward understanding.

As I believe the authors have a much clearer understanding of their own work, this weakness can be easily resolved as long as they provide this information during the rebuttal.

2. The external benchmark results are severely inconsistent with the original benchmarks

Some of the results look abnormal. For example, the CEO score that this paper reports for a small model (Qwen 27B) is higher than the best runs of large models (Opus 4.7, GPT-5.5) reported in the CEO-Bench paper. I believe the authors are responsible for explaining this phenomenon in the paper. Without a reasonable explanation, it is a major weakness.

The inconsistent/abnormal results are as follows:

  • The horizons differ from those of the original benchmarks: the paper's reference durations are 365 days for CEO and 30 days for Merchant, whereas the original benchmarks use 500 days for CEO-Bench (the original paper only defines a 50-day variant, not a 365-day one) and 365 days for MerchantBench (there is no 30-day simulation setting).

  • Merchant: the MerchantBench paper states "Each run starts with RMB 2,000 in cash, a RMB 1,000 security deposit" (RMB = CNY); the metric is final net assets over 365 days; all LLMs score between 6.56 and 59.46 thousand RMB (best: Qwen3.7-Max with Hermes, 59.46), humans score 217.61 thousand, and the rule-based policy scores 24.48 thousand; the original paper has no other starting-capital setting. In Table 11 of the paper, with only 30 days, 27B Base reaches 1,299,153.51 RMB, 35B Base 2,179,983.58 RMB, and 27B SFT 4,679,074.09 RMB; that is, Base is about 22 and 37 times the best LLM in the original benchmark, and about 6 and 10 times the human result. In Table 10, across 8 single-domain training conditions and 2,048 episodes, the lowest single-episode Merchant score is 1,291.2 thousand RMB (27B Network); even the worst episode is about 22 times the best original LLM and 6 times the human result, so the gap is not caused by a few outlier episodes.

  • CEO: in the CEO-Bench paper, each model starts with USD1M in cash, is scored on its cash at the end of 500 days, and is reported by its best of 3 runs; Claude Opus 4.7 reaches USD70,620, GPT-5.5 about USD33K, and Qwen 3.7 Max USD365,346; "GLM 5.1, DeepSeek V4 Pro, Gemini 3 Flash, and Grok 4.20 bankrupt on all runs"; the rule-based policy reaches USD15.76M, and the abstract states that "all evaluated models remain below the rule-based baseline". In the paper, 27B Base (not a pretrained-only base model; I think the authors mean the original Qwen model) goes from USD106,694 to USD202,036 after SFT, and 35B from USD94,738 to USD115,229. If the USD1M starting point is kept, both Base and SFT lose 80–90% of the capital, and "+89.4%" means losing somewhat less, which the paper does not mention. Moreover, the mean over 256 episodes of 27B Base is higher than the best of 3 runs of Opus 4.7 and GPT-5.5. Also, the lowest single-episode CEO score over 2,048 episodes is USD86.1 thousand (35B Grid), with no bankruptcy at all, whereas bankruptcy is common in the original benchmark. The paper reports no rule-based score on the external benchmarks.

That said, what still holds is that Base and SFT are compared under the same setting; thus, the finding that "SFT scores higher under the authors' setting" is not itself affected. What is affected is the claim of following the original benchmarks and the meaning of the external scores.

This is my major concern about this paper. As long as these abnormal/inconsistent issues can be addressed, I believe this paper can be a solid work.

3. The admission thresholds are very low

The threshold values can only be found in the code. In the repository, comparisons() in domains/inventory/synthesis.py requires the reference controller to beat the version with one capability removed by at least 0.01 in normalized score, and to beat the simple baseline by at least 0 (that is, not to be worse). In the acceptance certificates, the network and grid templates also use 0.01 for the capability gap and 0 for simple baselines. This threshold seems too low to me, unless the authors can justify it or the code is actually outdated (i.e., not the final version used for the paper).

4. The writing is hard to follow

The writing of the paper is not very good: the sentences and paragraphs lack transitions, and many self-coined terms are used without introduction, which makes the paper difficult to read. For example, the abstract and the introduction use terms such as "recipe", "task family", "admission" and "reference controller" before they are defined, and without explanation.


These four weaknesses, especially the second one, raise my major concerns, so my current inclination is to reject. I actually prefer to rate this paper 1.5, however, there's no such an option. I look forward to hearing back from the authors; as long as these issues can be resolved, I would be very glad to raise my score.

Questions for authors

Q1: Who wrote the simulators? Q2: Who wrote the reference controllers? Q3: Why does an LLM select the configuration? Q4: What is the purpose of rewriting the task description? Q5: Which agent harness is used for data collection and evaluation? Q6: What are the settings of the external benchmarks? (See Weakness 2.)

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.