acceptodds
Under review as a conference paper at ICLR 2027

TerraWeave: Grounding Policy-Skill Co-Evolution through Verified Execution for Earth Observation Agents

10 views

Abstract

Tool-augmented Earth observation (EO) agents solve multi-step geospatial tasks, yet successful tool execution does not guarantee that the dependencies required by downstream steps have been established. We introduce TerraWeave, which uses verified execution state as a shared reference for policy credit and skill evaluation. GeoState records observation-supported resources, relations, and readiness conditions. Grounded Dependency-Local Group Relative Policy Optimization (GDL-GRPO) assigns decision-local credit by normalizing verified progress among decisions sharing the same dependency opportunity. Skill candidates are compiled and execution-verified, then evaluated through frozen-policy ON/OFF continuations from the same executable snapshot to estimate incremental utility. Verified progress drives policy updates, while measured incremental utility selects skill versions for subsequent training. On the 1,169-task OpenEarthAgent test set, TerraWeave reaches 50.37/77.01 in native answer score (Ans.) and generation-tool execution (Gen.). It improves Ans./Gen. by 2.26/2.76 percentage points over a matched skill-augmented RL baseline and by 7.87/12.18 points over Standard GRPO. With the bank disabled, the co-evolved policy retains most gains over policy-only GDL-GRPO; enabling the bank adds runtime gains.

Primary area:
reinforcement learning
Keywords:
Earth Observation Agents, Tool-Using Agents, Reinforcement Learning, Skill Learning and Evolution, Credit Assignment

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_Z8HjOpenReview ↗
Rating
1
Confidence
2
Informativeness
2
Presentation
1
Soundness
1
Read review
Paper summary

The paper addresses the issue of step reward assignment by considering dependencies of subsequent steps in earth observation agents. The same step could be right or wrong depending on whether its dependencies are fulfilled, which previous methods do not capture. The paper also contributes a new skill bank that is evolved based on the utility it provides.

Critical strengths
  • The approach is intuitive and addresses weaknesses of previous approaches. Understanding how to assign rewards to each step is a challenging problem, and the work shows that building dependency checkers can improve on this front.
Critical weaknesses
  • The benefit over baselines for GDL-GRPO is not clearly established. The utility improvements' confidence intervals include zero, and improvements beyond utility are demonstrated on metrics that are part of the reward policy, which just shows that optimizing for this metric increases it.
  • The comparisons to baselines are not budget-matched. L. 288 states "TerraWeave also uses no-gradient admission and utility probes, leaving total execution cost unmatched," and Table 35 shows that TerraWeave ran 8,7058{,}705 environments, whereas the baselines only ran 4,3284{,}328, over twice as many. This confounds the comparison. Especially with the previous weakness, it's not clear where the gains come. It seems plausible that the improvements from GDL-GRPO are a statistical artifact and that TerraWeave performs better only because it has twice the budget.
Questions for authors
  • How would GDL-GRPO with the ReSkill skill bank perform?
  • How would TerraWeave's skill bank perform with SALT?
  • The code has not been provided, and descriptions from the paper raise implementation questions. For example, how has the decoding seed been implemented for API-hosted models such as GPT-5.5? Can authors provide the code?
  • Why are the confidence intervals in Tables 26 and 30 exactly symmetric?
  • How were the "Eligible / all" counts in Table 22 obtained?
Reviewer_86rROpenReview ↗
Rating
3
Confidence
3
Informativeness
3
Presentation
3
Soundness
2
Read review
Paper summary

TerraWeave couples policy learning and skill evolution through a verified execution representation for Earth-observation workflows. GeoState records supported resource bindings and requirements. GDL-GRPO assigns local advantages among decisions facing matched dependency opportunities. Candidate skills pass compilation and execution admission, then receive utility estimates from paired ON/OFF continuations restored from the same state under a frozen policy. Experiments compare credit-assignment and skill-RL baselines, separate retained policy gains from runtime bank effects, and test a compositional holdout. On OEA-TEST, the full system scores 50.37 Ans. and 77.01 Gen., exceeding the nearest ReSkill baseline by 2.26 and 2.76 points in the primary runs.

Critical strengths
  1. The shared execution representation connects two well-motivated decisions: which actions deserve training credit and which skills help from an applicable state. The skill evaluation is carefully designed. Paired branches restore the same executable state, freeze the policy, and keep writable outputs separate. Admission evidence is kept out of utility estimation, and repeated observations are clustered by task. These choices make the intervention results substantially more interpretable than a comparison of unrelated successful trajectories.

  2. The evaluation addresses several plausible alternative explanations. It includes same-feedback credit baselines, targeted grouping and skill-lifecycle ablations, crossed policy/bank evaluation, three additional training runs, and a held-out composition test. The independent verifier audit is also useful. The reported gain over ReSkill persists across the additional runs, and most improvement over policy-only GDL remains with the runtime bank disabled.

Critical weaknesses

The comparison does not establish how much of the gain comes from the method under a common total execution allowance. TerraWeave uses 4,328 training rollouts plus 4,377 bank-evidence executions, while D2Skill and ReSkill perform their comparisons inside the 4,328-rollout allowance. The paper is explicit about this, and the 1.28x accelerator-use figure is helpful, but it is measured against a GRPO reference and excludes API, CPU, and tool-server costs. The nearest-baseline gains are modest enough that a total-budget comparison could change the practical conclusion. This limits the comparative result rather than invalidating the reported scores.

Questions for authors

Please see weakness

Reviewer_17vUOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
1
Soundness
2
Read review
Paper summary

EO agents rely on multiple steps of tool calling and reasoning to solve geospatial tasks. However, not every successful tool call moves reasoning towards the specified goal. For example, if you want the segmented boundary of hospitals in a region, a tool that outputs area extent would not provide any valuable information. Thus, to improve overall EO reasoning performance, policy learning must prioritize actions that advance unresolved dependencies to solve a specific problem. Given the difficult and specialized nature of EO observation, practitioners have built skill banks containing procedures for common tasks. While this provides scaffolding for the aforementioned policy learning, it also introduces a new challenge. How can we use skill banks if the policy being learned changes the context in which tools are being called? To address all of these concerns, the authors propose TerraWeave, a novel framework that continually updates a central GeoState to balance stored skills in GeoSkillBank and assign credit only to truly influential actions.

Critical strengths

The authors posit a key disadvantage of prior works in agentic systems for EO is the unverified credit assignment of actions and poor coupling of skill libraries and policies. With TerraWeave, they aimed to rectify these shortcomings. Experimental results show impressive performance gains over standard GRPO (Table 1) and ablations of the proposed framework’s design choices (Table 2 and Table 3). Moreover, the authors provide Table 4 to show that the performance gains come from better policy learning and not just some niche evolution of the skill bank.

Critical weaknesses

The authors propose a new policy learning framework to improve credit assignment and policy-skill coupling. However, despite the performance gains, many aspects of the method and evaluation are not easily understood or motivated. For example, both policy learning and skill evolution rely on GeoState and GeoVerify, yet the text does not easily motivate their design or any other versions explored. Also, the reward setup and how different penalty and reward events are weighted were not motivated. On the evaluation side, the focus on OEA-test should be explained, especially due to the growing collection of EO evaluation datasets and environments. Additional clarification questions are included in the “questions” section and are relevant to my overall recommendation.

Outside of properly motivating the design choices, the paper should also provide a more in-depth error analysis of both existing methods and TerraWeave. For example, while we see performance gains and some performance stratification across tool calls (Table 32) and problem categories (Table 21), a more in-depth error analysis is needed to explore whether TerraWeave minimizes the credit assignment problem. For instance, are there certain tool calls that are likely to be called without progressing tasks in prior works that are no longer being called as often?

Questions for authors

I have a few clarification questions.

What is meant by “the comparison unit matters: the same tool can face different prerequisites while different actions can establish the same requirement” (Line 46)?

What does “first independent plan-changing valid negative” mean in CVNC^{\text{VN}} definition? (Line 1291)

How exactly is pp calculated? (Line 223)

How was the GeoSkillBank initialized? What kind of skills are learned, and how do they converge or diverge from common skill banks in EO? How fragile are trained skill banks (Line 482)? Qualitative examples beyond Figure 9 would be useful.

What is the compute overhead for each baseline presented in Table 2?

What are the quantitative differences between “frequently” and “periodically” in Line 269-270? Why was o3-mini used as a teacher model (Line 1550)?

Comments on the paper

1 comment
AuthorsOpenReview ↗

TerraWeave: Core Contributions and Controlled Evidence

We thank the reviewers for their careful assessment and constructive questions. We would like to highlight the central contribution of TerraWeave and how the additional evidence clarifies its scope.

TerraWeave addresses two coupled challenges in stateful Earth-observation workflows: successful tool execution does not necessarily resolve the dependencies required for task completion, and the usefulness of a procedural skill changes as the receiving policy improves.

Our framework connects these challenges through a shared, execution-verified state interface. GDL-GRPO assigns credit at comparable dependency opportunities, while GeoSkillBank compiles executable procedures and evaluates their incremental utility through matched, frozen-policy ON/OFF continuations. Utility-selected skills subsequently inform policy learning, creating a grounded co-evolution process.

The controlled results support both mechanisms. With the Bank OFF and the same GeoVerify feedback, GDL-GRPO improves Ans./Gen. by +1.84/+2.30 points over GRPO+GeoVerify. Removing dependency-based grouping reduces HELD-OUT Binary Answer from 50.5% to 47.1%. Under the execution-count-matched comparison, TerraWeave achieves 50.37/77.01 versus 49.02/75.40 for ReSkill-EqualExec.

Our reviewer-specific responses provide the corresponding controls, diagnostic analyses, and implementation clarifications. We will incorporate these clarifications into the revised manuscript.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.