acceptodds
Under review as a conference paper at ICLR 2027

Polar: Agentic RL on Any Harness at Scale

Abstract

Reinforcement learning for language agents increasingly depends on custom harnesses that manage long-running context, multi-turn tool use and multi-agent orchestration. However, porting these harnesses into RL environment interfaces remains difficult and often loses important training signals. We bridge this gap with polar, a rollout framework for scalable asynchronous RL over arbitrary agent harnesses. Polar treats the agent harness as a black box: it proxies LLM API calls, records token-level model interactions, and reconstructs token-faithful trajectories for training. Each rollout node efficiently manages runtime prewarming, agent execution, trajectory reconstruction, and evaluation in parallel, exposing asynchronous service endpoints that can be consumed by independent trainers at scale. This decoupled design makes polar agnostic to agent harnesses, training infrastructure, and RL algorithms while improving compute utilization for long-running agent workloads. We validate polar by training agents on software-engineering tasks with popular coding harnesses. Using simple GRPO, polar improves Qwen3.5-4B by 22.6, 4.8, 0.6 and 6.2 points on SWE-Bench Verified with the Codex, Claude Code, Qwen Code and Pi harnesses, respectively. We further demonstrate polar for offline data generation over custom harnesses and ablate trajectory reconstruction strategies.

Primary area:
reinforcement learning
Keywords:
agentic RL

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_b5fUOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

This paper presents POLAR, an asynchronous rollout framework for reinforcement learning over long-horizon language agents. Rather than requiring existing agent harnesses to be rewritten into the environment interface of a specific training framework, POLAR treats the harness as a black box and places a provider-compatible proxy at its model API boundary. The proxy forwards model requests and records prompt tokens, behavior-policy sampled tokens, log probabilities, and message structure, then reconstructs these interactions into trainer-ready trajectories. The system separates runtime preparation, harness execution, trajectory reconstruction, evaluation, and trainer callbacks into asynchronous stages, and provides two trajectory construction strategies: per-request and token-faithful prefix merging. The authors train Qwen3.5-4B with GRPO on SWE-Gym across four coding harnesses—Codex, Claude Code, Qwen Code, and Pi—and report absolute pass@1 gains of 22.6, 4.8, 0.6, and 6.2 points on SWE-Bench Verified, respectively. The paper also presents an efficiency ablation of prefix merging versus per-request reconstruction, and an offline SFT data generation case study using Qwen3.5-122B-A10B with the Pi harness.

Critical strengths
  • The paper identifies and addresses an important, practical systems bottleneck: complex or closed-source agent harnesses are difficult to port losslessly into the environment abstractions prescribed by training frameworks.
  • Using the model API as the unified observation and training boundary is a clean, pragmatic design. It covers harnesses shipped as CLIs, package-managed tools, or closed-source binaries without requiring changes to their internal event loops.
  • The system architecture cleanly separates task scheduling, runtime preparation, harness execution, trajectory reconstruction, evaluation, and trainer callbacks, which is well suited to long-tail rollouts and expensive environment startup.
  • Prefix merging handles token fidelity carefully: trainable tokens come from the behavior policy's raw sampled output, while canonical interstitial tokens are explicitly masked, avoiding drift from naive transcription and re-tokenization.
Critical weaknesses
  • The claim of supporting 'any harness' is qualified by important preconditions: the harness must allow its model endpoint to be redirected, model traffic must use a protocol the proxy can emulate, and the proxy must faithfully mimic authentication, errors, streaming events, and provider-specific semantics. The current experiments cover only coding-style CLI harnesses and do not establish broad generality.
  • All main online RL experiments use the same Qwen3.5-4B base model, the same SWE-Gym training set, and the same software-engineering task family. The paper does not validate on browser, OS, research, data-analysis, or true multi-agent settings, so external validity is limited.
  • There is no direct end-to-end comparison against Agent Lightning, rLLM, SkyRL-Agent, OpenClaw-RL, or other low-intrusion training approaches. Table 3 is a qualitative judgment based on the authors' reading of code and documentation, which cannot substitute for empirical comparisons of compatibility, throughput, integration cost, and training outcomes.
  • The main benchmark results do not report independent random seeds, confidence intervals, or variance. In particular, the 0.6-point gain on Qwen Code likely falls within training or SWE-Bench evaluation noise.
  • Key control experiments are missing: e.g., training the same model in a standard agent loop without the native harness, supervised fine-tuning only, per-request reconstruction with proper session-level normalization, or training on one harness and evaluating transfer to another. This makes it hard to attribute the gains to POLAR's system design, GRPO, harness-specific adaptation, or training budget.
  • The prefix-merging efficiency ablation is not fully fair. The two strategies produce 1,185 versus 218 trainer updates, which may also change effective batch construction, sequence lengths, optimizer steps, and reward propagation semantics. The 5.39× wall-clock difference therefore cannot be attributed solely to the trajectory reconstruction strategy as a pure systems efficiency gain.
Questions for authors
  • Please crisply define the applicability boundary of 'any harness'. Which harnesses cannot be trained via POLAR—e.g., those with fixed remote endpoints, certificate pinning, end-to-end encryption, custom websockets, client-side local sampling, or those that depend on precise streaming timing?
  • How many independent training runs and evaluations do the four SWE-Bench Verified results reflect? Can you report random seeds, standard deviations, or confidence intervals? In particular, is the 0.6-point gain on Qwen Code statistically meaningful?
  • Please provide a direct comparison with the closest systems, including at least integration effort, amount of harness code that must change, supported protocols, throughput, end-to-end training time, and final performance. Qualitative judgments in Table 3 alone make it hard to confirm a substantive advantage over Agent Lightning or rLLM.
  • In the 5.39× speedup experiment between prefix_merging and per_request: are total tokens, trainable tokens, optimizer steps, batch token caps, sequence-length distributions, and gradient update counts held constant? If not, how do you separate pure systems gains from changes in optimization semantics?
  • What exactly does 'rollout GPU utilization' in Figure 5b measure—inference engine busy time, sampled device utilization, or actual kernel utilization? Why would the choice of trajectory representation change this metric from 20.4% to 87.7%?
1 comment

We appreciate the reviewer for the specific review. These are critical comments that help us realized common questions in our approach. Here are our response to each of the concerns.

W1/Q1. Boundary of "any harness". Polar supports all common LLM API types (Anthropic Messages, OpenAI Chat, OpenAI Responses or Google generateContent over HTTP/SSE). By "any harness" it is not to say the proxy translation is fixed for any possible API types in the future, but to strengthen the innovation of using API-capture as the design boundary for modern Agentic RL rollout. To sustain our claim, we ctively maintain the framework's proxy for future API type changes. Harness url redirection is a native feature to existing agent harnesses to support multiple provider, and is assumed in Polar's proxy. Not supported: fixed remote endpoints, certificate pinning and proprietary weights. But these are intentional simplicity since we assume a RL framework must have access to its local policy and is self-served for token capture and weight update. Eleven mainstream harnesses run today (see preset examples) with no harness-side changes. We will add a "Scope and limitations" paragraph and qualify the abstract to "any harness with a redirectable, supported model API".

W2. Narrow example with Qwen 4B and SWE task. We further validated Polar over a large run: Qwen3.8-27B with the Pi harness, trained on the public Skill2Env corpus (7,496 terminal tasks), evaluated on Terminal-Bench 2.1: 49.4% → 58.4% pass@1 over 5 trials (details under Zf4Z W1). Multimodal, Web and Computer-uUe reconstruction statistics are under WgMP W3.

W3/Q3. Empirical comparison with other frameworks. We agree. Below is our additional comparison experiment using the same SWE tasks, Qwen3.5-4B with GRPO on 8 B200, each over Claude Code and Codex harnesses.

Agent LightningrLLMSkyRL-AgentOpenClaw-RLPOLAR
Claude Code RL?yes (per-request only)nononoys (prefix-merging)
Codex RL?nonononoyes (prefix-merging)
Train time (h)10.5 (cc)---7.4 (cc), 7.6 (codex)
SWE-Bench Verified32.2 (cc)---33.8 (c), 28.0 (codex)

"No" means no Anthropic/Responses endpoint, or the loop must be re-implemented inside the framework with non-trivial effort. We do not claim better policies where a harness fits these systems; the claim is that more harnesses become trainable with no per-harness integration.

W4/Q2. Variance. We agree. Table 1 is one training run and one pass over 500 instances. We add evaluation intervals and repeated evaluations across 4 independent runs.

HarnessGain
Codex+21.0 ± 4.2
Claude Code+5.2 ± 5.8
Qwen Code-0.4 ± 1.0
Pi+8.8 ± 2.6

Qwen Code gain is within noise. The correct interpretation is that Polar RL did not degrade a model on the harness it was already tuned for. We will add this note to the main experiment section.

W5. Controls. We answer by argument and will add (a) and (b) as experiments in the final version. On attribution: the policy gain is from GRPO; Polar's claim is that GRPO becomes applicable to all agent harnesses, so that policy learns to solve problems respecting harness behaviors during training. (a)/(b) Table 1 already shows capability does not carry across harnesses: the same base model scores 34.2% on Pi and Qwen Code but 3.8% on Codex, whose Responses protocol and patch-style tools it has not seen. Training on a generic loop cannot be expected to close that gap. (c) SFT needs successful trajectories on the target harness, which only a rollout system such as Polar can collect (Sec. 3.2); it complements RL but its stage during training (usualy mid-training) is orthogonal. (d) Session-level normalisation removes per-request's sample-weighting bias but keeps its redundant prefix computation (see Zf4Z Q1).

W6/Q4. Ablation fairness. 1,185 and 218 are trainer samples, not gradient updates, and both arms take 3 optimiser steps. We kept the key variants constant: model, hardware, rollout workload (192 sessions), per-GPU token cap, and trainable tokens (2.31M vs 2.3M; every sampled token is preserved for training in prefix-merging algorithm). The only difference by design is the number of tokens forwarded through the trainer (30.8M vs 6.3M). The ablation states trainer-side speedup for a given model, dataset and hardware consumed over the same epoch.

Q5. Fig. 5b. The metric is utilisation of the 4 inference GPUs, averaged over the 3-step window. The representation does not change inference cost. It changes how fast the trainer consumes rollouts. During Async RL, a max async step is necessary to limit staleness between training and inference. With per-request the trainer is the bottleneck and rollout turns idle to wait for training updates. With prefix merging the trainer keeps pace and they stay busy (87.7%).

Reviewer_WgMPOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
3
Presentation
2
Soundness
2
Read review
Paper summary

This paper introduces POLAR, a rollout framework designed for scalable, asynchronous reinforcement learning (RL) over arbitrary, off-the-shelf agent harnesses. POLAR treats the agent harness as a black box. It interposes a provider-compatible model API proxy at the LLM boundary to capture request/response data, track token-level interactions, and reconstruct token-faithful trajectories for training. Furthermore, POLAR decouples runtime initialization, harness execution, evaluation, and trajectory reconstruction through asynchronous in-node worker pools and a rollout-as-a-service design. The authors validate POLAR using standard GRPO on popular harnesses over SWE-Bench/SWE-Gym tasks, demonstrating performance gains and improved training throughput via token-faithful prefix merging.

Critical strengths
  1. The ability to treat complex agent harnesses as black boxes without internal code modifications is highly practical and lowers the barrier for applying RL to real-world software engineering agents.

  2. The decoupled, asynchronous rollout-as-a-service architecture effectively isolates CPU-heavy runtime setup and long-tail evaluation from GPU-bound training, improving system utilization and scalability.

  3. The paper provides extensive experiments across multiple coding harnesses on SWE-Bench Verified/SWE-Gym, accompanied by useful ablations on trajectory reconstruction strategies (e.g., token-faithful prefix merging vs. per-request).

Critical weaknesses
  1. The core contributions of this work are predominantly engineering and systems-oriented. While valuable for systems infrastructure, the paper lacks fundamental algorithmic breakthroughs in RL methodology or agentic reasoning. The techniques largely repurpose existing proxy-based tracing and asynchronous task queuing concepts for LLM agent workflows.

  2. Intercepting LLM API traffic via a proxy to capture traces is a well-known paradigm in systems engineering and observability (and shares similarities with concurrent concurrent logging tools like Agent Lightning and rLLM). The distinction in minimum integration point, while robust for CLI/binary tools, feels more like an engineering refinement than a conceptual leap.

  3. The evaluation is heavily tailored to software engineering tasks (SWE-Bench/SWE-Gym). It remains unclear how well the black-box proxy and prefix merging strategies generalize to other highly interactive, stateful domains (e.g., web navigation, multi-modal computer use) where tool feedback and environment states are vastly different.

Questions for authors
  1. Discuss and evaluate whether POLAR's proxy-based approach and prefix merging can be seamlessly generalized beyond coding agents to other modalities where token-interstitial structures might differ.

  2. Since the contribution is heavily systems-focused, the authors should re-frame the narrative to position POLAR explicitly as a robust systems infrastructure middleware for the community, providing deeper analysis on overhead, proxy latency, and scaling bottlenecks under larger cluster topologies.

2 comments

We thank the reviewer for the constructive suggestions, in particular Q2.

W1. Contributions are predominantly engineering.

We agree, and we do not claim an algorithmic contribution: all experiments deliberately use unmodified GRPO. Polar is a systems paper, and ICLR's call explicitly lists infrastructure and software libraries as in scope. We will make this positioning explicit in the title/abstract/introduction (see Q2).

Within that scope, two parts are unique contribution from Polar and did not exist in prior open literature or frameworks.

  1. Token-faithful reconstruction of black-box sessions. prefix-merging is a unique algorithm Polar contributes and validates in agentic RL experiments, that yields 5.39× training-time reduction than previously widely used "per-request" strategy in the community.
  2. Provider-protocol emulation in front of an open-weights backend. To run Claude Code, Codex or Gemini CLI unmodified against a local policy, the proxy must accept Anthropic Messages, OpenAI Responses and Google generateContent traffic, translate tools, reasoning fields, images and streaming events in both directions, and still return exact token IDs and log-probabilities to the trainer. The full set of double-direction proxy translation was not presented in any open framework before Polar.

W2. Proxy-based capture is a known paradigm and overlaps with Agent Lightning and rLLM.

We agree that intercepting LLM traffic is not new. The difference we claim is narrower and concrete:

Agent LightningrLLMPOLAR
Harness source changes0 lines~200 lines: agent lives in framework0 lines
Protocols acceptedOpenAI Chat onlyOpenAI Chat onlyAnthropic Messages, OpenAI Chat, OpenAI Responses, Google generateContent
Multi-turn sample constructionper-requestper "step", framework defined loopper-request, prefix-merging

W3. Evaluation is tailored to software engineering; unclear generalisation.

We agree that our original SWE example from the paper is narrow. In fact, the capture and reconstruction path in Polar is generalizable and agnostic to what the tools do. Since its original release, we have added these tested examples:

  • Image input. Our release includes a vision example where agent answers by reading an image.

  • Terminal Agent. Our release also include a 15k harbor-format terminal tasks from AI2's TMAX.

  • RL gains outside software engineering (new). We trained Qwen3.8-27B with POLAR and the Pi harness on the public Skill2Env corpus (7,496 harbor-format terminal RL tasks) and evaluated on Terminal-Bench 2.1: 49.4% → 58.4% pass@1 (+9.0,). Details are in our response to Reviewer Zf4Z, W1(a).

Thanks for your suggestion on web and computer-use harnesses. We additionally run with the following tasks through Polar using Qwen3.8-27B:

BenchmarkHarnessModalitySessionsMean requestsPrefix merged traces
WebArena-Litebrowser-usetext + image165241.2
OSWorldscreenshot agent keeping the last 3 framesimage300189.5

Q1. Can the proxy approach and prefix merging generalize to other modalities?

Yes. Prefix merging makes two assumptions: (i) within a chain, the harness appends to the conversation rather than rewriting it, and (ii) the chat template closes an assistant turn with an end-of-turn token. Neither concerns modality. Tool results of any kind (text, screenshots as image placeholders, structured observations) land in the masked interstitial.

Q2. Re-frame as systems infrastructure middleware.

We adopt this framing. We will revise the abstract and introduction to present Polar as rollout infrastructure, move the framework comparison into the main text, and add a systems-evaluation subsection with the following.

Proxy overhead. Polar's routing and gateway translation overhead is trivial compared to time spent on inference.

Protocol (harness)Gateway overhead p50 / p95 (ms)Backend generation p50 / p95 (s)Overhead share
Anthropic Messages (Claude Code)6 / 218.4 / 410.07%
OpenAI Responses (Codex)9 / 307.1 / 360.13%
OpenAI Chat (Qwen Code, Pi)3 / 116.5 / 330.05%

(Data from Polar's quick start example.)

Scaling and bottlenecks. The bottleneck is collectively influenced by the dataset (difficulty), policy model, GPU topology (for training and inference), Polar gateway load concurrency and rollout setups (eg. session timeout). We show three examples below, where the bottleneck can be differed by different topology settings, even using the same model and dataset during rollout.

Gateways × rollout GPUsConcurrent sessionsSessions / hourFirst saturated resource
1 × 464250inference GPUs
4 × 12256960trainer
8 × 245121,750host CPU and sandboxes memory
Reviewer_WgMPOpenReview ↗

Discussion

Thank you for your reply. I agree that building infrastructure is a meaningful and unconventional endeavor, and it holds unique significance in today's AI community.

Reviewer_Zf4ZOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
2
Soundness
3
Read review
Paper summary

This paper presents POLAR, a rollout framework designed to support scalable agentic RL with existing agent harnesses without modifying their internal implementations. Its key idea is to treat agent harnesses as black boxes and capture LLM API interactions through a proxy, reconstructing token-faithful training trajectories from the recorded model requests and responses. The framework also introduces an asynchronous rollout-as-a-service architecture that separates runtime preparation, agent execution, trajectory reconstruction, and evaluation. In particular, its prefix-merging strategy combines compatible multi-turn model interactions into longer training sequences while preserving the original sampled tokens and appropriate loss masks.

The authors validate POLAR through GRPO training of Qwen3.5-4B on SWE-Gym using four coding harnesses: Codex, Claude Code, Qwen Code, and Pi. Evaluation on SWE-bench Verified shows improvements of 22.6, 4.8, 0.6, and 6.2 percentage points, respectively. They also report a 5.39× training speedup from prefix merging compared with per-request reconstruction in a three-step comparison, and demonstrate the framework's applicability to offline SFT data generation.

Critical strengths
  1. Efficient trajectory reconstruction through prefix merging.

The proposed prefix-merging strategy is a useful contribution for long-horizon agentic RL, where repeated model calls often contain substantial overlapping context. By merging compatible requests into longer training sequences while preserving sampled token IDs and loss masks, POLAR reduces redundant computation and training overhead. The reported 5.39× speedup over per-request reconstruction demonstrates its potential practical value.

  1. Practical and minimally intrusive integration with existing agent harnesses.

The proxy-based design provides a practical approach to integrating existing agent harnesses into RL training without modifying their internal implementations. This is particularly valuable for complex coding agents such as Codex and Claude Code, whose execution logic, tool interfaces, and context management can be difficult to reproduce in conventional RL frameworks.

  1. End-to-end validation across multiple coding harnesses.

The experiments demonstrate that POLAR can successfully support GRPO training across four different coding harnesses. In particular, the improvements on Claude Code (+4.8 points) and Pi (+6.2 points) provide encouraging evidence that the framework enables meaningful RL training beyond simply adapting to an unfamiliar harness, as may be the case with Codex. The additional offline SFT data generation experiment further illustrates the framework's versatility.

Critical weaknesses

Limited scalability evaluation.

Although POLAR emphasizes scalable agentic RL, the experimental validation remains relatively limited. The RL experiments use only a 4B model trained on 293 tasks for a single epoch, without evaluating larger models or more demanding training workloads. Moreover, the paper lacks systematic scaling experiments across different computational resources or rollout concurrency levels, making it difficult to assess the framework's scalability in more realistic, large-scale RL training settings.

Questions for authors

The paper reports significant reward hacking when broadcasting outcome rewards to per-request traces, attributing it to noisy credit assignment. However, it is unclear why prefix merging would fundamentally alleviate this issue, since both strategies rely on the same session-level outcome reward. Could the authors clarify how reward assignment, loss normalization, and sample weighting differ between the two strategies, and explain what specifically causes the observed reward hacking under per-request reconstruction?

1 comment

We thank the reviewer for the careful reading and for the positive assessment of prefix merging and black-box integration. We answer the weakness and the question in turn.

W1. Limited scalability evaluation (4B model, 293 tasks, one epoch; no scaling sweep over resources or rollout concurrency)

We agree the submission under-reports scaling evidence. We add a larger-scale RL experiment, and three supporting analyses.

New experiment: Qwen3.8-27B model RL on 7,496 terminal agent tasks, trained on 32 GB200 GPUs (8 nodes on slurm), evaluated on Terminal-Bench 2.1. We trained Qwen3.8-27B with POLAR on the public Skill2Env dataset, which provides 7,496 RL tasks spanning far more than software engineering (literature review, data analysis, security audits, document and slide production, operations), each a containerised terminal task with a programmatic verifier. We train on all 7,496 tasks. Rollouts use the off-the-shelf Pi harness and prefix merging. Evaluation is on Terminal-Bench 2.1, held out from training, pass@1 over 5 trials per task.

Qwen3.8-27B, Pi harnessTerminal-Bench 2.1Gain
Base49.4%—
RL recipe A GRPO advantages, token-level trust-region mask54.1%+4.7
RL recipe B mean-baseline advantages, trace-level trust region58.4%+9.0

Training is stable and improves over the run. Mean programmatic reward over the 256 rollouts of each update, recipe B:

Updates1–2021–4041–8081–120121–160161–200201–232
Training reward0.5510.7020.7060.7100.7050.7040.723
Mean sampled tokens / session16.6k14.4k13.4k13.0k13.5k12.7k13.5k
Session timeout rate5.6%3.5%4.3%4.2%3.3%3.5%2.4%

Reward rises from 0.55 to 0.70 over the first 40 updates and holds at 0.70–0.72 through update 232, with shorter sessions and fewer timeouts. The trainer-versus-sampler log-probability gap stays between 5×10⁻⁴ and 1.6×10⁻³ for the whole run and the trust-region gate rejects under 0.1% of sequences, which is direct evidence that Polar's proxy-captured log-probabilities and prefix-merged traces remain faithful at this scale.

Q1. Why does prefix merging alleviate the reward hacking seen with per-request reconstruction, given both use the same session-level reward? How do reward assignment, loss normalisation and sample weighting differ?

The two builders carry the same reward and the same trainable tokens; yet per-request leads to imbalanced reward weighting among trainable traces in our original vanilla GRPO setup.

(a) Reward assignment. Identical. The evaluator produces one outcome reward per session, and it is attached to every trace emitted from that session.

(b) Trainable tokens. Identical by construction. Every sampled assistant token appears as a trainable (loss-mask = 1) token exactly once under either builder. Per-request emits one trace per model call, whose trainable part is that call's response. Prefix merging emits one trace per append-only chain, whose trainable part is the concatenation of the same responses, with harness-inserted tokens masked.

(c) Sample weighting — imbalanced under per-request. Under per-request, a session's influence on both the group baseline and the gradient therefore scales with its number of model calls. For example, a long and tedious agent rollout which consumes extraneous turns to solve a problem gets higher weights of positive reward during training step, than a neat and clean solution. In the original GRPO recipe from our paper, the gradient updates did not normalize within individual agent session, thus amplified this reward imbalance.

Our latest experiments introduce this trace-level normalization, so each agent rollout contributes equal-weighted gradient updates regardless of the number of reconstructed traces. We observed that this trace-level normalization effectively reduced reward hacking and policy collapsing during RL for per-request strategy.

Comments on the paper

1 comment
AuthorsOpenReview ↗

General Comment

Thank you all for the detailed reviews. Our paper presents Polar, a rollout framework that makes off-the-shelf agent harnesses trainable with RL by capturing token info at the model API boundary, including an innovative algorithm rebuilding training traces with prefix merging. We are encouraged that the reviewers agreed on the strengths of the work: that black-box integration through the model API is "practical and minimally intrusive" (Zf4Z) and"a clean, pragmatic design" (b5fU). We appreciate the reviewers' acknowledgement on the historical meaning of prefix-merging as a useful contribution for long-horizon agentic RL (Zf4Z) that handles token fidelity carefully (b5fU); and that the asynchronous rollout architecture cleanly separates runtime setup and evaluation from GPU-bound training (WgMP, b5fU).

In the individual replies we describe the changes made in response to the reviews, which we believe make the paper stronger. Two of them answer concerns raised by more than one reviewer:

  • Scale and task diversity (Zf4Z, b5fU). Beyond the SWE example presented, we further conduct terminal-agent RL by training Qwen3.8-27B on unmodified Pi harness on a public harbor dataset. Terminal-Bench 2.1 improves from 49.4% to 58.4%, and the sampler-versus-trainer log-probability gap stays small for the whole run (details in Zf4Z W1).
  • Scope of "any harness" (WgMP, b5fU). Polar's gateway is designed to seamlessly construct harness rollouts through API boundaries. Today, Polar still covers the fullest open standards: OpenAI ChatCompletion, Response, Anthropic Messages, and Google generateContent API from both static and streaming traffic in double direction. And we actively maintain the proxy to support upstream changes. By "any harness" we wanted to strengthen the importance of API proxy as agentic rollout boundaries. To avoid misinterpretation, we add a "Scope and limitations" paragraph to explicitly brings up the API types in open standard and that harness must supports provider URL redirecting.

We enjoyed working through the feedback and are happy to continue the discussion.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.