acceptodds
Under review as a conference paper at ICLR 2027

UnCurl: Reliability-Conditioned Partial Sharing for Joint Curriculum and Exploration Control

Abstract

Efficiently training large language model (LLM) agents requires deciding both which tasks to practice and how much to explore within each task. Existing controllers usually estimate these decisions independently, although both can depend on shared capabilities; forcing one shared uncertainty representation, however, can propagate proxy error and cause negative transfer. Our contributions are threefold. We formulate curriculum–exploration coordination as conditional partial sharing between task-level learning progress (LP) and state-action information gain (IG). We introduce UnCurl, which retains private LP and IG factors and admits a shared predictive factor only when calibrated dependence, reliability, held-out utility, effective sample size, and drift tests pass. A monotone soft gate controls sharing strength, while an independently trained shadow posterior provides fail-closed fallback. On ALFWorld, UnCurl reaches 91% unseen success versus 86% for interaction-matched Independent LP+IG, raises normalized AULC from 73 to 81, and reduces interactions to the 80% development threshold from 43k to 30k. It lowers harmed-task fraction from 0.48 under Full-share LP+IG to 0.16 and reaches 44% success on WebArena-Lite. UnCurl uses 37.7% more GPU-hours than Independent LP+IG.

Primary area:
reinforcement learning
Keywords:
LLM Agents, Curriculum Learning, Information Gain

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_GpVZOpenReview ↗
Rating
2
Confidence
2
Informativeness
3
Presentation
2(+1)
Soundness
2
Read review
Paper summary

This paper provides a controlled partial-sharing structure UnCurl for organizing training-time task-level curriculum control(Learning Progress) and state-action level exploration(Information Gain) for better efficiency and performance, while mitigating potential negative transfer. A synthesis environment(CLCS) is applied for controlled calibration for gate components in terms of dependence, reliability and sharing-strength, and shadow posterior is provided as fallback when hard-gate conditions are not satisfied. The resulting adjusted structure is further evaluated in ALFWorld and WebArena-Lite with inprovements over the vanilla LP+IG baseline, at the cost of additional GPU hours.

Critical strengths
  1. The overall logical flow of the paper is fluent and the motivation is clearly presented. The method discussion incrementally progresses, making the question and solution raised reasonable.
  2. The figure explaining the sharing mechanism (fig.1) and the training/routing (fig.2) are well designed, making it easy to understand the main route.
  3. The author clearly distinguishes two objectives (LP and IG), and specifically designs statistical and reliability tests for both objectives to guide sharing. An independent shadow posterior is provided as fallback when the sharing conditions are not satisfied.
  4. Experiments are carefully designed with multiple controlled baselines and ablations, providing signals (negative transfer/learning efficiency/sharing dynamics under shift) in addition of accuracy, providing source for analyzing the behavior and effectiveness of the provided method.
Critical weaknesses
  1. Undefined symbol and terms greatly harms readability. There are numerous mathematical symbols and terms introduced without clear definitions(across section 2), and displayed equations are not numbered. The methods are stacked in table 1 without proper grouping, and the columns abbreviations require referring back to main text. The font size in fig.3 is so small that is almost not readable.
  2. The reliability of LP estimation in real training remains insufficiently validated. Though CLCS proxy calibration results are provided, how accurately the estimated LP reflects actual post-update learning progress in ALFWorld remains unclear. Subsequent dependence detection and sharing decisions might also me affected as well.
  3. The core sharing mechanism can still be further isolated in the ablation study. E.g. the contribution of the independent shadow posterior. Also, the comparison between fixed and adaptive partial sharing changes both the gate strategy and fallback mechanism, making it difficult to attribute the observed improvements to individual components.
  4. Some statistical design lacks sufficient justifications. For example, the weighted mean aggregation of state-action IG may discard information relevant to dependence detection, while the normalization and weighting of LP/IG/outcome losses during posterior training and LP/IG losses during gate calibration are not sufficiently explained.
Questions for authors
  1. How is the LP estimation validated during actual training? Could the authors clarify how one-step gain probes are constructed and report the prediction accuracy of LP proxies on ALFWorld, rather than only on CLCS?
  2. How sensitive are the sharing decisions to the statistical design choices? In particular, could the authors further explain the weighted-mean choice of IG aggregation and clarify how LP/IG losses are normalized and weighted? Would alternative aggregation strategies or loss weights significantly affect the gate decisions?
  3. Could the authors provide more targeted ablations of the adaptive sharing mechanism? For example, comparing adaptive gating with/without the independent shadow posterior, as well as fixed/adaptive gating under the same fallback setting would help isolate the contribution of fallback and clarify whether the improvements come from adaptive sharing itself.
Reviewer_34YtOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
2
Soundness
3
Read review
Paper summary

This paper is about joint control of two decisions in agent training: which tasks to practise, and how much to explore within a task. Instead of estimating the task-level learning-progress signal and the state-action information-gain signal independently, or forcing them into one representation, the paper keeps private factors for each and admits a shared predictive factor only when a battery of diagnostics passes: dependence tests with calibrated power, reliability estimates, held-out predictive utility, effective sample size, and a drift test. Sharing strength is set by a monotone gate, and when the checks fail the system falls back to a separately trained shadow posterior. Evaluation is on a calibrated synthetic suite, ALFWorld with interaction-matched controls, and WebArena-Lite.

Critical strengths
  1. The experimental discipline is good: five seeds, paired comparisons, confidence intervals from a hierarchical bootstrap, and pre-registered endpoints.

  2. The framing is claiming that sharing always helps, the paper sets out why unconditional sharing is unsafe and makes sharing a decision that can fail.

Critical weaknesses
  1. The comparison that matters most is the one against fixed partial sharing, and there the margin is small: most of the improvement over the independent baseline seems to come from having private factors at all, not from the diagnostics that decide when to share. Since the adaptive version also costs more, I am not sure what the routing layer buys on the agent benchmark, as opposed to on the synthetic suite where it was calibrated.

  2. There are many thresholds in the gating rule, and from the tables I cannot tell which of them actually bind during ALFWorld training. If the gate mostly opens, or mostly stays shut, the system behaves like one of the simpler baselines, and the reported gain would need a different explanation.

Questions for authors
  1. How often does the gate open during ALFWorld training, and how do the results change if it is fixed open or fixed shut?

  2. Relative to fixed partial sharing at the same cost, what does the diagnostic layer add?

  3. Were the individual thresholds in the eligibility rule ablated on the agent benchmark, or only on the calibrated suite?

Reviewer_YW4gOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

UnCurl coordinates curriculum selection and action exploration through reliability-conditioned partial sharing, with private representations and an independent fallback. It improves interaction efficiency and success rates on ALFWorld, with additional computational cost.

Critical strengths
  1. Clearly distinguishes learning progress from information gain and explicitly addresses negative transfer.
  2. Includes useful independent/full-sharing controls and transparently reports computational overhead.
Critical weaknesses
  1. Reliability estimates and gate calibration are transferred from synthetic environments; their validity on real agent trajectories remains insufficiently established.
  2. The complex gating mechanism lacks simpler adaptive baselines and compute-matched comparisons, leaving its necessity and cost-effectiveness unclear.
Questions for authors
  1. How is the validity of CLCS-derived reliability estimates verified on ALFWorld and WebArena?
  2. Why does Figure 3(b) show substantial negative transfer in cells marked as independent fallback?
  3. Can you reconcile Table 1’s unseen-SR improvement of 5.4 percentage points with the reported paired improvement of 5.0?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.