HELICES: Adaptive Expansion of Capability Frontiers for Agent Self-Evolution

Self-evolution of large language model (LLM)-based agents aims to continually improve their capabilities by discovering and internalizing effective behaviors from past interactions. Yet existing approaches often separate these processes: experience extraction does not consistently adapt to what the agent still needs to learn. We argue that experience extraction and internalization should be coupled around a dynamic capability frontier. Accordingly, we introduce HELICES, a method for adaptive capability-frontier expansion that couples experience extraction and internalization as the policy evolves. HELICES probes this frontier with self-generated natural-language hints, using gains over unguided execution to identify effective behaviors not yet mastered. HELICES jointly optimizes hint generation and utilization through shared parameters, and then internalizes independently verified gains by fully supervising actions and selectively weighting reasoning tokens based on hint attribution. Together, adaptive probing and verified internalization continually expand the capability frontier for independent task solving. Across three agent benchmarks, HELICES surpasses strong baselines by up to 7.2 percentage points and generalizes to unseen tasks without hints.