Beyond Context: Parameter-Efficient Model-Level Evolution of Agents

Agents accumulate experience, and most frameworks keep it in context—editable skills, memories, and workflows that stay open to inspection and revision. That openness is worth paying for, but the payment recurs: text-carried behavior occupies the context of every call that uses it, long after it has stabilized. We organize persistent agent experience into versioned, callable domain capabilities, each encoded in a separately stored parameter-efficient update to a frozen shared backbone—a LoRA patch in our implementation. We train these patches from verified records of an agent's own work and teacher-assisted supervision; only those that pass evaluation are enabled for invocation by later agent versions. At run time the parent agent selects an enabled capability inside its own task-solving process, with no separately trained dispatch module; a deterministic harness assembles the specialist's request from the task, accepted artifacts, and runtime observations under an explicit output contract. Specialists cooperate through what they produce, sharing the backbone while keeping their patches separate; editable skills remain available for coordination and rapid revision. We evaluate a registry of eight domain patches on a Qwen3-8B backbone. Each patch specializes in its domain and improves on the base model on every evaluated benchmark that contributed no training rows. Behind the agentic interface, Ours uses patch-backed specialists with typed handoffs and outperforms the base model in every reported domain. On five constructed workflows that none of the patches was trained for, the registry grounds 274 of 500 cases against 173 on base weights, a 58% relative gain on tasks requiring two to four specialist stages to cooperate. Holding all eight patches resident on one accelerator uses 82.25% less process GPU memory than eight full-model replicas, with low runtime switching latency (32.84 ms median).