AERCA: Adaptive Experience Regulation and Credit Assignment for Long-Horizon Agents

Reliable end-to-end completion remains challenging for LLM agents in long-horizon tasks. We identify two recurring obstacles. First, retrieved experience can remain influential after becoming poorly matched to the agent's evolving state or information need, causing previously useful guidance to bias subsequent decisions. Second, long-horizon RL exhibits a credit assignment dilemma: some credit schemes improve strict completion while leaving many unresolved trajectories with limited progress, whereas milestone-aware credit can substantially improve intermediate progress without yielding correspondingly stronger completion. To address these challenges, we introduce AERCA (Adaptive Experience Regulation and Memory-Aware Credit Assignment), which couples adaptive experience use during execution with selective credit assignment during policy learning. AERCA structures past interactions into Task-level Completion Experience for how to complete and Gap-level Cognition Experience for how to know, while dynamically regulating whether retrieved experience should be used, retained, or discarded as execution evolves. For policy learning, relative trajectory progress determines how much memory-aware credit a trajectory receives, while breakthroughs during memory use events determine where that credit is assigned. On ScienceWorld, AERCA achieves $45.08\%$ full success with a task score of $30.70$, while reaching $78.13\%$ success and a task score of $87.83$ on WebShop. More importantly, even when all compared methods use our Adaptive Experience Regulation, AERCA improves full success over GiGPO by $4.67$ percentage points on ScienceWorld, isolating the contribution of Memory-Aware Credit Assignment. Ablation studies further support the complementary contributions of the two components. Our code and datasets are available at https://anonymous.4open.science/r/AERCA-6FF2/.