Modern computer-use agents suffer from costly operational amnesia: every session restarts from a blank slate, forcing multimodal backbones to repeatedly burn tokens rediscovering standard interface routines across environments like WebArena and OSWorld. Instead of treating procedural learning as an expensive, fragile fine-tuning problem, an engineering framework from Longtao Hu (UESTC), Xiao Liang, and Linchao Zhu (Zhejiang University) externalizes execution experience into an evolving, versioned code library.
Architecture of Online Skill Evolution
The framework systematically distills raw click-and-type trajectories and environment feedback into parameterized, executable Python-style procedures. To prevent performance drift and cyclic regressions during deployment, the architecture decouples execution from memory consolidation: the operational agent queries against a frozen library snapshot, while knowledge synthesis runs asynchronously between rollouts without altering underlying model weights.
"Each iteration executes against a frozen library snapshot, and evidence-guided skill updates become available in subsequent iterations without changing model parameters."
Context efficiency is built directly into retrieval. The agent's Executor inspects only a lightweight catalog of procedural summaries, loading detailed function bodies on demand to minimize context bloat while preserving long-horizon execution state.
Empirical Evaluation and Longitudinal Stability
Benchmarked against a configuration-matched zero-library baseline across four OSWorld application domains using a fixed grounding stack, the evolving-library pipeline demonstrated measurable gains. After a five-iteration warm-up phase, the evolving library generated post-warm-up mean evaluator score improvements ranging from 5.7 to 18.6 percentage points across all observed domains.
Enterprise Takeaways and Systemic Limits
For engineering leaders deploying computer-use automation, externalized procedural memory delivers tangible architectural benefits: enterprise-specific UI workflows accumulate into an inspectable, auditable codebase rather than opaque model checkpoints, significantly cutting inference latency and token burn. However, the study flags critical boundary conditions: in complex desktop environments like GIMP, cross-task retrieval noise and revision churn led to brittle recovery when interfaces drifted, confirming that procedural caching remains heavily dependent on interface stability rather than universal visual generalization.