Structured agent skills sound neat on paper, but they fail to survive contact with reality. According to a joint preprint from researchers at Princeton, Stanford, UC San Diego, USC, and Johns Hopkins, the modular promise of inference-time skills collapses as scope expands: actual retrieval precision plunges from 29.6% down to a dismal 3.3% when candidate skill pools grow from 5 to 100.
Analyzing 8,135 normalized trial records across multiple benchmarks, the team dissected how structured packages actually alter execution. The data reveals that procedural anchoring drives 65.7% of successful cases, while explicit knowledge injection accounts for just 4.5%. In other words, modular skills act mostly as procedural guardrails rather than repositories of missing domain knowledge, yielding a modest 6.06-point boost over raw Workflow Memory in matched runs.
For engineering leadership, the message is clear: aggregate benchmark gains mask fatal runtime brittleness. Zhiyuan Jiang, Fangrui Huang, and their co-authors show that skills buckle under ambiguous contexts and cross-instruction interference, creating cascading execution failures where ground-truth retrieval neither guarantees nor correlates cleanly with downstream task success. CTOs and AI architects must abandon blind skill-library scaling and focus on deterministic gating and strict operational boundary enforcement.