Modern LLM agents are stuck in "waiting for response" mode. While a system builds a reasoning chain, queries databases, or calls an API, businesses lose time. Researchers from UC Santa Barbara and LinkedIn have confirmed the obvious: traditional architectures are a bottleneck. An agent undergoes a painfully long "thinking" process before it ever initiates a network request. For the real-world sector, this translates to sluggish interfaces and inefficient automation pipelines where inference costs are multiplied by hardware downtime.
The failure of external drafts
Attempts to fix this via speculative execution—trying to guess a tool call before the agent finishes formulating it—have long hit a wall. According to Jiabao Ji and Yujian Liu, using external "draft" models creates a critical speculator–agent gap. A lightweight assistant model might generate a logical request, but if it doesn't align with the primary agent's internal logic, the result is a mismatch. This triggers a prediction reset, wasting resources without gaining a millisecond of speed.
The gap between the speculator and the agent arises because off-the-shelf lightweight models mimic an abstract assistant rather than the specific agent deployed in your system.
This provides a vital lesson: external predictors are too imprecise for complex business logic. Instead of building a complex stable of disparate models, researchers propose making the agent its own prophet. This "self-speculative" agent is a single model operating in two modes. In "agent" mode, it solves the task; in "speculator" mode, it predicts the next call based on partial trajectories. This architecture allows for the reuse of the prefix KV cache, radically slashing computational overhead.
Joint RL and the bottom line
To teach a model to excel at both roles, the team implemented Joint Agent-Speculator Reinforcement Learning (Joint RL). The system trains on its own data, simultaneously honing task-solving quality and prediction accuracy. It functions like a grandmaster who doesn't just make a move but prepares the board for the next combination in advance, saving time on context switching.
Our method significantly improves the Hit@1 metric for the next tool call.
Data confirms that joint training works. In tests involving agentic search (QA) and conversational tool use, first-guess accuracy (Hit@1) surged. Crucially, the authors emphasize that this speed doesn't come at the expense of quality—the success rate for task completion remains stable. You get a responsive system that hasn't grown duller just because it started moving faster.
This research sends a clear signal to AI architects: stop trying to prop up heavy models with small surrogates. The future lies in optimizing internal self-prediction capabilities. While the method requires high-quality trajectory data and still penalizes prediction errors with computation resets, it represents the first real step toward autonomous systems that are no longer frustratingly slow.