Vision-Language-Action (VLA) models promise end-to-end reasoning and trajectory generation for autonomous driving, but their severe computational latency has kept them stranded in simulation. A research team from UC San Diego and Princeton—led by Zekai Li, Yihao Liang, Hongfei Zhang, Jian Chen, Yesheng Liang, and Zhijian Liu—aims to change that with FlashDrive. As detailed in their preprint, real-time VLA execution collapses under four compounding bottlenecks: redundant visual encoding across sequential video frames, repeated language model prefilling at each timestep, sluggish autoregressive token generation, and indiscriminate flow-matching denoising applied across non-uniform velocity fields.
FlashDrive tackles this pipeline drag through an algorithm-system co-design. At the infrastructure layer, the framework integrates kernel fusion and CUDA Graph compilation. Algorithmically, it eliminates waste across all four failure points: introducing streaming KV-cache reuse across temporal windows, deploying a non-autoregressive diffusion drafter for speculative decoding to exploit the predictably low token entropy of driving rationale, and applying adaptive step caching to focus trajectory denoising strictly where velocity gradients actually shift.
Tested on the 10-billion-parameter Alpamayo 1.5 model with W4A8 quantization on a single NVIDIA RTX PRO 6000 GPU, FlashDrive slashed end-to-end latency from 717 ms to 151 ms. This 4.7x speedup boosts operational control frequency from an unviable 1.4 Hz to a practical 6.6 Hz without compromising safety metrics: minADE6@6.4s drifted by a negligible ~0.08 m, while closed-loop collision and off-road rates improved in simulation. By dismantling redundant inference overhead rather than brute-forcing hardware capacity, the work shows how heavy multimodal reasoning models can realistically fit inside real-time onboard vehicle budgets.