Can autonomous vehicle developers dynamically adapt driving policies to new operational constraints without rewriting the base weights of their underlying models?
Recent world action models couple the driving policy and the visual world model in a single generative network, reaching planning scores above 90 PDMS on NAVSIM v1. However, world action models freeze driving objectives after training, requiring costly retraining for every new task. When an autonomous fleet requires operational changes—such as maintaining wider margins to road edges on narrow streets, keeping larger clearances around vulnerable road users, or respecting temporary speed limits and passenger comfort preferences—teams face severe compute costs. As Xiyan Su, Finn Rasmus Schäfer, Hojun Lim, Frank Diermeyer, Johannes Betz, and Markus Lienkamp point out, adding a single new objective conventionally means collecting supervision for it and retraining a billion-parameter model from scratch.
Energy-guided sampling replaces capital-heavy retraining of world models by steering existing trajectories during generation. A research team from the Technical University of Munich and the Munich Institute of Robotics and Machine Intelligence introduced GuidedWAM, a model-agnostic framework designed to inject new driving objectives into frozen world action models directly at sampling time. The central engineering question has been whether external energy guidance can reliably alter complex trajectory streams without destabilizing end-to-end multi-task policies.
The Bottleneck of Frozen World Models
Modern end-to-end driving architectures integrate video generation with trajectory planning. Systems like DriveWAM and UNIVERSE train video and ego action tokens in one shared attention sequence under a single flow-matching objective. Other architectures split computation differently: SimWAM co-trains a video expert with a light action expert and drops the video branch at inference for run-time, while DriveLaW plans from the latent space of its video generator. Architectural divergence has accelerated across the sector: on 2026-03-26, the LeWM world model reduced model size to 15 million parameters with computational savings of 30 to 40 percent compared with traditional world model simulators, whereas on 2026-09-07, Alibaba introduced Qwen-Drive 1.0 to unify 3D spatial perception, conversational interaction, and path planning inside a single multimodal framework.
"Adding one means collecting supervision for it and retraining a billion-parameter model, which has to be repeated for every new objective."
Yet across these systems, the core trajectory plan remains constrained by a flow-matching sampler conditioned solely on fixed training context channels: past frames, ego vehicle status, and navigation commands. If an operating rule was omitted during the initial training run, modifying vehicle behavior demands an entire retraining pipeline.
Steering Trajectories Without Backpropagation
GuidedWAM bypasses model retraining by converting operational goals into mathematical energy functions evaluated during action generation. The framework constructs each driving objective as a differentiable energy evaluated on the sampler's one-step clean estimate of the action stream. The resulting gradient is added directly to the frozen model's velocity field at sampling time, requiring only forward passes through the frozen network without computing Jacobians.
To ensure that external guidance does not derail multi-task action streams during safety-critical maneuvers, the authors implement three stabilizing mechanisms: sampling-time guidance scheduling, per-energy normalization, and velocity-bounded correction. In fact, injecting new rules into autonomous planners does not require recomputing foundational network weights or altering base representations.
"On the NAVSIM v2 hard-split, guidance improves all three frozen, publicly released WAMs by 1.8 to 3.6 EPDMS, averaged over three sampler seeds, and adds less than 10% to the sampling time per step."
On the benchmark NAVSIM v2 hard-split, GuidedWAM improved all three evaluated frozen, publicly released WAMs by 1.8 to 3.6 EPDMS across three sampler seeds. Crucially for real-time onboard deployment, the guidance adds less than 10% to the sampling time per step, resolving the trade-off between control fidelity and inference latency.
Post-training energy guidance establishes an operational control layer above frozen world action models. By avoiding repetitive full-scale model retraining for localized policy adjustments, autonomous mobility programs can decouple fleet behavioral updates from foundation-scale compute budgets. The next operational test lies in scaling multi-energy guidance across production fleet edge hardware.
Side by side
Can an autonomous driving fleet adapt to strict road constraints without continuously burning budgets on retraining? The GuidedWAM framework demonstrates that tweaking driving policies no longer demands retraining billion-parameter world action models from scratch. While alternative setups cut compute by muting video generation during inference, steering trajectory generation directly at sampling bypasses the lock-in of fixed training objectives altogether. The bypass around retraining rests on a solid baseline: generative models already score above 90 PDMS on the NAVSIM v1 benchmark.