For decades, NVIDIA’s ecosystem has held the market by the throat through its monopoly on low-level expertise. Writing GPU kernels—the fundamental building blocks of modern neural networks—has become a sequestered art accessible only to a narrow circle of engineers. Shiyi Cao and the Berkeley Sky Lab team rightly point out that the industry has fallen into a "gilded cage" of CUDA-specific optimizations. While the hardware market diversifies, developers remain hostages to legacy code. Porting Attention or SSM models to new architectures previously meant reinventing the wheel, creating an insurmountable barrier for any NVIDIA competitor.
Automated destruction of a monopoly
To crack this cage open, researchers have refined K-Search—an evolutionary kernel search framework—to support Apple MLX. The system, developed by Cao at UC Berkeley, uses AI to optimize GPU kernels through an iterative cycle of trial and error. It mirrors the workflow of a human engineer but at speeds no human can match. Instead of starting from scratch, K-Search treats decades of existing CUDA developments as a knowledge base, adapting them for Apple Silicon architecture.
The CUDA ecosystem has accumulated decades of hard-won experience: manual optimizations that took thousands of man-hours are now merely input data for an algorithm.
The key breakthrough lies in the translation layer. K-Search doesn't just copy code; it re-engineers the logic to suit the unified memory architecture of M-series chips. By adhering to hardware rules and mathematical constraints, the framework ensures the output is not just functional code, but a physically efficient solution tailored to Apple’s hardware specifics. This transforms NVIDIA’s accumulated legacy from a competitive advantage into a public resource.
The economics of local inference
Despite millions of devices sold, Apple Silicon has long underdelivered on performance. Critical kernels, such as Paged Attention or MoE routing, were either missing or poorly implemented. K-Search closes this gap with startling efficiency. Benchmarks show the system achieves 0.97x the speed of native MLX Attention kernels—effectively reaching the level of expert manual optimization automatically. Even more striking are the results for Mamba SSM kernels: a 20x speedup in the prefill stage compared to the popular mlx-lm library.
Our approach achieves near-expert performance on Apple Silicon, demonstrating 0.97x the speed of the native MLX Attention kernel.
For businesses, this translates to a direct path for reducing CapEx. This level of optimization allows R&D tasks and even production workloads to be moved from expensive cloud-based H100s to local Mac hardware. The K-Search methodology is not vendor-locked; the same logic can be applied to expand onto Intel, AMD chips, or custom ASICs. The bottleneck is no longer a scarce human resource, but merely the efficiency of an evolutionary search. The AI industry’s dependence on a single chip supplier due to software legacy is finally beginning to loosen.