A 27-billion-parameter model packed into 16 bits takes up about 54 GB, requiring heavy server GPU clusters, whereas one-bit weight compression packs a comparable volume into 4 GB of VRAM.
Cloud providers and developers of heavy architectures have spent years building their business models around infinite parameter growth and expensive hardware rentals, but the emergence of efficient quantization schemes calls into question the very necessity of cloud infrastructure for inference.
Compression architecture and the dead end of naive approaches
The economics of local deployment hinges on the physical size of the weights, and attempts to pack models down to a minimal footprint have traditionally run into a degradation of generation quality. Simple layer-by-layer weight pruning kills the model, turning it into a random noise generator because crude rounding destroys internal dependencies.
Bonsai from PrismML uses precisely this weight storage scheme in transformer blocks while maintaining full operational capability.
As explained by company head Babak Hassibi, one of the creators of the Optimal Brain Surgeon method, attempts to replicate this result via a Hessian-based layer solver show that optimization for calibration data leads to overfitting, where the error on third-party text increases two-to-threefold due to unconstrained matrix directions.
Financial consequences for the cloud market
The shift to one-bit quantization strikes at a key revenue stream for cloud giants: selling compute hours for heavy inference tasks. When a model with billions of parameters fits onto a consumer graphics card thanks to proper scaling selection and training, companies no longer need to pay for cloud clusters.
As the documentation shows, Bonsai-1.7B with this storage scheme exhibits a perplexity of 16.5, whereas the original Qwen3-1.7B sits at 12.3.
The developers released the format, compute kernels for llama.cpp and MLX, and a simple quantization tool, though they did not disclose the method for deriving the weights themselves, leaving the market room for independent experimentation with distillation. Meanwhile, the cosine similarity of such models to the original baseline versions drops to 0.49 with a mismatching sign share of 29%, which proves a departure from simple copying in favor of finding a fundamentally different point with minimal efficiency loss.
PrismML registered a drop in perplexity with closed weight training methods and open kernels for llama.cpp. In our view, this establishes a new standard for local cost savings that will force cloud providers to revise their pricing policies.