Serving frontier language models under strict latency and VRAM limits has turned 4-bit weights-and-activations (W4A4) quantization into a baseline requirement. Modern architectures increasingly rely on hybrid configurations that replace standard softmax attention with linear operators to bound memory footprints over extended sequence lengths. In models such as Qwen3.8-27B, 48 of the 64 layers are recurrent Gated DeltaNet (GDN) blocks, while only 16 remain standard attention layers. Until recently, standard engineering consensus left recurrent components in higher precision out of fear that quantization noise inside the recurrent loop would compound uncontrollably across thousands of processing steps.
Challenging the Recurrence Fragility Assumption
AI researchers Sergii Kozyrev and Davyd Maiboroda challenged this assumption by developing Minima, an open recipe that applies NVFP4 W4A4 quantization to all 496 linear layers of Qwen3.8-27B, including its GDN projections. The NVFP4 format pairs E2M1 4-bit values with an E4M3 scale per 16-element block to execute natively on modern accelerators, shrinking the model footprint to 17.5 GiB.
"We test that intuition by building Minima: NVFP4 W4A4 on all 496 linear layers, GDN included."
Rather than degrading as context length scales, the quantized architecture maintains benchmark parity with full-precision baselines. Across MMLU-Pro, GSM8K, AIME'25, GPQA-Diamond, LiveCodeBench, and RULER retrieval out to 64K tokens, Minima matches BF16 precision within seed noise, recording a statistically negligible 5-task average difference of just -0.52 points while preserving perplexity stability out to 32K and 64K context windows.
Structural Mechanisms Behind 4-Bit Stability
The resilience of the recurrent layers stems from specific mathematical properties embedded within Gated DeltaNet. First, while inputs share residual stream outliers, NVFP4 16-element block scaling isolates these spikes locally. Second, the gate projections turn out to be the least vulnerable components because their softplus, exponential, and sigmoid parameterizations naturally compress an approximate 11% matrix multiplication error down to an approximate 2% output error.
Furthermore, the delta-rule recurrence mechanism bounds injected quantization noise at a flat plateau over 32,000 tokens. The state updates actively overwrite earlier error traces along the current key direction, allowing the network to discard state impulses within hundreds of steps instead of letting deviations compound over long horizons.
Efficiency Gains and Serving Optimization
This footprint reduction to 17.5 GiB accelerates the prefill phase by 14% to 19% relative to standard comparison baselines. To preserve long-context performance during deployment, Kozyrev and Maiboroda demonstrated that calibrated FP8 KV-cache scales recover 83% of the quantized model's long-context KV cache penalty without additional compute overhead. The complete quantized checkpoint is hosted publicly on Hugging Face under minima-ai/mnma_qwen3.8_27b_nvfp4.
The takeaway is clear: recurrent layers in hybrid models do not require fragile, high-precision handling during low-bit deployment. Because Gated DeltaNet inherently suppresses quantization drift, uniform 4-bit quantization turns hybrid architectures into a formidable, cost-effective alternative to pure transformers for high-throughput enterprise inference.