The open-source inference engine llama.cpp has officially merged pull request #27342 by contributor Jian Chen, baking in native speculative decoding via DFlash2. The implementation bundles grouped dynamic depthwise convolutions with an edge-ranking candidate selector, auto-activating whenever a compatible GGUF checkpoint is loaded—sparing engineers the usual headache of juggling fragile manual runtime flags.
Benchmark results from llama-benchy 0.4.0 on Qwen3.8-27B (UD-Q4_K_XL) indicate that DFlash2 effectively doubles decoding throughput across long context windows without accuracy degradation. At baseline depth zero, DFlash2 delivered 26.39 tokens per second against 11.81 for unassisted decoding. More crucially for long-context retrieval, at 32,000 tokens of depth, a draft width of 4 sustained 21.11 tokens per second, whereas vanilla decoding stagnated at 10.54 and legacy DFlash v1 collapsed to 10.75. Validation runs on Apple M5 Pro silicon running Qwen3.8-27B at Q4_K_M precision across GSM8K workloads confirmed stability using inco's Apache-2.0 drafter weights.
For teams running self-hosted LLM infrastructure, this shifts the unit economics of 27B-parameter models immediately. You effectively double long-context throughput on existing workstation and edge silicon without redesigning orchestration pipelines. However, wider draft widths suffer diminishing returns under massive context pressure, meaning ML platform teams must actively profile draft width against target prompt lengths rather than relying on default settings in production.