inclusionAI has open-sourced its Ling-3.0 foundation model family, and rather than just dumping opaque final weights, the team released key intermediate checkpoints across the entire training pipeline. Covering two tiers—Ling-3.0-flash and Ling-3.0-tiny—the release exposes pre-trained (30T tokens), mid-trained, and Warmup-Stable and Merge (WSM) checkpoints on Hugging Face, ModelScope, and OpenRouter, giving engineers a transparent canvas for custom fine-tuning and continual pretraining.
The flagship Ling-3.0-flash-base leverages a sparse 1/64 Mixture-of-Experts architecture: 124B total parameters, but only 5.1B non-embedding parameters activated per token. By combining 512 routed experts (activating 8 per token alongside one shared expert) with a native hybrid linear attention mechanism—pairing KDA and Gated MLA layers in a 5:1 ratio—the model delivers long-context inference on budget server hardware without the usual VRAM penalties. Dropping standard learning-rate decay in favor of WSM checkpoint merging further lets teams test domain-specific offline decay profiles without burning compute on repeat runs.
From a systems architecture perspective, releasing granular 30T and mid-stage weights shifts the unit economics for enterprise AI. High-performance small and sparse models increasingly make expensive proprietary frontier APIs redundant for specialized agentic workflows. Because both Flash and Tiny share identical training recipes, R&D teams can validate domain adaptation and distillation pipelines cheaply on Ling-3.0-tiny before deploying directly to Flash, cementing the move toward cost-effective on-premise execution.