Yandex has released the base pre-trained model Alice AI-T5-35B-A0.6B as open source, powering fast responses in Yandex Search. Speaking at the Practical ML Conf, Alice AI Search CTO Artur Petrosyan noted that the team focused strictly on applied problems with stringent millisecond latency requirements.
Betting on a Forgotten Architecture
At the core of the new system is a combination of Encoder-Decoder and Mixture of Experts (MoE) architectures. Such hybrid setups have barely appeared on the market since 2021–2023, making this a high-stakes engineering gamble. The pragmatic logic is simple: the encoder reads heavy document context in a single pass, while a compact decoder rapidly outputs short text. Meanwhile, the Mixture of Experts provides the general quality and knowledge volume of a 35-billion-parameter model while keeping actual compute costs at the level of a 600-million active parameter model.
The development was not without technical hurdles: during pre-training, routing in the encoder collapsed, forcing engineers to completely redesign the expert balancing scheme.
The Mixture of Experts gives Alice AI-T5 the knowledge level of a 35-billion-parameter model with the compute load of a 600-million-parameter model, solving long-document processing with minimal latency.
To reduce the load on the decoder and prevent redundant text arrays from passing through the language model, the team built a lightweight context-compression model. They trained a small 80-million-parameter BERT-like network from scratch to determine the relevance of each token to the user query. Cross-Entropy RL turns it into an extractor that leaves behind only missing facts.
Product Metrics
During SFT fine-tuning and Reinforcement Learning from Human Feedback (RLHF), developers entirely abandoned editor-written golden answers. The system automatically selected the best options from a pool of generations from various models using internal quality signals, while the RL phase further trained the model on actual user behavior within the service.
This pipeline delivered measurable business results: implementing the system added +0.34% to the Sessions Per User product metric. Furthermore, in blind pairwise comparisons, Alice AI-T5 outperforms Google AI Overview in 56.7% of cases.
Replicating this inference economics on third-party open weights without an equivalent context-compression pipeline and user signal loop will prove extremely difficult for other market players.