Attempting to offload an entire search traffic stream onto heavy neural networks quickly hits the hard reality of infrastructure economics and latency SLAs. In production, search remains the backbone of the customer journey: an extra half-second delay directly chips away at conversion rates. As explained by Dmitry, lead of semantics at service marketplace Profi.ru, their search engine solves three core operational tasks: identifying the requested service, finding a specific specialist's profile, or filtering out jobs the platform does not support.
The architectural backbone for such volumes is inevitably built around classic lexical search engines. At Profi.ru, Elasticsearch serves as the primary data source, with content managers updating the search index daily through an internal admin dashboard. Reliable filtering and exact match queries remain safely anchored in a proven database.
Strict Latency Limits and Monitoring
Output quality requirements dictate architectural constraints directly. The platform's product team maintains an uncompromising benchmark: over 90% of user selections must fall strictly within the top five search results.
"At peak, it handles around 150,000 queries per hour, which comes down to just over 60 queries per second."
User Behavior and Resource Optimization
Routing every standard, high-frequency query through a large language model is an easy way to burn inference compute budgets for zero added value. LLM-based semantic parsing is only commercially viable for complex, rare, and unstructured long-tail queries.
The pragmatic takeaway for CTOs and product leads is straightforward: in a mature enterprise environment, an LLM functions purely as an add-on layer on top of a lexical search engine, never as its wholesale replacement. When a system handles 150,000 queries an hour at peak, predictable latency and stable SLAs easily outweigh generative novelties.