Discussions about purchasing in-house hardware for neural networks arise in companies with enviable regularity. The logic is simple: API bills are climbing, so isn't it time to buy a card and run everything locally? At this point, enthusiasts' eyes light up, while an experienced CTO reaches for a calculator. The problem is that "expensive" is not a fixed figure but a subjective feeling. Without a precise understanding of token consumption volume, any debate over GPU procurement devolves into an exchange of fantasies. In practice, the cost of the same million tokens on a brand-new RTX 5090 fluctuates between 1 and 20,726 rubles. The card is the same, the model is identical, yet the price difference is twenty-thousand-fold. And it isn't just about the electricity bills.
Idle Math and the VRAM Trap
Let’s dissect the object of desire: the RTX 5090 with 32 GB of memory. This flagship card is tempting because of the "plug and play" appeal. Based on market estimates for mid-2026, even a base model will cost 429,990 rubles, while top-tier custom versions can easily exceed 730,000. But the magic of these numbers is deceptive. Inference is a finicky process; it hits bottlenecks in video memory (VRAM) capacity and speed far sooner than it exhausts the "muscles" of the chip itself. Attempting to run a heavy model on a 5090 is like hauling bricks in a supercar: impressive, but pointless. Furthermore, a gigabyte of memory on this model costs 3.5 times more than on the time-tested 3090.
Inference relies on VRAM volume and speed far more than on raw compute power. Therefore, it is more accurate to compare cards by price per gigabyte of VRAM rather than by the marketing name of the generation.
Calculating the economics requires pragmatism, using two variables: the hourly cost of the hardware and the throughput. An owned card generates a fixed monthly loss regardless of whether it is under load or gathering dust. An API, however, costs exactly what you consume. For the hardware to pay for itself, the data flow must exceed the break-even point. If the card sits idle, you are simply watching money burn along with depreciation.
The Break-Even Point and Hidden TCO
When calculating Total Cost of Ownership (TCO), many make the amateur mistake of looking only at the receipt. The RTX 5090 requires an appropriate "entourage": a 1200W power supply, at least 128 GB of RAM (to prevent model weights from taking an eternity to load), and a fast NVMe drive. This adds approximately 150,000 rubles to the estimate. Then come the operating expenses. With a TDP of 575W, the system pulls about 750W under load. At a rate of 8 rubles per kWh and 24/7 operation, the electricity bill becomes a significant line item. However, the component market situation hits even harder: SK Hynix has already contracted its HBM output through 2026, and standard DRAM prices jumped 60% in a single quarter.
A card costs a fixed amount every month, whether a million tokens or zero tokens pass through it. An API costs exactly what you actually process through the system.
The gap between "owned" and "cloud" tokens only vanishes at extremely high utilization rates. For a business with irregular queries, local hardware turns into a CTO's expensive toy. The only rational argument for overpaying for local inference is a deliberate privacy tax. If data is so sensitive that it cannot be trusted to third-party providers, the purchase becomes an insurance policy. In all other cases, the math is merciless: until your generation volume covers the monthly TCO and the payroll for engineers to maintain this "zoo," cloud solutions remain the only sane choice for a CFO.