When 28 out of 67 curated articles turned out to be irrelevant for the blog, Claude Sonnet wasted its limits on every single one of them.
In a lightweight pipeline where a script fetches fresh data once a day and Claude Sonnet writes a breakdown instruction for each piece, a cascaded architecture with a cheap classifier solves the problem of exhausting expensive LLM limits on an unfiltered stream. The issue is that everything gets saved to bookmarks indiscriminately: useful guides, work messages, personal notes, and images without a single word of text. When weekly limits run out mid-workweek and the entire system stalls, a cheap filtering mechanism is required before making calls to the heavy model.
Cheap classifier at the input
To weed out informational noise before hitting the heavy model, the specialized Jev model releasing startup TypeSafe on September 15, 2026, is deployed. The tool is built for closed-domain questions: the system accepts text and a dictionary of questions, returning probability values instead of a text paragraph.
Input costs $0.042 per million tokens, with zero output cost due to the absence of generated text.
The model is available on Cloudflare Workers AI under the identifier typesafe/jev with a Zero data retention tag. Jev's accuracy sits around 68%, creating the operational risk of filtering out useful materials alongside noise. According to TypeSafe benchmarks, Jev's outputs match ground truth from large models in roughly two out of three cases.
Fine-tuning the prompt questions
Data filtration efficiency directly depends on the precision of question formulation and rejection criteria. An initial generic question such as "is this a useful fact?" yielded an AUC of 0.46 on a labeled dataset, performing worse than random guessing. Specifying concrete rejection reasons pushed the AUC up to 0.95.
At a threshold of 0.2, the model retained all useful materials while dropping 10 out of 28 irrelevant ones. On borderline decisions above 0.72–0.77, performance improvements stalled, restricting the filter's utility to a cautious preliminary barrier.
The cascaded approach utilizing cheap classifiers transitions from an abstract experiment to operational necessity when hitting production environment constraints. Hard token cost reduction is achieved through precise prompt engineering applied to the primary model.
How the picture shifted
Back in late July, buying surplus intelligence in bulk already showed up as a massive operational expense rather than a rounding error. Cascaded inference turns token savings from a neat theoretical benchmark into straightforward pipeline plumbing.
Our stories: AI Economics: Cutting LLM Costs by 70x (July 21).