Council Post: Are LLMs Becoming Too Expensive? How You Can Fight Back

Posted on July 14, 2026 in business-finance

Komninos Chatzipapas (Κομνηνός Χατζηπαπάς), Founder of Omicron AI Software.

With the release of Fable 5 (which is essentially Mythos 5 with guardrails), it's becoming increasingly apparent that frontier AI labs are willing to disregard pricing in order to squeeze every little bit of power out of their large language models (LLMs). Even though most of the cost is subsidized, there are still reports of agentic developers going through up to 50 times more than what they had spent on their flat subscriptions.

The numbers back this up. Fable 5 costs twice as much as GPT 5.5 for input tokens while delivering very similar results. Anthropic has also announced its intention to have Fable series models operate on usage-based billing outside its subscriptions, which signals where the industry is heading.

It makes sense that they're doing this. AI is a winner-takes-all space, so being a little bit better can make you a lot more money. Labs are racing each other to the top of the leaderboards, and cost efficiency simply isn't the priority right now.

So, why is usage being subsidized at all? Because it lets labs gather data they can later use to distill smaller, cheaper models. There are widespread suspicions that AI labs sometimes distill models without public announcements, leading to degraded performance for users who never agreed to the trade. (To be fair, many of these reports can be attributed to other causes, such as a change in the default system prompt, but the suspicion alone tells you something about the trust gap between labs and their customers.)

The Open-Source Alternative

One route to reduce costs is to use an open-source model instead of a proprietary one. Generally, open-source models are about one year behind the state of the art, which sounds bad until you realize that for most business tasks, last year's frontier is more than good enough.

There's another advantage: A company can fine-tune an open-source model with its own data to make it perform better on its specific workload. For companies that use very long system prompts (or, as they're now sometimes called, developer prompts), fine-tuning has an extra benefit. Once it's done, that long prompt is no longer required. The instructions are baked into the model itself. I appreciate that most frontier AI labs have developed prompt caching, but even so, having no system prompt at all is better than caching one.

There are some disadvantages, though. Fine-tuning requires an upfront investment, as it needs to be done by an expert, and there's the ongoing maintenance cost of running your own infrastructure. You're trading a monthly API bill for GPU management, which isn't a trade worth it at low scale.

But the savings can be dramatic. Done well, I've found that this approach can reduce LLM costs by over 90%. For a small startup making a handful of API calls a day, it's not worth the hassle. At sufficient scale, however, it deserves serious consideration, because the labs have made one thing clear: They're not going to lower prices for you. You'll have to fight back yourself.

Forbes Technology Council is an invitation-only community for world-class CIOs, CTOs and technology executives. Do I qualify?