The future of AI will be smaller and not bigger. My take on where AI’s headed? Forget bigger, flashier, more bloated models. The future’s going to be smaller, smarter, and way more focused. This is especially with the advent of agentic AI, since this will be driving business processes, and business finances are finite and nobody will want to pay the real cost of using all of these LLMs.
I have been teaching a short course at the Australian Graduate School of Management (AGSM) for the past year called AI for Organisational Innovation and have been exposed to people in leadership from many organisations who are grappling with the challenges of implementing AI. One thing I have been telling them all about the future is that I think the real future is smaller and more focused models rather than this arms race for ever larger and fancier LLMs and LRMs*.
We have all seen the huge investments in data centres and the unsustainable use of water and power to drive all these large models (Inside the relentless race for AI capacity, Financial Times, 31 July 2025). At some stage everyone will realise that using our scarce resources to fuel data centres is not a sensible thing to do (at least outside the USA).
Small Language Models
So, while I have been saying that I think the future is smaller models, it is nice to see some research coming out to support my thinking. A recent paper titled Small Language Models are the Future of Agentic AI, where the lay out their arguments. Now I believe these folks for a few reasons – (1) I agree with their messaging, and (2) they are all with NVIDIA Research.
“Here we lay out the position that small language models (SLMs) are sufficiently powerful, inherently more suitable, and necessarily more economical for many invocations in agentic systems, and are therefore the future of agentic AI. Our argumentation is grounded in the current level of capabilities exhibited by SLMs, the common architectures of agentic systems, and the economy of LM deployment. We further argue that in situations where general-purpose conversational abilities are essential, heterogeneous agentic systems (i.e., agents invoking multiple different models) are the natural choice. We discuss the potential barriers for the adoption of SLMs in agentic systems and outline a general LLM-to-SLM agent conversion algorithm.”
The real cost of AI
Right now, no one’s really footing the true bill for AI – whether it’s GenAI, LLMs, or LRMs. But that party won’t last forever. Sooner or later, the people bankrolling this tech are going to want us to cough up the real costs. And when that invoice lands? Cue the sticker shock. And that’s when folks will start shopping around for cheaper, smarter options, and that’s the moment SLMs and agentic AI will really start to shine.
* Large Language Model (LLM), Large Reasoning Model (LRM)