The Two-Layer AI Stack: Cost Discipline Meets Strategic Leverage

We need to stop thinking about AI as one big blob. As I’ve been saying for ages we need to think critically about what kind of AI we are deploying, the purpose for which we are deploying it, and how much it is going to cost us. If an organisation is planning to deploy AI at scale then it will be worth thinking about this, if only to mitigate the horrific bill 💸 shock that is headed your way.

A useful mental model is emerging for organisations trying to balance AI capability with cost control: treat AI as a two-layer system. The base layer is your “engine” - the cheapest capable model that reliably executes tasks. The upper layer is your “steering” - frontier models that guide, critique, and refine outputs where it matters.
This framing, articulated nicely by Nate B. Jones, captures a shift from chasing raw model power toward architecting systems that optimise for both cost and performance. The key insight from Jones is simple: execution is becoming commoditised, while judgement remains scarce.

The two-layer AI stack

A useful way to think about modern AI systems is as a two-layer stack.

• The bottom layer is your engine: the models that actually execute work at scale - summarising, classifying, drafting, transforming, and wiring into workflows.
• The top layer is your steering: higher-capability frontier models that shape, critique, and govern what that engine does.

Most organisations today overpay by using frontier models as both engine and steering. They send every task, trivial or complex, through the most expensive path, then wonder why AI costs look like a second cloud bill.

The emerging playbook reverses that. You push as much execution as possible into cheap, capable models and reserve frontier systems for the points where judgment, nuance, or risk really matter. The twist is that the engine layer should be open-weight wherever it can be - not just cheap, but under your control.

Why open weight matters in the engine

An “open-weight” model is one where you can download the weights and run inference on your own infrastructure. You might still pay for hosting or managed services, but you are not forced to send every request through a single vendor’s API on their terms.

For the engine layer, this has three big implications:

• Cost predictability: When you control deployment, you can decide whether to run on your own GPUs, a low-cost cloud, or a specialised inference provider. Your bill is driven by infrastructure economics, not by per-token pricing that can change overnight.
• Portability: You can move the engine wherever it needs to run - into different regions for data residency, closer to critical systems, or inside regulated environments - without rewriting your stack for a new proprietary API.
• Resilience: No single provider can “switch off” your core execution capability. Contracts can end, features can change, or geopolitical events can disrupt access - but downloaded weights keep running.

In other words, open weight turns the engine from a rented capability into an owned asset. That changes the balance of power.

Cost leverage: nobody owns your execution

When you build your engine layer on open-weight models, you gain leverage in several ways:

• You can benchmark multiple models for a given task and choose the best cost-quality trade-off, instead of being locked into one provider’s pricing curve.
• You can upgrade or swap out models as new releases arrive, without changing your application logic; the interface between your engine and your workflows stays largely the same.
• You can negotiate with proprietary providers from a position of strength: their frontier models become optional steering components, not existential dependencies.

This is the economic heart of the two-layer idea: the more you de-risk and commoditise execution, the more freedom you have to spend selectively on frontier steering.

Frontier models as steering, not engines

Frontier systems still matter. They tend to be better at:

• Complex, multi-hop reasoning
• Handling ambiguous instructions and edge cases
• Giving higher-quality critiques, reviews, or design-level suggestions
• Navigating natural language interactions with non-technical users

But their role changes. Instead of doing every task end-to-end, frontier models sit above the engine as:

• Reviewers: checking outputs from the cheaper engine for correctness, coherence, policy alignment, or brand tone.
• Architects: helping design workflows, prompts, and evaluation criteria that govern how the engine operates.
• Escalation paths: handling the small minority of tasks that the engine cannot safely or reliably complete.

Because this steering layer touches far fewer tokens and tasks, you can afford to use more expensive models here without blowing up the budget. The value comes from leverage: one good steering decision can improve thousands of cheap engine executions.

Open weight as a defence against lock-in

Most AI lock-in is subtle. It does not just come from using a proprietary model; it accumulates through:

• Prompts tailored to a single provider’s behaviour
• Fine-tunes that cannot be exported
• Evaluation suites that assume one model’s quirks
• Integrations intertwined with a single API

Open-weight engines give you a way to push back. When your everyday execution runs on models you can host and swap, you are less dependent on any one vendor for day-to-day operations. Frontier systems become plug-ins you integrate and can remove, rather than the spine of your stack.

This matters for:

• Enterprises that need to manage long-term costs and avoid being unable to switch providers when commercial terms change.
• Public-sector and critical infrastructure environments that need continuity even if external APIs are disrupted.
• Smaller teams and creators who cannot absorb large, unpredictable bills but still want strong AI capabilities.

Open weight does not magically solve governance or safety. But it does give you a more controllable substrate on which to build those systems.

Governance, evaluation, and the real competitive edge

Once your engine is cheap, capable, and under your control, the real differentiators move elsewhere:

• Evaluation: the ability to measure whether outputs meet your standards - accuracy, bias, security, compliance, tone - and to do so systematically.
• Orchestration: the way you break tasks into steps and route them between models, tools, and humans.
• Governance and policy: the guardrails you design around AI use, from access controls and logging to approval flows and human-in-the-loop interventions.

Two organisations might use the same open-weight models and similar frontier systems. The one that invests in evaluation and governance will usually get better outcomes at lower cost, because it can safely push more work into the cheap engine and reserve steering for where it adds the most value.

The message for leaders is clear: AI advantage is becoming less about having special access to magical models and more about designing a stack where execution is cheap, resilient, and vendor-independent - and then building strong steering on top.