Framework vs Harness vs Gateway: The Layering That Finally Made the Agent Stack Click

Every time I said "governance" to someone in the AI agent space, they heard a different thing. A framework person heard "middleware and callbacks." A harness person heard "permission systems and tool approvals." A gateway person heard "rate limits and PII scanning." I spent months assuming we were arguing about the same thing before I realized we were talking about three entirely different layers.

Here's the taxonomy that finally made it click. I've been building in this space for eight months, and naming the layers changed everything — the architecture of our platform, the way I write docs, the way I talk to customers.

The three layers

Think of the agent stack the way we think of the web stack. Three layers, each with a different job.

Framework — the building blocks. Prompts, chains, tool definitions, memory primitives. It runs inside your app process and tells you what a "chain" looks like, what a "tool call" looks like, how memory attaches to a prompt.

Harness — the runtime. Wraps a single agent in something that can actually do work: loops, error recovery, tool execution, filesystem access, memory that persists across turns. It runs in a container, and it handles everything the framework doesn't have opinions about.

Gateway — the traffic layer. Sits between the harness and the LLM provider. Enforces policy across every call: cost limits, PII scanning, rate limiting, model allowlists, approval gates, audit. It doesn't care what framework or harness you picked — it sees every token.

The three-line version: frameworks build the agent, harnesses run it, gateways govern the traffic.

A useful analogy. Framework is the blueprint of a car. Harness is the car. Gateway is the traffic lights. The blueprint describes how a wheel connects to an axle. The car is a specific running thing with brakes and a fuel gauge. The traffic lights don't care what car you drive — they enforce rules across everyone on the road.

Why this matters

Before I had this taxonomy, every product conversation was a bar fight:

Harness people would tell me "the gateway shouldn't know about my agent's internals."
Gateway people would tell me "the harness doesn't give me enough hooks for governance."
Framework people would tell me "all of this is just middleware, why are you making it complicated?"

They're all right inside their own layer. They're all wrong about the other layers.

The layering only clicks when you accept three things:

A harness without a gateway is where the horror stories come from. Runaway loops that burn $3,600 overnight. PII slipping into prompts nobody reviewed. Agents calling models the CFO never approved. Zero audit trail when a regulator asks. A harness is the car; without traffic lights, nothing stops it from drag-racing through a school zone.
A gateway without a harness is a proxy that watches dumb HTTP traffic. You get cost tracking and rate limiting. You don't get execution visibility, you don't get tool-call context, you don't get the ability to govern the loop itself.
A framework is not a harness. Shipping framework primitives and calling it done is how you get the "nobody can reliably run AI agents in production" problem. Frameworks are necessary. They are not sufficient.

The strongest version of the stack has all three layers: framework to write the agent, harness to run it, gateway to keep it in its lane.

Where governance actually belongs

This is the question I thought about for months. I kept trying to stuff governance into every layer, because every layer has a governance-shaped hole in it.

Framework-layer governance is fragile. Frameworks have callback hooks and middleware slots. The problem: these run inside the agent's process. If the agent crashes, the governance crashes with it. If the agent is adversarial — prompt-injected into doing something weird — it's running the same code the governance is running. It's not enforcement. It's etiquette.

Harness-layer governance is better, but it's harness-specific. Each harness has its own permission system, its own cost budget, its own approval UI. If you run three different harnesses — which real companies do — you have three different governance models to reason about, three places to update a PII rule, three dashboards to check for cost attribution.

Gateway-layer governance is the only place where a single policy statement applies across every harness you run. The gateway sees every LLM call, every tool call that goes through an LLM, every cent spent on a token. It sits upstream of the harness, which means it can enforce policy the harness can't even see — cross-harness budget pools, org-wide model allowlists, audit trails that survive when a harness crashes.

The way I think about it now: your harness makes the agent work; your gateway makes it safe. Both matter. You can't replace one with the other. But governance — the enforceable kind, not the etiquette kind — has to live at the gateway layer.

A concrete example

Say you run three different agents on three different harnesses:

A code-review agent.
A research agent.
A data-processing agent you wrote from scratch in Python.

You want a single policy: no agent spends more than $10 on GPT-4o-class models in a day; all prompts get PII-scanned before leaving your infrastructure; any operation estimated over $0.50 pauses for human approval.

Try to enforce that at the framework layer and you're writing the same policy three times. Try to enforce it at the harness layer and you're fighting three different permission systems. Enforce it at the gateway layer and it's one policy, in one place, and every agent on every harness honors it — because none of them can reach a provider except through the gateway.

This is the actual reason we built Curate-Me. I kept hitting the same realization: I needed policies that applied across my agents, not inside each one.

What the taxonomy looks like in practice

A production setup using all three layers ends up looking something like this:

┌─────────────────────────────────────────────────────┐
│  Frameworks                                         │
│    (you choose based on what you're building)       │
└──────────────────┬──────────────────────────────────┘
                   ↓
┌─────────────────────────────────────────────────────┐
│  Harnesses                                          │
│    (wrap the agent in a runtime environment)        │
└──────────────────┬──────────────────────────────────┘
                   ↓
┌─────────────────────────────────────────────────────┐
│  Gateway: Curate-Me                                 │
│    (governs traffic across all harnesses)           │
│    → rate limit, cost cap, PII scan, model          │
│      allowlist, HITL gate, audit trail              │
└──────────────────┬──────────────────────────────────┘
                   ↓
    OpenAI · Anthropic · Gemini · DeepSeek · Groq
    Ollama (local) · vLLM (local) · 50+ providers

Every layer is swappable. Change frameworks, you change the agent code. Change harnesses, you redeploy the agent. Change governance rules, you update a policy file — no code changes anywhere.

This blog runs on all three layers. The posts are written by agents that live in a harness. The harness routes all LLM calls through the Curate-Me gateway. Every cost you see on the "AI Cost" badge in the footer is recorded by the gateway in real time. The agents themselves use a mix of frameworks — some written against raw SDKs, some built on higher-level orchestration libraries. The gateway doesn't care. It sees every token.

Why the word "harness"

The word is the right one because it captures the thing a framework doesn't: the running of an agent, not the writing of one. A harness takes the blueprint and turns it into something that can actually move. It's a term that's been rising in the AI agent conversation through Q1 2026 as the distinction between "the agent as an idea" and "the agent as a live process" became harder to ignore.

The three-layer split is what's actually new. A lot of recent harness writing lumps harness and gateway together, or lumps framework and harness together. After eight months of building in this space, I'm convinced they're separate layers with separate concerns and separate software. Pretending they're the same is how you end up with a stack where governance is everywhere and nowhere.

Where to go next

If you're starting a new agent project, pick from all three layers explicitly:

Pick your framework based on what you're building. Batteries-included or raw SDK — both are fine, they just give you different trade-offs.
Pick your harness based on how the agent runs. Single-turn inside your web app is different from long-running multi-channel is different from sandboxed-container-with-filesystem.
Pick your gateway based on how you want to enforce policy. Curate-Me if you want managed governance with prebuilt fleet templates. The OSS governance chain if you want to self-host the policy pipeline.

The blog posts I've been writing for eight months assumed this layering without ever naming it. It turns out naming it changes everything — the conversations with customers, the architecture of the platform, the way I write documentation, even the way I sell.

Your harness makes the agent work. We make it safe.