The Context Budget Is Your Agent's Real Architecture — Everything Else Is Plumbing

Every architectural decision in your agent system — subagent delegation, memory, tool design, model choice — boils down to managing a single finite resource: the context window. Here's how to treat the context budget as a first-class constraint, with concrete patterns that ship.

Most architectural debates about AI agents are really debates about one thing nobody names directly: how to spend the context budget.

Should you decompose into subagents? Context budget. Should you use a vector database or stuff documents inline? Context budget. Should this tool return full payloads or summaries? Context budget. Should you use GPT-5 or Haiku? Context budget — because a smarter model can do more with fewer tokens, but costs more per token.

Every other constraint — latency, cost, reliability, observability — cascades from this one. If you blow the context budget, latency spikes, costs compound, and the agent forgets why it was called in the first place.

And yet most teams treat context as an implementation detail. It isn’t. It’s the architect’s first constraint.

The Math They Don’t Put in the README

Here’s the uncomfortable reality of production agent economics. In 2025, the industry learned that agentic workflows consume between 5x and 30x more tokens per task than a standard chatbot query. By mid-2026, Uber’s CTO revealed Claude Code adoption across 5,000 engineers burned through the entire annual AI budget in four months — monthly API costs per engineer ran between $500 and $2,000 (Cockroach Labs, June 2026).

The multiplier isn’t the model. It’s context accumulation.

Every tool call, every retry, every guardrail check, every handoff re-feeds the accumulated window. Token consumption grows roughly quadratically with task depth. A single agent session doing multi-hop reasoning over tool results can burn 100,000+ tokens without anyone noticing until the invoice arrives.

This isn’t a pricing problem. It’s an architecture problem: the context window is a finite budget, and most agent designs spend it like it’s infinite.

The Context Budget as a First-Class Constraint

Here’s a mental model that has served our team well. Treat the context window as exactly what it is: a fixed-capacity working memory shared across everything the agent needs to do in a single session. Budget allocation looks like this:

Consumer	Typical share	What drives it
System prompt / persona	1,000–3,000 tokens	Agent role, rules, tool descriptions
Conversation history	Variable, grows linearly	User turns, assistant responses
Tool call + results	500–10,000+ per call	Payload size, number of calls
Guardrail / validation passes	100–500 per check	Input/output guardrails, hooks
Handoff / delegation overhead	200–1,000 per handoff	Serialized state, summary injection
Reasoning / thinking budget	Variable	Extended thinking, chain-of-thought

A typical mid-complexity agent run: system prompt (~2,000), five turns of conversation (~8,000), eight tool calls averaging 3,000 tokens each (~24,000), two guardrail passes (~800), and one subagent handoff (~500). That’s 35,300 tokens — before any actual reasoning output.

Now run that 40 times in parallel across an enterprise support workflow. You’ve burned 1.4 million tokens on what a naive dashboard calls “one interaction.”

This is why context discipline isn’t optional. It’s the difference between a $20/month agent and a $2,000/month agent.

Pattern 1: Subagent Delegation Is Context Compression, Not Parallelism

The subagent pattern is everywhere now — OpenAI Agents SDK, Claude Agent SDK, LangGraph, CrewAI. Most teams adopt it for parallelism. That’s missing the point.

Dan Farrelly, co-founder of Inngest, has argued persuasively that the primary value of subagents is context compression. A parent agent spawns a child with an isolated context window, the child executes tool calls and reasoning entirely within its own budget, and returns a 750-token summary. The parent never sees the 15 tool calls, eight file reads, and three validation errors the child slogged through. It only sees the result.

As the Epsilla blog documented in their analysis of production subagent architectures, implementing this properly reduced tokens added to the parent agent’s context by over 90%.

Here’s what this looks like in the Claude Agent SDK:

from claude_agent_sdk import Agent, tool

research_agent = Agent(
    name="researcher",
    description="Researches a topic and returns a concise summary with sources",
    tools=[web_search, fetch_page],
    system_prompt="""You are a research specialist. Your ONLY job is to:
1. Search for the requested information
2. Fetch and read relevant sources
3. Return a 500-token summary with citations

Do NOT include your search process, dead ends, or intermediate results.
Your entire output must fit in 500 tokens.""",
)

orchestrator = Agent(
    name="orchestrator",
    tools=[delegate_to(research_agent), format_report],
    subagents=[research_agent],
    system_prompt="""You coordinate research. When you need information,
delegate to the researcher subagent. The researcher returns compressed
summaries — use those. Never fetch data directly if a subagent can do it.""",
)

The critical detail is in the subagent’s system prompt: “return a 500-token summary.” Without that constraint, the subagent will happily return 5,000 tokens of process narrative, and you’ve gained nothing. Context compression must be explicit.

We covered this dynamic in our analysis of agent architecture convergence: every framework now implements some form of subagent delegation, but the frameworks that win in production are the ones that make context isolation enforceable rather than advisory.

Pattern 2: Tool Design Is a Budget Allocation Exercise

Every tool your agent can call is a line item in the context budget. The return payload from a tool gets injected into the conversation — and stays there. If your search_database tool returns full row payloads with all 47 columns, you’re burning budget on data the agent will never use.

The fix is counterintuitive but consistent across teams we’ve worked with: tools should return summaries by default, with full data available on explicit request.

# Bad: returns everything, burns budget every call
@tool
def search_customers(query: str) -> list[dict]:
    """Search customer database. Returns full records."""
    results = db.search(query, limit=50)
    return results  # 50 records × 15 fields = ~5,000 tokens

# Better: returns summaries, lets the agent request details
@tool
def search_customers(
    query: str,
    fields: list[str] | None = None,
    summary_only: bool = True,
) -> dict:
    """Search customer database.
    
    By default returns a compact summary. Use summary_only=False
    for full records, and fields to select specific columns.
    """
    results = db.search(query, limit=50)
    
    if summary_only:
        return {
            "count": len(results),
            "ids": [r["id"] for r in results],
            "names": [r["name"] for r in results],
            "summary": f"{len(results)} customers found. Top match: {results[0]['name']}",
        }
    
    if fields:
        return {"count": len(results), "data": [
            {f: r[f] for f in fields} for r in results
        ]}
    
    return {"count": len(results), "data": results}

The pattern generalizes: every tool is a budget decision. If you’re returning more than the agent needs, you’re paying for tokens that don’t improve outcomes. The Zylos Research cost engineering analysis found that 60–85% of agent API spend is recoverable through disciplined design — and tool payload optimization is the single highest-leverage lever after prompt caching.

Pattern 3: Model Routing as Budget Triage

Not every agent turn needs a frontier model. A classification step can use Haiku or Gemini Flash. A tool selection step can use a smaller model. Save the big models for reasoning and synthesis.

This is budget triage: match model capability to the complexity of what the context window contains at each step.

Budget allocation by agent lifecycle stage:

[Classification]  →  Flash / Haiku          (~500 tokens)
[Tool Selection]  →  Sonnet / GPT-5.2        (~2,000 tokens)
[Tool Execution]  →  Model doesn't matter    (tool call overhead only)
[Reasoning]       →  Opus / GPT-5            (~8,000 tokens)
[Synthesis]       →  Sonnet / GPT-5.2        (~3,000 tokens)

Our team routes each agent step through a lightweight classifier that decides the model tier based on task complexity and current context size. When the context is small (early in the session), we use cheaper models more aggressively. As it grows and the agent needs more reasoning power to navigate accumulated information, we upgrade.

This is the inverse of what most teams do — they pick one model for the entire session and accept whatever it costs. Context-aware model routing can cut per-task token cost by 40–60% with no measurable quality degradation.

Pattern 4: Hard Budget Enforcement (Before Your CFO Does It for You)

The industry has a growing collection of runaway agent horror stories. Zylos Research documents incidents ranging from $15 burned in ten minutes to $47,000 over eleven days. Cockroach Labs’ analysis of the Uber incident frames the core issue: “The pilot numbers were one thing, and the production numbers were a different animal entirely.”

Soft limits don’t work. The agent doesn’t know about your budget. It will keep reasoning, keep calling tools, keep accumulating context until it runs out of window or runs out of your money. You need hard circuit breakers at the infrastructure layer:

from dataclasses import dataclass
from datetime import datetime

@dataclass
class ContextBudget:
    max_tokens: int
    tokens_used: int = 0
    max_tool_calls: int = 50
    tool_calls_made: int = 0
    max_cost_cents: int = 500  # $5.00 hard cap
    cost_cents: int = 0

    def consume(self, tokens: int, cost_cents: int = 0) -> None:
        self.tokens_used += tokens
        self.cost_cents += cost_cents
        
        if self.tokens_used > self.max_tokens:
            raise BudgetExceededError(
                f"Context budget exhausted: {self.tokens_used}/{self.max_tokens} tokens"
            )
        if self.cost_cents > self.max_cost_cents:
            raise BudgetExceededError(
                f"Cost budget exhausted: ${self.cost_cents/100:.2f}"
            )

class BudgetExceededError(Exception):
    """Hard stop — agent cannot continue."""
    pass


# Usage in an agent loop
budget = ContextBudget(max_tokens=100_000, max_cost_cents=500)

for step in agent.run(prompt):
    budget.consume(
        tokens=step.token_count,
        cost_cents=step.cost_cents,
    )
    if isinstance(step, ToolCall) and step.count > budget.max_tool_calls:
        raise BudgetExceededError("Tool call limit exceeded")

This isn’t elegant. It’s infrastructure. But it’s the kind of infrastructure that separates teams with predictable AI bills from teams whose CFOs are asking uncomfortable questions after the quarterly cloud invoice.

The Architectural Litmus Test

When we evaluate an agent architecture — our own or a client’s — we use a single litmus test:

Where does the context budget grow, and who’s responsible for compressing it?

If the answer is “every tool call adds unbounded data and nobody compresses anything,” the architecture is incomplete, regardless of how elegant the orchestration pattern looks on a whiteboard.

If the answer is “subagents compress, tools return summaries, models are routed by complexity, and circuit breakers cap runaway sessions,” you have an architecture that will survive production.

The frameworks are converging. The models are commoditizing. The context budget is the architecture. Build accordingly.

---

Further reading:

• Agent Architecture Is Converging — and That Changes How You Build — How every major framework now shares the same primitives
• The Inference Cost Paradox: Models Are Nearly Free, But Your Agent Bill Just Hit $100K — Why falling per-token prices don’t solve the budget problem
• AI Agent Cost Engineering — Production Token Economics (Zylos Research) — Data on 60–85% recoverable spend through context discipline
• The Bill Arrives: How to Manage Agentic AI Costs at Scale (Cockroach Labs) — The Uber case study and multiplier math
• The 3 Essential Sub-Agent Patterns for Production-Grade AI Systems (Epsilla) — 90% context reduction through subagent isolation