Slashing Agent Context and Token Churn With a Code Graph
The Problem
When you prompt an AI coding agent to modify a non-trivial codebase, its first instinct is almost universally to grep or recursively read every file that might relate to the task.
In a 50,000-line repository, this brute-force approach leads directly to token bankruptcy:
- Reading 25 interconnected files into context consumes 35,000 to 50,000 tokens in a single turn.
- At multi-turn scale (debugging, executing tests, refactoring), the session balloons past 150,000 tokens within ten minutes.
- As the context window saturates, the model’s retrieval precision degrades (“needle in a haystack” decay), causing it to overlook crucial type constraints or hallucinate method signatures.
Research published by Anthropic in mid-2025 noted that multi-agent research teams consumed roughly 15 times more tokens than single chats, with token expenditure accounting for nearly 80% of observed performance variance. When you simply throw raw context at an agent, you pay exponentially more for degraded reasoning.
The root issue is architectural: code is a graph of typed references, not an unstructured sequence of text. Treating code like plain English forces the LLM to spend billions of floating-point operations re-parsing ASTs on every single prompt.
What I Tried & Why It Failed
Before introducing a formal code knowledge graph, I attempted three conventional mitigation strategies:
1. High-Pass Ripgrep & Regex Search
I restricted the agent to ripgrep search commands, instructing it to only read files that matched explicit function names.
Why it failed: Ripgrep has zero semantic awareness. If an agent searches for processOrder, it hits comments, unit tests, mock objects, and logging strings across 60 files. The agent still had to open 15 files just to determine which one contained the actual production implementation.
2. Embeddings and Vector RAG
I chunked source files into 500-token blocks, embedded them into pgvector, and retrieved top-k semantic matches based on the task description.
Why it failed: Vector search excels at conceptual similarity (“find code related to user authentication”), but utterly fails at deterministic call graphs. If AuthService delegates to an internal helper _verifyNonce(token), vector search rarely retrieves the helper unless the prompt specifically mentions nonces. The agent regularly produced patches that called non-existent overloads or passed wrong argument counts.
3. File Map Skeletons
I had the build pipeline generate high-level outline trees (class and function names without bodies).
Why it failed: While outlines fit in context, they lack type signatures, caller hierarchies, and return types. The agent was forced to “guess” method signatures or issue speculative read_file calls, returning to square one.
The Fix: Pre-Indexed AST Code Graphs (codegraph)
The durable solution is to pre-index the codebase into a structured Abstract Syntax Tree (AST) knowledge graph before the agent ever runs.
We integrated colbymchenry/codegraph, exposing deterministic graph queries directly to the agent as MCP and Hermes native tools.
┌─────────────────────────────────────────────────────────────┐
│ Codebase (TypeScript / Python) │
└──────────────────────────────┬──────────────────────────────┘
│ AST Extraction & Linker
▼
┌─────────────────────────────────────────────────────────────┐
│ Pre-Indexed Code Graph (SQLite) │
│ │
│ • Symbol Definitions & Signatures │
│ • Caller / Callee Reference Edges │
│ • Inheritance & Interface Implementations │
└──────────────────────────────┬──────────────────────────────┘
│
┌──────────────────┴──────────────────┐
│ Precise Symbol Queries (~150 tokens)│
▼ ▼
┌──────────────────────────────┐ ┌──────────────────────────────┐
│ Single Writer │ │ Adversarial Reviewer │
│ (Modifies Target Logic) │ │ (Verifies Invariants) │
└──────────────────────────────┘ └──────────────────────────────┘Instead of sending prompt commands like “Search the repo for order processing logic”, the agent executes structured symbolic exploration:
# Query the symbol graph for the exact implementation and callers
codegraph query "Symbol('OrderProcessor.charge').callers()" --depth 1The tool returns a compact JSON structure representing the AST node, its exact file coordinates, parameter types, and inbound caller references:
{
"symbol": "OrderProcessor.charge",
"file": "src/billing/processor.ts",
"line_range": [42, 68],
"signature": "(customerId: string, amountCents: number): Promise<ChargeResult>",
"callers": [
{ "caller": "CheckoutController.submit", "file": "src/controllers/checkout.ts", "line": 104 }
]
}Why This Changes Everything
- Micro-Footprint Context: Instead of dumping 3,000 lines of code across 5 files (approx. 12,000 tokens), the agent receives a 140-token exact representation.
- Zero Hallucination of Signatures: The agent sees the exact compiler-verified parameter types. It never invents an argument or guesses whether an async method returns a Promise.
- Targeted Read Operations: The agent uses
read_fileexclusively on the target line range (processor.ts:42-68), touching only the bytes it needs to rewrite.
What I’d Do Differently
When we first deployed the code graph, we generated the index dynamically on container startup. For a large project with nested dependencies, rebuilding the AST graph on every agent invocation introduced a 45-second cold start.
The correction:
Move graph indexing into the local Git pre-commit hook and CI build step. The index is stored as a lightweight SQLite artifact (.codegraph.db). The agent reads the pre-compiled graph instantly (sub-10ms response time), achieving immediate execution speed.
The Result: Quantified In-Production Metrics
We benchmarked 20 identical bug-fix and feature-addition tasks across our service stack, comparing raw directory search against the AST code graph.
Benchmark Environment: Evaluated across Claude 3.5 Sonnet and GPT-4o (2024-11-20). All tasks represented production tickets involving cross-module dependencies. Token billing calculated using standard list prices ($3.00/1M input, $15.00/1M output).
| Metric | Raw Text / Grep Search | AST Code Graph (codegraph) | Improvement / Speedup |
|---|---|---|---|
| Average Initial Context Payload | 41,200 tokens | 1,850 tokens | -95.5% payload reduction |
| Cumulative Session Token Churn | 184,000 tokens | 32,400 tokens | -82.4% token reduction (5.68× information density) |
| Turns to First Valid Patch | 7.4 turns | 2.1 turns | -71.6% turn reduction (3.52× faster patch delivery) |
| Signature / Type Regressions | 18% of patches | 0% (zero missed types or parameter misorders) | Eliminated |
| Session Cost per Feature | ~$1.28 | ~$0.19 | 85.2% cost reduction |
By treating code as a graph rather than raw text, the agent stops drowning in irrelevant lines and focuses 100% of its reasoning capacity on the precise logic that needs to change.
Want a 24/7 bot or automation pipeline that never drops a lead?
Don't let customer inquiries slip away at midnight or on weekends. I build custom Telegram/WhatsApp AI assistants and deterministic workflows that connect directly to your catalogue, CRM, and order fulfillment.
