About This Session
Every AI coding agent I've used does the same thing: it reads your entire file, every single time. On a mid-size Python project we work on, that's 45,000 tokens per query. The agent only needed about 4,900. We were burning through input tokens and didn't even realise how much until the bill landed.
So we built something to fix it. An open-source tool that indexes your codebase locally using tree-sitter for AST-aware chunking and sqlite-vec for vector search. It plugs into your editor through the Model Context Protocol, so the agent gets exactly the code it needs instead of everything. The result: 94% fewer tokens, measured across 20 real-world coding queries.
What made this tricky was getting retrieval quality right. Pure vector search missed too much; keyword search returned noise. We ended up combining both with Reciprocal Rank Fusion and adding confidence scoring so the agent knows when it should fall back to reading the full file.
In this talk, I'll walk through the architecture decisions we made and the ones we got wrong. Why we picked sqlite-vec over LanceDB. Why we chose truncation over LLM-based summarisation. How we handle secret redaction before anything hits the embedding model. And I'll show live benchmarks comparing retrieval with and without the index.
You'll walk away knowing how to build your own local RAG for code, how to measure token savings properly, and practical ways to cut your AI tooling costs without changing how you work.