--- title: "In-Browser RAG & Model Context Protocol: Sub-10ms Semantic Retrieval at Zero Cloud Cost" description: "How we built a serverless, zero-cloud-cost in-browser RAG retrieval engine and Model Context Protocol (MCP) server indexing 50+ essays, podcasts, and software architectures directly inside dh.sh." url: "https://herrington.ai/cujs/in-browser-rag-engine/" markdown_url: "https://herrington.ai/cujs/in-browser-rag-engine/index.md" author: "Daniel Herrington" date: "2026-09-11" updated: "2026-10-31" markdown_tokens: 5729 content_signal: "ai-train=yes, search=yes, ai-input=yes" mcp_endpoint: "https://herrington.ai/mcp" --- # In-Browser RAG & Model Context Protocol: Sub-10ms Semantic Retrieval at Zero Cloud Cost *How we built a serverless, zero-cloud-cost in-browser RAG retrieval engine and Model Context Protocol (MCP) server indexing 50+ essays, podcasts, and software architectures directly inside `dh.sh`.* --- ## The Floating Shell: What Is That Thing? You may have noticed a shell floating on my website and wondered what that was. Even if you haven't clicked on it, I'd love to tell you a little more about the technology. When visitors land on [herrington.ai](https://herrington.ai), they often see a subtle pill in the bottom corner reading `dh.sh`. Most personal sites that feature floating widgets are embedding off-the-shelf third-party chat widgets—an iframe that connects to an OpenAI API wrapper, sends every keystroke to a hosted server, takes three seconds to respond, and costs the site owner recurring cloud tokens on every visit. The floating shell on this site is fundamentally different: **it is a live, transparent client-side execution environment for Retrieval-Augmented Generation (RAG) and the Model Context Protocol (MCP).** When you open `dh.sh` and ask a question, no backend server is spun up. No third-party LLM API endpoint is billed. Instead, your browser downloads a precomputed 1MB inverted index, executes a multi-factor BM25 / TF-IDF ranking algorithm across 50 documents and 730 semantic passages in vanilla JavaScript, runs a deterministic MCP tool call (`search_article_insights`), and renders a verified answer with exact quotes and direct citations in **under 8 milliseconds**. Furthermore, because the retrieval engine adheres strictly to the open **Model Context Protocol (MCP)** specification, this exact same engine is exposed as an MCP server that you can plug directly into **Claude Desktop**, **Cursor**, or autonomous agent swarms. Here is why we built it, how the architecture functions, and the Critical User Journeys (CUJs) it unlocks for autonomous agents, developers, journalists, and human readers. --- ## The Problem Space: Why Traditional RAG Fails Personal Sites Over the past few years, this website has evolved into an extensive technical repository: - **28 deep architectural essays** (covering TPU cluster scaling, deterministic agent harnesses, vibe coding economics, insurtech data privacy, and brand share-of-voice). - **6 Critical User Journey (CUJ) engineering case studies**. - **4 full-length podcast transcripts and media profiles** (including *Software Leaders Uncensored* and *CIO.com* features). - **Multiple open-source repositories and architectural blueprints**. When someone visits the site wanting to know: *"What is Daniel's stance on vibe coding vs. agent harnesses?"*, *"How does he measure ML Goodput?"*, or *"What did he say about actuarial models for autonomous driving?"*, standard web navigation falls apart. No one wants to manually click through five different archive carousels, open 20 tabs, and `Ctrl+F` through thousands of words. ### The Standard Industry Playbook: Cloud Vector RAG The standard recommendation for natural language search across custom content is a cloud RAG pipeline: 1. Ingest markdown into an external vector database (Pinecone, Qdrant, Weaviate). 2. Host an API server (FastAPI, Express) that receives incoming user queries. 3. Call an embedding model (e.g., `text-embedding-3-small`) to convert the query into vectors. 4. Perform approximate nearest neighbor (ANN) search across the cloud index. 5. Pass the retrieved passages to a large language model to stream an answer. While viable for enterprise document sets with millions of pages, this architecture introduces massive friction for a public portfolio: 1. **The Vector DB Subscription Tax:** Cloud vector databases cost between $50 and $200 per month just to stay alive in RAM, even when nobody is querying the site at 3 AM. 2. **Open-Ended Token Vulnerability:** A public web form calling OpenAI or Anthropic directly exposes your API keys and monthly credit card limits to scrape bots, script kiddies, and recursive prompt injection loops. 3. **Severe Cold-Start Latency:** Between embedding generation, remote network round-trips to the database, vector similarity compute, and LLM first-token generation, typical cloud RAG pipelines clock **1,200ms to 3,500ms** of latency. 4. **Black-Box Opacity:** Standard chat widgets show an animated spinner and dump text into a bubble. You have no visibility into what documents were retrieved, what scoring metric was used, or what tools were executed. We decided to invert the paradigm: **precompute semantic weights at static site build time, ship the inverted index to the browser, execute retrieval on the edge in sub-10ms, and expose the tools through the Model Context Protocol.** --- ## Critical User Journeys (CUJs) To guide the engineering of this system, we defined four distinct Critical User Journeys following our standardized CUJ framework: ``` ┌────────────────────────────────────────────────────────────────────────┐ │ CRITICAL USER JOURNEYS (CUJ FRAMEWORK) │ ├────────────────────────────────────────────────────────────────────────┤ │ 1. Autonomous Research Agent │ Machine-to-machine zero-token retrieval│ │ 2. Human Developer / MCP User │ Native querying inside Claude / IDE │ │ 3. Tech Journalist & Media │ High-precision sourcing & quotes │ │ 4. In-Browser Visitor │ Sub-10ms interactive shell (dh.sh) │ └────────────────────────────────────────────────────────────────────────┘ ``` ### CUJ 1: Autonomous Research Agent CUJ (Machine-to-Machine Discovery) * **Statement:** I want an **autonomous agent to query Daniel's complete knowledge base via structured RPCs** while avoiding **brute-force scraping and noisy DOM parsing** so that **it can extract high-precision citations and verified stances with zero token bloat**. * **User Scenario:** An autonomous research agent (running on Perplexity, Claude 3.7 Sonnet, or an AutoGPT swarm) is tasked with synthesizing expert perspectives on enterprise AI governance. Instead of downloading 50 HTML pages, stripping navigation menus, fighting DOM mutations, and burning 150,000 tokens on boilerplate, the agent hits our structured MCP endpoints or queries `article_index.json`. It receives clean, deduplicated semantic chunks with exact confidence scores in milliseconds, preserving its context window for synthesis. ### CUJ 2: Human Power-User via MCP CUJ (Claude Desktop & IDE Integration) * **Statement:** I want to **interact with Daniel's site through the standard Model Context Protocol (MCP) in Claude Desktop or my IDE** while avoiding **context-switching into a web browser** so that **his essays, case studies, and engineering blueprints are queryable tools directly inside my local AI workflow**. * **User Scenario:** A staff software engineer working in Cursor or Claude Desktop is architecting a multi-agent orchestration engine. Rather than leaving their IDE, opening a browser, searching `herrington.ai`, and copy-pasting code snippets, they have `herrington-ai-mcp` configured in their `claude_desktop_config.json`. They simply ask Claude: *"According to Daniel Herrington's writings, how should we structure state machine transitions in multi-agent frameworks?"* Claude calls `search_article_insights`, retrieves the exact blueprint, and writes the code inline. ### CUJ 3: Tech Journalist & Media Researcher CUJ (The Future of Sourcing) * **Statement:** I want an **AI research assistant with MCP access to Daniel's verified body of work** while avoiding **outdated press releases or scouring fragmented LinkedIn posts** so that **I can instantly extract authentic quotes, track topic history, and cite his perspectives with verified references**. * **User Scenario:** A technology reporter at *Wired* or *CIO Magazine* is preparing a feature on how Chief AI Officers manage board governance and autonomous coding risks. Instead of sending emails back and forth or combing through old podcasts, the journalist's AI sourcing assistant connects to Daniel's knowledge base via MCP. It cross-references his recent CIO article on *Harnessing Unleashed AI Agents* with his podcast interview on *Software Leaders Uncensored*, extracting verified direct quotes with exact publication dates and source URLs. ### CUJ 4: In-Browser Human Visitor CUJ (Instant Interactive Terminal) * **Statement:** I want to **query Daniel's extensive portfolio of essays, media appearances, and architecture case studies through natural language directly in my browser** while avoiding **hallucinated answers and slow cloud API cold starts** so that **I receive exact, high-confidence quotes and verified hyperlinks in sub-10ms**. * **User Scenario:** An engineering leader visiting the site clicks the floating `dh.sh` terminal badge. They type: *"What is the Free Beer Paradox?"* Without any network latency or external API calls, the terminal executes the inverted index search in 6ms, displays the internal JSON-RPC 2.0 tool execution log, and synthesizes an answer highlighting Daniel's essay on open-source economics with a direct clickable link. --- ## Dual Architecture: Edge Inverted Index + Model Context Protocol The engine operates across two decoupled planes: **Build-Time Precomputation** (Python SSG) and **Dual-Mode Consumption** (Client-Side Vanilla JS + External MCP Server). ``` ┌────────────────────────────────────────────────────────────────────────┐ │ BUILD-TIME PIPELINE (Python SSG) │ │ │ │ 28 Native Essays 6 CUJ Studies 4 Media Transcripts Project Docs│ │ │ │ │ │ │ │ ▼ ▼ ▼ ▼ │ │ ┌────────────────────────────────────────────────────────────────┐ │ │ │ Heading-Aware Semantic Chunker & Markdown Normalizer │ │ │ └────────────────────────────────┬───────────────────────────────┘ │ │ ▼ │ │ ┌────────────────────────────────────────────────────────────────┐ │ │ │ TF-IDF Weighting & Inverted Index Generator (6,583 Terms) │ │ │ └────────────────────────────────┬───────────────────────────────┘ │ │ ▼ │ │ public/data/article_index.json (1.0MB / 180KB gzip) │ └────────────────────────────────────┬───────────────────────────────────┘ │ ┌─────────────────────────┴─────────────────────────┐ ▼ ▼ ┌───────────────────────────────────────┐ ┌───────────────────────────────────────┐ │ CLIENT-SIDE ENGINE (Vanilla JS) │ │ EXTERNAL MCP SERVER (TypeScript) │ │ │ │ │ │ Runs inside dh.sh Terminal │ │ Runs via Stdio / SSE for Claude │ │ - Tokenizer & Stop-Word Filter │ │ - @modelcontextprotocol/sdk │ │ - Postings Accumulator (3ms) │ │ - Tools: search_article_insights │ │ - Exact-Phrase & Title Boosting │ │ - get_career_timeline, get_writings │ │ - JSON-RPC 2.0 Observable Traces │ │ - Plugs into claude_desktop_config │ │ │ │ │ │ Output: Sub-10ms UI Answers │ │ Output: Native Agent Tool Execution │ └───────────────────────────────────────┘ └───────────────────────────────────────┘ ``` --- ## Technical Deep-Dive ### 1. Build-Time Semantic Chunking & Indexing When `python3 scripts/build_all.py` compiles the static site, it runs `scripts/build_rag_index.py`. The indexer scans all markdown assets across `skills/cujs/references/`, `public/insights/`, and structured records in `public/data/`. Rather than using crude fixed-length character splitters (which cut sentences and destroy semantic context), our chunker is **heading-aware**: ```python def chunk_markdown(slug, title, url, body, category="insight", tags=None): """Splits markdown into coherent semantic chunks bounded by headings.""" chunks = [] lines = body.split('\n') current_heading = title current_paras = [] for line in lines: heading_match = re.match(r'^(#{1,3})\s+(.+)$', line) if heading_match: # Flush existing section before starting new heading if current_paras: content = "\n".join(current_paras).strip() if len(content) > 60: chunks.append({ "slug": slug, "title": title, "heading": current_heading, "url": url, "content": content, "category": category, "tags": tags or [] }) current_paras = [] current_heading = heading_match.group(2).strip() else: if line.strip(): current_paras.append(line.strip()) ``` Each chunk retains its parent title, current section heading, canonical URL, and category badge (`Native Essay`, `CUJ Case Study`, `Media & Interview`, or `Project Architecture`). ### 2. Inverted Index with BM25 / TF-IDF Weighting For each term $t$ in chunk $d$, we compute its Term Frequency ($TF$), and across the entire corpus of $N$ documents, its Inverse Document Frequency ($IDF$): $$\text{IDF}(t) = \ln\left(1 + \frac{N - n_t + 0.5}{n_t + 0.5}\right)$$ Terms appearing frequently in a specific passage but rarely across the general corpus (e.g., *"actuarial"*, *"goodput"*, *"unleashed"*, *"vibe"*, *"diarization"*) receive strong positive weights, while conversational stopwords are eliminated. The resulting compiled artifact (`public/data/article_index.json`) contains: - **`total_documents`**: 50 indexed assets - **`total_chunks`**: 730 semantic passages - **`unique_terms`**: 6,583 indexed terms - **`inverted_index`**: Map of `term -> [chunk_id_1, chunk_id_2, ...]` - **`idf`**: Precomputed term weights ready for $O(1)$ lookup ### 3. Client-Side Query Execution & Heuristic Re-Ranking When a user submits a query to `dh.sh`, `searchArticleInsights(query)` executes directly in the visitor's browser: ```javascript // Heuristic Multi-Factor Re-Ranking in Vanilla JS scores.forEach((score, chunkIdx) => { const chunk = this.articleIndex.chunks[chunkIdx]; const titleLower = chunk.title.toLowerCase(); const headingLower = chunk.heading.toLowerCase(); rawTokens.forEach(token => { if (titleLower.includes(token)) { scores.set(chunkIdx, scores.get(chunkIdx) + 2.5); // Boost title matches } if (headingLower.includes(token)) { scores.set(chunkIdx, scores.get(chunkIdx) + 1.5); // Boost section heading } }); // Exact contiguous phrase match bonus if (chunk.content.toLowerCase().includes(queryLower)) { scores.set(chunkIdx, scores.get(chunkIdx) + 4.0); } }); ``` To prevent a single long essay from dominating all top spots, the engine applies **diversity deduplication**, guaranteeing that the user receives the top-scoring excerpt from multiple distinct works. --- ## The Model Context Protocol (MCP) Integration The retrieval engine is not locked to a web browser. We implemented the official **`@modelcontextprotocol/sdk`** in TypeScript (`mcp-server/src/index.ts`), supporting both **Stdio** and **SSE** (Server-Sent Events) transports. ### Exposed MCP Tools The server exposes 7 distinct tools: 1. `search_article_insights`: Semantic search and RAG retrieval across Daniel's essays, media appearances, and architecture case studies. 2. `get_career_timeline`: Structured executive career history (The Zebra, Google, Vroom, Capital One, Priceline, MSU). 3. `get_latest_writings`: Most recent articles, LinkedIn publications, and arXiv research papers. 4. `get_interviews_and_media`: Podcast appearances, press interviews, and thought leadership profiles. 5. `get_projects`: Open-source and architectural project portfolio details. 6. `get_personal_facts`: Hobbies, setup preferences, and verified personal background. 7. `get_social_links`: Verified social and code repository links. ### How to Hook Daniel's MCP Server into Claude Desktop Developers, researchers, and journalists can query this site directly inside **Claude Desktop** by adding the server to `claude_desktop_config.json`: ```json { "mcpServers": { "herrington-ai": { "command": "node", "args": [ "/path/to/herrington.ai-site/mcp-server/build/index.js" ] } } } ``` Once configured, Claude gains native tool-calling access to the entire body of work: > **Human:** *"What does Daniel Herrington say about the economics of vibe coding?"* > **Claude:** *Calls `search_article_insights(query="vibe coding economics")`* > **Tool Output:** *Returns passages from 'The End of Vibe Coding' and 'Bricks, Bots, and Vibe Coding'.* > **Claude:** *"Daniel argues that while vibe coding accelerates initial prototyping, it incurs a massive maintenance debt once systems encounter production constraints..."* --- ## Interactive Try-It-Live Showcase You can test the RAG engine right now on this page! Click any of the query buttons below to launch the floating `dh.sh` terminal and watch it retrieve verified insights in real time:
--- ## Architectural Benchmarks: Edge RAG vs. Cloud RAG | Metric | Traditional Cloud RAG (Vector DB + LLM) | `dh.sh` In-Browser RAG Engine | Local MCP Server (Claude Desktop) | | :--- | :--- | :--- | :--- | | **Retrieval Latency** | 1,200ms – 3,500ms | **4ms – 8ms** (~250x faster) | **10ms – 25ms** (Local stdio RPC) | | **Monthly Hosting Cost** | $50 – $200 / month | **$0.00 / month** (Static CDN) | **$0.00 / month** (Local execution) | | **Marginal Cost per Query** | ~$0.01 – $0.05 / query | **$0.00** (Zero API tokens) | Included in Claude subscription | | **Offline Resiliency** | ❌ Fails without network | **✔ 100% offline capable** | **✔ Works with local LLMs** | | **Vulnerability to Scrape Bots** | High (Exposes API credits) | **Zero risk** (Client compute) | Zero risk (Local stdio) | | **Protocol Standardization** | Proprietary REST endpoints | **JSON-RPC 2.0 (MCP)** | **Official Model Context Protocol** | --- ## The Future of Personal Portfolios & Autonomous Sourcing The web is undergoing a rapid phase transition from human-browsed HTML documents to agent-synthesized intelligence. In this emerging paradigm: 1. **Personal websites must become queryable APIs:** Simply presenting a resume or blog list is no longer sufficient. If autonomous research agents cannot query your body of work through structured protocols, your ideas will be ignored by the synthesis engines that inform enterprise decision-makers. 2. **Journalists and researchers will expect verified MCP endpoints:** Rather than emailing PR representatives or reading outdated boilerplate, reporters will use MCP-connected assistants to explore an executive's published record, verify past quotes, and analyze topic trajectories. 3. **Edge compute wins on cost and privacy:** By shipping precomputed inverted indexes directly to the client, site creators can deliver conversational, agentic discovery to millions of visitors without spending a single dollar on cloud vector databases or token APIs. The floating shell on `herrington.ai` is a live proof-of-concept for this future. Click `dh.sh`, type a question, and experience sub-10ms retrieval for yourself.