Build a Fully Offline RAG App in Your Browser

Browser RAG is useful when documents should remain on the reader’s device. There is no upload endpoint, hosted vector database, or inference API to secure. The trade-off is that the browser must download, cache, and run every part of the pipeline.

This tutorial builds that pipeline. The finished application accepts two to five TXT, Markdown, text-based PDF, or DOCX files; creates embeddings locally; retrieves relevant passages; and generates an answer with a compact WebLLM model. After one provisioning run, it survives a network-disabled hard reload.

Complete source code: ihiteshsharma/offline-browser-rag
Clone the tested implementation or use it to compare each step in this tutorial.

Understand the system before writing code

The application has three flows:

1. Provision: download and cache the application, generation model, embedding model, tokenizers, and runtimes.
2. Index: parse documents into located sections, split them into chunks, create embeddings, and store the index.
3. Answer: embed a question, retrieve matching chunks, give that evidence to the generator, and validate its citations.

WebLLM is not a hosted API. It loads a model already converted to MLC artifacts, moves its quantized weights into GPU memory through WebGPU, tokenizes the prompt, and generates tokens inside the browser. A web worker keeps that computation away from the interface thread. Its API resembles OpenAI chat completions, but no prompt leaves the page. WebLLM’s documentation describes the WebGPU runtime and worker interfaces.

This lab uses a second runtime for retrieval. Transformers.js runs all-MiniLM-L6-v2 through WASM. Keeping one fixed embedding model means changing the chat model does not invalidate every stored vector. It also leaves the GPU to WebLLM while the smaller embedding workload runs on the CPU.

Offline readiness spans four stores:

• the service worker caches the application shell and runtime assets;
• WebLLM caches its generation model in IndexedDB;
• Transformers.js caches its model, configuration, and tokenizer;
• the application’s IndexedDB stores chunks, vectors, and index metadata.

A cached page is not an offline RAG system if its first question still downloads a tokenizer or model shard.

Prerequisites and budget

The executed lab uses Node.js 24.13.1, pnpm 10.31.0, Vite 8.1.5, vite-plugin-pwa 1.3.0, WebLLM 0.2.84, Transformers.js 4.2.0, PDF.js 6.1.200, and Mammoth 1.12.0.

Allow 60–90 minutes, a WebGPU-capable browser, and several gigabytes of browser-managed storage. Model downloads use significant bandwidth, GPU memory, and battery even though there is no per-query server bill.

Step 1: create the application

To start with the completed implementation:

git clone https://github.com/ihiteshsharma/offline-browser-rag.git
cd offline-browser-rag
pnpm install
pnpm test
pnpm dev

To build it incrementally while following the tutorial:

mkdir offline-browser-rag
cd offline-browser-rag
pnpm init
pnpm add @mlc-ai/web-llm@0.2.84 @huggingface/transformers@4.2.0
pnpm add pdfjs-dist@6.1.200 mammoth@1.12.0
pnpm add --save-dev vite@8.1.5 vite-plugin-pwa@1.3.0 workbox-window@7.4.1

Use separate modules for document parsing, retrieval, and model execution:

src/
├── document-parser.js
├── main.js
├── model-worker.js
├── retrieval.js
└── style.css

The service worker must include .mjs; Vite emits the PDF.js worker with that extension:

VitePWA({
  registerType: "autoUpdate",
  workbox: {
    globPatterns: ["**/*.{html,js,mjs,css,wasm}"],
    maximumFileSizeToCacheInBytes: 32 * 1024 * 1024,
  },
});

The service worker handles the shell and bundled runtimes. Do not place gigabyte-scale model weights in a generic Workbox precache.

Step 2: provision a WebLLM model

WebLLM cannot load an arbitrary Hugging Face identifier. It needs compatible MLC weights and a matching model library, so expose a tested allowlist:

const MODEL_ALLOWLIST = new Set([
  "Qwen2.5-0.5B-Instruct-q4f16_1-MLC",
  "Llama-3.2-1B-Instruct-q4f16_1-MLC",
]);

Create a WebWorkerMLCEngine and keep WebLLM’s prebuilt model registry while selecting IndexedDB caching:

engine = await CreateWebWorkerMLCEngine(worker, selectedModel, {
  appConfig: { ...prebuiltAppConfig, cacheBackend: "indexeddb" },
  initProgressCallback,
});

const persistent = await navigator.storage.persist();

Before downloading, display navigator.storage.estimate() and the model choice. persist() only asks the browser to reduce automatic eviction; it can be denied, and users can always clear site data.

Step 3: parse every document inside the browser

All formats should cross one boundary:

{ source: "handbook.pdf", locator: "page-12", text: "Extracted text…" }

Everything after that boundary remains independent of PDF or DOCX.

For a text PDF, pass file.arrayBuffer() to PDF.js, iterate over pages, and preserve page numbers:

const pdf = await getDocument({
  data: new Uint8Array(await file.arrayBuffer()),
  useSystemFonts: true,
}).promise;

for (let pageNumber = 1; pageNumber <= pdf.numPages; pageNumber += 1) {
  const page = await pdf.getPage(pageNumber);
  const content = await page.getTextContent();
  const text = content.items.map((item) => item.str).join(" ").trim();
  if (text) sections.push({
    source: file.name,
    locator: `page-${pageNumber}`,
    text,
  });
}

Configure GlobalWorkerOptions.workerSrc with Vite’s ?url import so PDF parsing uses the emitted browser worker. PDF.js performs this parsing locally; no file upload is involved.

For DOCX, Mammoth accepts the same browser ArrayBuffer:

const { value } = await mammoth.extractRawText({
  arrayBuffer: await file.arrayBuffer(),
});

const sections = value
  .split(/\n\s*\n/)
  .map((text) => text.trim())
  .filter(Boolean)
  .map((text, index) => ({
    source: file.name,
    locator: `paragraph-${index + 1}`,
    text,
  }));

Raw text is appropriate for retrieval and avoids rendering untrusted HTML. Mammoth warns that its generated HTML is not sanitized. Tables, embedded images, and exact layout require a richer extraction policy rather than silently flattening them. Mammoth’s browser API documents the ArrayBuffer input.

Reject unsupported types, empty extraction results, and files over an explicit size limit. Page and paragraph locators later become inspectable citations.

Optional extension: scanned-PDF OCR

PDF.js cannot extract text that exists only as an image. A browser-only extension can render each page to a canvas and send it to a Tesseract.js worker. Tesseract.js does not consume PDFs directly, so PDF rendering and OCR are separate steps. Its worker, WASM core, and language data must also be cached or self-hosted for offline use. OCR adds enough download, accuracy, and performance work to remain outside the main lab. Tesseract.js documentation

Step 4: chunk, embed, and store

Chunk each located section into roughly 180 words with 30 words of overlap. Carry the locator into the source marker, such as handbook.pdf#page-12#0.

Use the repository’s available q4 MiniLM artifact:

const embedder = await pipeline(
  "feature-extraction",
  "onnx-community/all-MiniLM-L6-v2-ONNX",
  { device: "wasm", dtype: "q4" },
);

const output = await embedder(
  chunks.map(({ text }) => text),
  { pooling: "mean", normalize: true },
);

Store chunks and vectors in IndexedDB. Also record the embedding model, dtype, chunk size, overlap, and document checksum. Any change to those values should trigger a clean re-index.

Step 5: retrieve and generate

Embed the question with the same MiniLM pipeline. Compute cosine similarity against stored vectors, sort descending, and retain the top three chunks. Linear search is adequate for this small lab and keeps retrieval understandable.

Build a prompt that treats retrieved text as evidence:

Answer only from the supplied context.
If the context is insufficient, say the documents do not contain the answer.
Cite every supported claim using its [source#locator#chunk] marker.

Stream the prompt through engine.chat.completions.create() with a low temperature. Always display the retrieved passages and scores beside the generated answer.

Step 6: enforce grounded answers

Prompt instructions do not guarantee citations. Request structured output containing an answer plus citation objects when the selected model supports WebLLM’s response_format.

Accept a response only when every citation names a currently retrieved chunk and its evidence quotation occurs verbatim in that chunk. An answer without a valid citation should be retried once, then replaced with an abstention and the raw retrieved passages. Calibrate any retrieval threshold on answerable and unanswerable test questions; similarity is not proof of support.

Step 7: verify failure and offline recovery

Run pnpm test and pnpm build. The current suite executes nine tests, including real PDF text extraction, DOCX paragraph extraction, locator preservation, chunking, retrieval ordering, and grounded-prompt construction. The production build precaches eight entries, including the PDF worker and ONNX WASM runtime.

Before provisioning, disable networking and confirm the model action reports that a connection is required. Then provision, index documents, answer a baseline question, disable networking, and hard-reload.

A reference run in Brave on macOS 26 restored Qwen 2.5 0.5B without persistent-storage permission and answered after the hard reload. Retrieval correctly ranked support.md#0 at 0.791, but the model omitted an inline citation—exactly why Step 6 requires application-side validation.

Ship it to production

Run pnpm build and deploy dist to an HTTPS static origin; vite preview is not a production server. Pin dependency and model revisions, confirm licenses, use integrity metadata where supported, and stage service-worker and index migrations.

Use a strict Content Security Policy and avoid third-party scripts that can read local documents. Treat retrieved text as untrusted input, cap file size and page count, and expose no consequential model tools. Maintain a browser/device matrix, storage budget, cache-completeness check, reprovision action, and delete-all-data action.

Useful observability stays local or opt-in: browser capabilities, storage quota, parser warnings, page and chunk counts, embedding duration, retrieved source markers, similarity scores, time to first token, generation speed, cache misses, and GPU device loss. Do not log document text or sensitive filenames by default.

The tutorial does not provide encrypted browser storage, multi-user synchronization, perfect PDF reading order, DOCX layout fidelity, OCR guarantees, or large-corpus indexing. Those are explicit production gaps, not properties supplied by local inference.

For a larger corpus, measure linear-search latency before adding an approximate index or lexical prefilter. A desktop runtime is the simpler alternative when browser compatibility, storage eviction, or local model size becomes the dominant constraint.

Cleanup

Clear the chunks, vectors, and metadata object stores. Remove model artifacts and service-worker caches through the browser’s site-data controls for the application origin.

Sources

• WebLLM 0.2.84
• WebLLM API reference
• Transformers.js
• Transformers.js model cache registry
• Mozilla PDF.js
• Mammoth.js
• Tesseract.js
• Vite static deployment
• OWASP RAG Security Cheat Sheet