Skip to main content

Documentation Index

Fetch the complete documentation index at: https://mintlify.com/StarTrail-org/PixelRAG/llms.txt

Use this file to discover all available pages before exploring further.

PixelRAG is an open-source visual retrieval-augmented generation system from Berkeley SkyLab, BAIR, and the Berkeley NLP Group. Instead of parsing documents to text, PixelRAG renders every page to screenshot tiles and embeds those images into a searchable vector index — so the retrieval stage operates directly on what the page looks like, not on a lossy text extraction of it.

​
The problem with text-based RAG

Traditional RAG pipelines parse HTML or PDFs into plain-text chunks. That process silently destroys structure: a table becomes a scrambled list of numbers, a chart becomes nothing at all, and a carefully laid-out comparison matrix becomes an unordered wall of prose. When those chunks are later retrieved and passed to a reader model, the context needed to answer the question is already gone. PixelRAG sidesteps the parsing step entirely. Each page is rendered to one or more JPEG tiles — a faithful screenshot of exactly what a person would see — and a Qwen3-VL-Embedding model (LoRA-fine-tuned on screenshot data) embeds those tiles into a space where visual content is retrievable by natural-language queries. The reader model receives the actual image, sees the table or chart, and can answer questions about it.

​
Two ways to use PixelRAG

PixelRAG ships in two complementary modes: Hosted Wikipedia index — A pre-built index of 8.28M Wikipedia pages is served live at api.pixelrag.ai. No setup and no API key are required. Send a POST request with a text (or image) query and receive ranked screenshot tiles back immediately. This is the fastest way to try visual RAG. Self-hosted with custom documents — Use the pixelrag pipeline to render your own PDFs, web pages, or local files; embed them with the same vision-language model; and serve a private search API backed by FAISS or Qdrant. The full pipeline runs on Linux (CUDA) and macOS (Apple Silicon / MPS).

​
Key features

Visual structure preservation

Pages are captured as screenshot tiles before any text extraction, keeping tables, charts, infographics, and layout intact for both retrieval and reading.

8.28M-page Wikipedia index

A hosted FAISS index covering all of English Wikipedia is available at api.pixelrag.ai — no download, no GPU, and no API key required.

Claude plugin (pixelbrowse)

The pixelbrowse skill gives Claude Code the ability to screenshot any URL with pixelshot and read the image directly, so it sees charts and tables the way a person does.

FAISS and Qdrant backends

Local indexes use FAISS by default. Switch to Qdrant for configurable quantization, disk-backed vectors, payload filtering, and a single collection shared across multiple servers.

LoRA fine-tuned embedding model

Retrieval is powered by Qwen/Qwen3-VL-Embedding-2B with LoRA adapters trained on screenshot data and published at Chrisyichuan/wiki-screenshot-embedding-lora.

Multi-GPU embedding

The pixelrag embed stage accepts a --gpu-ids flag so embedding jobs can be distributed across multiple GPUs to speed up large-scale index builds.

​
Architecture overview

Every PixelRAG pipeline follows four stages. Each stage is independently installable so you only pull in the dependencies you need.
1

Render

The pixelshot command (or pixelrag_render.render_url in Python) captures URLs, PDFs, and local HTML files as tiled JPEG images using a headless Chrome/Chromium browser via the Chrome DevTools Protocol. Each tile is at most 8 192 px tall, giving the vision model a high-fidelity window into each section of the page.
2

Chunk

pixelrag chunk organises the raw tile directories produced by the render stage into shards ready for embedding — assigning sequential IDs and writing the metadata that links each vector back to its source tile.
3

Embed

pixelrag embed runs Qwen3-VL-Embedding-2B (with optional LoRA adapters) over each tile shard and writes .npy embedding files. On Linux the model uses CUDA; on macOS it uses the MPS backend; and device: auto in pixelrag.yaml picks the best available option automatically.
4

Index

pixelrag build-index aggregates the embedding shards into a FAISS flat index (or pushes vectors to Qdrant) and writes a summary.json that pixelrag serve reads at startup. The pixelrag index build orchestrator command runs all four stages end-to-end from a single pixelrag.yaml config file.

​
Next steps

Quickstart

Query the hosted Wikipedia index in 30 seconds, or build a local index from your own documents in under five minutes.

Concepts

Deeper explanation of tile rendering, the embedding model, and how FAISS and Qdrant indexes are structured.

Build docs developers (and LLMs) love

Get started for freeTalk to us