Skip to main content

Documentation Index

Fetch the complete documentation index at: https://mintlify.com/StarTrail-org/PixelRAG/llms.txt

Use this file to discover all available pages before exploring further.

PixelRAG is built on a single fundamental insight: when you parse an HTML page or PDF into text for a RAG pipeline, the parser silently discards all visual structure — tables become ambiguous prose, charts lose their axes, infographic layouts collapse into disconnected sentences. Rendering the document to an image first and retrieving over screenshots instead preserves that structure exactly as a human sees it, so the reader model can answer questions directly from the visual output.

​
The problem with text-based RAG

Standard RAG pipelines extract text from source documents, chunk it, embed it, and retrieve the closest chunks to a query. This works well for plain prose, but fails in predictable ways the moment a document has meaningful visual structure:
  • Tables — a benchmark comparison table becomes a run of raw numbers with no column headers attached. The reader has no way to reconstruct which number belongs to which row and column.
  • Charts and graphs — axis labels, trend lines, and annotated data points are lost or scrambled. A bar chart showing quarterly revenue becomes a few isolated text fragments.
  • Infographics and multi-column layouts — the spatial relationships that communicate meaning are simply not present in the extracted text stream.
  • Mixed-content pages — a Wikipedia article with an infobox table, a sidebar, and body text gets merged into a single undifferentiated text blob whose chunks can be misleading.
Consider a concrete example: a table with benchmark numbers, where each row is a model and each column is a task. Parsed to text, it becomes something like "Model A 84.2 91.3 78.6 Model B 87.1 89.0 81.4" — a sequence of numbers with no structural anchoring. When a user asks “What is Model B’s score on Task 2?”, a text-based retriever may return the right chunk, but the reader model cannot reliably locate the correct cell. The visual representation of the same table is unambiguous: rows and columns are immediately readable.

​
PixelRAG’s approach

PixelRAG replaces the parse-to-text step with a render-to-image step, then embeds and retrieves the images directly.
1

Render documents to screenshot tiles

The pixelshot command (backed by a headless Chromium CDP process) renders each URL or PDF to a full-page screenshot at 875px viewport width. Because pages are often tall, the output is split into 8192px-tall JPEG tiles stored as {output_dir}/{stem}.png.tiles/tile_NNNN.jpg, alongside a tiles.json manifest. PDF files are rendered via PyMuPDF (bundled) or pdf2image (poppler).
2

Chunk tiles into 1024px strips

Each 8192px tile is split into 1024px-tall strips called chunks (files named chunk_XXXX_YY.png). Strips shorter than 28 pixels — one Qwen3-VL patch — are merged into the previous chunk rather than discarded. Embedding 1024px chunks instead of full 8192px tiles reduces the visual token count roughly 8× and significantly improves throughput.
3

Embed chunks with Qwen3-VL-Embedding-2B

Each chunk image is embedded by Qwen/Qwen3-VL-Embedding-2B, a vision-language model LoRA-fine-tuned on screenshot data from Wikipedia. Embeddings are stored as float16 in .npz shards alongside article IDs, tile indices, chunk indices, and Y offsets.
4

Build a FAISS or Qdrant vector index

All shard .npz files are merged, deduplicated by MD5 hash, and used to build a FAISS IndexIVFFlat index (default) or a Qdrant collection. The output is an index.faiss (or Qdrant collection), a metadata.npz, a summary.json, and an articles.json mapping article IDs to URLs.
5

Retrieve and read with a VLM

At query time, the same Qwen3-VL model encodes the text query into the same embedding space. The top-K matching chunk images are retrieved from the index. The reader model — a vision-language model — receives the actual rendered screenshot, and can read numbers, tables, and diagrams exactly as they appear on the page.

​
When to use PixelRAG vs text RAG

Use PixelRAG when…

  • Your documents contain tables, benchmark results, or structured data grids
  • You need answers from charts, diagrams, or infographics
  • You’re indexing Wikipedia or other web pages with rich layout
  • Visual structure carries semantic meaning (e.g. sidebars, infoboxes, callouts)
  • You want to search PDFs that were designed to be read visually

Use text RAG when…

  • Your corpus is plain prose with no meaningful visual structure
  • Documents are already clean, well-structured text (e.g. code, logs, transcripts)
  • You need to search inside text that is not rendered (e.g. raw markdown, config files)
  • Latency or cost constraints rule out a vision model at query time
When in doubt, PixelRAG is the safer default for web pages and PDFs — the renderer preserves structure you might not have realized was important until a text-based system got it wrong.

​
Pre-built vs custom indexes

You do not need to run the pipeline to start using PixelRAG.

Hosted index (api.pixelrag.ai)

The live endpoint at https://api.pixelrag.ai serves a pre-built index of 8.28 million Wikipedia pages. No setup, no API key. Send a POST to /search with a text or image query and receive rendered chunk images back immediately.
curl -X POST https://api.pixelrag.ai/search \
  -H "Content-Type: application/json" \
  -d '{"queries": [{"text": "What is the capital of France?"}], "n_docs": 5}'

Custom index (your own docs)

Run pixelrag index build with a pixelrag.yaml config file pointing at your own documents — local files, PDFs, web URLs, or a kiwix ZIM archive. The pipeline runs all four stages automatically and produces a local FAISS or Qdrant index ready to serve.
pixelrag index build
pixelrag serve --index-dir ./my_index --port 30001

Build docs developers (and LLMs) love

Get started for freeTalk to us