Skip to main content

Documentation Index

Fetch the complete documentation index at: https://mintlify.com/StarTrail-org/PixelRAG/llms.txt

Use this file to discover all available pages before exploring further.

pixelrag_index orchestrates the full pipeline (source → render → chunk → embed → index) from a pixelrag.yaml config. It is the highest-level entry point in the PixelRAG toolkit. The pixelrag index build CLI command and the build() Python function both call the same four-stage pipeline: render documents to tiles, chunk tiles into 1024 px strips, embed chunks with Qwen3-VL-Embedding-2B, and build a FAISS (or Qdrant) index. Install with pip install 'pixelrag[index]'.

​
build()

from pixelrag_index.pipelines import build
from pixelrag_index.config import load_config

config = load_config("pixelrag.yaml")
output_path = build(config, limit=100, force=False)
Runs the full four-stage pipeline and returns the path to the output directory.

​
Parameters

​
config
dict
required
Configuration dictionary as returned by load_config(). Controls source, rendering, embedding, and index parameters.
​
limit
integer or None
default:"None"
Process only the first N documents from the source. Useful for testing a config without running the full build. None (default) processes all documents.
​
force
boolean
default:"False"
When True, delete any existing tiles/ and embeddings/ directories before starting. Useful for a clean rebuild when the source set has changed significantly. Without --force, the pipeline skips already-rendered tiles and reuses existing embeddings.

​
Returns

Path — the output directory (same as config["output"]), containing index.faiss, metadata.npz, summary.json, and articles.json.

​
Stages

  1. Render — calls pixelrag_render.render_urls (for web/kiwix sources) or render_pdf (for PDF sources) to produce tiles/ from each document.
  2. Chunk — runs pixelrag_embed.chunk to split 8192 px tiles into 1024 px strips and write chunks.json manifests.
  3. Embed — runs pixelrag_embed.embed (GPU) or pixelrag_embed.embed_cpu (CPU/MPS) to produce .npz embedding shards in embeddings/.
  4. Index — runs pixelrag_embed.index build to merge shards and build the FAISS index (or upload to Qdrant).

​
Example

from pixelrag_index.pipelines import build
from pixelrag_index.config import load_config

# Load config and run the pipeline on the first 50 documents
config = load_config("pixelrag.yaml")
output = build(config, limit=50, force=False)
print(f"Index written to: {output}")

# Force a clean rebuild
output = build(config, force=True)

​
load_config()

from pixelrag_index.config import load_config

config = load_config()              # auto-discover pixelrag.yaml / pixelrag.yml
config = load_config("my.yaml")     # explicit path
Load a pixelrag.yaml configuration file and merge it with the built-in defaults. Returns a plain dict.

​
Parameters

​
path
string or None
default:"None"
Path to a YAML config file. When None, the function searches for pixelrag.yaml then pixelrag.yml in the current working directory. If neither is found, the built-in defaults are returned as-is.

​
Returns

dict with the merged config. Built-in defaults are: 
{
    "ingest": {"backend": "cdp", "quality": 85, "tile_height": 8192},
    "embed":  {"model": "Qwen/Qwen3-VL-Embedding-2B", "device": "cuda"},
    "output": "./index",
}
Any keys present in the YAML file override the corresponding defaults.

​
Example pixelrag.yaml

source:
  type: local       # local | web | kiwix | pdf
  path: ./my_docs

embed:
  model: Qwen/Qwen3-VL-Embedding-2B
  device: auto      # cuda on Linux, mps on macOS, cpu as fallback

output: ./my_index

​
CLI Reference

​
pixelrag index build

Run the full pipeline from a config file. 
pixelrag index build [OPTIONS]
OptionDefaultDescription
--config, -cauto-discoverPath to pixelrag.yaml
--source, -sfrom configSource path (overrides source.path in config)
--source-typefrom configSource type: local, web, kiwix, pdf
--output, -ofrom configOutput directory
--devicefrom configEmbedding device: auto, cpu, mps, cuda
--limit, -nnoneMaximum number of documents to process
--force, -ffalseDelete existing tiles and embeddings for a clean rebuild
Examples: 
# Build from config
pixelrag index build

# Limit to 100 documents for a test run
pixelrag index build --limit 100

# Force rebuild from scratch
pixelrag index build --force

# Override config path and output directory
pixelrag index build --config ./configs/wiki.yaml --output ./wiki_index

​
pixelrag monitor

Monitor live pipeline progress. Shows per-stage status and throughput. 
pixelrag monitor
pixelrag monitor tracks progress by watching the output directory structure. Run it in a separate terminal while pixelrag index build is running.

​
End-to-End Example

from pixelrag_index.pipelines import build
from pixelrag_index.config import load_config

# 1. Write a config
import yaml
from pathlib import Path

config_path = Path("pixelrag.yaml")
config_path.write_text(yaml.dump({
    "source": {"type": "local", "path": "./docs"},
    "embed": {"model": "Qwen/Qwen3-VL-Embedding-2B", "device": "auto"},
    "output": "./my_index",
}))

# 2. Load config and build
config = load_config(str(config_path))
output = build(config)
print(f"Done! Serve with: pixelrag serve --index-dir {output}")
After the build completes, serve the index: 
pixelrag serve --index-dir ./my_index --port 30001

Build docs developers (and LLMs) love

Get started for freeTalk to us