pixelrag CLI: Stage Commands, Options, and pip Extras

pixelrag is the umbrella CLI that dispatches to each pipeline stage. Each stage is a subcommand backed by an independently installable package; install the relevant extra to enable it. Stage 0 (rendering pages to screenshot tiles) is handled by the standalone pixelshot command, not a subcommand here.

Synopsis

pixelrag <stage> [args...]

Stages

Stage	Description	Requires extra
`chunk`	Split 8192 px tiles into 1024 px chunks	`embed`
`embed`	Embed tile shards with Qwen3-VL	`embed`
`build-index`	Merge embeddings into a FAISS or Qdrant index	`embed`
`index`	Run the full pipeline: source → render → chunk → embed → index	`index`
`monitor`	Monitor pipeline progress	`index`
`serve`	Start the FastAPI search server	`serve`

pixelrag chunk

Pre-chunks tile images into 1024 px strips on disk. For each *.png.tiles/ directory, it reads every tile_XXXX.jpg and splits it into chunk_XXXX_YY.png files, then writes a chunks.json manifest. Requires the embed extra.

--tiles-dir

string

Process all shard directories found under this path. Mutually exclusive with --shard-dir. Use this for batch operations over a full tiles output directory.

--shard-dir

string

Process a single shard directory. Mutually exclusive with --tiles-dir.

Exactly one of --tiles-dir or --shard-dir is required.

--workers

integer

default:"96"

Number of parallel shard worker processes when using --tiles-dir.

--force

boolean

Rechunk even when chunks.json already exists. Compares tile MD5 hashes against the stored values and skips unchanged tiles, so this is safe to run incrementally.

--delete-tiles

boolean

Delete the original tile_*.jpg files after each shard is chunked. Saves significant disk space when tiles are no longer needed for re-rendering.

--dry-run

boolean

Count chunks without writing any files to disk. Useful for estimating output size before committing.

Example

pixelrag chunk --tiles-dir ./tiles --workers 8

pixelrag embed

Embeds tile shards using Qwen3-VL-Embedding-2B and writes per-shard .npz files. Requires the embed extra.

--shard-dir

string

required

Path to the input shard directory containing *.png.tiles/ subdirectories.

--output-dir

string

required

Directory where the output shard .npz embedding files are written.

--gpu-ids

string

default:"all"

Comma-separated GPU IDs to use, or all to use every visible GPU. For example: 0,1,2,3.

--batch-size

integer

default:"128"

Number of images passed to the embedding model per call.

--backend

string

default:"sglang"

Inference backend for embedding.

Value	Description
`sglang`	SGLang engine — recommended for GPU, highest throughput
`vllm`	vLLM pooling engine
`direct_gpu`	Direct Transformers inference; required for `--adapter`

--mode

string

default:"chunks"

Embedding granularity.

Value	Description
`chunks`	Embed pre-chunked 1024 px strips (default, ~8× fewer tokens per tile)
`tiles`	Embed full 8192 px tiles

Example

pixelrag embed --shard-dir ./tiles --output-dir ./embeddings --gpu-ids 0,1

pixelrag build-index

Merges all shard .npz embedding files, deduplicates vectors, and builds a FAISS or Qdrant vector index. Requires the embed extra.

--embeddings-dir

string

default:"./data/embeddings"

Directory containing the shard .npz files produced by pixelrag embed.

--output-dir

string

default:"./output/search_index"

Output directory where index files and summary.json are written.

--nlist

integer

default:"4096"

Number of IVF clusters for the FAISS index. A good starting point is n_vectors / 40. The pipeline auto-tunes this when called from pixelrag index build.

--nprobe

integer

default:"128"

Default search nprobe stored in the index. Higher values increase recall at the cost of query latency.

--backend

string

default:"faiss"

Vector index backend.Choices: faiss, qdrant

--qdrant-url

string

Qdrant server or Cloud URL. Required when --backend qdrant.

--collection

string

default:"pixelrag"

Qdrant collection name.

--append

boolean

Upsert into an existing Qdrant collection rather than failing. Mutually exclusive with --recreate.

--recreate

boolean

Delete and recreate the Qdrant collection before uploading. Mutually exclusive with --append.

--qdrant-quantization-config

string

Path to a JSON file containing a Qdrant quantization_config object. Applied when creating a new or recreated collection.

Example

pixelrag build-index --embeddings-dir ./embeddings --output-dir ./index

pixelrag index

The full-pipeline orchestrator. Reads pixelrag.yaml, then runs all four stages in order: source ingestion → render (via pixelshot) → chunk → embed → build index. Requires the index extra.

command

string

required

Sub-command to run. Currently build is the only supported command.

--config

string

Path to a pixelrag.yaml configuration file. When omitted, the pipeline searches the current directory and common parent directories for pixelrag.yaml. Short form: -c.

--limit

integer

Process only the first N documents from the source. Useful for smoke-testing a configuration before committing to a full index build. Short form: -n.

--force

boolean

Delete existing tiles and embeddings and rebuild from scratch. Without this flag, the pipeline skips documents whose tile directories already exist and whose source identity hasn’t changed. Short form: -f.

pixelrag index build

pixelrag serve

Starts the FastAPI search server backed by a pre-built FAISS or Qdrant index. Requires the serve extra.

--index-dir

string

default:"./index"

Directory containing index.faiss (or summary.json for Qdrant) and metadata.npz. Overrides the PIXELRAG_INDEX_DIR environment variable.

--tiles-dir

string

default:"./tiles"

Directory of rendered tile images, served at /tile/{article_id}/{tile_index}/{chunk_index}. Overrides the PIXELRAG_TILES_DIR environment variable.

--articles-json

string

default:"./articles.json"

Path to the articles.json mapping file produced by pixelrag index build. Overrides the PIXELRAG_ARTICLES_JSON environment variable.

--model

string

default:"Qwen/Qwen3-VL-Embedding-2B"

HuggingFace model name or local path for query embedding. Must match the model used during pixelrag embed.

--device

string

default:"cpu"

Device for query embedding inference.Choices: cpu, cuda

--peft-adapter

string

Path to a PEFT/LoRA adapter directory. The adapter is merged into the base model weights at startup via merge_and_unload, so inference runs at base-model speed.

--host

string

default:"0.0.0.0"

Host address for the uvicorn server.

--port

integer

default:"30001"

Port for the uvicorn server.

--backend

string

Vector backend override. When omitted, the backend is read from summary.json in --index-dir, falling back to faiss.Choices: faiss, qdrant

--qdrant-url

string

Qdrant server or Cloud URL. Required when using the Qdrant backend.

--qdrant-api-key

string

Qdrant API key. Can also be set via the QDRANT_API_KEY environment variable.

--qdrant-client-config

string

Path to a JSON file containing QdrantClient constructor arguments. Merged with --qdrant-url and --qdrant-api-key.

--collection

string

Qdrant collection name. When omitted, the value from summary.json is used, falling back to pixelrag.

--render-on-demand

boolean

Render tile images on the fly from a running kiwix-serve instance instead of reading a pre-materialized tiles/ directory. Only pages that are actually retrieved get rendered and cached locally.

--kiwix-url

string

default:"http://localhost:30900"

Base URL of a running kiwix-serve, used when --render-on-demand is set. Overrides the PIXELRAG_KIWIX_URL environment variable.

--zim-book

string

kiwix book ID for /content/<book>/ URL construction (e.g. wikipedia_en_all_maxi_2024-01). Auto-derived from the kiwix catalog when omitted. Overrides the PIXELRAG_ZIM_BOOK environment variable.

pixelrag serve --index-dir ./index --port 30001

Install Hints

If you run a stage whose package is not installed, pixelrag prints the exact install command and exits with code 1:

pixelrag: stage 'embed' is not installed.
  → uv sync --package pixelrag-embed   (dev)
  → pip install 'pixelrag[embed]'   (published)

Which extra installs which stage?

pip extra	Stages enabled
(none, base install)	`pixelshot` command only
`embed`	`chunk`, `embed`, `build-index`
`index`	`index`, `monitor`
`serve`	`serve`

Install all stages at once:

pip install 'pixelrag[embed,index,serve]'

REST API

Python SDK

CLI Reference

pixelrag CLI: Stage Commands, Options, and pip Extras

Synopsis

Stages

pixelrag chunk

pixelrag embed

pixelrag build-index

pixelrag index

pixelrag serve

Install Hints

Build docs developers (and LLMs) love

REST API

Python SDK

CLI Reference

Documentation Index

​Synopsis

​Stages

​pixelrag chunk

​pixelrag embed

​pixelrag build-index

​pixelrag index

​pixelrag serve

​Install Hints

Build docs developers (and LLMs) love

Synopsis

Stages

pixelrag chunk

pixelrag embed

pixelrag build-index

pixelrag index

pixelrag serve

Install Hints