Documentation Index
Fetch the complete documentation index at: https://mintlify.com/StarTrail-org/PixelRAG/llms.txt
Use this file to discover all available pages before exploring further.
pixelrag serve exposes a FastAPI search server that loads a visual index into memory and
answers text, image, or pre-computed-embedding queries via a simple HTTP API. It works with both
locally built indexes and the pre-built Wikipedia indexes from Hugging Face, and it supports
both FAISS (default) and Qdrant backends — the backend is detected automatically from the index
summary.json.
Starting the server
Install the serve extra, then point the server at your index directory:
pip install 'pixelrag[serve]'
pixelrag serve --index-dir ./my_index --port 30001
0.0.0.0:30001 by default and starts accepting requests as soon as the
model and index finish loading.
All flags:
| Flag | Default | Description |
|---|
--index-dir | ./index | Directory containing index.faiss and summary.json |
--tiles-dir | ./tiles | Directory with tile images |
--articles-json | ./articles.json | Path to articles.json with title and URL metadata |
--model | Qwen/Qwen3-VL-Embedding-2B | Embedding model for query encoding |
--device | cpu | Inference device: cpu or cuda |
--peft-adapter | — | Path to a LoRA adapter directory (merged into the model at load time) |
--backend | (from summary.json) | Vector backend: faiss or qdrant (default: read from index summary.json, else faiss) |
--qdrant-url | — | Qdrant server or Cloud URL (required when backend is qdrant) |
--qdrant-api-key | $QDRANT_API_KEY | API key for Qdrant Cloud |
--qdrant-client-config | — | Path to a JSON file of QdrantClient constructor arguments |
--collection | (from summary.json) | Qdrant collection name (default: from summary.json, else pixelrag) |
--host | 0.0.0.0 | Bind address |
--port | 30001 | Bind port |
--render-on-demand | off | Render tile images from a Kiwix ZIM on the fly — no materialized tiles directory required |
--kiwix-url | http://localhost:30900 | Base URL of a running kiwix-serve (for --render-on-demand) |
--zim-book | (auto-derived) | Kiwix book ID for /content/{book}/; auto-derived from --kiwix-url if omitted |
| Variable | Description |
|---|
PIXELRAG_INDEX_MMAP=1 | Memory-map the FAISS index file instead of loading it into RAM — useful for multi-100 GB indexes where startup time and resident memory both matter |
Searching
Text query
Image query
Pre-computed embedding
Post a JSON body with a queries array. Each query object needs a text field:curl -X POST http://localhost:30001/search \
-H "Content-Type: application/json" \
-d '{"queries": [{"text": "capital of France"}], "n_docs": 5}'
Encode the image as base64 and pass it in the image field. The server also accepts
data URLs (data:image/png;base64,...):import base64, json, urllib.request
with open("query_image.png", "rb") as f:
b64 = base64.b64encode(f.read()).decode()
payload = json.dumps({"queries": [{"image": b64}], "n_docs": 5}).encode()
req = urllib.request.Request(
"http://localhost:30001/search",
data=payload,
headers={"Content-Type": "application/json"},
method="POST",
)
with urllib.request.urlopen(req) as r:
print(json.load(r))
If you have already computed a query embedding (e.g. from a batch job), pass it
directly as a list of floats in the embedding field. Every query in the request
must use embedding — mixing it with text or image fields in the same request
is not supported:payload = {
"queries": [{"embedding": [0.012, -0.034, ...]}],
"n_docs": 5,
}
Response structure
A successful /search response contains a results array with one QueryResult per query.
Each result holds a hits list:
{
"results": [
{
"hits": [
{
"score": 0.912,
"vector_id": 104832,
"article_id": 12,
"tile_index": 0,
"chunk_index": 2,
"y_offset": 2048,
"tile_height": 1024,
"path": "12.png.tiles/chunk_0000_02.png",
"url": "https://en.wikipedia.org/wiki/Paris"
}
]
}
]
}
| Field | Description |
|---|
score | Cosine similarity between the query and the retrieved chunk (higher is better) |
vector_id | Raw vector row ID in the FAISS index |
article_id | Document index in articles.json |
tile_index | Which tile within the document (0-based) |
chunk_index | Which 1024-pixel strip within the tile (0-based) |
y_offset | Pixel offset from the top of the tile to this chunk |
tile_height | Pixel height of this chunk |
path | Relative path to the tile image file |
url | Source URL or file path of the document |
article_pages | Tile/chunk coordinates that exist on disk for this article (e.g. "0:0-8,1:0-4"), or null for on-demand indexes |
image_base64 | Base64-encoded tile image, present only when include_images: true |
Search options
All options are fields on the search request body:
| Option | Type | Description |
|---|
n_docs | int | Number of results to return per query (default: 10) |
nprobe | int | FAISS nprobe override — higher values improve recall at the cost of speed |
articles_only | bool | Filter out Wikipedia meta pages (Portal:, Category:, List of …, disambiguation) |
department | string | Restrict search to a subdirectory department (set during pixelrag index build) |
include_images | bool | Include base64-encoded tile images in every hit |
min_tile_height | int | Exclude chunks shorter than this height — filters out partial or blank tiles at page bottoms |
instruction | string | Override the default query embedding instruction sent to the model |
department is a pre-filter that runs inside FAISS via an IDSelector (or as a payload
filter in Qdrant), not a post-filter. This means you always get the full n_docs results as
long as the department has enough indexed tiles.
Hosted Wikipedia API
api.pixelrag.ai serves a pre-built index of 8.28M Wikipedia pages. No authentication is
required — send the same search payload you would send to a local server:
curl -X POST https://api.pixelrag.ai/search \
-H "Content-Type: application/json" \
-d '{"queries": [{"text": "What is the capital of France?"}], "n_docs": 5}'
curl https://api.pixelrag.ai/status
"image" with a base64-encoded PNG or
JPEG just as you would against a local server.