Skip to main content

Documentation Index

Fetch the complete documentation index at: https://mintlify.com/jundot/omlx/llms.txt

Use this file to discover all available pages before exploring further.

oMLX exposes three top-level commands — serve, launch, and diagnose — each designed for a distinct part of the workflow. serve starts the inference server and accepts the bulk of the configuration flags. launch wires an external coding tool (such as Codex or OpenCode) to a running oMLX server. diagnose checks installation health and prints recovery steps. All flags shown below reflect the actual argparse defaults in omlx/cli.py.

​
omlx serve

Start a multi-model OpenAI-compatible inference server. Models are discovered automatically from subdirectories of --model-dir. Each subdirectory must contain a valid MLX model with a config.json and one or more .safetensors files. 
omlx serve --model-dir ~/models
omlx serve --model-dir ~/models --max-model-memory 32GB --port 8080
omlx serve --model-dir ~/models --paged-ssd-cache-dir ~/.omlx/cache --hot-cache-max-size 8GB
omlx serve --model-dir ~/models --api-key your-secret-key --log-level debug
FlagTypeDefaultDescription
--model-dirstring~/.omlx/modelsDirectory containing model subdirectories. Accepts a comma-separated list for multiple directories.
--max-model-memorystringauto (80% of system RAM)Maximum memory budget for loaded models. Accepts human-readable sizes like 32GB or the special value disabled to remove the limit entirely.
--max-process-memorystringauto (RAM − 8 GB)Total process memory ceiling. Accepts a percentage (80%), the value auto, or disabled. Valid percentage range is 10–99.
FlagTypeDefaultDescription
--hoststring127.0.0.1Host address to bind. Use 0.0.0.0 to expose the server on all interfaces.
--portinteger8000TCP port to listen on.
--log-levelchoiceinfoVerbosity level. Options: trace, debug, info, warning, error. The trace level includes full message content and maps to debug inside uvicorn.
--sse-keepalive-modechoicechunkControls how SSE keepalive events are emitted. chunk sends protocol-aware no-op events compatible with strict clients such as OpenClaw; comment sends the legacy : keep-alive comment; off disables keepalive entirely.
--api-keystring(none)Optional API key. When set, all requests must include Authorization: Bearer <key>.
--base-pathstring~/.omlxRoot directory for all oMLX data (models, logs, cache, settings).
FlagTypeDefaultDescription
--max-concurrent-requestsinteger8Maximum number of requests processed simultaneously. Higher values increase throughput but consume more memory.
FlagTypeDefaultDescription
--paged-ssd-cache-dirstring(none)Directory for paged SSD cache storage. Providing this flag enables the oMLX prefix cache. Defaults to ~/.omlx/cache when enabled via settings.
--paged-ssd-cache-max-sizestring100GBMaximum disk space the SSD cache may occupy. Accepts sizes like 100GB or 50GB.
--hot-cache-max-sizestring0 (disabled)In-memory hot cache budget. Accepts sizes like 8GB. When set to 0, only the cold SSD tier is used.
--no-cacheflagoffDisables the oMLX paged SSD cache entirely. mlx-lm’s BatchGenerator still manages KV state internally.
--initial-cache-blocksinteger256Number of cache blocks pre-allocated at startup. Higher values reduce dynamic allocation overhead for large contexts.
FlagTypeDefaultDescription
--mcp-configstring(none)Path to a Model Context Protocol configuration file in JSON or YAML format.
FlagTypeDefaultDescription
--hf-endpointstring(none)Custom HuggingFace Hub endpoint. Use https://hf-mirror.com for regions with restricted access to huggingface.co.
--ms-endpointstring(none)Custom ModelScope Hub endpoint URL.
--http-proxystring(none)HTTP proxy URL, e.g. http://proxy.company.com:8080. Sets both HTTP_PROXY and http_proxy environment variables.
--https-proxystring(none)HTTPS proxy URL. Sets both HTTPS_PROXY and https_proxy environment variables.
--no-proxystring(none)Comma-separated list of hosts or IPs to bypass the proxy, e.g. localhost,127.0.0.1.
--ca-bundlestring(none)Path to a CA bundle PEM file for TLS interception environments. Sets REQUESTS_CA_BUNDLE and SSL_CERT_FILE.
When you pass any non-default flag to omlx serve, oMLX automatically saves the resulting configuration to ~/.omlx/settings.json. Subsequent omlx serve invocations without flags will pick up the saved values.

​
omlx launch

Configure and launch an external tool to use the running oMLX server. The command checks that the server is reachable before launching, and if --model is omitted it prompts you to choose from the available models. 
omlx launch opencode
omlx launch codex --model qwen3-coder
omlx launch openclaw --tools-profile full
omlx launch list
ArgumentDescription
toolThe tool to launch. One of claude, codex, opencode, openclaw, pi, or the literal string list to print all available integrations with their install status.
FlagTypeDefaultDescription
--modelstring(interactive)Model to pass to the tool. If omitted and only one model is loaded, it is selected automatically. If multiple models are loaded, you are prompted to choose.
--hoststringfrom settings or 127.0.0.1oMLX server host. Resolved in order: CLI flag → settings.json → default.
--portintegerfrom settings or 8000oMLX server port. Resolved in order: CLI flag → settings.json → default.
--api-keystringfrom settingsAPI key for authenticating with the oMLX server.
--tools-profilechoicecodingOpenClaw tools profile. Options: minimal, coding, messaging, full.

​
omlx diagnose

Run diagnostic checks and print recovery steps for known issues. 
omlx diagnose menubar
ArgumentValueDescription
targetmenubarDiagnoses why the oMLX menubar icon may be hidden on macOS Tahoe (26.x). Reports macOS version, app install path, running PID, and recent ControlCenter visibility log entries. Prints manual recovery steps because Apple’s sandbox policy prevents third-party apps from programmatically re-enabling their own status item.
Example output 
oMLX menubar diagnostics
========================================
macOS:          26.0
Bundle ID:      com.omlx.app
App installed:  yes
Menubar app:    running
PID:            1234

If the icon is missing on macOS Tahoe (26.x):
  1. Open System Settings > Menu Bar
  2. Find 'oMLX' and set it to 'Show in Menu Bar'
  3. If oMLX isn't in the list, quit the menubar app and relaunch oMLX.app

Build docs developers (and LLMs) love

Get started for freeTalk to us