Skip to main content

Documentation Index

Fetch the complete documentation index at: https://mintlify.com/jundot/omlx/llms.txt

Use this file to discover all available pages before exploring further.

oMLX uses a layered configuration system: every setting has a compiled-in default, which can be overridden by ~/.omlx/settings.json, then by OMLX_* environment variables, and finally by CLI flags passed to omlx serve. Values flow in that order — CLI always wins. When you pass any non-default CLI flag, oMLX writes the merged result back to settings.json automatically so the next bare omlx serve picks it up without requiring you to repeat the flags.

​
Priority order

CLI flags  >  OMLX_* environment variables  >  ~/.omlx/settings.json  >  built-in defaults

​
Settings file

The settings file lives at ~/.omlx/settings.json (or <base-path>/settings.json if you use --base-path). It is created automatically on first run and updated whenever you pass non-default CLI flags. 
{
  "version": "1.0",
  "server": {
    "host": "127.0.0.1",
    "port": 8000,
    "log_level": "info",
    "sse_keepalive_mode": "chunk"
  },
  "model": {
    "model_dirs": ["/Users/you/models"],
    "max_model_memory": "32GB"
  },
  "memory": {
    "max_process_memory": "auto"
  },
  "scheduler": {
    "max_concurrent_requests": 8
  },
  "cache": {
    "enabled": true,
    "ssd_cache_max_size": "auto",
    "hot_cache_max_size": "0",
    "initial_cache_blocks": 256
  },
  "auth": {
    "api_key": null
  },
  "mcp": {
    "config_path": null
  }
}
You can also edit settings.json directly in a text editor. oMLX reads it on every omlx serve invocation. Invalid values cause a validation error at startup with a clear message.

​
Environment variables

All environment variables are prefixed with OMLX_. They override the settings file but are themselves overridden by CLI flags.

​
Server

VariableTypeDefaultDescription
OMLX_HOSTstring127.0.0.1Bind address for the HTTP server.
OMLX_PORTinteger8000TCP port to listen on.
OMLX_LOG_LEVELstringinfoLog verbosity: trace, debug, info, warning, or error.

​
Model

VariableTypeDefaultDescription
OMLX_MODELstring(none)Model name passed to single-model configurations (used by OMLXConfig.from_env).
OMLX_MODEL_DIRstring~/.omlx/modelsComma-separated list of model directories.
OMLX_MAX_MODEL_MEMORYstringautoMaximum memory for loaded models, e.g. 32GB or disabled.
OMLX_MAX_PROCESS_MEMORYstringautoTotal process memory limit: a percentage like 80%, auto, or disabled.
OMLX_TRUST_REMOTE_CODEbooleanfalseAllow execution of custom Python shipped with a model repository (modeling_*.py). Disabled by default for security — only enable for models you trust explicitly. See trust_remote_code security note.

​
Generation

VariableTypeDefaultDescription
OMLX_MAX_TOKENSinteger32768Default maximum tokens to generate per request.
OMLX_TEMPERATUREfloat1.0Default sampling temperature.

​
Scheduler

VariableTypeDefaultDescription
OMLX_MAX_CONCURRENT_REQUESTSinteger8Maximum number of requests processed simultaneously. Also accepted as OMLX_MAX_NUM_SEQS.

​
Cache

VariableTypeDefaultDescription
OMLX_CACHE_ENABLEDbooleanfalseEnable the paged SSD cache (set to true to activate).
OMLX_SSD_CACHE_DIRstring~/.omlx/cacheDirectory for the SSD cold cache tier.
OMLX_SSD_CACHE_MAX_SIZEstringautoMaximum disk space for the SSD cache tier.
OMLX_HOT_CACHE_ONLYbooleanfalseWhen true, serve from the in-memory hot cache only (no disk writes).
OMLX_INITIAL_CACHE_BLOCKSinteger256Pre-allocated cache blocks at startup.

​
MCP

VariableTypeDefaultDescription
OMLX_MCP_CONFIGstring(none)Path to MCP configuration file (JSON or YAML).

​
Authentication

VariableTypeDefaultDescription
OMLX_API_KEYstring(none)API key required on all requests when set.

​
HuggingFace and ModelScope

VariableTypeDefaultDescription
OMLX_HF_ENDPOINTstring(none)Custom HuggingFace Hub URL, e.g. https://hf-mirror.com.
OMLX_MS_ENDPOINTstring(none)Custom ModelScope endpoint URL.

​
Network

VariableTypeDefaultDescription
OMLX_HTTP_PROXYstring(none)HTTP proxy URL.
OMLX_HTTPS_PROXYstring(none)HTTPS proxy URL.
OMLX_NO_PROXYstring(none)Comma-separated bypass list.
OMLX_CA_BUNDLEstring(none)Path to CA bundle PEM for TLS interception.

​
Logging

VariableTypeDefaultDescription
OMLX_LOG_DIRstring~/.omlx/logsOverride the directory where log files are written.
OMLX_LOG_RETENTION_DAYSinteger7Days to keep rotated log files before deletion.

​
Directories

oMLX creates the following directories automatically under ~/.omlx/ (or the directory set by --base-path):
DirectoryPurpose
~/.omlx/modelsDefault model storage. Scanned for MLX model subdirectories.
~/.omlx/logsStructured server log and crash log.
~/.omlx/cacheDefault paged SSD cache directory.
If a model directory cannot be created (for example, a disconnected external drive), oMLX logs a warning and skips it rather than failing. The logs and cache directories are required — a failure to create them is fatal.

​
Log locations

When running brew services start omlx, logs go to two places:
LogLocationContents
Service log$(brew --prefix)/var/log/omlx.logstdout and stderr from the service process
Server log~/.omlx/logs/server.logStructured application log with request IDs
Crash log~/.omlx/logs/crash.logPython thread dumps captured on SIGABRT/SIGSEGV
# Follow the service log
tail -f $(brew --prefix)/var/log/omlx.log

# Follow the structured server log
tail -f ~/.omlx/logs/server.log

​
Homebrew service configuration

When running oMLX as a Homebrew service, CLI flags cannot be passed directly. Use environment variables instead, or run omlx serve once with your desired flags to persist them to settings.json before starting the service. 
# Option 1: persist settings first, then start the service
omlx serve --model-dir ~/models --port 8000 &
# (Ctrl-C after "Saved CLI arguments to settings.json" appears)
brew services start omlx

# Option 2: use environment variables in a launchd plist override
# Set OMLX_MODEL_DIR, OMLX_PORT, etc. in your shell profile or a
# launchd EnvironmentVariables block
The Homebrew service uses zero-config defaults (~/.omlx/models, port 8000) unless settings.json is present or environment variables are set.

Build docs developers (and LLMs) love

Get started for freeTalk to us