Local model inference lets you generate images and videos entirely on your own hardware, without sending prompts to any cloud API. This capability is only available in the desktop app — the hosted web version at muapi.ai always uses cloud APIs. If you are running the Next.js development server or the hosted version, the local engines are not available. The desktop app ships two independent local engines. They do not overlap: sd.cpp handles image models on the same machine the app runs on, while Wan2GP is an HTTP client to a separate GPU server you operate yourself. Pick whichever fits your hardware.Documentation Index
Fetch the complete documentation index at: https://mintlify.com/anil-matcha/open-generative-ai/llms.txt
Use this file to discover all available pages before exploring further.
Local inference is exclusive to the desktop app (macOS, Windows, Linux). The hosted web version always routes requests through cloud APIs. Download the desktop app to use local models.
Engines at a Glance
| Engine | What it is | Best for |
|---|---|---|
| sd.cpp (bundled) | C++ engine from stable-diffusion.cpp, runs on the same machine as the app. Metal GPU on Apple Silicon; CUDA, Vulkan, or ROCm on Linux/Windows. | Image-only models. Works on Mac M-series without any external server. |
| Wan2GP (BYO server) | HTTP client to a user-run Wan2GP Gradio server. The server runs Python + PyTorch on a CUDA/ROCm GPU; the desktop app only sends prompts and receives results. | Video models (Wan 2.2, Hunyuan Video, LTX Video) and large image models (Flux.1 Dev, Qwen Image). Requires a NVIDIA/AMD GPU on the server; the desktop app itself can run on a Mac. |
Hardware Notes
- sd.cpp runs on CPU (all platforms) with optional Metal GPU acceleration on Apple Silicon (M1/M2/M3/M4). CUDA, Vulkan, and ROCm are supported on Linux and Windows.
- Metal GPU acceleration is built into the macOS desktop binary — it is significantly faster than CPU-only. On an M2 Mac, expect roughly 1–2 s/step with Metal active for SD 1.5 models. If you see ~10 s/step instead, the binary may have fallen back to CPU.
- Z-Image models (Turbo and Base) are memory-intensive. 16 GB RAM is recommended (the weights alone total ~7.4 GB, plus a 2.4 GB compute buffer). On a base 8 GB M-series Mac, Z-Image is known to hang the system — use an SD 1.5 model instead.
- Wan2GP requires a CUDA or ROCm GPU on the server machine. It does not support Apple Silicon MPS, so it cannot run on the same Mac as the desktop app — but it can run on a LAN PC, a rented RunPod instance, or any other CUDA/ROCm machine you can reach over HTTP.
How to Configure
Open Settings → Local Models inside the desktop app. From there you can:- Install the sd.cpp inference engine with a single click
- Download individual model weights (and auxiliary files for Z-Image models)
- Enter and test the URL of a running Wan2GP Gradio server
- See the resolved model folder path on disk
Explore the Engines
sd.cpp Engine
Bundled C++ engine for local image generation on Mac, Windows, and Linux. Supports SD 1.5, SDXL, and Z-Image Turbo/Base with Metal, CUDA, Vulkan, or ROCm acceleration.
Wan2GP Server
Connect the desktop app to a self-hosted Wan2GP Gradio server for video generation (Wan 2.2, Hunyuan, LTX) and large image models (Flux.1 Dev, Qwen Image) on CUDA/ROCm GPUs.