Overview
Off Grid’s Android implementation uses native Kotlin modules bridged to React Native via JNI. The platform leverages Qualcomm’s QNN NPU for image generation, OpenCL for GPU-accelerated LLM inference, and Android’s native APIs for PDF extraction and background downloads.Native Modules
LocalDreamModule
Manages Stable Diffusion inference on Android using the local-dream C++ library with MNN and QNN backends. File:android/app/src/main/java/ai/offgridmobile/localdream/LocalDreamModule.kt
Architecture:
- Spawns
libstable_diffusion_core.soas a subprocess - Subprocess runs HTTP server on
localhost:18081 - TypeScript layer communicates via HTTP for generation requests
- Module handles: process lifecycle, QNN library extraction, RGB→PNG conversion
- MNN (CPU)
- QNN (NPU)
All ARM64 devices
- Alibaba’s MNN framework
- CPU-only inference
- Models use
.mnnfile extension (e.g.,unet.mnn,clip.mnn,vae_decoder.mnn) - Typical performance: ~15s for 512×512 @ 20 steps (Snapdragon 8 Gen 3)
Build.SOC_MODEL (API 31+). Examples: SM8450 (8 Gen 1), SM8550 (8 Gen 2), SM8650 (8 Gen 3).
QNN Runtime Libraries:
Extracted from assets/qnnlibs/ to app’s filesDir/runtime_libs/ on first load:
libQnnHtp.so— Main QNN HTP (Hexagon Tensor Processor) backendlibQnnHtpV68.sothroughlibQnnHtpV81.so— Chipset-specific variantslibQnnSystem.so— System backend- Skel/Stub variants for FastRPC communication with DSP
Setting
DSP_LIBRARY_PATH and ADSP_LIBRARY_PATH is critical for QNN to locate DSP firmware. Without these, QNN initialization fails with SELinux errors.- TypeScript calls
LocalDreamModule.generateImage(params) - Module checks server health (
http://127.0.0.1:18081/health) - Posts JSON request to
/generateendpoint - Server returns Server-Sent Events (SSE) stream
- Module parses SSE events:
progress→ emitsLocalDreamProgresswith step count and optional preview imagecomplete→ returns final base64 RGB image
- RGB data converted to PNG via
Bitmap.createBitmap()and saved tofilesDir/generated_images/
DownloadManager Module
Handles background model downloads using Android’s nativeDownloadManager API.
File: android/app/src/main/java/ai/offgridmobile/download/DownloadManagerModule.kt
Features:
- Background downloads survive app kill and device reboot
- Native notification with progress bar
- Automatic retry on network interruption
- Download state persisted in
SharedPreferences - 500ms polling interval for progress updates
- startDownload
- Progress Polling
PdfExtractorModule
Extracts text from PDF files using PdfiumAndroid (Chromium’s PDF renderer). File:android/app/src/main/java/ai/offgridmobile/pdf/PDFExtractorModule.kt
Implementation:
Hardware Acceleration
OpenCL GPU Offloading (LLM Inference)
Llama.cpp supports OpenCL GPU acceleration on Qualcomm Adreno GPUs via thellama.rn native module.
Configuration:
- User sets GPU layers (0-99) in model settings
- llama.cpp offloads first N transformer layers to GPU
- Remaining layers run on CPU
- Automatic fallback to CPU if OpenCL initialization fails
- Flagship devices (Adreno 740+): 20-40 tok/s with GPU layers
- CPU-only: 15-30 tok/s (ARM NEON, i8mm, dotprod SIMD)
- Flash attention automatically disabled when GPU layers > 0 (llama.cpp limitation)
- Devices with ≤4GB RAM: GPU layers forced to 0 to prevent Metal/OpenCL
abort()crashes
QNN NPU Acceleration (Image Generation)
Qualcomm AI Engine Direct (QNN) accelerates Stable Diffusion inference on the Hexagon DSP (Neural Processing Unit). Supported Chipsets:- Snapdragon 8 Gen 1 (SM8450)
- Snapdragon 8 Gen 2 (SM8550)
- Snapdragon 8 Gen 3 (SM8650)
- Snapdragon 8 Gen 4 (SM8750)
- Snapdragon 8 Gen 5 (SM8850)
min— Non-flagship SoCs (8 Gen 1)8gen1— 8 Gen 1 optimized8gen2— 8 Gen 2/3/4/5 optimized (uses V75+ HTP)
- QNN: ~5-10s for 512×512 @ 20 steps
- MNN (CPU fallback): ~15s on same device
- 2-3x speedup from NPU acceleration
Build Configuration
Gradle Build
File:android/app/build.gradle
Native Libraries
Location:android/app/src/main/jniLibs/arm64-v8a/
libstable_diffusion_core.so— local-dream Stable Diffusion inference engine
assets/qnnlibs/— QNN runtime libraries (19 files, ~50MB total)assets/ggml-hexagon/— Optional GGML Hexagon backend for LLM inference
- React Native Hermes bytecode bundle
- Native Kotlin/Java bytecode
- llama.rn ARM64 binaries (llama.cpp + OpenCL)
- whisper.rn ARM64 binaries (whisper.cpp)
- local-dream C++ library
- QNN runtime libraries
Performance Tuning
Image Generation
CPU Threads:Download Management
Parallel Downloads: Android’s DownloadManager supports multiple concurrent downloads. Off Grid limits to 2 simultaneous downloads to prevent network saturation. Network Policy:References
- llama.cpp OpenCL:
https://github.com/ggerganov/llama.cpp/blob/master/docs/backend/OpenCL.md - Qualcomm QNN:
https://www.qualcomm.com/developer/software/qualcomm-ai-engine-direct - PdfiumAndroid:
https://github.com/mshockwave/PdfiumAndroid - Android DownloadManager:
https://developer.android.com/reference/android/app/DownloadManager - local-dream:
https://github.com/xingchensong/local-dream(MNN/QNN Stable Diffusion)