Skip to main content

Overview

Off Grid’s Android implementation uses native Kotlin modules bridged to React Native via JNI. The platform leverages Qualcomm’s QNN NPU for image generation, OpenCL for GPU-accelerated LLM inference, and Android’s native APIs for PDF extraction and background downloads.

Native Modules

LocalDreamModule

Manages Stable Diffusion inference on Android using the local-dream C++ library with MNN and QNN backends. File: android/app/src/main/java/ai/offgridmobile/localdream/LocalDreamModule.kt Architecture:
  • Spawns libstable_diffusion_core.so as a subprocess
  • Subprocess runs HTTP server on localhost:18081
  • TypeScript layer communicates via HTTP for generation requests
  • Module handles: process lifecycle, QNN library extraction, RGB→PNG conversion
Backend Selection:
All ARM64 devices
  • Alibaba’s MNN framework
  • CPU-only inference
  • Models use .mnn file extension (e.g., unet.mnn, clip.mnn, vae_decoder.mnn)
  • Typical performance: ~15s for 512×512 @ 20 steps (Snapdragon 8 Gen 3)
NPU Detection:
Detects Qualcomm Snapdragon SoCs via Build.SOC_MODEL (API 31+). Examples: SM8450 (8 Gen 1), SM8550 (8 Gen 2), SM8650 (8 Gen 3). QNN Runtime Libraries: Extracted from assets/qnnlibs/ to app’s filesDir/runtime_libs/ on first load:
  • libQnnHtp.so — Main QNN HTP (Hexagon Tensor Processor) backend
  • libQnnHtpV68.so through libQnnHtpV81.so — Chipset-specific variants
  • libQnnSystem.so — System backend
  • Skel/Stub variants for FastRPC communication with DSP
Environment Setup:
Setting DSP_LIBRARY_PATH and ADSP_LIBRARY_PATH is critical for QNN to locate DSP firmware. Without these, QNN initialization fails with SELinux errors.
Image Generation Flow:
  1. TypeScript calls LocalDreamModule.generateImage(params)
  2. Module checks server health (http://127.0.0.1:18081/health)
  3. Posts JSON request to /generate endpoint
  4. Server returns Server-Sent Events (SSE) stream
  5. Module parses SSE events:
    • progress → emits LocalDreamProgress with step count and optional preview image
    • complete → returns final base64 RGB image
  6. RGB data converted to PNG via Bitmap.createBitmap() and saved to filesDir/generated_images/

DownloadManager Module

Handles background model downloads using Android’s native DownloadManager API. File: android/app/src/main/java/ai/offgridmobile/download/DownloadManagerModule.kt Features:
  • Background downloads survive app kill and device reboot
  • Native notification with progress bar
  • Automatic retry on network interruption
  • Download state persisted in SharedPreferences
  • 500ms polling interval for progress updates
Key Methods:
Race Condition Fix:
On slow emulators, download completion notification could arrive before React Native received the event. Fixed by tracking event delivery separately.

PdfExtractorModule

Extracts text from PDF files using PdfiumAndroid (Chromium’s PDF renderer). File: android/app/src/main/java/ai/offgridmobile/pdf/PDFExtractorModule.kt Implementation:
Dependency:
PdfiumAndroid provides native bindings to Chromium’s PDFium library (same engine as Chrome’s PDF viewer).

Hardware Acceleration

OpenCL GPU Offloading (LLM Inference)

Llama.cpp supports OpenCL GPU acceleration on Qualcomm Adreno GPUs via the llama.rn native module. Configuration:
  • User sets GPU layers (0-99) in model settings
  • llama.cpp offloads first N transformer layers to GPU
  • Remaining layers run on CPU
  • Automatic fallback to CPU if OpenCL initialization fails
Performance:
  • Flagship devices (Adreno 740+): 20-40 tok/s with GPU layers
  • CPU-only: 15-30 tok/s (ARM NEON, i8mm, dotprod SIMD)
OpenCL backend is experimental. Some devices may crash during layer offload initialization. Start with 0 GPU layers and incrementally increase while monitoring stability.
Compatibility Notes:
  • Flash attention automatically disabled when GPU layers > 0 (llama.cpp limitation)
  • Devices with ≤4GB RAM: GPU layers forced to 0 to prevent Metal/OpenCL abort() crashes

QNN NPU Acceleration (Image Generation)

Qualcomm AI Engine Direct (QNN) accelerates Stable Diffusion inference on the Hexagon DSP (Neural Processing Unit). Supported Chipsets:
  • Snapdragon 8 Gen 1 (SM8450)
  • Snapdragon 8 Gen 2 (SM8550)
  • Snapdragon 8 Gen 3 (SM8650)
  • Snapdragon 8 Gen 4 (SM8750)
  • Snapdragon 8 Gen 5 (SM8850)
Model Variants:
  • min — Non-flagship SoCs (8 Gen 1)
  • 8gen1 — 8 Gen 1 optimized
  • 8gen2 — 8 Gen 2/3/4/5 optimized (uses V75+ HTP)
QNN Architecture:
Performance Characteristics:
  • QNN: ~5-10s for 512×512 @ 20 steps
  • MNN (CPU fallback): ~15s on same device
  • 2-3x speedup from NPU acceleration

Build Configuration

Gradle Build

File: android/app/build.gradle

Native Libraries

Location: android/app/src/main/jniLibs/arm64-v8a/
  • libstable_diffusion_core.so — local-dream Stable Diffusion inference engine
APK Assets:
  • assets/qnnlibs/ — QNN runtime libraries (19 files, ~50MB total)
  • assets/ggml-hexagon/ — Optional GGML Hexagon backend for LLM inference
Build Output: APK includes:
  • React Native Hermes bytecode bundle
  • Native Kotlin/Java bytecode
  • llama.rn ARM64 binaries (llama.cpp + OpenCL)
  • whisper.rn ARM64 binaries (whisper.cpp)
  • local-dream C++ library
  • QNN runtime libraries
Typical APK size: ~150MB (after compression)

Performance Tuning

Image Generation

CPU Threads:
MNN backend parallelizes UNet inference across CPU threads. Optimal thread count: 4-6 on octa-core SoCs. QNN Cache Warmup: First generation after model load takes 120s+ for QNN to build and cache the compute graph. Subsequent generations are 5-10s.

Download Management

Parallel Downloads: Android’s DownloadManager supports multiple concurrent downloads. Off Grid limits to 2 simultaneous downloads to prevent network saturation. Network Policy:
Downloads proceed over cellular data and roaming. Users can pause via notification.

References

  • llama.cpp OpenCL: https://github.com/ggerganov/llama.cpp/blob/master/docs/backend/OpenCL.md
  • Qualcomm QNN: https://www.qualcomm.com/developer/software/qualcomm-ai-engine-direct
  • PdfiumAndroid: https://github.com/mshockwave/PdfiumAndroid
  • Android DownloadManager: https://developer.android.com/reference/android/app/DownloadManager
  • local-dream: https://github.com/xingchensong/local-dream (MNN/QNN Stable Diffusion)