Linux Platform Support

Documentation Index

Fetch the complete documentation index at: https://mintlify.com/AlexsJones/llmfit/llms.txt

Use this file to discover all available pages before exploring further.

​

GPU Detection Methods

​

NVIDIA GPUs

​

1. nvidia-smi (Primary)

# Multi-GPU query with addressing mode (unified memory detection)
nvidia-smi --query-gpu=addressing_mode,memory.total,name --format=csv,noheader,nounits

# Fallback query (older nvidia-smi versions)
nvidia-smi --query-gpu=memory.total,name --format=csv,noheader,nounits

• NVIDIA Grace/DGX Spark (GB10, GB20): addressing_mode returns “ATS” (Address Translation Services)
• When VRAM is unavailable, falls back to /proc/meminfo for total RAM

• Aggregates same-model GPUs (e.g., 2x RTX 4090 → count=2, per-card VRAM=24GB)
• Total VRAM = per-card VRAM × count (for tensor splitting across cards)

​

2. sysfs Fallback (Containers/Toolbx)

# Check vendor ID in /sys/class/drm/card*/device/
cat /sys/class/drm/card0/device/vendor
# 0x10de = NVIDIA

# Read VRAM
cat /sys/class/drm/card0/device/mem_info_vram_total

• lspci -nn queries PCI slot from sysfs uevent file
• Matches GPU model from lspci output (e.g., “GeForce RTX 2060”)
• Fallback to flatpak-spawn --host lspci in Flatpak environments

• Checks DRIVER= in sysfs uevent file
• nvidia driver → CUDA backend
• Other drivers (nouveau, etc.) → Vulkan backend

​

AMD GPUs

​

1. rocm-smi (ROCm installed)

# VRAM detection
rocm-smi --showmeminfo vram
# Parses "Total Memory (B): 8589934592"

# GPU name
rocm-smi --showproductname
# Looks for "Card Series" or "Card Model" lines

• Tracks per-GPU VRAM from multiple “Total” lines
• Backend: ROCm

​

2. sysfs (No ROCm)

# Check vendor ID
cat /sys/class/drm/card0/device/vendor
# 0x1002 = AMD

# Read VRAM
cat /sys/class/drm/card0/device/mem_info_vram_total

• lspci -nn for AMD/ATI VGA/3D controllers
• Backend: Vulkan (no ROCm)

​

3. Unified Memory APUs

# Detection: CPU name contains "Ryzen AI"
lscpu | grep "Model name"
# AMD Ryzen AI MAX+ 395 w/ Radeon 8060S
# AMD Ryzen AI 9 HX 370 w/ Radeon 890M

• VRAM = total system RAM (unified pool)
• Backend: Vulkan
• unified_memory flag set to true

​

Intel Arc GPUs

​

Discrete GPUs (Arc A770, A370M)

# Check vendor ID
cat /sys/class/drm/card0/device/vendor
# 0x8086 = Intel

# Read VRAM
cat /sys/class/drm/card0/device/mem_info_vram_total

​

Integrated GPUs (Arc Graphics in Meteor Lake)

# lspci detection
lspci -nn | grep -i "intel.*arc"
# 00:02.0 VGA compatible controller [0300]: Intel Corporation Arc Graphics [8086:7d55]

• Discrete: Reports exact VRAM
• Integrated: Reports 0 VRAM (shares system memory)
• Backend: SYCL (Intel oneAPI)

​

Ascend NPUs

# List NPU IDs
npu-smi info -l
# NPU ID : 0
# NPU ID : 1

# Per-NPU memory info
npu-smi info -t memory -i 0
# HBM Capacity(MB) : 65536

• Each NPU tracked separately (count=1 per NPU)
• VRAM = HBM capacity in MB → GB
• Backend: Ascend

​

Installation Methods

​

Homebrew (Recommended)

brew install llmfit

​

Quick Install Script

# Install to /usr/local/bin (requires sudo)
curl -fsSL https://llmfit.axjns.dev/install.sh | sh

# Install to ~/.local/bin (no sudo)
curl -fsSL https://llmfit.axjns.dev/install.sh | sh -s -- --local

​

From Source

git clone https://github.com/AlexsJones/llmfit.git
cd llmfit
cargo build --release
sudo cp target/release/llmfit /usr/local/bin/

​

Troubleshooting

​

GPU Not Detected

1. Check if nvidia-smi works:
   nvidia-smi
   
2. If nvidia-smi fails, check sysfs:
   ls /sys/class/drm/card*/device/vendor
   cat /sys/class/drm/card0/device/vendor  # Should be 0x10de
   
3. Install nvidia-utils package:
   # Debian/Ubuntu
   sudo apt install nvidia-utils
   
   # Fedora
   sudo dnf install nvidia-driver
   
   # Arch
   sudo pacman -S nvidia-utils
   
4. Manual override:
   llmfit --memory=24G system
   

1. Check if rocm-smi works:
   rocm-smi --showmeminfo vram
   
2. Install ROCm:
   # Ubuntu
   wget https://repo.radeon.com/amdgpu-install/latest/ubuntu/jammy/amdgpu-install_*.deb
   sudo dpkg -i amdgpu-install_*.deb
   sudo amdgpu-install --usecase=rocm
   
   # Check /sys/class/drm for AMD cards
   grep -l 0x1002 /sys/class/drm/card*/device/vendor
   
3. Sysfs fallback (no ROCm):
   cat /sys/class/drm/card0/device/mem_info_vram_total
   

1. Check sysfs vendor ID:
   grep -l 0x8086 /sys/class/drm/card*/device/vendor
   
2. Check lspci:
   lspci | grep -i "intel.*arc"
   
3. Install Intel compute runtime:
   # Ubuntu
   sudo apt install intel-opencl-icd
   

​

VRAM Shows 0 or Wrong Value

# Check if driver is loaded
lsmod | grep nvidia

# Reinstall driver
sudo apt reinstall nvidia-driver-535  # or your version

llmfit --memory=16G

ls /sys/class/drm/card0/device/
# Should see vendor, mem_info_vram_total, etc.

​

WSL-Specific Issues

echo $WSL_DISTRO_NAME
cat /proc/version | grep -i microsoft

• Install NVIDIA CUDA on WSL2: https://docs.nvidia.com/cuda/wsl-user-guide/
• nvidia-smi should work once CUDA drivers are installed

• Limited support; ROCm not officially supported in WSL
• Use CPU-only mode or run llmfit natively on Windows

​

Multi-GPU Detection Issues

nvidia-smi --query-gpu=memory.total,name --format=csv,noheader,nounits

24564, NVIDIA GeForce RTX 4090
24564, NVIDIA GeForce RTX 4090

llmfit system
# GPU 1: NVIDIA GeForce RTX 4090 (24.00 GB VRAM, CUDA)
# GPU 2: NVIDIA GeForce RTX 3060 (12.00 GB VRAM, CUDA)

​

Next Steps

• Platform Overview
• macOS Platform Guide
• Windows Platform Guide