Skip to main content

Documentation Index

Fetch the complete documentation index at: https://mintlify.com/Deepak-Sangle/TornadoVM/llms.txt

Use this file to discover all available pages before exploring further.

TornadoVM ships three production backends that translate Java bytecode into native GPU kernels at runtime: the CUDA backend for NVIDIA hardware, the OpenCL backend for cross-vendor coverage (AMD, Intel, NVIDIA, CPUs, and FPGAs), and the Metal backend for Apple Silicon. All three share the same TaskGraph API and KernelContext programming model — the same Java code runs unchanged across every backend — but each backend has distinct capabilities, installation requirements, and performance characteristics that make it suited to particular deployment targets.

​
Compilation Pipeline

Every backend follows the same multi-stage pipeline: TornadoVM lowers Java bytecode through the Graal compiler infrastructure into a backend-specific intermediate representation, then JIT-compiles it to a native binary on first execution.
1

Java Bytecode Ingestion

TornadoVM intercepts the Java method designated by a TaskGraph task at the JVM bytecode level. No source code changes or annotation processors are required.
2

Graal IR Lowering

The bytecode is fed through the GraalVM compiler pipeline. High-level Java IR nodes (loops, array accesses, arithmetic) are lowered to backend-specific LIR nodes — CUDALIRStmt, MetalLIRStmt, or the OpenCL equivalent.
3

Backend Code Generation

The LIR is serialised into the target shader language:
  • CUDA → CUDA PTX assembly
  • OpenCL → OpenCL C kernel source
  • Metal → Metal Shading Language (MSL) source
4

Native Compilation

The generated source is handed to the platform’s native compiler:
  • CUDA → NVRTC JIT-compiles PTX to a device-specific cubin binary.
  • OpenCL → The vendor OpenCL runtime compiles OpenCL C to a platform binary.
  • Metal → The Metal framework compiles MSL to an MTLLibrary pipeline.
5

Kernel Dispatch & Data Management

TornadoVM dispatches the compiled kernel, automatically managing host-to-device and device-to-host data transfers according to the DataTransferMode declared in the TaskGraph.

​
Backend Selection at Runtime

TornadoVM discovers all installed backends via Java ServiceLoader at startup. When you call tornado --devices, it queries every loaded backend and enumerates the platforms and devices each one exposes. The backend used for a given task is determined by which device the TornadoExecutionPlan targets. 
# List all available backends and their devices
tornado --devices
A typical multi-backend output looks like: 
Number of Tornado drivers: 2
Driver: CUDA
  0: CUDA -- GeForce RTX 4090 (available)

Driver: OpenCL
  0: Intel(R) OpenCL -- 13th Gen Core i9 (available)
  1: NVIDIA OpenCL  -- GeForce RTX 4090 (available)
If both CUDA and OpenCL backends are installed, the unit test suite may silently default to the OpenCL device. Build with only the CUDA backend (make BACKEND=cuda) when running CUDA-specific tests to avoid false positives.

​
Backend Capability Matrix

FeatureCUDAOpenCLMetal
JIT kernel compilation✅✅✅
@Parallel loop annotation✅✅✅
KernelContext (thread IDs, local mem, barriers)✅✅✅
cuBLAS / cuBLASLt library tasks✅❌❌
cuFFT library tasks✅❌❌
cuDNN library tasks✅❌❌
cuSPARSE library tasks✅❌❌
CUTLASS library tasks✅❌❌
Tensor Core MMA intrinsics (mma.sync)✅❌❌
SIMDGROUP matrix multiply-accumulate❌❌✅
CUDA Graphs (withCUDAGraph())✅❌❌
CPU fallback device❌✅❌
FPGA support❌✅❌
AMD GPU support❌✅❌
Apple Silicon (M1–M4)❌✅†✅
† OpenCL support on Apple Silicon via the Apple OpenCL runtime; Metal is the preferred backend on macOS.

​
Installing Multiple Backends

All three backends can be installed in a single step using the full variant or by passing a comma-separated backend list to the installer. 
# All backends in a single SDK
sdk install tornadovm 5.2.0-full

# Or pick individual backends
sdk install tornadovm 5.2.0-cuda
sdk install tornadovm 5.2.0-opencl
sdk install tornadovm 5.2.0-metal

​
SDKMAN! Version Reference

5.2.0-opencl

OpenCL backend — the default choice. Targets NVIDIA, AMD, and Intel GPUs; integrated GPUs (Apple M-series, Intel HD, ARM Mali); multi-core CPUs; and FPGAs. Install with sdk install tornadovm 5.2.0-opencl.

5.2.0-cuda

CUDA backend — for NVIDIA GPUs. Includes PTX code generation, NVRTC JIT compilation, Tensor Core MMA intrinsics, CUDA library tasks (cuBLAS, cuFFT, cuDNN, cuSPARSE, CUTLASS), and CUDA Graphs. Install with sdk install tornadovm 5.2.0-cuda.

5.2.0-metal

Metal backend — for Apple Silicon (M1–M4). Generates Metal Shading Language (MSL) and compiles via the Metal framework. Includes SIMDGROUP matrix multiply-accumulate operations. Install with sdk install tornadovm 5.2.0-metal.

5.2.0-full

All backends — ships CUDA, OpenCL, and Metal together. Use this for heterogeneous environments or when you need to test across multiple backends on the same machine. Install with sdk install tornadovm 5.2.0-full.

​
Verifying Your Installation

After installing any backend, confirm the runtime can see your hardware: 
tornado --devices
If the command returns no devices, check that the appropriate GPU driver or runtime is installed:
  • CUDA: NVIDIA driver + CUDA Toolkit with nvrtc library
  • OpenCL: Vendor OpenCL runtime (NVIDIA, AMD ROCm, Intel OpenCL, or ocl-icd)
  • Metal: Xcode Command Line Tools on macOS 12+ with Apple Silicon
Use tornado --printKernel to dump the generated PTX, OpenCL C, or MSL source to stdout. This is the fastest way to verify that your Java kernel is being compiled correctly by a given backend.

Build docs developers (and LLMs) love

Get started for freeTalk to us