TornadoVM: JIT-Compile Java to CUDA, OpenCL, and Metal

TornadoVM is a plug-in to OpenJDK and compatible JDK distributions that lets you accelerate Java programs on GPUs, integrated GPUs, and multi-core CPUs — without writing CUDA C, OpenCL, or any native code. You annotate a loop or use the KernelContext API, wrap your data in a TaskGraph, and TornadoVM’s JIT compiler does the rest: it compiles your ordinary Java bytecode to a native GPU kernel at runtime and manages every host-to-device data transfer on your behalf.

What Problem Does It Solve?

Offloading compute-intensive Java workloads to a GPU has traditionally meant writing C/CUDA or OpenCL kernels, maintaining a JNI bridge, keeping two codebases in sync, and repeating the work for every target vendor. TornadoVM collapses that stack. The same Java source file that runs on the JVM also becomes a CUDA kernel on NVIDIA hardware, an OpenCL kernel on AMD/Intel, and a Metal shader on Apple Silicon — with no code changes and no recompilation.

How the JIT Pipeline Works

Java Bytecode → Graal IR

TornadoVM hooks into the Graal JIT compiler inside the JVM. When an execution plan runs for the first time, TornadoVM intercepts the bytecode of the annotated method and converts it into Graal’s intermediate representation (IR).

Graal IR → Backend IR

TornadoVM’s compiler back-ends lower the Graal IR into the target language: CUDA PTX for NVIDIA, OpenCL C for AMD/Intel/CPU, or Metal Shading Language (MSL) for Apple Silicon.

Native Compilation

The generated source is compiled to a native GPU binary at runtime: PTX is compiled through NVRTC to a cubin; OpenCL C is compiled by the driver’s online compiler; MSL is compiled by the Metal runtime. The binary is cached and reused for every subsequent execution.

Execution & Data Management

TornadoVM transfers DataTransferMode-annotated buffers to the device, dispatches the compiled kernel, and copies results back — all coordinated by the TornadoExecutionPlan you created.

TornadoVM does not replace your JVM. It is a transparent plug-in that adds GPU offload capability while everything else in your application runs on the JVM as normal.

Supported Hardware

TornadoVM targets any device reachable via one of its three backends.

NVIDIA GPUs

Full CUDA backend: PTX codegen via NVRTC, cuBLAS, cuFFT, cuDNN library tasks, Tensor Core mma.sync intrinsics, and CUDA Graph capture. Also addressable through the OpenCL backend.

AMD & Intel GPUs

OpenCL backend covers discrete AMD Radeon, Intel Arc, and Intel HD/Iris integrated GPUs. The same kernel source runs unchanged across all of them.

Apple Silicon (M1–M4)

Native Metal backend introduced in 5.2.0. Java kernels are compiled to Metal Shading Language and executed directly on the GPU cores of M1, M2, M3, and M4 chips.

Multi-Core CPUs & FPGAs

The OpenCL backend also targets multi-core CPUs from Intel and AMD, giving you portable SIMD acceleration. FPGA targets (Intel, Xilinx/AMD) are also reachable via the OpenCL backend.

Two Programming Styles

TornadoVM exposes two complementary APIs. Both styles can be mixed inside the same TaskGraph.

@Parallel Annotation Style
KernelContext Explicit Style

The simplest path to GPU acceleration. Add @Parallel to a loop variable and TornadoVM infers the thread mapping automatically — no knowledge of thread IDs or work-groups required.

import uk.ac.manchester.tornado.api.annotations.Parallel;
import uk.ac.manchester.tornado.api.types.arrays.FloatArray;

public class VectorAdd {
    // Annotate the loop index — TornadoVM maps each iteration
    // to a separate GPU thread automatically.
    public static void add(FloatArray a, FloatArray b, FloatArray c) {
        for (@Parallel int i = 0; i < c.getSize(); i++) {
            c.set(i, a.get(i) + b.get(i));
        }
    }
}

Best for: beginners, embarrassingly parallel loops, porting existing Java code.

Provides full GPU programming control: explicit global/local thread IDs, local (shared) memory, and barriers — the same model as CUDA/OpenCL/SYCL.

import uk.ac.manchester.tornado.api.KernelContext;
import uk.ac.manchester.tornado.api.types.arrays.FloatArray;

public class VectorAddKernel {
    // ctx.globalIdx is the CUDA/OpenCL equivalent of
    // blockIdx.x * blockDim.x + threadIdx.x
    public static void add(KernelContext ctx,
                           FloatArray a, FloatArray b, FloatArray c) {
        int i = ctx.globalIdx;
        if (i < c.getSize()) {
            c.set(i, a.get(i) + b.get(i));
        }
    }
}

Best for: fine-grained thread control, local memory tiling, Tensor Core intrinsics, performance-critical kernels.

Wrapping a Kernel in a TaskGraph

Regardless of which style you choose, you orchestrate execution through a TaskGraph and a TornadoExecutionPlan:

import uk.ac.manchester.tornado.api.*;
import uk.ac.manchester.tornado.api.enums.DataTransferMode;
import uk.ac.manchester.tornado.api.types.arrays.FloatArray;

FloatArray a = new FloatArray(1024);
FloatArray b = new FloatArray(1024);
FloatArray c = new FloatArray(1024);
// ... fill a and b ...

TaskGraph tg = new TaskGraph("s0")
    .transferToDevice(DataTransferMode.FIRST_EXECUTION, a, b)
    .task("t0", VectorAdd::add, a, b, c)
    .transferToHost(DataTransferMode.EVERY_EXECUTION, c);

ImmutableTaskGraph itg = tg.snapshot();
try (TornadoExecutionPlan plan = new TornadoExecutionPlan(itg)) {
    plan.execute();  // JIT-compiled to your GPU on first call
}

Key Differentiators

No JNI to Maintain

Unlike raw GPU bindings (e.g., JCUDA, LWJGL’s OpenCL), TornadoVM generates and compiles the GPU kernel from your Java bytecode. There is no native C/CUDA bridge to write, version, or keep in sync with your Java logic.

One Codebase, Four Backends

A TaskGraph written once runs on CUDA, OpenCL, Metal, or a CPU — you switch targets at runtime with plan.withDevice(device). No rewrite, no recompilation, no new build artefacts.

Native NVIDIA Library Ecosystem

On NVIDIA hardware, TornadoVM goes beyond PTX generation. cuBLAS, cuFFT, and cuDNN calls are available as .libraryTask() entries in the same TaskGraph as your JIT-compiled kernels, sharing device buffers on a single CUDA stream with zero extra copies.

Transparent Device Memory Management

TornadoVM’s off-heap FloatArray, IntArray, and friends are allocated outside the Java heap, enabling zero-copy pinned transfers on supported hardware. The DataTransferMode enum controls when buffers are moved, letting you amortise transfer cost across iterations.

Dynamic Reconfiguration

The execution plan can migrate between devices at runtime, fall back to the JVM if no GPU is available, and be re-optimised per device based on live profiler data — all without restarting the application.

Licensing

TornadoVM uses a two-tier licensing model designed for broad commercial use.

Modules	License
`tornado-api`, `tornado-annotation`, examples, benchmarks, matrices, OpenCL headers	Apache 2.0
`tornado-runtime`, `tornado-drivers`	GPLv2 with Classpath Exception

The Classpath Exception on the runtime and driver modules is the same one used by OpenJDK itself. It means your application code is not subject to copyleft obligations — just as running on OpenJDK does not make your application GPL.

What People Build With TornadoVM

TornadoVM is used in production for machine learning inference, computer vision, physics simulation, financial modelling, and signal processing. Notable open-source projects include GPULlama3.java (Llama 3 / Qwen / Mistral inference at 117 tok/s on an RTX 5090), TornadoVM-Ray-Tracer (real-time ray tracing in pure Java), and the ESA Gaia Mission data-processing pipeline.

Start Here

Installation

Install TornadoVM via SDKMAN!, build from source, or add the Maven dependency. Covers all backends and platforms.

Quickstart

Run your first GPU-accelerated Java program in minutes. Covers both the @Parallel annotation style and the KernelContext API.

Get Started

Programming Guide

Backends

NVIDIA Integration

Guides

Contributing

TornadoVM: JIT-Compile Java to CUDA, OpenCL, and Metal

What Problem Does It Solve?

How the JIT Pipeline Works

Supported Hardware

NVIDIA GPUs

AMD & Intel GPUs

Apple Silicon (M1–M4)

Multi-Core CPUs & FPGAs

Two Programming Styles

Wrapping a Kernel in a TaskGraph

Key Differentiators

Licensing

What People Build With TornadoVM

Start Here

Installation

Quickstart

Build docs developers (and LLMs) love

Get Started

Programming Guide

Backends

NVIDIA Integration

Guides

Contributing

Documentation Index

​What Problem Does It Solve?

​How the JIT Pipeline Works

​Supported Hardware

NVIDIA GPUs

AMD & Intel GPUs

Apple Silicon (M1–M4)

Multi-Core CPUs & FPGAs

​Two Programming Styles

​Wrapping a Kernel in a TaskGraph

​Key Differentiators

​Licensing

​What People Build With TornadoVM

​Start Here

Installation

Quickstart

Build docs developers (and LLMs) love

What Problem Does It Solve?

How the JIT Pipeline Works

Supported Hardware

Two Programming Styles

Wrapping a Kernel in a TaskGraph

Key Differentiators

Licensing

What People Build With TornadoVM

Start Here