Skip to main content

Documentation Index

Fetch the complete documentation index at: https://mintlify.com/Deepak-Sangle/TornadoVM/llms.txt

Use this file to discover all available pages before exploring further.

TornadoVM is designed to target any heterogeneous hardware that a JVM process can see — discrete GPUs, integrated GPUs, multi-core CPUs exposed through OpenCL, and anything else a backend driver enumerates. When a TaskGraph contains independent tasks (no shared data dependencies between them), TornadoVM can run each task on a different device, either sequentially or truly concurrently. This makes it possible to saturate all available accelerators simultaneously: one task processes a red colour channel on an NVIDIA GPU while another processes a green channel on an Intel CPU, for example. Understanding how to list, select, and assign devices is fundamental to extracting peak hardware utilisation from TornadoVM.

​
Listing Available Devices

​
Command-Line Enumeration

The quickest way to see what TornadoVM can access is the --devices flag: 
tornado --devices
Example output on a system with one GPU and two CPU OpenCL devices: 
Number of Tornado drivers: 1
Driver: OpenCL
  Total number of OpenCL devices  : 3
  Tornado device=0:0  (DEFAULT)
    OPENCL --  [NVIDIA CUDA] -- NVIDIA GeForce RTX 3070
        Global Memory Size: 7.8 GB
        Local Memory Size: 48.0 KB
        Workgroup Dimensions: 3
        Total Number of Block Threads: [1024]
        Max WorkGroup Configuration: [1024, 1024, 64]
        Device OpenCL C version: OpenCL C 1.2

  Tornado device=0:1
    OPENCL --  [Intel(R) OpenCL] -- 13th Gen Intel(R) Core(TM) i7-13700
        Global Memory Size: 62.5 GB
        Local Memory Size: 32.0 KB
        Workgroup Dimensions: 3
        Total Number of Block Threads: [8192]
        Max WorkGroup Configuration: [8192, 8192, 8192]
        Device OpenCL C version: OpenCL C 3.0

  Tornado device=0:2
    OPENCL --  [Intel(R) OpenCL] -- Intel(R) Core(TM) CPU
        Global Memory Size: 62.5 GB
        Local Memory Size: 256.0 KB
        Device OpenCL C version: OpenCL C 1.2
The notation driverIndex:deviceIndex (e.g. 0:0, 0:1) is used throughout TornadoVM to identify a device uniquely.

​
Programmatic Enumeration

Use TornadoRuntimeProvider to inspect devices at runtime without leaving Java: 
import uk.ac.manchester.tornado.api.runtime.TornadoRuntimeProvider;
import uk.ac.manchester.tornado.api.TornadoBackend;
import uk.ac.manchester.tornado.api.common.TornadoDevice;

// Iterate over all backends (drivers) and their devices
int numBackends = TornadoRuntimeProvider.getTornadoRuntime().getNumBackends();

for (int d = 0; d < numBackends; d++) {
    TornadoBackend backend = TornadoRuntimeProvider.getTornadoRuntime().getBackend(d);
    System.out.println("Backend: " + backend.getBackendType());

    int numDevices = backend.getNumDevices();
    for (int i = 0; i < numDevices; i++) {
        TornadoDevice device = backend.getDevice(i);
        System.out.printf("  [%d:%d] %s (%s)%n",
            d, i,
            device.getDeviceName(),
            device.getDeviceType());

        long maxMem = device.getMaxAllocMemory();
        System.out.printf("         Max alloc: %.1f MB%n", maxMem * 1e-6);
    }
}

​
Device Query via TornadoExecutionPlan

TornadoExecutionPlan exposes a static convenience method for direct device retrieval by index: 
import uk.ac.manchester.tornado.api.TornadoExecutionPlan;
import uk.ac.manchester.tornado.api.common.TornadoDevice;

// Fetch device at backend 0, device index 1
TornadoDevice gpu  = TornadoExecutionPlan.getDevice(0, 0);
TornadoDevice cpu  = TornadoExecutionPlan.getDevice(0, 1);
TornadoDevice def  = TornadoExecutionPlan.DEFAULT_DEVICE;

​
Selecting a Device for the Whole Execution Plan

The simplest device-selection API routes all graphs in the plan to a single device. 
TornadoDevice targetGpu = TornadoExecutionPlan.getDevice(0, 0);

try (TornadoExecutionPlan plan = new TornadoExecutionPlan(itg)) {
    plan.withDevice(targetGpu)
        .execute();
}

​
Per-Task Device Assignment

When a TaskGraph contains multiple tasks, you can route each individual task to a different device using the overloaded .withDevice(String taskName, TornadoDevice device) form. The task name uses the format "graphName.taskName". 
TornadoDevice gpu = TornadoExecutionPlan.getDevice(0, 0);
TornadoDevice cpu = TornadoExecutionPlan.getDevice(0, 1);

try (TornadoExecutionPlan plan = new TornadoExecutionPlan(itg)) {
    plan.withDevice("blur.red",   gpu)   // red channel → GPU
        .withDevice("blur.green", cpu)   // green channel → CPU
        .withDevice("blur.blue",  gpu)   // blue channel → GPU
        .execute();
}

​
Command-Line Alternative

Per-task device assignment can also be specified at launch time using JVM system properties, without modifying source code: 
tornado \
  --jvm="-Dblur.red.device=0:0 -Dblur.green.device=0:1 -Dblur.blue.device=0:0" \
  -m tornado.examples/uk.ac.manchester.tornado.examples.compute.BlurFilter

​
Sequential vs. Concurrent Multi-Device Execution

By default, TornadoVM runs each independent task sequentially — one device finishes before the next starts. This mode is useful for debugging and profiling individual tasks.
Tasks assigned to different devices still run one after the other in the order they appear in the TaskGraph.
tornado --threadInfo \
    --jvm="-Dblur.red.device=0:0 -Dblur.green.device=0:1 -Dblur.blue.device=0:0" \
    -m tornado.examples/uk.ac.manchester.tornado.examples.compute.BlurFilter
All bytecodes execute from the main Java thread. One device is always idle while the other is working.
In concurrent mode, TornadoVM spawns one interpreter instance per device, each running inside a thread-pool thread. The debug flag --printBytecodes will print the bytecode stream for each interpreter instance labelled by the device and thread name (e.g. pool-1-thread-1).

​
Complete Multi-Device Example

The following self-contained example demonstrates device enumeration, per-task assignment, and concurrent execution in a single program. 
import uk.ac.manchester.tornado.api.*;
import uk.ac.manchester.tornado.api.annotations.Parallel;
import uk.ac.manchester.tornado.api.common.TornadoDevice;
import uk.ac.manchester.tornado.api.enums.DataTransferMode;
import uk.ac.manchester.tornado.api.runtime.TornadoRuntimeProvider;
import uk.ac.manchester.tornado.api.types.arrays.FloatArray;

public class MultiDeviceExample {

    /** Kernel A: scale by a factor (runs on device 0) */
    public static void scale(FloatArray data, float factor) {
        for (@Parallel int i = 0; i < data.getSize(); i++) {
            data.set(i, data.get(i) * factor);
        }
    }

    /** Kernel B: add an offset (runs on device 1) */
    public static void offset(FloatArray data, float bias) {
        for (@Parallel int i = 0; i < data.getSize(); i++) {
            data.set(i, data.get(i) + bias);
        }
    }

    public static void main(String[] args) throws Exception {

        // --- 1. Enumerate and print available devices ---
        int numBackends = TornadoRuntimeProvider.getTornadoRuntime().getNumBackends();
        System.out.println("Available backends: " + numBackends);
        for (int b = 0; b < numBackends; b++) {
            var backend = TornadoRuntimeProvider.getTornadoRuntime().getBackend(b);
            for (int d = 0; d < backend.getNumDevices(); d++) {
                TornadoDevice dev = backend.getDevice(d);
                System.out.printf("  [%d:%d] %s | max alloc: %.0f MB%n",
                    b, d, dev.getDeviceName(),
                    dev.getMaxAllocMemory() * 1e-6);
            }
        }

        // --- 2. Select two devices (fall back to same device if only one exists) ---
        TornadoDevice dev0 = TornadoExecutionPlan.getDevice(0, 0);
        TornadoDevice dev1 = TornadoRuntimeProvider
            .getTornadoRuntime().getBackend(0).getNumDevices() > 1
            ? TornadoExecutionPlan.getDevice(0, 1)
            : dev0;

        // --- 3. Prepare data ---
        final int SIZE = 65_536;
        FloatArray arrayA = new FloatArray(SIZE);
        FloatArray arrayB = new FloatArray(SIZE);
        arrayA.init(4.0f);
        arrayB.init(10.0f);

        // --- 4. Build TaskGraph with two independent tasks ---
        TaskGraph tg = new TaskGraph("multi")
            .transferToDevice(DataTransferMode.EVERY_EXECUTION, arrayA, arrayB)
            .task("scale",  MultiDeviceExample::scale,  arrayA, 2.0f)
            .task("offset", MultiDeviceExample::offset, arrayB, 5.0f)
            .transferToHost(DataTransferMode.EVERY_EXECUTION, arrayA, arrayB);

        ImmutableTaskGraph itg = tg.snapshot();

        // --- 5. Execute with per-task device assignment ---
        try (TornadoExecutionPlan plan = new TornadoExecutionPlan(itg)) {
            plan.withDevice("multi.scale",  dev0)
                .withDevice("multi.offset", dev1)
                .withConcurrentDevices()
                .execute();
        }

        System.out.printf("arrayA[0] = %.1f (expected 8.0)%n", arrayA.get(0));
        System.out.printf("arrayB[0] = %.1f (expected 15.0)%n", arrayB.get(0));
    }
}

​
Dynamic Reconfiguration (Research Feature)

TornadoVM includes a dynamic reconfiguration mode (DRMode) that can evaluate execution across all available devices. This feature is not enabled automatically and is maintained as a research capability rather than a core runtime feature. DRMode is an enum in uk.ac.manchester.tornado.api with two values:
  • DRMode.SERIAL — The runtime evaluates all devices sequentially (compiles and runs each ImmutableTaskGraph one after another) before making a device-switching decision.
  • DRMode.PARALLEL — The runtime evaluates all devices in parallel, mapping each physical accelerator to a separate Java thread.
Dynamic reconfiguration only explores single-device execution paths. It does not distribute tasks across multiple devices simultaneously.

​
Multi-Device Limitations

Supported

  • Independent tasks (no shared data) routed to different devices
  • Sequential and concurrent multi-device execution modes
  • Per-task device selection via API or JVM system properties
  • Mix of GPU and CPU tasks in the same execution plan

Not Supported

  • Tasks with shared data dependencies across devices (must run on a single device)
  • Batch processing across multiple devices (single device only)
  • Dynamic reconfiguration with multi-device task distribution
  • Automatic dependency analysis between tasks on different devices
When designing for multi-device execution, structure your TaskGraph so that independent work items are expressed as separate tasks. The natural decomposition for image processing (one task per colour channel, one task per image tile) or physics simulation (one task per particle subsystem) maps cleanly onto independent multi-device scheduling.

Build docs developers (and LLMs) love

Get started for freeTalk to us