TornadoVM Off-Heap Primitive Array Types for the GPU

TornadoVM’s primitive array types are purpose-built containers for GPU-bound data. The JVM garbage collector cannot safely pin ordinary heap arrays at a stable physical address—while a GC compaction move is in flight, the GPU would read corrupted memory. Every class in the uk.ac.manchester.tornado.api.types.arrays package allocates its payload in a java.lang.foreign.MemorySegment backed by off-heap native memory, giving TornadoVM a stable pointer it can hand directly to OpenCL, CUDA, or SPIR-V without any intermediate copy. All classes extend the abstract sealed base TornadoNativeArray, which enforces a small fixed-size array header (16 or 24 bytes, depending on JVM compressed-pointer settings) prepended to the data region so the TornadoVM runtime can locate each buffer in device memory unambiguously.

The header size defaults to 16 bytes when compressed OOPs are active and 24 bytes otherwise. Override it with -Dtornado.panama.objectHeader=<bytes> if your JVM is configured non-standardly.

Why off-heap arrays?

GC-safe device transfer

Off-heap memory has a fixed physical address for its lifetime—the GC never moves it—so TornadoVM can DMA-transfer the buffer without a pinning phase.

Zero-copy sharing

A FloatArray produced by one TaskGraph task can be consumed by a subsequent library task (e.g. cuBLAS) on the same stream with no host roundtrip.

Direct segment access

getSegment() and getSegmentWithHeader() expose the raw MemorySegment for interop with Panama foreign-memory APIs.

Slicing & concat

slice(offset, length) returns a view over a sub-region; concat(arrays...) merges multiple arrays into one contiguous buffer.

Complete type reference

The table below maps each TornadoVM array class to its Java primitive equivalent, element size, and the concrete element type used in get/set calls.

Class	Replaces	Element type	Bytes/element
`FloatArray`	`float[]`	`float`	4
`IntArray`	`int[]`	`int`	4
`DoubleArray`	`double[]`	`double`	8
`LongArray`	`long[]`	`long`	8
`ShortArray`	`short[]`	`short`	2
`ByteArray`	`byte[]`	`byte`	1
`CharArray`	`char[]`	`char`	2
`HalfFloatArray`	(none — FP16)	`HalfFloat`	2
`BFloat16Array`	(none — BF16)	`short` (raw bits)	2
`FP8Array`	(none — FP8)	`byte` (raw bits)	1
`Int8Array`	`byte[]` (int8)	`byte`	1

All array classes are final and @SegmentElementSize-annotated so the TornadoVM compiler backend can compute aligned byte offsets without reflection at runtime.

`TornadoNativeArray` — base class API

Every concrete array type inherits the following abstract methods from TornadoNativeArray:

getSize()

int

Returns the number of logical elements (not bytes) in the array.

getSegment()

MemorySegment

Returns the off-heap MemorySegment excluding the TornadoVM array header. Use this slice when you need a pointer to the raw data region for interop.

getSegmentWithHeader()

MemorySegment

Returns the full MemorySegment including the array header. This is the pointer the TornadoVM runtime resolves to device addresses internally.

getNumBytesOfSegment()

long

Total bytes occupied by the data region, excluding the header: getSize() * getElementSize().

getNumBytesOfSegmentWithHeader()

long

Total bytes of the full native allocation, including the header prefix.

getElementSize()

int

Size in bytes of a single element (e.g. 4 for FloatArray, 2 for HalfFloatArray).

clear()

void

Resets all elements to the zero value for the element type.

`FloatArray` — detailed API

FloatArray is the most commonly used off-heap type. Its API is representative of all numeric array classes.

Constructors

FloatArray(int numberOfElements)

constructor

Allocates a new off-heap float array of the given size, zero-initialized.

FloatArray(FloatArray... arrays)

constructor

Constructs a new array by concatenating all provided arrays in order.

Factory methods (static)

FloatArray.fromArray(float[] values)

static factory

Copies an on-heap float[] into a new off-heap FloatArray. Useful for initializing buffers from existing Java data.

FloatArray.fromElements(float... values)

static factory

Varargs convenience wrapper over fromArray.

FloatArray.fromSegment(MemorySegment segment)

static factory

Copies raw float bytes from an existing Panama MemorySegment (without the TornadoVM header) into a new FloatArray.

FloatArray.fromSegmentShallow(MemorySegment segment)

static factory

Wraps an existing segment (which must include the TornadoVM header) without copying. Useful when a native library has already allocated the buffer.

FloatArray.fromFloatBuffer(FloatBuffer buffer)

static factory

Initializes a FloatArray from a NIO FloatBuffer.

FloatArray.concat(FloatArray... arrays)

static factory

Returns a new FloatArray containing all elements of the inputs, concatenated in argument order.

Instance methods

get(int index)

float

Returns the float at the given element index.

set(int index, float value)

void

Stores value at the given element index.

init(float value)

void

Sets every element to value. Prefer this over a manual loop—it is JIT-friendly and can execute as a @Parallel task.

slice(int offset, int length)

FloatArray

Returns a new FloatArray backed by a sub-range of this array’s data. Throws IllegalArgumentException if the slice is out of bounds.

toHeapArray()

float[]

Copies the off-heap contents back to a standard on-heap float[]. Only use this for diagnostic output—repeated calls are expensive.

FloatArray.initialize(FloatArray array, float value)

static void

Parallel-annotated factory initializer intended to be used as a TaskGraph.task(...) target. The runtime dispatches it on the GPU as a fill kernel.

`HalfFloatArray` — FP16 specifics

HalfFloatArray stores IEEE 754 half-precision (FP16) values, each occupying 2 bytes. There is no Java primitive for FP16; the element type is the wrapper class uk.ac.manchester.tornado.api.types.HalfFloat.

FP16 arithmetic has limited precision (~3 decimal digits) and a reduced dynamic range (max ≈ 65504). Always verify numerical correctness with an FP32 reference when switching to FP16 weights.

Key methods beyond the base API

get(int index)

HalfFloat

Returns a HalfFloat wrapper holding the FP16 bit pattern at index. Call .getFloat32() on the result to decode to a Java float.

set(int index, HalfFloat value)

void

Stores the FP16 bit pattern of value at index.

getHalf2(int index)

Half2

Loads two consecutive half-float elements as a packed Half2 vector. index must be even for 4-byte alignment. On backends with native half2 support this maps to a single 32-bit load.

setHalf2(int index, Half2 value)

void

Stores a Half2 into two consecutive elements starting at index (must be even).

init(HalfFloat value)

void

Fills every element with the given HalfFloat value.

toShortArray()

short[]

Returns the raw FP16 bit patterns as a short[]—useful for debugging or interop with libraries that accept short buffers.

HalfFloatArray is the primary input/output type for cuBLASLt FP16 GEMM (ltMatmulFP16, ltMatmulGeluBiasFP16) and CUTLASS tensor-core GEMM (cutlassHgemm).

`BFloat16Array` and `FP8Array`

BFloat16Array (Brain Float 16) stores BF16 values—the same 8-bit exponent as FP32 with only 7 mantissa bits. Its raw element type is short (the bit pattern); use getFloat(int) / setFloat(int, float) for host-side float↔BF16 conversion via the BFloat16 codec. FP8Array stores 8-bit floating-point values (1 byte each). The format is E4M3 by default; the class provides separate getE4M3(int) / getE5M2(int) decoders so you can choose the interpretation. FP8 arrays are used as weight and activation storage for cuBLASLt FP8 GEMM (ltMatmulFP8).

Migration guide: `float[]` → `FloatArray`

Migrating existing Java kernel code from float[] to FloatArray is mechanical. The three changes are: allocation, element access, and TaskGraph registration.

Before (float[])
After (FloatArray)

// Allocation
float[] a = new float[N];
float[] b = new float[N];
float[] c = new float[N];

// Initialization
Arrays.fill(a, 1.0f);
Arrays.fill(b, 2.0f);

// Kernel (runs on CPU)
public static void vectorAdd(float[] a, float[] b, float[] c) {
    for (@Parallel int i = 0; i < a.length; i++) {
        c[i] = a[i] + b[i];
    }
}

// TaskGraph
TaskGraph graph = new TaskGraph("s0")
    .transferToDevice(DataTransferMode.FIRST_EXECUTION, a, b)
    .task("add", MyClass::vectorAdd, a, b, c)
    .transferToHost(DataTransferMode.EVERY_EXECUTION, c);

import uk.ac.manchester.tornado.api.types.arrays.FloatArray;

// Allocation — off-heap, GC-safe
FloatArray a = new FloatArray(N);
FloatArray b = new FloatArray(N);
FloatArray c = new FloatArray(N);

// Initialization
a.init(1.0f);
b.init(2.0f);

// Kernel — identical body, new parameter types
public static void vectorAdd(FloatArray a, FloatArray b, FloatArray c) {
    for (@Parallel int i = 0; i < a.getSize(); i++) {
        c.set(i, a.get(i) + b.get(i));
    }
}

// TaskGraph — identical structure
TaskGraph graph = new TaskGraph("s0")
    .transferToDevice(DataTransferMode.FIRST_EXECUTION, a, b)
    .task("add", MyClass::vectorAdd, a, b, c)
    .transferToHost(DataTransferMode.EVERY_EXECUTION, c);

Full end-to-end example

The example below creates two FloatArray inputs, runs a parallel vector-add kernel through a TaskGraph, and reads back the result.

import uk.ac.manchester.tornado.api.TaskGraph;
import uk.ac.manchester.tornado.api.TornadoExecutionPlan;
import uk.ac.manchester.tornado.api.annotations.Parallel;
import uk.ac.manchester.tornado.api.enums.DataTransferMode;
import uk.ac.manchester.tornado.api.types.arrays.FloatArray;

public class VectorAddExample {

    // Kernel: element-wise a[i] + b[i] → c[i]
    public static void vectorAdd(FloatArray a, FloatArray b, FloatArray c) {
        for (@Parallel int i = 0; i < a.getSize(); i++) {
            c.set(i, a.get(i) + b.get(i));
        }
    }

    public static void main(String[] args) throws Exception {
        final int N = 1 << 20; // 1 M elements

        // 1. Allocate off-heap buffers
        FloatArray a = FloatArray.fromElements(/* seed */ new float[N]);
        FloatArray b = new FloatArray(N);
        FloatArray c = new FloatArray(N);

        // 2. Initialize on the host
        for (int i = 0; i < N; i++) {
            a.set(i, (float) i);
            b.set(i, (float) (N - i));
        }

        // 3. Build and execute the TaskGraph
        TaskGraph graph = new TaskGraph("vadd")
            .transferToDevice(DataTransferMode.FIRST_EXECUTION, a, b)
            .task("add", VectorAddExample::vectorAdd, a, b, c)
            .transferToHost(DataTransferMode.EVERY_EXECUTION, c);

        try (TornadoExecutionPlan plan = new TornadoExecutionPlan(graph.snapshot())) {
            plan.execute();
        }

        // 4. Read result — every c[i] should be N
        System.out.printf("c[0] = %.0f  (expected %.0f)%n", c.get(0), (float) N);
    }
}

Use FloatArray.fromArray(existingFloatArray) when you have data in an existing float[]. The factory does a single bulk copy into off-heap memory and is faster than element-by-element set calls.

Core API

Data Types

Library Tasks

TornadoVM Off-Heap Primitive Array Types for the GPU

Why off-heap arrays?

GC-safe device transfer

Zero-copy sharing

Direct segment access

Slicing & concat

Complete type reference

`TornadoNativeArray` — base class API

`FloatArray` — detailed API

Constructors

Factory methods (static)

Instance methods

`HalfFloatArray` — FP16 specifics

Key methods beyond the base API

`BFloat16Array` and `FP8Array`

Migration guide: `float[]` → `FloatArray`

Full end-to-end example

Build docs developers (and LLMs) love

Core API

Data Types

Library Tasks

Documentation Index

​Why off-heap arrays?

GC-safe device transfer

Zero-copy sharing

Direct segment access

Slicing & concat

​Complete type reference

​TornadoNativeArray — base class API

​FloatArray — detailed API

​Constructors

​Factory methods (static)

​Instance methods

​HalfFloatArray — FP16 specifics

​Key methods beyond the base API

​BFloat16Array and FP8Array

​Migration guide: float[] → FloatArray

​Full end-to-end example

Build docs developers (and LLMs) love

Why off-heap arrays?

Complete type reference

`TornadoNativeArray` — base class API

`FloatArray` — detailed API

Constructors

Factory methods (static)

Instance methods

`HalfFloatArray` — FP16 specifics

Key methods beyond the base API

`BFloat16Array` and `FP8Array`

Migration guide: `float[]` → `FloatArray`

Full end-to-end example