Skip to main content

Documentation Index

Fetch the complete documentation index at: https://mintlify.com/happyme531/ztu_somemodelruntime_ez_rknn_async/llms.txt

Use this file to discover all available pages before exploring further.

The layout option tells the session how to interpret the dimension order of input tensors you pass via run, run_async, or run_pipeline. Most RKNN models are exported with NHWC tensors internally, but the runtime can accept inputs in a different order and perform the necessary reordering — either in hardware (by setting the RKNN input format field) or in software (by NumPy transpose before the data is handed to the driver). Choosing the right layout avoids a mismatch between the shape you provide and the shape the model expects.
Layout conversion only applies to 4-D tensors (n_dims == 4). Tensors with any other rank are passed through with RKNN_TENSOR_UNDEFINED as their format, regardless of the layout setting.

​
Layout values at a glance

ValueAccepted aliasInput dimension orderHardware format fieldSoftware transpose
"nchw""original"N, C, H, WRKNN_TENSOR_NCHWNo
"nhwc"—N, H, W, CRKNN_TENSOR_NHWCNo
"nchw_software""original_software"N, C, H, W (input) → N, H, W, C (sent to driver)RKNN_TENSOR_NHWCYes
"any"—As-isRKNN_TENSOR_UNDEFINEDNo

​
"nchw" / "original" (default)

The session sets the RKNN input format field to RKNN_TENSOR_NCHW for every 4-D input. This is the native RKNN format: the model was compiled expecting channel-first tensors. The input shape reported by get_inputs() is displayed as [N, C, H, W]. Use this when your model was exported with NCHW inputs, or when you are already producing channel-first arrays (e.g., PyTorch’s default). 
from ztu_somemodelruntime_ez_rknn_async import InferenceSession, make_provider_options
import numpy as np

sess = InferenceSession(
    "model.rknn",
    provider_options=make_provider_options(layout="nchw"),
)

# Shape: [batch, channels, height, width]
x = np.random.rand(1, 3, 224, 224).astype(np.float32)
outputs = sess.run(None, {"input": x})

​
"nhwc"

The session sets the RKNN input format field to RKNN_TENSOR_NHWC. No data reordering is performed in Python; you must supply arrays whose last axis is the channel dimension. Use this when your pipeline produces channel-last arrays (e.g., images decoded by OpenCV or Pillow, which return (H, W, C) or (N, H, W, C)). 
from ztu_somemodelruntime_ez_rknn_async import InferenceSession, make_provider_options
import numpy as np

sess = InferenceSession(
    "model.rknn",
    provider_options=make_provider_options(layout="nhwc"),
)

# Shape: [batch, height, width, channels]
x = np.random.rand(1, 224, 224, 3).astype(np.uint8)
outputs = sess.run(None, {"input": x})

​
"nchw_software" / "original_software"

This layout accepts NCHW arrays from the caller but converts them to NHWC before passing the data to the RKNN driver. The conversion is performed in Python via array.transpose(0, 2, 3, 1) followed by a contiguous copy. The hardware format field is set to RKNN_TENSOR_NHWC. When this layout is active, get_inputs() reports shapes in NCHW order (the format you provide) even though the model internally expects NHWC. This makes the session’s reported shape match your array shape so inference code does not need to track two different conventions. Use this when:
  • Your model was compiled for NHWC input, but your pipeline produces NCHW arrays.
  • You want a single code path that looks channel-first throughout, with the session handling the reorder transparently.
Software transpose adds a copy on every inference call. For high-throughput pipelines, prefer "nhwc" and produce channel-last arrays directly from your image source to avoid the overhead.
from ztu_somemodelruntime_ez_rknn_async import InferenceSession, make_provider_options
import numpy as np

sess = InferenceSession(
    "model.rknn",
    provider_options=make_provider_options(layout="nchw_software"),
)

# Provide NCHW — the session transposes to NHWC internally
x = np.random.rand(1, 3, 224, 224).astype(np.float32)
outputs = sess.run(None, {"input": x})

​
"any"

No format conversion is applied. The RKNN input format field is set to RKNN_TENSOR_UNDEFINED for all inputs, leaving interpretation entirely to the driver. Use this for non-image models or when you have already arranged the data exactly as the model expects and do not want the session to reinterpret the layout. 
from ztu_somemodelruntime_ez_rknn_async import InferenceSession, make_provider_options
import numpy as np

sess = InferenceSession(
    "model.rknn",
    provider_options=make_provider_options(layout="any"),
)

# Provide data in whatever arrangement the model expects
x = np.random.rand(1, 128).astype(np.float32)
outputs = sess.run(None, {"input": x})

​
Choosing the right layout

PyTorch produces tensors in NCHW order by default. If your model was exported directly from PyTorch without transposing the inputs, use "nchw".
opts = make_provider_options(layout="nchw")
# Input: (N, C, H, W) — matches PyTorch convention

Build docs developers (and LLMs) love

Get started for freeTalk to us