Skip to main content

Documentation Index

Fetch the complete documentation index at: https://mintlify.com/happyme531/ztu_somemodelruntime_ez_rknn_async/llms.txt

Use this file to discover all available pages before exploring further.

RKNN models can reference operators not built into the runtime. These are supplied as shared library (.so) files that export a get_rknn_custom_op() function. EZ RKNN Async loads and registers these plugins at session creation time via the custom_op_paths and custom_op_default_path provider options.

​
Build-time requirement

Custom op support is compiled in only when rknn_custom_op.h is present at build time. The C++ preprocessor macro ZTU_EZRKNN_ASYNC_HAS_CUSTOM_OP is set to 1 when the header is found. If your build lacks the header, calling any custom-op option emits a UserWarning and raises RuntimeError.

​
Loading a specific plugin file

Pass one or more .so paths to custom_op_paths: 
from ztu_somemodelruntime_ez_rknn_async import InferenceSession, make_provider_options

opts = make_provider_options(
    custom_op_paths="/usr/lib/rknpu/op_plugins/librkcst_myop.so",
)

sess = InferenceSession("model_with_custom_op.rknn", provider_options=opts)
You can also pass a list of paths to load multiple plugins in one session: 
opts = make_provider_options(
    custom_op_paths=[
        "/usr/lib/rknpu/op_plugins/librkcst_op_a.so",
        "/usr/lib/rknpu/op_plugins/librkcst_op_b.so",
    ],
)
The plugin .so must export a symbol named exactly get_rknn_custom_op. If the symbol is missing, session creation raises a RuntimeError with the dlsym error detail. If the file cannot be opened at all, a UserWarning is emitted first (to surface the potential RKNN stability risk), then a RuntimeError is raised.

​
Loading from the default plugin directory

Set custom_op_default_path=True to scan the platform’s standard plugin directory and load every file matching librkcst_*.so:
PlatformScanned directory
Linux/usr/lib/rknpu/op_plugins/
Android (64-bit)/vendor/lib64/
Android (32-bit)/vendor/lib/
opts = make_provider_options(
    custom_op_default_path=True,
)

sess = InferenceSession("model.rknn", provider_options=opts)
If the directory does not exist on your system, a UserWarning is emitted and no plugins are loaded. Individual plugin load failures emit per-file warnings rather than aborting the whole scan.
You can combine custom_op_paths and custom_op_default_path=True in a single session to load specific plugins alongside any platform defaults.

​
Why custom ops force disable_dup_context

By default, EZ RKNN Async creates additional RKNN contexts with rknn_dup_context to share the loaded model across worker threads. There is a known stability issue in the RKNN runtime when custom operators are registered on duplicated contexts. To avoid this:
  • Requesting any custom-op loading automatically sets disable_dup_context=True for that session, regardless of the value you pass explicitly.
  • Each worker thread initializes its own independent context via rknn_init.
  • The session emits a UserWarning at startup to make this behavior visible:
    Custom op loading requested; dup_context is disabled for this session to avoid known RKNN stability issues. 
# Both lines below produce identical behavior — the flag is forced on
# when custom ops are requested.
opts_a = make_provider_options(custom_op_default_path=True)
opts_b = make_provider_options(custom_op_default_path=True, disable_dup_context=True)
Using rknn_dup_context with custom ops can cause crashes or silent incorrect results. Do not attempt to override this safety measure by patching the provider options dict after calling make_provider_options.

​
Complete example

from ztu_somemodelruntime_ez_rknn_async import InferenceSession, make_provider_options
import numpy as np

opts = make_provider_options(
    custom_op_paths="/path/to/librkcst_my_layer.so",
    schedule=[0, 1],          # data-parallel across two cores
    threads_per_core=1,
)

sess = InferenceSession("model_with_my_layer.rknn", provider_options=opts)

input_data = np.random.rand(1, 3, 256, 256).astype(np.float32)
outputs = sess.run(None, {"input": input_data})
print(outputs[0].shape)

​
Provider option keys

Both spelling variants are accepted and behave identically:
KeyAliasTypeDescription
custom_op_pathscustom_op_pathstr or List[str]Path(s) to .so plugin files.
custom_op_default_pathload_custom_ops_from_default_pathboolScan the platform default plugin directory.

Build docs developers (and LLMs) love

Get started for freeTalk to us