Model Functions

The model functions provide the core capabilities for loading and configuring ONNX Runtime GenAI models.

Model Creation

OgaCreateModel

function

OgaResult* OGA_API_CALL OgaCreateModel(const char* config_path, OgaModel** out);

Creates a model from a configuration directory.Parameters:

config_path: Path to the model configuration directory (UTF-8 encoded)
out: Pointer to store the created model

Returns: NULL on success, or OgaResult* containing error message on failureExample:

OgaModel* model = NULL;
OgaResult* result = OgaCreateModel("/path/to/model", &model);
if (result != NULL) {
    fprintf(stderr, "Error: %s\n", OgaResultGetError(result));
    OgaDestroyResult(result);
    return -1;
}

// Use the model...

OgaDestroyModel(model);

OgaCreateModelFromConfig

function

OgaResult* OGA_API_CALL OgaCreateModelFromConfig(const OgaConfig* config, OgaModel** out);

Creates a model from an OgaConfig object.Parameters:

config: Configuration object to use for the model
out: Pointer to store the created model

Returns: NULL on success, or OgaResult* containing error message on failure

OgaCreateModelWithRuntimeSettings

function

OgaResult* OGA_API_CALL OgaCreateModelWithRuntimeSettings(
    const char* config_path,
    const OgaRuntimeSettings* settings,
    OgaModel** out
);

Creates a model with runtime settings and device configuration.Parameters:

config_path: Path to the model configuration directory (UTF-8 encoded)
settings: Runtime settings to use for the model
out: Pointer to store the created model

Returns: NULL on success, or OgaResult* containing error message on failure

OgaDestroyModel

function

void OGA_API_CALL OgaDestroyModel(OgaModel* model);

Destroys a model and frees its resources.Parameters:

model: The model to destroy

Model Properties

OgaModelGetType

function

OgaResult* OGA_API_CALL OgaModelGetType(const OgaModel* model, const char** out);

Returns the type of the model (e.g., “gpt2”, “llama”).Parameters:

model: The model to query
out: Pointer to store the model type string

Returns: NULL on success, or OgaResult* containing error message on failureNote: The returned string must be destroyed with OgaDestroyString()Example:

const char* model_type = NULL;
OgaResult* result = OgaModelGetType(model, &model_type);
if (result == NULL) {
    printf("Model type: %s\n", model_type);
    OgaDestroyString(model_type);
}

OgaModelGetDeviceType

function

OgaResult* OGA_API_CALL OgaModelGetDeviceType(const OgaModel* model, const char** out);

Returns the device type of the model (e.g., “CPU”, “CUDA”, “DML”).Parameters:

model: The model to query
out: Pointer to store the device type string

Returns: NULL on success, or OgaResult* containing error message on failureNote: The returned string must be destroyed with OgaDestroyString()

Configuration Objects

OgaConfig

OgaCreateConfig

function

OgaResult* OGA_API_CALL OgaCreateConfig(const char* config_path, OgaConfig** out);

Creates a configuration object from a configuration directory.Parameters:

config_path: Path to the configuration directory (UTF-8 encoded)
out: Pointer to store the created config

Returns: NULL on success, or OgaResult* containing error message on failure

OgaDestroyConfig

function

void OGA_API_CALL OgaDestroyConfig(OgaConfig* config);

Destroys a configuration object.Parameters:

config: The config to destroy

OgaConfigOverlay

function

OgaResult* OGA_API_CALL OgaConfigOverlay(OgaConfig* config, const char* json);

Overlays JSON configuration on top of the existing config.Parameters:

config: The config to modify
json: JSON string to overlay

Returns: NULL on success, or OgaResult* containing error message on failureExample:

const char* json_overlay = "{\"max_length\": 200}";
OgaResult* result = OgaConfigOverlay(config, json_overlay);
if (result != NULL) {
    fprintf(stderr, "Failed to apply overlay: %s\n", OgaResultGetError(result));
    OgaDestroyResult(result);
}

Provider Configuration

OgaConfigClearProviders

function

OgaResult* OGA_API_CALL OgaConfigClearProviders(OgaConfig* config);

Clears the list of execution providers in the configuration.Parameters:

config: The config to modify

Returns: NULL on success, or OgaResult* containing error message on failure

OgaConfigAppendProvider

function

OgaResult* OGA_API_CALL OgaConfigAppendProvider(OgaConfig* config, const char* provider);

Adds a provider to the end of the provider list. If the provider already exists, this function does nothing.Parameters:

config: The config to modify
provider: The provider name to add (e.g., “CPU”, “CUDA”, “DML”)

Returns: NULL on success, or OgaResult* containing error message on failureExample:

// Add CUDA provider
OgaResult* result = OgaConfigAppendProvider(config, "CUDA");
if (result != NULL) {
    fprintf(stderr, "Failed to add provider: %s\n", OgaResultGetError(result));
    OgaDestroyResult(result);
}

OgaConfigSetProviderOption

function

OgaResult* OGA_API_CALL OgaConfigSetProviderOption(
    OgaConfig* config,
    const char* provider,
    const char* key,
    const char* value
);

Sets a provider-specific option.Parameters:

config: The config to modify
provider: The provider name
key: The option key
value: The option value

Returns: NULL on success, or OgaResult* containing error message on failure

Hardware Device Filtering

OgaConfigSetDecoderProviderOptionsHardwareDeviceType

function

OgaResult* OGA_API_CALL OgaConfigSetDecoderProviderOptionsHardwareDeviceType(
    OgaConfig* config,
    const char* provider,
    const char* hardware_device_type
);

Filters execution provider devices by hardware device type (e.g., “CPU”, “GPU”, “NPU”).Parameters:

config: The config to modify
provider: The provider name
hardware_device_type: The hardware device type

Returns: NULL on success, or OgaResult* containing error message on failure

OgaConfigSetDecoderProviderOptionsHardwareDeviceId

function

OgaResult* OGA_API_CALL OgaConfigSetDecoderProviderOptionsHardwareDeviceId(
    OgaConfig* config,
    const char* provider,
    uint32_t hardware_device_id
);

Filters execution provider devices by hardware device ID.Parameters:

config: The config to modify
provider: The provider name
hardware_device_id: The hardware device ID

Returns: NULL on success, or OgaResult* containing error message on failure

OgaConfigSetDecoderProviderOptionsHardwareVendorId

function

OgaResult* OGA_API_CALL OgaConfigSetDecoderProviderOptionsHardwareVendorId(
    OgaConfig* config,
    const char* provider,
    uint32_t hardware_vendor_id
);

Filters execution provider devices by hardware vendor ID.Parameters:

config: The config to modify
provider: The provider name
hardware_vendor_id: The hardware vendor ID

Returns: NULL on success, or OgaResult* containing error message on failure

OgaConfigClearDecoderProviderOptionsHardwareDeviceType

function

OgaResult* OGA_API_CALL OgaConfigClearDecoderProviderOptionsHardwareDeviceType(
    OgaConfig* config,
    const char* provider
);

Clears the hardware device type filter.

OgaConfigClearDecoderProviderOptionsHardwareDeviceId

function

OgaResult* OGA_API_CALL OgaConfigClearDecoderProviderOptionsHardwareDeviceId(
    OgaConfig* config,
    const char* provider
);

Clears the hardware device ID filter.

OgaConfigClearDecoderProviderOptionsHardwareVendorId

function

OgaResult* OGA_API_CALL OgaConfigClearDecoderProviderOptionsHardwareVendorId(
    OgaConfig* config,
    const char* provider
);

Clears the hardware vendor ID filter.

Model Data from Memory

OgaConfigAddModelData

function

OgaResult* OGA_API_CALL OgaConfigAddModelData(
    OgaConfig* config,
    const char* model_filename,
    const void* model_data,
    size_t model_data_length
);

Adds model data to load the model from memory instead of from disk.Parameters:

config: The config to modify
model_filename: The name of the model file as defined in the config
model_data: Pointer to the model data (must remain valid until model is created)
model_data_length: Length of the model data in bytes

Returns: NULL on success, or OgaResult* containing error message on failureNote: The model data must remain valid at least until the model is created. If using session.use_ort_model_bytes_directly, the data must remain valid until the model is destroyed.

OgaConfigRemoveModelData

function

OgaResult* OGA_API_CALL OgaConfigRemoveModelData(
    OgaConfig* config,
    const char* model_filename
);

Removes previously added model data.Parameters:

config: The config to modify
model_filename: The name of the model file to remove

Returns: NULL on success, or OgaResult* containing error message on failure

Runtime Settings

OgaCreateRuntimeSettings

function

OgaResult* OGA_API_CALL OgaCreateRuntimeSettings(OgaRuntimeSettings** out);

Creates a runtime settings object.Parameters:

out: Pointer to store the created runtime settings

Returns: NULL on success, or OgaResult* containing error message on failure

OgaDestroyRuntimeSettings

function

void OGA_API_CALL OgaDestroyRuntimeSettings(OgaRuntimeSettings* settings);

Destroys a runtime settings object.Parameters:

settings: The runtime settings to destroy

OgaRuntimeSettingsSetHandle

function

OgaResult* OGA_API_CALL OgaRuntimeSettingsSetHandle(
    OgaRuntimeSettings* settings,
    const char* handle_name,
    void* handle
);

Sets a runtime handle (e.g., device context) for the runtime settings.Parameters:

settings: The runtime settings to modify
handle_name: The name of the handle to set
handle: Pointer to the handle value

Returns: NULL on success, or OgaResult* containing error message on failure

Complete Example

#include "ort_genai_c.h"
#include <stdio.h>

int main() {
    // Create configuration
    OgaConfig* config = NULL;
    OgaResult* result = OgaCreateConfig("/path/to/model", &config);
    if (result != NULL) {
        fprintf(stderr, "Error: %s\n", OgaResultGetError(result));
        OgaDestroyResult(result);
        return -1;
    }
    
    // Configure providers
    result = OgaConfigClearProviders(config);
    if (result == NULL) {
        result = OgaConfigAppendProvider(config, "CUDA");
    }
    if (result == NULL) {
        result = OgaConfigAppendProvider(config, "CPU");
    }
    if (result != NULL) {
        fprintf(stderr, "Error configuring providers: %s\n", OgaResultGetError(result));
        OgaDestroyResult(result);
        OgaDestroyConfig(config);
        return -1;
    }
    
    // Create model from config
    OgaModel* model = NULL;
    result = OgaCreateModelFromConfig(config, &model);
    if (result != NULL) {
        fprintf(stderr, "Error creating model: %s\n", OgaResultGetError(result));
        OgaDestroyResult(result);
        OgaDestroyConfig(config);
        return -1;
    }
    
    // Get model properties
    const char* model_type = NULL;
    const char* device_type = NULL;
    
    result = OgaModelGetType(model, &model_type);
    if (result == NULL) {
        printf("Model type: %s\n", model_type);
        OgaDestroyString(model_type);
    }
    
    result = OgaModelGetDeviceType(model, &device_type);
    if (result == NULL) {
        printf("Device type: %s\n", device_type);
        OgaDestroyString(device_type);
    }
    
    // Use the model for generation...
    
    // Cleanup
    OgaDestroyModel(model);
    OgaDestroyConfig(config);
    
    return 0;
}

C API Overview

Learn about memory management and error handling

Generator Functions

Generate text with your loaded model

Python API

C++ API

C# API

C API

Model Creation

Model Properties

Configuration Objects

OgaConfig

Provider Configuration

Hardware Device Filtering

Model Data from Memory

Runtime Settings

Complete Example

See Also

C API Overview

Generator Functions

Build docs developers (and LLMs) love

Python API

C++ API

C# API

C API

Documentation Index

​Model Creation

​Model Properties

​Configuration Objects

​OgaConfig

​Provider Configuration

​Hardware Device Filtering

​Model Data from Memory

​Runtime Settings

​Complete Example

​See Also

C API Overview

Generator Functions

Build docs developers (and LLMs) love

Model Creation

Model Properties

Configuration Objects

OgaConfig

Provider Configuration

Hardware Device Filtering

Model Data from Memory

Runtime Settings

Complete Example

See Also