litert::Options

#include <litert_options.h>

Manages the configuration options for compiling a LiteRT model.

Summary

This class provides methods to set hardware accelerators, add custom operations, bind external tensors, and configure various backend-specific options (e.g., GPU, CPU, Qualcomm, MediaTek, etc.).

Inheritance

Inherits from: internal::Handle< LiteRtOptions, LiteRtDestroyOptions >

Constructors and Destructors
Options()
Options(LiteRtOptions compilation_options, OwnHandle owned)
Constructs an Options object from a C handle.

Public types
ScopedWeightSectionMap using
absl::flat_hash_map< std::string, ScopedWeightSection >
A map from a group name to a weight section.

Friend classes
CompiledModel
friend class
CompiledModelNext
friend class
tools::ApplyPlugin
friend LiteRtStatus

Public functions
AddCustomOpKernel(const std::string & custom_op_name, int custom_op_version, const LiteRtCustomOpKernel & custom_op_kernel, void *custom_op_kernel_user_data)
Expected< void >
Adds a custom operator kernel.
AddCustomOpKernel(CustomOpKernel & custom_op_kernel)
Expected< void >
Adds a custom operator kernel.
AddExternalTensorBinding(const std::string & signature_name, const std::string & tensor_name, void *data, size_t size_bytes)
Expected< void >
Binds an external memory buffer to a specific tensor in the model.
AddOpaqueOptions(OpaqueOptions && options)
Expected< void >
Deprecated. Use the GetXXXOptions() methods instead.
GetCompilerOptions()
Expected< CompilerOptions & >
Returns a reference to the compiler options.
GetCpuOptions()
Expected< CpuOptions & >
Returns a reference to the CPU options.
GetGoogleTensorOptions()
Expected< google_tensor::GoogleTensorOptions & >
Returns a reference to the Google Tensor options.
GetGpuOptions()
Expected< GpuOptions & >
Returns a reference to the GPU options.
GetHardwareAccelerators()
Expected< LiteRtHwAcceleratorSet >
Retrieves the currently set hardware accelerators.
GetIntelOpenVinoOptions()
Expected< intel_openvino::IntelOpenVinoOptions & >
Returns a reference to the Intel OpenVINO options.
GetMediatekOptions()
Expected< mediatek::MediatekOptions & >
Returns a reference to the MediaTek options.
GetOpaqueOptions()
Retrieves the opaque options.
GetQualcommOptions()
Expected< qualcomm::QualcommOptions & >
Returns a reference to the Qualcomm options.
GetRuntimeOptions()
Expected< RuntimeOptions & >
Returns a reference to the runtime options.
SetExternalWeightScopedFile(ScopedFile & scoped_file, ScopedWeightSectionMap sections)
Expected< void >
Registers a ScopedFile that contains all external buffer groups.
SetHardwareAccelerators(HwAccelerators accelerators)
Expected< void >
Sets the hardware accelerators to be used for the model.
SetHardwareAccelerators(HwAcceleratorSet accelerators)
Expected< void >
Sets the hardware accelerators to be used for the model.

Public static functions
Create()
Creates a new Options object.

Public types

ScopedWeightSectionMap

absl::flat_hash_map< std::string, ScopedWeightSection > ScopedWeightSectionMap

A map from a group name to a weight section.

A weight section contains the offset and length of a contiguous region inside a ScopedFile that backs a single external buffer group. This map provides the mapping between the group name and its section.

Friend classes

CompiledModel

friend class CompiledModel

CompiledModelNext

friend class CompiledModelNext

tools::ApplyPlugin

friend LiteRtStatus tools::ApplyPlugin(std::unique_ptr< tools::ApplyPluginRun > run)

Public functions

AddCustomOpKernel

Expected< void > AddCustomOpKernel(
  const std::string & custom_op_name,
  int custom_op_version,
  const LiteRtCustomOpKernel & custom_op_kernel,
  void *custom_op_kernel_user_data
)

Adds a custom operator kernel.

Details
Parameters
custom_op_name
The name of the custom operator.
custom_op_version
The version of the custom operator.
custom_op_kernel
The custom operator kernel implementation.
custom_op_kernel_user_data
User data to be passed to the kernel.
Returns
An Expected object that is empty on success, or contains an error.

AddCustomOpKernel

Expected< void > AddCustomOpKernel(
  CustomOpKernel & custom_op_kernel
)

Adds a custom operator kernel.

Details
Parameters
custom_op_kernel
The custom operator kernel to add.
Returns
An Expected object that is empty on success, or contains an error.

AddExternalTensorBinding

Expected< void > AddExternalTensorBinding(
  const std::string & signature_name,
  const std::string & tensor_name,
  void *data,
  size_t size_bytes
)

Binds an external memory buffer to a specific tensor in the model.

This function sets the tensor's allocation type to kTfLiteCustom, making it appear as a constant tensor with a pre-allocated buffer.

Details
Parameters
signature_name
The name of the signature containing the tensor.
tensor_name
The name of the tensor to bind.
data
A pointer to the external memory buffer.
size_bytes
The size of the external memory buffer in bytes.
Returns
An Expected object that is empty on success, or contains an error.

AddOpaqueOptions

Expected< void > AddOpaqueOptions(
  OpaqueOptions && options
)

Deprecated. Use the GetXXXOptions() methods instead.

GetCompilerOptions

Expected< CompilerOptions & > GetCompilerOptions()

Returns a reference to the compiler options.

GetCpuOptions

Expected< CpuOptions & > GetCpuOptions()

Returns a reference to the CPU options.

Use this to configure CPU-specific settings.

GetGoogleTensorOptions

Expected< google_tensor::GoogleTensorOptions & > GetGoogleTensorOptions()

Returns a reference to the Google Tensor options.

Use this to configure Google Tensor-specific settings.

GetGpuOptions

Expected< GpuOptions & > GetGpuOptions()

Returns a reference to the GPU options.

Use this to configure GPU-specific settings.

GetHardwareAccelerators

Expected< LiteRtHwAcceleratorSet > GetHardwareAccelerators()

Retrieves the currently set hardware accelerators.

Details
Returns
An Expected object containing the set of hardware accelerators, or an error.

GetIntelOpenVinoOptions

Expected< intel_openvino::IntelOpenVinoOptions & > GetIntelOpenVinoOptions()

Returns a reference to the Intel OpenVINO options.

Use this to configure Intel OpenVINO-specific settings.

GetMediatekOptions

Expected< mediatek::MediatekOptions & > GetMediatekOptions()

Returns a reference to the MediaTek options.

Use this to configure MediaTek-specific settings.

GetOpaqueOptions

Expected< OpaqueOptions > GetOpaqueOptions()

Retrieves the opaque options.

Details
Returns
An Expected object containing the opaque options, or an error.

GetQualcommOptions

Expected< qualcomm::QualcommOptions & > GetQualcommOptions()

Returns a reference to the Qualcomm options.

Use this to configure Qualcomm-specific settings.

GetRuntimeOptions

Expected< RuntimeOptions & > GetRuntimeOptions()

Returns a reference to the runtime options.

Options

 Options()=default

Options

 Options(
  LiteRtOptions compilation_options,
  OwnHandle owned
)

Constructs an Options object from a C handle.

Details
Parameters
compilation_options
The C handle to the LiteRT options.
owned
Indicates whether this object should take ownership of the provided handle.

SetExternalWeightScopedFile

Expected< void > SetExternalWeightScopedFile(
  ScopedFile & scoped_file,
  ScopedWeightSectionMap sections
)

Registers a ScopedFile that contains all external buffer groups.

Details
Parameters
scoped_file
The file containing the external weights.
sections
A map from group names to their respective sections in the file.
Returns
An Expected object that is empty on success, or contains an error.

SetHardwareAccelerators

Expected< void > SetHardwareAccelerators(
  HwAccelerators accelerators
)

Sets the hardware accelerators to be used for the model.

Details
Parameters
accelerators
A bitmask of hardware accelerators.
Returns
An Expected object that is empty on success, or contains an error.

SetHardwareAccelerators

Expected< void > SetHardwareAccelerators(
  HwAcceleratorSet accelerators
)

Sets the hardware accelerators to be used for the model.

Details
Parameters
accelerators
A set of hardware accelerators.
Returns
An Expected object that is empty on success, or contains an error.

Public static functions

Create

Expected< Options > Create()

Creates a new Options object.

Details
Returns
An Expected object containing the new Options instance, or an error if creation fails.