TfLiteContext

Retrieves the corresponding TfLiteContext of a subgraph that the given subgraph_index points to and switches to the delegate context for that subgraph.

If an invalid subgraph index is given, returns kTfLiteError.

NOTE: This function is expected to be paired with ReleaseSubgraphContext() once the delegate preparation is done and/or the delegate context functions are no longer needed.

WARNING: This is an experimental interface that is subject to change.

Add tensors_to_add tensors, preserving pre-existing Tensor entries.

If non-null, the value pointed to by first_new_tensor_index will be set to the index of the first new tensor.

Allocate a buffer which will be deallocated right after invoke phase.

The memory is allocated from heap in TFL, and from volatile arena in TFLM. This method is only available in invoke stage.

NOTE: If possible use RequestScratchBufferInArena method to avoid memory allocation during inference time.

WARNING: This is an experimental interface that is subject to change.

Allocate persistent buffer which has the same life time as the interpreter.

Returns nullptr on failure. The memory is allocated from heap for TFL, and from tail in TFLM. This method is only available in Init or Prepare stage.

WARNING: This is an experimental interface that is subject to change.

Returns a TfLiteEvalTensor struct for a given index.

WARNING: This is an experimental interface that is subject to change.

WARNING: This method may not be available on all platforms.

The execution plan contains a list of the node indices in execution order.

execution_plan->size is the current number of nodes. And, execution_plan->data[0] is the first node that needs to be run. TfLiteDelegates can traverse the current execution plan by iterating through each member of this array and using GetNodeAndRegistration() to access details about a node. i.e.

TfLiteIntArray* execution_plan;
TF_LITE_ENSURE_STATUS(context->GetExecutionPlan(context,
                                                &execution_plan));
for (int exec_index = 0; exec_index < execution_plan->size;
      exec_index++) {
   int node_index = execution_plan->data[exec_index];
   TfLiteNode* node;
   TfLiteRegistration* reg;
   context->GetNodeAndRegistration(context, node_index, &node, ®);
}

Note: the memory pointed by '*execution_plan is OWNED by TfLite runtime. Future calls to GetExecutionPlan invalidates earlier outputs. The following code snippet shows the issue of such an invocation pattern. After calling CheckNode, subsequent access to plan_1st is undefined.

void CheckNode(const TfLiteNode* node) {
  ...
  TfLiteIntArray* plan_2nd;
  TF_LITE_ENSURE_STATUS(
      context->GetExecutionPlan(context, &plan_2nd)
  );
  ...
}

TfLiteIntArray* plan_1st;
TF_LITE_ENSURE_STATUS(context->GetExecutionPlan(context, &plan_1st));
for (int exec_index = 0; exec_index < plan_1st->size; exec_index++) {
   int node_index = plan_1st->data[exec_index];
   TfLiteNode* node;
   TfLiteRegistration* reg;
   context->GetNodeAndRegistration(context, node_index, &node, ®);
   CheckNode(node);
}

WARNING: This is an experimental interface that is subject to change.

Access external contexts by type.

WARNING: This is an experimental interface that is subject to change.

Retrieves named metadata buffer from the TFLite model.

Returns kTfLiteOk if metadata is successfully obtained from the flatbuffer Model: that is, there exists a metadata entry with given name string. (see TFLite's schema.fbs). The corresponding buffer information is populated in ptr & bytes. The data from ptr is valid for the lifetime of the Interpreter.

WARNING: This is an experimental interface that is subject to change.

Get a Tensor node by node_index.

WARNING: This is an experimental interface that is subject to change.

Get the scratch buffer pointer.

This method is only available in Eval stage.

WARNING: This is an experimental interface that is subject to change.

Returns a TfLiteTensor struct for a given index.

WARNING: This is an experimental interface that is subject to change.

WARNING: This method may not be available on all platforms.

This method provides a preview of post-delegation partitioning.

Each TfLiteDelegateParams in the referenced array corresponds to one instance of the delegate kernel. Example usage:

TfLiteIntArray* nodes_to_replace = ...;
TfLiteDelegateParams* params_array;
int num_partitions = 0;
TF_LITE_ENSURE_STATUS(context->PreviewDelegatePartitioning(
   context, delegate, nodes_to_replace, &params_array,
   &num_partitions));
for (int idx = 0; idx < num_partitions; idx++) {
   const auto& partition_params = params_array[idx];
   ...
}

NOTE: The context owns the memory referenced by partition_params_array. It will be cleared with another call to PreviewDelegatePartitioning, or after TfLiteDelegateParams::Prepare returns.

WARNING: This is an experimental interface that is subject to change.

Releases the subgraph context by switching back to the TFLite kernel context for the subgraph that the given subgraph_index points to.

NOTE: This function is expected to be used after AcquireSubgraphContext() once the delegate preparation is done and/or the delegate context functions are no longer needed.

WARNING: This is an experimental interface that is subject to change.

Replace ops with one or more stub delegate operations.

This function does not take ownership of nodes_to_replace.

Request that an error be reported with format string msg.

Request a scratch buffer in the arena through static memory planning.

This method is only available in Prepare stage and the buffer is allocated by the interpreter between Prepare and Eval stage. In Eval stage, GetScratchBuffer API can be used to fetch the address.

WARNING: This is an experimental interface that is subject to change.

Request memory pointer be resized.

Updates dimensions on the tensor. NOTE: ResizeTensor takes ownership of newSize.

Resize the memory pointer of the tensor.

This method behaves the same as ResizeTensor, except that it makes a copy of the shape array internally so the shape array could be deallocated right afterwards.

WARNING: This is an experimental interface that is subject to change.

Set the value of a external context.

Does not take ownership of the pointer.

WARNING: This is an experimental interface that is subject to change.

Flag for allowing float16 precision for FP32 calculation.

default: false.

WARNING: This is an experimental API and subject to change.

opaque full context ptr (an opaque c++ data structure)

Pointer to the op-level profiler, if set; nullptr otherwise.

Number of threads that are recommended to subsystems like gemmlowp and eigen.

An array of tensors in the interpreter context (of length tensors_size)

Number of tensors in the context.