Performs embedding extraction on images.
Signature:
export declare class ImageEmbedder extends VisionTaskRunner
Extends: VisionTaskRunner
Methods
| Method | Modifiers | Description |
|---|---|---|
| cosineSimilarity(u, v) | static |
Utility function to compute cosine similarity[1] between two Embedding objects.[1]: https://en.wikipedia.org/wiki/Cosine_similarity |
| createFromModelBuffer(wasmFileset, modelAssetBuffer) | static |
Initializes the Wasm runtime and creates a new image embedder based on the provided model asset buffer. |
| createFromModelPath(wasmFileset, modelAssetPath) | static |
Initializes the Wasm runtime and creates a new image embedder based on the path to the model asset. |
| createFromOptions(wasmFileset, imageEmbedderOptions) | static |
Initializes the Wasm runtime and creates a new image embedder from the provided options. |
| embed(image, imageProcessingOptions) | Performs embedding extraction on the provided single image and waits synchronously for the response. Only use this method when the ImageEmbedder is created with running mode image. |
|
| embedForVideo(imageFrame, timestamp, imageProcessingOptions) | Performs embedding extraction on the provided video frame and waits synchronously for the response. Only use this method when the ImageEmbedder is created with running mode video. |
|
| setOptions(options) | Sets new options for the image embedder.Calling setOptions() with a subset of options only affects those options. You can reset an option back to its default value by explicitly setting it to undefined. |
ImageEmbedder.cosineSimilarity()
Utility function to compute cosine similarity[1] between two Embedding objects.
[1]: https://en.wikipedia.org/wiki/Cosine_similarity
Signature:
static cosineSimilarity(u: Embedding, v: Embedding): number;
Parameters
| Parameter | Type | Description |
|---|---|---|
| u | Embedding | |
| v | Embedding |
Returns:
number
Exceptions
if the embeddings are of different types(float vs. quantized), have different sizes, or have an L2-norm of 0.
ImageEmbedder.createFromModelBuffer()
Initializes the Wasm runtime and creates a new image embedder based on the provided model asset buffer.
Signature:
static createFromModelBuffer(wasmFileset: WasmFileset, modelAssetBuffer: Uint8Array): Promise<ImageEmbedder>;
Parameters
| Parameter | Type | Description |
|---|---|---|
| wasmFileset | WasmFileset | A configuration object that provides the location of the Wasm binary and its loader. |
| modelAssetBuffer | Uint8Array | A binary representation of the TFLite model. |
Returns:
Promise<ImageEmbedder>
ImageEmbedder.createFromModelPath()
Initializes the Wasm runtime and creates a new image embedder based on the path to the model asset.
Signature:
static createFromModelPath(wasmFileset: WasmFileset, modelAssetPath: string): Promise<ImageEmbedder>;
Parameters
| Parameter | Type | Description |
|---|---|---|
| wasmFileset | WasmFileset | A configuration object that provides the location of the Wasm binary and its loader. |
| modelAssetPath | string | The path to the TFLite model. |
Returns:
Promise<ImageEmbedder>
ImageEmbedder.createFromOptions()
Initializes the Wasm runtime and creates a new image embedder from the provided options.
Signature:
static createFromOptions(wasmFileset: WasmFileset, imageEmbedderOptions: ImageEmbedderOptions): Promise<ImageEmbedder>;
Parameters
| Parameter | Type | Description |
|---|---|---|
| wasmFileset | WasmFileset | A configuration object that provides the location of the Wasm binary and its loader. |
| imageEmbedderOptions | ImageEmbedderOptions | The options for the image embedder. Note that either a path to the TFLite model or the model itself needs to be provided (via baseOptions). |
Returns:
Promise<ImageEmbedder>
ImageEmbedder.embed()
Performs embedding extraction on the provided single image and waits synchronously for the response. Only use this method when the ImageEmbedder is created with running mode image.
Signature:
embed(image: ImageSource, imageProcessingOptions?: ImageProcessingOptions): ImageEmbedderResult;
Parameters
| Parameter | Type | Description |
|---|---|---|
| image | ImageSource | The image to process. |
| imageProcessingOptions | ImageProcessingOptions | the ImageProcessingOptions specifying how to process the input image before running inference. The classification result of the image |
Returns:
ImageEmbedder.embedForVideo()
Performs embedding extraction on the provided video frame and waits synchronously for the response. Only use this method when the ImageEmbedder is created with running mode video.
Signature:
embedForVideo(imageFrame: ImageSource, timestamp: number, imageProcessingOptions?: ImageProcessingOptions): ImageEmbedderResult;
Parameters
| Parameter | Type | Description |
|---|---|---|
| imageFrame | ImageSource | The image frame to process. |
| timestamp | number | The timestamp of the current frame, in ms. |
| imageProcessingOptions | ImageProcessingOptions | the ImageProcessingOptions specifying how to process the input image before running inference. The classification result of the image |
Returns:
ImageEmbedder.setOptions()
Sets new options for the image embedder.
Calling setOptions() with a subset of options only affects those options. You can reset an option back to its default value by explicitly setting it to undefined.
Signature:
setOptions(options: ImageEmbedderOptions): Promise<void>;
Parameters
| Parameter | Type | Description |
|---|---|---|
| options | ImageEmbedderOptions | The options for the image embedder. |
Returns:
Promise<void>