Introducing LiteRT : Google's high-performance runtime for on-device AI, formerly known as TensorFlow Lite.
Learn more
Send feedback
tflite_support.metadata_writers.metadata_info.RegexTokenizerMd
A container for the Regex tokenizer 1 metadata information.
View aliases
Main aliases
tflite_support.metadata_writers.audio_classifier.metadata_info.RegexTokenizerMdtflite_support.metadata_writers.audio_classifier.metadata_writer.metadata_info.RegexTokenizerMdtflite_support.metadata_writers.bert_nl_classifier.metadata_info.RegexTokenizerMdtflite_support.metadata_writers.bert_nl_classifier.metadata_writer.metadata_info.RegexTokenizerMdtflite_support.metadata_writers.image_classifier.metadata_info.RegexTokenizerMdtflite_support.metadata_writers.image_classifier.metadata_writer.metadata_info.RegexTokenizerMdtflite_support.metadata_writers.image_segmenter.metadata_info.RegexTokenizerMdtflite_support.metadata_writers.image_segmenter.metadata_writer.metadata_info.RegexTokenizerMdtflite_support.metadata_writers.nl_classifier.metadata_info.RegexTokenizerMdtflite_support.metadata_writers.nl_classifier.metadata_writer.metadata_info.RegexTokenizerMdtflite_support.metadata_writers.object_detector.metadata_info.RegexTokenizerMdtflite_support.metadata_writers.object_detector.metadata_writer.metadata_info.RegexTokenizerMd
tflite_support . metadata_writers . metadata_info . RegexTokenizerMd (
delim_regex_pattern : str , vocab_file_path : str
)
Args
delim_regex_pattern
the regular expression to segment strings and create
tokens.
vocab_file_path
path to the vocabulary file.
Methods
View source
create_metadata () -> tflite_support . metadata_schema_py_generated . ProcessUnitT Creates the Bert tokenizer metadata based on the information.
Returns
A Flatbuffers Python object of the Bert tokenizer metadata.
Send feedback
Except as otherwise noted, the content of this page is licensed under the Creative Commons Attribution 4.0 License , and code samples are licensed under the Apache 2.0 License . For details, see the Google Developers Site Policies . Java is a registered trademark of Oracle and/or its affiliates.
Last updated 2024-05-08 UTC.
Need to tell us more?
[[["Easy to understand","easyToUnderstand","thumb-up"],["Solved my problem","solvedMyProblem","thumb-up"],["Other","otherUp","thumb-up"]],[["Missing the information I need","missingTheInformationINeed","thumb-down"],["Too complicated / too many steps","tooComplicatedTooManySteps","thumb-down"],["Out of date","outOfDate","thumb-down"],["Samples / code issue","samplesCodeIssue","thumb-down"],["Other","otherDown","thumb-down"]],["Last updated 2024-05-08 UTC."],[],[]]
View source on GitHub