google-ai-edge/litert-lm v0.19.0 added logit masking for GPU-constrained decoding, a web SDK that supports image and audio input, and support for several speech models. This project is Google's open-source inference framework for deploying large language models on edge devices. The source of these changes is the v0.19.0 release in the GitHub repository. This update covers decoding features, input paths, and speech model support together.
Logit masking was added for GPU-constrained decoding, and the web SDK supports image and audio input. With the addition of support for several speech models, the scope of the release extends to web input and speech models. This is not simply a version number bump of the inference framework, but a release that also includes the features and API changes stated in the release summary. However, the specific operating conditions of each feature and the list of supported models are not given in the provided release information.
This release also includes a breaking compatibility change in C API 1.0.0. Because the return types of functions that can fail and the exported symbols change, existing code that uses the C API may be affected in terms of compatibility. Performance figures and per-model memory usage are not in the provided information. Therefore, what can be confirmed in this version are the feature additions and the C API changes, and there is no basis for judging the extent of any actual performance improvement.