bitsandbytes-foundation/bitsandbytes is a library that quantizes PyTorch model weights to 4 or 8 bits to reduce the memory required for inference and training. It is a project for working with large language models using less memory, and its latest release is 0.50.2. However, the change list for this version is not contained in the given material, so it cannot be known what was added and changed.
Quantization is a method of approximating weight values with fewer bits to reduce storage space. The README states that this library supports 8-bit inference via LLM.int8(), quantization-based training via QLoRA, and reduction of optimizer-state memory via 8-bit optimizers. It ships quantization primitives for 8·4-bit operations together with the bitsandbytes.optim module, and the Linear4bit and Linear8bitLt layers reduce the memory occupied by weights during inference. The license is MIT, the primary language is Python, it was created on June 4, 2021, and the recorded star count is 8529.
However, this record contains no figures showing the magnitude of the memory savings and no adoption cases. In the evaluation scores, novelty, technology, and quality are left unfilled, with only momentum remaining at 1.3. Broad support and large real-world savings are not the same statement, and unless measured figures can be confirmed from the material, this library is a project worth comparing only as a candidate.