github.devotd.
NewAI

decoderesearch/saelens v6.54.12 — Interpreting Language Model Internals with SAEs

decoderesearch/saelens is a Python library that trains and analyzes sparse autoencoders (SAEs) on language model activations to interpret internal representations.

Repositorydecoderesearch/saelens ↗

decoderesearch/saelens is a Python library that trains and analyzes sparse autoencoders (SAEs) on language model activations to interpret internal representations.

GitHub description: Training Sparse Autoencoders on Language Models

stars
★ 1.6K
Forks
295
Language
Python
License
MIT
Latest release
v6.54.12
Read original source ↗View on GitHub ↗

decoderesearch/saelens is a Python library that trains sparse autoencoders (SAEs) on language model activations to analyze the representations inside the model. The latest public release is v6.54.12, and the collected records contain no changelog for this version, so what exactly changed is not given.

An SAE works by tracing a single activation back to a small number of features, and this library helps learn and analyze those features so as to study mechanistic interpretability. The README states that inference is possible with all PyTorch-based models, that the library is deeply integrated with TransformerLens, and that activations extracted from Hugging Face Transformers or NNsight can be passed to the SAE's encode() and decode() methods. It also explains how to load a pretrained SAE for analysis, and how to train one yourself and then build a feature dashboard with the SAE-Vis Library. The license is MIT, the primary language is Python, and the repository has 1566 GitHub stars.

However, no novelty, technology, or quality scores have been assigned, so the only evaluation value available as a comparison baseline is a momentum score of 1.4 out of 10. The features described in the README are not results confirmed by independent measurement, and the changelog for v6.54.12 is also unknown. In practice, it is better to check the release notes first before starting.