ndif-team/nnsight is a library that intervenes in the internal computation of trained models to read and modify activations, layer outputs, attention patterns, and gradients. The language is Python and the license is MIT, and the latest release is v0.8.1. However, no information available to confirm the change list for this version, so this post does not specify it, and the exact contents have to be checked directly on the GitHub release page.
It works by recording an execution path (trace). Writing ordinary Python code inside a model.trace(...) block in execution order lets you observe, overwrite, or save the value at each step. It provides wrappers not only for PyTorch models but also for HuggingFace, diffusers, and vLLM, and the same trace can run on local models or on remote models in the NDIF infrastructure. It also handles sweeping over generation steps, processing multiple prompts at once, and recomputing gradients. The repository was created in October 2023 and has collected 1125 stars.
There is no evidence that it makes models run better in performance or accuracy. There are no comparison figures, most score items are unfilled, and the only filled momentum is as low as 1 out of 10. It is best regarded as a tool for looking inside the internals of a model you already have, and there is no basis for expecting it as a library that lifts training or inference performance.