baruquerodrigues/llm-legislativo is a research project that bundles into a single repository a large corpus of Brazilian legislative documents and a causal transformer model trained from scratch in pure PyTorch, without any pretrained weights. Its goal is to structure the official records of the Brazilian Congress as auditable data. It is published under the MIT license and was created on October 13, 2026. There are no public releases.
The data comes from the official historical records of the Brazilian Chamber of Deputies (Câmara dos Deputados) and the Federal Senate (Senado Federal). A byte-level BPE (byte-level BPE) tokenizer with a vocabulary of 8192 was trained from scratch; byte-level BPE splits text into bytes and then merges frequently used byte sequences into vocabulary entries. The causal masked transformer was implemented directly. Corpus dumping, validation and deduplication were handled in R, Scanbot OCR in Swift, and the tokenizer, model, training and evaluation in Python. The project shows a path for training directly on a large specialized corpus in a single language.
Tests are attached at every layer, and the data carries SHA-256 integrity verification. To prevent vocabulary contamination, the tokenizer is trained on the training split only, and leakage between the training and evaluation splits is checked. The model and dataset artifacts — six in total — are on the Hugging Face Hub, so the figures stated in the README can be verified from outside, and evaluation follows the byte_nll_v1 protocol.
That said, the trained model itself is small — 6 layers, d_model 384, context 512 — so its practical value is limited. There are also no performance figures comparing it with other models. In the scores, novelty 7.0, technical 8.0 and quality 8.0 are on the high side, while momentum 5.7 is on the low side, and the star count is 451. What is worth expecting is closer to the language-split pipeline, the per-layer tests and the design of learning the tokenizer only on the training split than to a release.