Devotd Github
NewRAG & retrieval

the-semq-group/semq — Add SimHash precision comparison to the floor benchmark

the-semq-group/semq adds SimHash to the rebuild benchmark and measures on an NVIDIA A10G how much the hash bits differ when the same FP32 projection is computed with different libraries, batch sizes, and precisions.

Repositorythe-semq-group/semq ↗

the-semq-group/semq adds SimHash to the rebuild benchmark and measures on an NVIDIA A10G how much the hash bits differ when the same FP32 projection is computed with different libraries, batch sizes, and precisions.

GitHub description: Deterministic symbolic codes for embedding vectors. Byte-identical across CPUs and languages, content-addressed, diffable and verifiable. C core with Python, Rust, Go and TypeScript bindings.

stars
★ 13
Forks
0
Language
Python
License
NOASSERTION
Latest release
v1.1.0
Read original source ↗View on GitHub ↗

the-semq-group/semq is a repository of deterministic symbolic codes that turn embedding vectors into the same byte string regardless of CPU or language. The 2026-10-22 commit f4ac419(#30) adds SimHash as a new comparison target to the rebuild benchmark, re-examines how closely it gives the same verdict as the baseline floor, and measures how much the hash bits differ when the same FP32 projection is computed with different libraries and different precisions. The evidence remains as-is on the commit page: https://github.com/The-SEMQ-Group/semq/commit/f4ac41952ef980c2b126ca1f3428a2e5636cccbf

SimHash is a method that projects a vector onto a random hyperplane and folds it into a string of sign bits. This commit assigns SimHash the bit budget of floor, 3 bits per dimension. This is because quant4 quantization uses 2×4 symbols, i.e. 3 bits. The verdict takes the distribution seen in an unchanged rebuild as the reference(null) and uses two statistics together: the number of changed rows and the row-wise Hamming distance(how many bit positions differ).

The measurement was done on a single NVIDIA A10G. Taking the FP32 projection computed with NumPy as the reference, PyTorch CPU, GPU batch sizes 64 and 4,096, and TF32 were applied to the same input and the rows whose bits differed were counted; out of a total of 1,150,307 vectors(including 1,000,000 unit vectors drawn with a seed), the PyTorch side differed in 3 to 49 rows per million rows, and TF32 differed in 68,794 rows. Batch size did not affect the number of changed rows on the A10G. A bit difference is not automatically a false positive, so compared against the floor hashed with NumPy, an unchanged rebuild was rejected only in 0.05% of draws under the FP32 hash and 100% under the TF32 hash. FP16 and BF16 were excluded from the study because they deliberately lower precision.

However, this figure is the result of a single A10G. The precision study explicitly states that without CUDA there are no results, so the page fails at that point, and the spec pins down only the quantization stage. Therefore, it is more appropriate to read the 100% rejection of TF32 as a value arising from a precision difference rather than a GPU defect. A way of reporting that a difference whose 95% intervals overlap is overlapping was also brought in, but it is still too early to judge whether the same figures will recur on other GPUs or with other library versions. The changes remained in the benchmarks and documents under benchmarks/ and docs/, and the core library code was not touched.