What this file is: the correspondence and the draft behind one
upstream move — PeetPedro/quantal-ternary (Qwen2.5-0.5B continued-trained as
BitNet b1.58, masked val 0.5597, 168 packed ternary matrices) running natively in
PyTorch through transformers.models.bitnet — no Rust runner required. Ten rounds
of review, one PR draft, and one story about a stranger who counted zeros. Seated here
2026-10-07, public, so nothing stays stranded on a local line.
BitNet checkpoints come in two forward variants; the weight-quant-only one needs its per-projection sub-layer norms to be absent, not neutralised:
use_sub_norms: bool = True # default keeps every existing model card unchanged self.ffn_sub_norm = BitNetRMSNorm(...) if config.use_sub_norms else None self.attn_sub_norm = BitNetRMSNorm(...) if config.use_sub_norms else None
An RMSNorm initialised to ones still normalises — the module has to not exist. Forced on, a weight-quant-only checkpoint flips the argmax. With the flag:
| prompt | max_abs vs golden (Rust-deployed) forward | argmax |
|---|---|---|
| prompt 1 | 6.497e-06 | 71703 = 71703 · PASS |
| prompt 2 | 6.676e-06 | 71703 = 71703 · PASS |
Parity gate: 1e-5. Files: configuration_bitnet.py (+flag),
modular_bitnet.py (conditional construction + application; the source of truth),
modeling_bitnet.py (generated mirror via make fix-repo), plus a
use_sub_norms=False smoke test.
Dipankar Sarkar range-read the published model — no download, just headers and a tensor slice — and unpacks one exported matrix: the scale array held two distinct values ("a scalar wearing a shape"); the codes were a sign matrix, 12 zeros in 4.36M weights. The third state of a 2-bit format, used once in 360,000. His question changed the quantizer:
It wants zeros — about a third of it (34.25% of the first layer's 21M weights sit below 0.5·scale). A real thresholded ternary, per 64-column group, restored the state:
| quantizer | MAE | zero fraction |
|---|---|---|
| sign-based (old) | 0.01005 | 0.00% |
| ternary t=0.5 | 0.00753 | 31.8% |
| ternary t=1.0 | 0.00961 | 58.5% |
dipankar-reply r1 → r10 (r8/r9/r10 also in EN) · grite-reply r2 — ten rounds of correspondence with the readers who audit bytes, archived in 8b-is/hf-mac/docs.