THE TRANSFORMERS LANE

8b-is ⨯ peterlodri-sec ⨯ the sovereign library · a ternary loader for huggingface/transformers
🜂 quantal-ternary → ⨯ → 🤗 transformers

What this file is: the correspondence and the draft behind one upstream move — PeetPedro/quantal-ternary (Qwen2.5-0.5B continued-trained as BitNet b1.58, masked val 0.5597, 168 packed ternary matrices) running natively in PyTorch through transformers.models.bitnet — no Rust runner required. Ten rounds of review, one PR draft, and one story about a stranger who counted zeros. Seated here 2026-10-07, public, so nothing stays stranded on a local line.

01 · THE DRAFT — [bitnet] Support use_sub_norms=False

BitNet checkpoints come in two forward variants; the weight-quant-only one needs its per-projection sub-layer norms to be absent, not neutralised:

use_sub_norms: bool = True   # default keeps every existing model card unchanged

self.ffn_sub_norm  = BitNetRMSNorm(...) if config.use_sub_norms else None
self.attn_sub_norm = BitNetRMSNorm(...) if config.use_sub_norms else None

An RMSNorm initialised to ones still normalises — the module has to not exist. Forced on, a weight-quant-only checkpoint flips the argmax. With the flag:

promptmax_abs vs golden (Rust-deployed) forwardargmax
prompt 16.497e-0671703 = 71703 · PASS
prompt 26.676e-0671703 = 71703 · PASS

Parity gate: 1e-5. Files: configuration_bitnet.py (+flag), modular_bitnet.py (conditional construction + application; the source of truth), modeling_bitnet.py (generated mirror via make fix-repo), plus a use_sub_norms=False smoke test.

02 · THE THIRD STATE IS REAL

Dipankar Sarkar range-read the published model — no download, just headers and a tensor slice — and unpacks one exported matrix: the scale array held two distinct values ("a scalar wearing a shape"); the codes were a sign matrix, 12 zeros in 4.36M weights. The third state of a 2-bit format, used once in 360,000. His question changed the quantizer:

"Does this model ever want a zero? If the trained weights are all at plus or minus one scale, b1.58 is doing b1 work — and the third state is a slot you are paying for in every file."

It wants zeros — about a third of it (34.25% of the first layer's 21M weights sit below 0.5·scale). A real thresholded ternary, per 64-column group, restored the state:

quantizerMAEzero fraction
sign-based (old)0.010050.00%
ternary t=0.50.0075331.8%
ternary t=1.00.0096158.5%
"Sometimes the win is not a lower loss. It is knowing the loss is real."

03 · THE ROUNDS

dipankar-reply r1 → r10 (r8/r9/r10 also in EN) · grite-reply r2 — ten rounds of correspondence with the readers who audit bytes, archived in 8b-is/hf-mac/docs.

04 · THE WIRE

docs home: 8b-is/hf-mac · E2E state: build + 13 tests / 2 suites green on the keychain-save branch (PR #1, verified 2026-10-07) · sibling file: HF-MAC ⨯ APPLE · wired with: 8b-is · peterlodri-sec · standardgalactic · the sovereign library

the lap's song: TouchTalk — Change It
"If you don't like something, change it. … Life is simple."