A language model from its GGUF file, and weights that stay quantized

An experimental local-model plugin overlay arrives, and the GPU protocol learns to keep a checkpoint's Q4_K and Q6_K blocks and decode them inside the matmul on every backend.

277f30fa applies a local-model Python plugin overlay verbatim, checked against its own SHA256SUMS, and says plainly that its integration gates had never run. The rest of the day made it real. 90e0a9c7 and 9472d78f gave the Python frontend relative imports and a no-op torch.autocast, which with 341 lines of shim was enough for Hugging Face's Qwen3, Qwen2, Llama and Mistral modelling files to run exactly as published, within 2e-7 of transformers. The commit makes a point worth keeping: what stood in the way was never numpy, but four tensor operations, autocast, relative imports and those 341 lines.

379a04cc reads GGUF without holding the file: a 6.8 GB checkpoint opens in 35 ms from a 16 MiB header read. ad64fce3 is the decision that matters: a Q4_K weight is 144 bytes per 256 values and 1,024 once expanded, so an input may now carry q4_k or q6_k blocks and every backend decodes them inside a transposed matmul, held to bit-for-bit equality with the same matmul over decoded values. e6c33be3 made the browser side Rust, 6f847191 added a Qwen3 plugin that reads the checkpoint where it lies, and by the evening 03b5c283 moved the container, block formats and tokenizer out to their own project, gguf-wasm, keeping only what is ZIPP's: the kernels and the protocol.