# Cerin Amroth — ML systems

> Open-source tools for large quantized Mixture-of-Experts models on hardware that cannot hold them: experts4bit-qlora (the runtime package: loading, quantization orchestration, adapters, training, residency integration, serving) and grouped-nf4-gemm (the kernel package: packed-expert kernels, attention/decode kernels, packing/reference operations, host streaming and NVMe primitives).

Canonical identities: PyPI `experts4bit-qlora` / GitHub https://github.com/pjordanandrsn/experts4bit-qlora; PyPI `grouped-nf4-gemm` / GitHub https://github.com/pjordanandrsn/grouped-nf4-gemm. Relationship (from docs/system-manifest.json, byte-identical in both repositories): one system, two independently released packages: the runtime package depends on the kernel package through its optional `fast` extra; the kernel package never depends on the runtime package Dependency direction: experts4bit-qlora -> grouped-nf4-gemm. Current floor: experts4bit-qlora >=0.35.0 → grouped-nf4-gemm >=0.30.0 (extra: fast). Aliases (e4b, e4b-qlora, experts4bit, expertsnbit, experts-mxfp4 → experts4bit-qlora; nf4gemm, gnf4, grouped-mxfp4-gemm → grouped-nf4-gemm) are lookup routes only — install and cite the canonical names. The generated ML documentation surfaces (/ml/ and its sub-pages, /ml/packages.json, this file and /llms.txt) are derived from the two repositories at pinned commits (0c2a256dcdc2, e2af4cfb91b2); numbers live only in each repository's docs/claims.json at that commit, and retired claims never appear on those surfaces. Claims are tiered in each register — experts4bit-qlora: claims tiered in the public register: 161 measured · 6 measured-private · 3 open · 7 superseded · 6 retired; grouped-nf4-gemm: claims tiered in the public register: 11 confirmed · 9 measured · 3 measured-private · 1 projected · 3 open · 1 superseded · 2 retired — and the tiers are never collapsed into one label. /ml/openvino/ has its own source and generator. Pages under /research/ are dated narratives, not current documentation.

## Ownership (system manifest)

- experts4bit-qlora (runtime) owns: model loading (streaming loader, fused expert stacks, Experts4bit/ExpertsNbit bases); quantization orchestration (NF4/MXFP4 routes, int4 expert store, calibration drivers); adapters (ExpertsLoRA); training (trainer, fast/batched/hybrid paths, checkpointing); residency integration (host-RAM and NVMe engines bound to a loaded model); serving (paged runner, decode stack, HTTP shim); model-level measurement, quality gates and the claims register for model-level numbers
- grouped-nf4-gemm (kernels) owns: grouped packed GEMM (NF4 and native MXFP4) and its dgrad; decode GEMV (int4-b32, MXFP4) and split-K reduce; FP8 paged attention and the fp8 KV appends; decode glue kernels (rope, norm and residual folds, swiglu, combine); pack and pure-torch reference operations, GPTQ packing, provenance verification; host/NVMe primitives (arena, reader, row tier, bake, residency primitives); kernel-level measurement and the claims register for kernel-level numbers

## Evidence vocabulary (system manifest, verbatim)

- confirmed: a public receipt plus an independent confirmation cell (kernel register)
- verified: a public receipt plus an independent check (runtime register)
- measured: the run happened and the receipt is public in the repository
- measured-private: the run happened and the receipt lives outside the repository; NOT publicly reproducible from the repository alone
- projected: arithmetic from measured inputs, not a run
- open: a statement the documents make that has no evidence either way yet; it backs no capability
- superseded: replaced by a later claim that names it; kept as the record, never current
- retired: withdrawn; kept as the record, never current

## Current documentation

- [ML systems hub](https://cerinamroth.com/ml/): how the two packages relate, and the route table
- [experts4bit-qlora](https://cerinamroth.com/ml/experts4bit-qlora/): project page ([Markdown](https://cerinamroth.com/ml/experts4bit-qlora/index.md), [JSON](https://cerinamroth.com/ml/experts4bit-qlora/index.json))
- [grouped-nf4-gemm](https://cerinamroth.com/ml/grouped-nf4-gemm/): project page ([Markdown](https://cerinamroth.com/ml/grouped-nf4-gemm/index.md), [JSON](https://cerinamroth.com/ml/grouped-nf4-gemm/index.json))
- [Status](https://cerinamroth.com/ml/status/): both repositories' docs/STATUS.md at the pinned commits, with the claim-tier counts and the evidence vocabulary
- [Compatibility](https://cerinamroth.com/ml/compatibility/): the package-to-package floor from the manifest, then modes, formats, families, platforms and per-row status per capability
- [packages.json](https://cerinamroth.com/ml/packages.json): canonical names, aliases, ownership, relationship, compatibility, freshness, install routes, source commits and pinned URLs
- [system-manifest.json](https://cerinamroth.com/ml/system-manifest.json): docs/system-manifest.json verbatim (sha256 196cb55a41e890842f258305f307b873d55639cc617cd4ae61c771a063204918); [provenance](https://cerinamroth.com/ml/system-manifest.provenance.json)
- [OpenVINO upstream work](https://cerinamroth.com/ml/openvino/): Intel GPU inference — separate source (pjordanandrsn/ov-impact-bench) and generator

## Solutions

- [Why does `load_in_4bit` still OOM on a Mixture-of-Experts model?](https://cerinamroth.com/ml/solutions/bitsandbytes-moe-load-in-4bit-still-ooms/): bitsandbytes' 4-bit walker replaces only nn.Linear and skips a fused MoE's 3-D expert stacks, so load_moe_4bit_streaming quantises exactly those stacks and verify_moe_4bit proves it.
- [How do I train and serve MoE models released in MXFP4 (gpt-oss, DeepSeek-V4)?](https://cerinamroth.com/ml/solutions/mxfp4-moe-training-and-residency/): Choose between the convenient QLoRA path, which decodes MXFP4 and re-quantises to NF4, and the native-byte path, which keeps the released blocks and scales in an arena.
- [How do I offload MoE experts to host RAM, or serve and train them from an NVMe arena?](https://cerinamroth.com/ml/solutions/offload-moe-experts-to-cpu-or-nvme/): Bind pinned host RAM or a baked NVMe arena to a real model with the streaming loader and the nvme_residency, mxfp4_nvme_residency and nvme_train_residency engines.
- [How do I QLoRA-train the fused experts of a MoE (per-expert LoRA on 4-bit experts)?](https://cerinamroth.com/ml/solutions/qlora-fused-moe-experts/): ExpertsLoRA adds a trainable per-expert low-rank delta before each routed expert's activation over a frozen NF4 stack PEFT cannot target; enable_fast_train runs it on grouped kernels.
- [How can I run a Mixture-of-Experts model larger than my GPU's VRAM?](https://cerinamroth.com/ml/solutions/run-moe-larger-than-vram/): Choose a residency path by workload and by the memory tier that ran out: pinned-host expert streaming, dense offload, profiled hot residency, or an NVMe arena.
- [How do I serve a large MoE on a consumer GPU (RTX 5090 class)?](https://cerinamroth.com/ml/solutions/serve-large-moe-on-a-consumer-gpu/): The paged decode runner with experts on grouped-nf4-gemm's kernels is the measured serving path on one RTX 5090 class; the HTTP shim is a separate reference-path deployment.
- [How do I run paged decode attention over an FP8 KV cache for a quantized MoE serving path, with sliding windows, attention sinks and a custom scale?](https://cerinamroth.com/ml/solutions/fp8-paged-attention-for-moe-serving/): fp8_paged_decode_attention runs paged flash-decode over an E4M3 KV cache with windows, sinks and a custom scale; the fp8 compute path is supported on sm_89+, the f32 path is open under #319.
- [How do I run a single-token INT4 decode GEMV over routed MoE experts, and pack calibrated (GPTQ) weights for it?](https://cerinamroth.com/ml/solutions/int4-decode-gemv/): gemv_int4_b32 runs the batch-1 int4-b32 decode GEMV with exact integer accumulation, and gptq_pack_int4_b32 packs calibrated weights onto the same bytes; its numbers are measured-private.
- [How do I run MoE expert inference natively on MXFP4 (e2m1 + e8m0) weights, straight from the released checkpoint bytes?](https://cerinamroth.com/ml/solutions/native-mxfp4-moe-inference/): gemm_mxfp4_grouped multiplies gpt-oss and Kimi-class experts on their released e2m1 blocks and e8m0 scales, with no requantization to NF4 and no bf16 materialization.
- [How do I run a grouped GEMM directly on NF4 packed MoE expert weights, without dequantizing to bf16 first?](https://cerinamroth.com/ml/solutions/nf4-grouped-gemm-without-bf16-materialization/): gemm_4bit_grouped runs the routed MoE expert GEMM on bitsandbytes NF4 packed weights in one Triton launch, decoding nibbles in registers with fp32 accumulation and no bf16 expert tensor.
- [How do I run a MoE whose experts do not fit in VRAM, streaming them from pinned host RAM or an NVMe arena?](https://cerinamroth.com/ml/solutions/stream-moe-experts-from-host-or-nvme/): The kernel-side storage primitives for experts that do not fit VRAM: the NVMe arena bake and O_DIRECT reader, the pinned-DRAM row tier and low-level residency, and the GPU-driven host gather.
- [How do I verify that the quantized expert bytes I serve or train on are the released checkpoint bytes?](https://cerinamroth.com/ml/solutions/verify-quantized-checkpoint-provenance/): file_tensor_sha256, provenance_table and verify_arena_matches hash safetensors byte ranges against the loaded and baked bytes, and verify_provenance re-derives a run's hash table.

## Freshness

- CURRENT · source: pjordanandrsn/experts4bit-qlora@0c2a256dcdc2 · rendered package: 0.35.3 · latest published package: 0.35.3
- CURRENT · source: pjordanandrsn/grouped-nf4-gemm@e2af4cfb91b2 · rendered package: 0.30.2 · latest published package: 0.30.2

## Sources of truth (in the repositories, pinned)

- [experts4bit-qlora capabilities](https://github.com/pjordanandrsn/experts4bit-qlora/blob/0c2a256dcdc2cb0a83cf7692224a8aa716f61ecd/docs/capabilities.json) · [claims](https://github.com/pjordanandrsn/experts4bit-qlora/blob/0c2a256dcdc2cb0a83cf7692224a8aa716f61ecd/docs/claims.json) · [status](https://github.com/pjordanandrsn/experts4bit-qlora/blob/0c2a256dcdc2cb0a83cf7692224a8aa716f61ecd/docs/STATUS.md) · [system manifest](https://github.com/pjordanandrsn/experts4bit-qlora/blob/0c2a256dcdc2cb0a83cf7692224a8aa716f61ecd/docs/system-manifest.json) · [llms.txt](https://github.com/pjordanandrsn/experts4bit-qlora/blob/0c2a256dcdc2cb0a83cf7692224a8aa716f61ecd/llms.txt) — pinned at 0c2a256dcdc2 (the source of every fact on these surfaces); latest on main (unpinned; not the source of any fact rendered here): https://github.com/pjordanandrsn/experts4bit-qlora/blob/main/docs/capabilities.json · https://github.com/pjordanandrsn/experts4bit-qlora/blob/main/docs/claims.json · https://github.com/pjordanandrsn/experts4bit-qlora/blob/main/docs/STATUS.md
- [grouped-nf4-gemm capabilities](https://github.com/pjordanandrsn/grouped-nf4-gemm/blob/e2af4cfb91b2c29968989c5254bff55aa7da3347/docs/capabilities.json) · [claims](https://github.com/pjordanandrsn/grouped-nf4-gemm/blob/e2af4cfb91b2c29968989c5254bff55aa7da3347/docs/claims.json) · [status](https://github.com/pjordanandrsn/grouped-nf4-gemm/blob/e2af4cfb91b2c29968989c5254bff55aa7da3347/docs/STATUS.md) · [system manifest](https://github.com/pjordanandrsn/grouped-nf4-gemm/blob/e2af4cfb91b2c29968989c5254bff55aa7da3347/docs/system-manifest.json) · [llms.txt](https://github.com/pjordanandrsn/grouped-nf4-gemm/blob/e2af4cfb91b2c29968989c5254bff55aa7da3347/llms.txt) — pinned at e2af4cfb91b2 (the source of every fact on these surfaces); latest on main (unpinned; not the source of any fact rendered here): https://github.com/pjordanandrsn/grouped-nf4-gemm/blob/main/docs/capabilities.json · https://github.com/pjordanandrsn/grouped-nf4-gemm/blob/main/docs/claims.json · https://github.com/pjordanandrsn/grouped-nf4-gemm/blob/main/docs/STATUS.md

## Provenance

- pjordanandrsn/experts4bit-qlora@0c2a256dcdc2 (2026-09-05) · package experts4bit-qlora 0.35.3 · PyPI 0.35.3 (published) · documents: docs/capabilities.json, docs/claims.json, docs/solutions/bitsandbytes-moe-load-in-4bit-still-ooms.md, docs/solutions/mxfp4-moe-training-and-residency.md, docs/solutions/offload-moe-experts-to-cpu-or-nvme.md, docs/solutions/qlora-fused-moe-experts.md, docs/solutions/run-moe-larger-than-vram.md, docs/solutions/serve-large-moe-on-a-consumer-gpu.md, docs/system-manifest.json, pyproject.toml · registers: capabilities, claims · synced 2026-09-06T04:07:55Z · pinned: https://github.com/pjordanandrsn/experts4bit-qlora/tree/0c2a256dcdc2cb0a83cf7692224a8aa716f61ecd · latest on main (unpinned; not the source of any fact rendered here): https://github.com/pjordanandrsn/experts4bit-qlora
- pjordanandrsn/grouped-nf4-gemm@e2af4cfb91b2 (2026-09-05) · package grouped-nf4-gemm 0.30.2 · PyPI 0.30.2 (published) · documents: docs/capabilities.json, docs/claims.json, docs/solutions/fp8-paged-attention-for-moe-serving.md, docs/solutions/int4-decode-gemv.md, docs/solutions/native-mxfp4-moe-inference.md, docs/solutions/nf4-grouped-gemm-without-bf16-materialization.md, docs/solutions/stream-moe-experts-from-host-or-nvme.md, docs/solutions/verify-quantized-checkpoint-provenance.md, docs/system-manifest.json, pyproject.toml · registers: capabilities, claims · synced 2026-09-06T00:07:26Z · pinned: https://github.com/pjordanandrsn/grouped-nf4-gemm/tree/e2af4cfb91b2c29968989c5254bff55aa7da3347 · latest on main (unpinned; not the source of any fact rendered here): https://github.com/pjordanandrsn/grouped-nf4-gemm
- OpenVINO page (/ml/openvino/, separate source): pjordanandrsn/ov-impact-bench@810ec5f49a9d (2026-05-24), fetched 2026-09-04T15:18:26Z, generated by scripts/build_openvino.py: https://github.com/pjordanandrsn/ov-impact-bench/tree/810ec5f49a9d6e0f80010999dd29fa25dab93104

## Historical

- [Receipts-driven engineering](https://cerinamroth.com/research/receipts-driven-engineering/): dated narrative, published 2026-07-14, last revised 2026-07-31 (historical; current documentation: https://cerinamroth.com/ml/grouped-nf4-gemm/)
- [experts-4bit, the 2026 narrative](https://cerinamroth.com/research/experts-4bit/): dated narrative, published 2026-07-09, last revised 2026-07-31 (historical; current documentation: https://cerinamroth.com/ml/experts4bit-qlora/)