{
  "schema_version": "1.0",
  "url": "https://cerinamroth.com/ml/",
  "title": "MoE Training on Consumer GPUs — Loggetta & Cerin Amroth",
  "description": "Train large MoE models on one GPU with Loggetta. Install the stack, check model support, and compare public benchmarks for speed and memory use.",
  "date_modified": "2026-10-24",
  "representations": {
    "json": "https://cerinamroth.com/data/pages/ml.json",
    "markdown": "https://cerinamroth.com/data/pages/ml.md"
  },
  "headings": [
    {
      "level": 1,
      "id": null,
      "text": "Large models. Smaller machines."
    },
    {
      "level": 2,
      "id": "measured-speed",
      "text": "A result you can check"
    },
    {
      "level": 2,
      "id": "released-and-research",
      "text": "What you can use today"
    },
    {
      "level": 3,
      "id": "models-supported-vs-tested",
      "text": "Check your model"
    },
    {
      "level": 2,
      "id": "how-the-stack-relates",
      "text": "One install, three projects"
    },
    {
      "level": 2,
      "id": "upstream-work",
      "text": "Contributions upstream"
    },
    {
      "level": 2,
      "id": "start-with-the-need",
      "text": "Choose your next step"
    },
    {
      "level": 3,
      "id": "evidence-vocabulary",
      "text": "Evidence vocabulary"
    },
    {
      "level": 3,
      "id": "solutions",
      "text": "Task guides"
    },
    {
      "level": 2,
      "id": "identity",
      "text": "Data and documentation"
    }
  ],
  "links": [
    {
      "text": "Home",
      "url": "https://cerinamroth.com/"
    },
    {
      "text": "Get started on GitHub",
      "url": "https://github.com/pjordanandrsn/loggetta"
    },
    {
      "text": "PyPI 0.7.0",
      "url": "https://pypi.org/project/loggetta/0.7.0/"
    },
    {
      "text": "Read the benchmark",
      "url": "https://github.com/pjordanandrsn/experts4bit-qlora/blob/605d21e67958fa94d5e73fc3a95ffcb5c8eae02d/bench/h2h-2026-10-02/tc1/RESULTS-tc1-pos69.md"
    },
    {
      "text": "Latest results and earlier readings",
      "url": "https://cerinamroth.com/research/#update-2026-10-08"
    },
    {
      "text": "guide",
      "url": "https://github.com/pjordanandrsn/loggetta/blob/v0.7.0/docs/TRAINING.md"
    },
    {
      "text": "supported and tested model matrix",
      "url": "https://github.com/pjordanandrsn/loggetta/blob/v0.7.0/PYPI.md#models"
    },
    {
      "text": "Loggetta",
      "url": "https://github.com/pjordanandrsn/loggetta"
    },
    {
      "text": "experts4bit-qlora",
      "url": "https://cerinamroth.com/ml/experts4bit-qlora/"
    },
    {
      "text": "grouped-nf4-gemm",
      "url": "https://cerinamroth.com/ml/grouped-nf4-gemm/"
    },
    {
      "text": "PR #852",
      "url": "https://github.com/unslothai/unsloth-zoo/pull/852"
    },
    {
      "text": "Plan and run training",
      "url": "https://github.com/pjordanandrsn/loggetta"
    },
    {
      "text": "Solve a memory, training, or kernel problem",
      "url": "https://cerinamroth.com/ml/solutions/"
    },
    {
      "text": "Check model and hardware requirements",
      "url": "https://cerinamroth.com/ml/compatibility/"
    },
    {
      "text": "Discuss an integration or research project",
      "url": "https://jordananderson.work/#contact"
    },
    {
      "text": "v0.58.1 · current status →",
      "url": "https://cerinamroth.com/ml/status/#experts4bit-qlora"
    },
    {
      "text": "v0.51.0 · current status →",
      "url": "https://cerinamroth.com/ml/status/#grouped-nf4-gemm"
    },
    {
      "text": "experts4bit-qlora >=0.55.0 → grouped-nf4-gemm >=0.43.0 (extra: fast)",
      "url": "https://cerinamroth.com/ml/compatibility/#package-compatibility"
    },
    {
      "text": "Why does `load_in_4bit` still OOM on a Mixture-of-Experts model?",
      "url": "https://cerinamroth.com/ml/solutions/bitsandbytes-moe-load-in-4bit-still-ooms/"
    },
    {
      "text": "How do I train and serve MoE models released in MXFP4 (gpt-oss, DeepSeek-V4)?",
      "url": "https://cerinamroth.com/ml/solutions/mxfp4-moe-training-and-residency/"
    },
    {
      "text": "How do I offload MoE experts to host RAM, or serve and train them from an NVMe arena?",
      "url": "https://cerinamroth.com/ml/solutions/offload-moe-experts-to-cpu-or-nvme/"
    },
    {
      "text": "How do I QLoRA-train the fused experts of a MoE (per-expert LoRA on 4-bit experts)?",
      "url": "https://cerinamroth.com/ml/solutions/qlora-fused-moe-experts/"
    },
    {
      "text": "How can I run a Mixture-of-Experts model larger than my GPU's VRAM?",
      "url": "https://cerinamroth.com/ml/solutions/run-moe-larger-than-vram/"
    },
    {
      "text": "How do I serve a large MoE on a consumer GPU (RTX 5090 class)?",
      "url": "https://cerinamroth.com/ml/solutions/serve-large-moe-on-a-consumer-gpu/"
    },
    {
      "text": "How do I run paged decode attention over an FP8 KV cache for a quantized MoE serving path, with sliding windows, attention sinks and a custom scale?",
      "url": "https://cerinamroth.com/ml/solutions/fp8-paged-attention-for-moe-serving/"
    },
    {
      "text": "How do I run a single-token INT4 decode GEMV over routed MoE experts, and pack calibrated (GPTQ) weights for it?",
      "url": "https://cerinamroth.com/ml/solutions/int4-decode-gemv/"
    },
    {
      "text": "How do I run MoE expert inference natively on MXFP4 (e2m1 + e8m0) weights, straight from the released checkpoint bytes?",
      "url": "https://cerinamroth.com/ml/solutions/native-mxfp4-moe-inference/"
    },
    {
      "text": "How do I run a grouped GEMM directly on NF4 packed MoE expert weights, without dequantizing to bf16 first?",
      "url": "https://cerinamroth.com/ml/solutions/nf4-grouped-gemm-without-bf16-materialization/"
    },
    {
      "text": "How do I run a MoE whose experts do not fit in VRAM, streaming them from pinned host RAM or an NVMe arena?",
      "url": "https://cerinamroth.com/ml/solutions/stream-moe-experts-from-host-or-nvme/"
    },
    {
      "text": "How do I verify that the quantized expert bytes I serve or train on are the released checkpoint bytes?",
      "url": "https://cerinamroth.com/ml/solutions/verify-quantized-checkpoint-provenance/"
    },
    {
      "text": "Site index",
      "url": "https://cerinamroth.com/site.json"
    },
    {
      "text": "Package data",
      "url": "https://cerinamroth.com/ml/packages.json"
    },
    {
      "text": "Compatibility manifest",
      "url": "https://cerinamroth.com/ml/system-manifest.json"
    },
    {
      "text": "Source record",
      "url": "https://cerinamroth.com/ml/system-manifest.provenance.json"
    },
    {
      "text": "Latest research data",
      "url": "https://cerinamroth.com/research/latest.json"
    },
    {
      "text": "AI reading guide",
      "url": "https://cerinamroth.com/ml/llms.txt"
    },
    {
      "text": "OpenVINO upstream fixes",
      "url": "https://cerinamroth.com/ml/openvino/"
    },
    {
      "text": "Research notes",
      "url": "https://cerinamroth.com/research/"
    },
    {
      "text": "Hugging Face",
      "url": "https://huggingface.co/pjordanandrsn"
    },
    {
      "text": "pjordanandrsn/experts4bit-qlora@605d21e67958",
      "url": "https://github.com/pjordanandrsn/experts4bit-qlora/commit/605d21e67958fa94d5e73fc3a95ffcb5c8eae02d"
    },
    {
      "text": "pjordanandrsn/grouped-nf4-gemm@30adb1bfa76f",
      "url": "https://github.com/pjordanandrsn/grouped-nf4-gemm/commit/30adb1bfa76f95f9ce4a4c29734b8f36de907eca"
    },
    {
      "text": "pjordanandrsn/experts4bit-qlora@605d21e67958",
      "url": "https://github.com/pjordanandrsn/experts4bit-qlora/commit/605d21e67958fa94d5e73fc3a95ffcb5c8eae02d"
    },
    {
      "text": "docs/capabilities.json",
      "url": "https://github.com/pjordanandrsn/experts4bit-qlora/blob/605d21e67958fa94d5e73fc3a95ffcb5c8eae02d/docs/capabilities.json"
    },
    {
      "text": "docs/claims.json",
      "url": "https://github.com/pjordanandrsn/experts4bit-qlora/blob/605d21e67958fa94d5e73fc3a95ffcb5c8eae02d/docs/claims.json"
    },
    {
      "text": "docs/solutions/bitsandbytes-moe-load-in-4bit-still-ooms.md",
      "url": "https://github.com/pjordanandrsn/experts4bit-qlora/blob/605d21e67958fa94d5e73fc3a95ffcb5c8eae02d/docs/solutions/bitsandbytes-moe-load-in-4bit-still-ooms.md"
    },
    {
      "text": "docs/solutions/mxfp4-moe-training-and-residency.md",
      "url": "https://github.com/pjordanandrsn/experts4bit-qlora/blob/605d21e67958fa94d5e73fc3a95ffcb5c8eae02d/docs/solutions/mxfp4-moe-training-and-residency.md"
    },
    {
      "text": "docs/solutions/offload-moe-experts-to-cpu-or-nvme.md",
      "url": "https://github.com/pjordanandrsn/experts4bit-qlora/blob/605d21e67958fa94d5e73fc3a95ffcb5c8eae02d/docs/solutions/offload-moe-experts-to-cpu-or-nvme.md"
    },
    {
      "text": "docs/solutions/qlora-fused-moe-experts.md",
      "url": "https://github.com/pjordanandrsn/experts4bit-qlora/blob/605d21e67958fa94d5e73fc3a95ffcb5c8eae02d/docs/solutions/qlora-fused-moe-experts.md"
    },
    {
      "text": "docs/solutions/run-moe-larger-than-vram.md",
      "url": "https://github.com/pjordanandrsn/experts4bit-qlora/blob/605d21e67958fa94d5e73fc3a95ffcb5c8eae02d/docs/solutions/run-moe-larger-than-vram.md"
    },
    {
      "text": "docs/solutions/serve-large-moe-on-a-consumer-gpu.md",
      "url": "https://github.com/pjordanandrsn/experts4bit-qlora/blob/605d21e67958fa94d5e73fc3a95ffcb5c8eae02d/docs/solutions/serve-large-moe-on-a-consumer-gpu.md"
    },
    {
      "text": "docs/system-manifest.json",
      "url": "https://github.com/pjordanandrsn/experts4bit-qlora/blob/605d21e67958fa94d5e73fc3a95ffcb5c8eae02d/docs/system-manifest.json"
    },
    {
      "text": "pyproject.toml",
      "url": "https://github.com/pjordanandrsn/experts4bit-qlora/blob/605d21e67958fa94d5e73fc3a95ffcb5c8eae02d/pyproject.toml"
    },
    {
      "text": "files at this commit",
      "url": "https://github.com/pjordanandrsn/experts4bit-qlora/tree/605d21e67958fa94d5e73fc3a95ffcb5c8eae02d"
    },
    {
      "text": "latest on main",
      "url": "https://github.com/pjordanandrsn/experts4bit-qlora"
    },
    {
      "text": "pjordanandrsn/grouped-nf4-gemm@30adb1bfa76f",
      "url": "https://github.com/pjordanandrsn/grouped-nf4-gemm/commit/30adb1bfa76f95f9ce4a4c29734b8f36de907eca"
    },
    {
      "text": "docs/capabilities.json",
      "url": "https://github.com/pjordanandrsn/grouped-nf4-gemm/blob/30adb1bfa76f95f9ce4a4c29734b8f36de907eca/docs/capabilities.json"
    },
    {
      "text": "docs/claims.json",
      "url": "https://github.com/pjordanandrsn/grouped-nf4-gemm/blob/30adb1bfa76f95f9ce4a4c29734b8f36de907eca/docs/claims.json"
    },
    {
      "text": "docs/solutions/fp8-paged-attention-for-moe-serving.md",
      "url": "https://github.com/pjordanandrsn/grouped-nf4-gemm/blob/30adb1bfa76f95f9ce4a4c29734b8f36de907eca/docs/solutions/fp8-paged-attention-for-moe-serving.md"
    },
    {
      "text": "docs/solutions/int4-decode-gemv.md",
      "url": "https://github.com/pjordanandrsn/grouped-nf4-gemm/blob/30adb1bfa76f95f9ce4a4c29734b8f36de907eca/docs/solutions/int4-decode-gemv.md"
    },
    {
      "text": "docs/solutions/native-mxfp4-moe-inference.md",
      "url": "https://github.com/pjordanandrsn/grouped-nf4-gemm/blob/30adb1bfa76f95f9ce4a4c29734b8f36de907eca/docs/solutions/native-mxfp4-moe-inference.md"
    },
    {
      "text": "docs/solutions/nf4-grouped-gemm-without-bf16-materialization.md",
      "url": "https://github.com/pjordanandrsn/grouped-nf4-gemm/blob/30adb1bfa76f95f9ce4a4c29734b8f36de907eca/docs/solutions/nf4-grouped-gemm-without-bf16-materialization.md"
    },
    {
      "text": "docs/solutions/stream-moe-experts-from-host-or-nvme.md",
      "url": "https://github.com/pjordanandrsn/grouped-nf4-gemm/blob/30adb1bfa76f95f9ce4a4c29734b8f36de907eca/docs/solutions/stream-moe-experts-from-host-or-nvme.md"
    },
    {
      "text": "docs/solutions/verify-quantized-checkpoint-provenance.md",
      "url": "https://github.com/pjordanandrsn/grouped-nf4-gemm/blob/30adb1bfa76f95f9ce4a4c29734b8f36de907eca/docs/solutions/verify-quantized-checkpoint-provenance.md"
    },
    {
      "text": "docs/system-manifest.json",
      "url": "https://github.com/pjordanandrsn/grouped-nf4-gemm/blob/30adb1bfa76f95f9ce4a4c29734b8f36de907eca/docs/system-manifest.json"
    },
    {
      "text": "pyproject.toml",
      "url": "https://github.com/pjordanandrsn/grouped-nf4-gemm/blob/30adb1bfa76f95f9ce4a4c29734b8f36de907eca/pyproject.toml"
    },
    {
      "text": "files at this commit",
      "url": "https://github.com/pjordanandrsn/grouped-nf4-gemm/tree/30adb1bfa76f95f9ce4a4c29734b8f36de907eca"
    },
    {
      "text": "latest on main",
      "url": "https://github.com/pjordanandrsn/grouped-nf4-gemm"
    },
    {
      "text": "Loggetta 0.7.0",
      "url": "https://pypi.org/project/loggetta/0.7.0/"
    },
    {
      "text": "/ml/openvino/",
      "url": "https://cerinamroth.com/ml/openvino/"
    },
    {
      "text": "pjordanandrsn/ov-impact-bench@810ec5f49a9d",
      "url": "https://github.com/pjordanandrsn/ov-impact-bench/commit/810ec5f49a9d6e0f80010999dd29fa25dab93104"
    }
  ],
  "structured_data": [
    {
      "@context": "https://schema.org",
      "@type": "CollectionPage",
      "name": "Large Mixture-of-Experts models on smaller hardware",
      "url": "https://cerinamroth.com/ml/",
      "isPartOf": {
        "@type": "WebSite",
        "name": "Cerin Amroth Research",
        "url": "https://cerinamroth.com/"
      },
      "about": [
        {
          "@type": "SoftwareSourceCode",
          "name": "Loggetta",
          "codeRepository": "https://github.com/pjordanandrsn/loggetta"
        },
        {
          "@type": "SoftwareSourceCode",
          "name": "experts4bit-qlora",
          "codeRepository": "https://github.com/pjordanandrsn/experts4bit-qlora"
        },
        {
          "@type": "SoftwareSourceCode",
          "name": "grouped-nf4-gemm",
          "codeRepository": "https://github.com/pjordanandrsn/grouped-nf4-gemm"
        }
      ],
      "inLanguage": "en",
      "description": "Train large MoE models on one GPU with Loggetta. Install the stack, check model support, and compare public benchmarks for speed and memory use.",
      "mainEntityOfPage": "https://cerinamroth.com/ml/"
    },
    {
      "@context": "https://schema.org",
      "@type": "BreadcrumbList",
      "itemListElement": [
        {
          "@type": "ListItem",
          "position": 1,
          "name": "Home",
          "item": "https://cerinamroth.com/"
        },
        {
          "@type": "ListItem",
          "position": 2,
          "name": "ML systems"
        }
      ]
    }
  ],
  "content_markdown": "[Home](https://cerinamroth.com/) › ML systems \n\n# Large models. Smaller machines.\n\n \n\nFine-tune Mixture-of-Experts models on a single GPU. Start with Loggetta: check the hardware, plan the run, and keep a report of what happened.\n\n \n\n```\npip install loggetta\nloggetta inspect\nloggetta plan Qwen/Qwen3-30B-A3B --seq 2048\n```\n\n[Get started on GitHub](https://github.com/pjordanandrsn/loggetta) · [PyPI 0.7.0](https://pypi.org/project/loggetta/0.7.0/)\n\n \n\n## A result you can check\n\n \n\nUnsloth spends 1.92× e4b's GPU time per training step on Qwen3-30B-A3B / RTX 5090, and 2.80× its wall-clock time on an AMD EPYC 7713 host. Matched adapters, initialization, tokens, and software; comparable held-out loss. Unsloth peaked lower (24.27 vs 26.16 GB).\n\n[Read the benchmark](https://github.com/pjordanandrsn/experts4bit-qlora/blob/605d21e67958fa94d5e73fc3a95ffcb5c8eae02d/bench/h2h-2026-10-02/tc1/RESULTS-tc1-pos69.md) · [Latest results and earlier readings](https://cerinamroth.com/research/#update-2026-10-08)\n\n \n\nGPU time is the host-independent figure: Unsloth spends more of each step on the CPU, so its wall-clock ratio grows on a slower host. Measured on torch 2.12.1+cu130 / transformers 5.5.0.\n\n \n\n## What you can use today\n\n \n\nLoggetta 0.7.0 plans and runs supported single-GPU MoE training on your own data, and keeps reusable adapters ([guide](https://github.com/pjordanandrsn/loggetta/blob/v0.7.0/docs/TRAINING.md)). It also plans serving placement; server launch uses the runtime separately.\n\n \n\n### Check your model\n\n \n\nStart with the [supported and tested model matrix](https://github.com/pjordanandrsn/loggetta/blob/v0.7.0/PYPI.md#models). Memory estimates are checked against real runs; they are estimates, not a guarantee that every workload fits.\n\n \n\n## One install, three projects\n\n \n\n | Project | Job \n | [Loggetta](https://github.com/pjordanandrsn/loggetta) | Inspect, plan, run, and save reports. \n | [experts4bit-qlora](https://cerinamroth.com/ml/experts4bit-qlora/) | Load, fine-tune, serve, and offload model weights. \n | [grouped-nf4-gemm](https://cerinamroth.com/ml/grouped-nf4-gemm/) | Compute on packed weights and move expert data. \n\n \n\n## Contributions upstream\n\n \n\nUnsloth Zoo merged [PR #852](https://github.com/unslothai/unsloth-zoo/pull/852) on 15 October 2026. It adds opt-in `mask_out_tokens` support to response-only training: chosen token sequences, such as `</think>`, are excluded from loss without altering default behavior. The change covers list/tensor data, vision collators, and MLX delegation, with regression tests. This records an upstream merge, not a packaged-release claim.\n\n \n\n## Choose your next step\n\n \n- [Plan and run training](https://github.com/pjordanandrsn/loggetta)\n- [Solve a memory, training, or kernel problem](https://cerinamroth.com/ml/solutions/)\n- [Check model and hardware requirements](https://cerinamroth.com/ml/compatibility/)\n- [Discuss an integration or research project](https://jordananderson.work/#contact)\n\n \n\nVersions, evidence, and source details\n\n \n\n`experts4bit-qlora` [v0.58.1 · current status →](https://cerinamroth.com/ml/status/#experts4bit-qlora) — claims tiered in the public register: 465 measured · 8 measured-private · 3 open · 19 superseded · 9 retired. `grouped-nf4-gemm` [v0.51.0 · current status →](https://cerinamroth.com/ml/status/#grouped-nf4-gemm) — claims tiered in the public register: 11 confirmed · 36 measured · 3 measured-private · 1 projected · 2 open · 1 superseded · 4 retired.\n\n### Evidence vocabulary\n\nconfirmeda public receipt plus an independent confirmation cell (kernel register)verifieda public receipt plus an independent check (runtime register)measuredthe run happened and the receipt is public in the repositorymeasured-privatethe run happened and the receipt lives outside the repository; NOT publicly reproducible from the repository aloneprojectedarithmetic from measured inputs, not a runopena statement the documents make that has no evidence either way yet; it backs no capabilitysupersededreplaced by a later claim that names it; kept as the record, never currentretiredwithdrawn; kept as the record, never current\n\n \n\nRuntime/kernel compatibility: [`experts4bit-qlora >=0.55.0 → grouped-nf4-gemm >=0.43.0 (extra: fast)`](https://cerinamroth.com/ml/compatibility/#package-compatibility).\n\n \n\n### Task guides\n\n | Problem | Package\n | [Why does `load_in_4bit` still OOM on a Mixture-of-Experts model?](https://cerinamroth.com/ml/solutions/bitsandbytes-moe-load-in-4bit-still-ooms/) | `experts4bit-qlora`\n | [How do I train and serve MoE models released in MXFP4 (gpt-oss, DeepSeek-V4)?](https://cerinamroth.com/ml/solutions/mxfp4-moe-training-and-residency/) | `experts4bit-qlora`\n | [How do I offload MoE experts to host RAM, or serve and train them from an NVMe arena?](https://cerinamroth.com/ml/solutions/offload-moe-experts-to-cpu-or-nvme/) | `experts4bit-qlora`\n | [How do I QLoRA-train the fused experts of a MoE (per-expert LoRA on 4-bit experts)?](https://cerinamroth.com/ml/solutions/qlora-fused-moe-experts/) | `experts4bit-qlora`\n | [How can I run a Mixture-of-Experts model larger than my GPU's VRAM?](https://cerinamroth.com/ml/solutions/run-moe-larger-than-vram/) | `experts4bit-qlora`\n | [How do I serve a large MoE on a consumer GPU (RTX 5090 class)?](https://cerinamroth.com/ml/solutions/serve-large-moe-on-a-consumer-gpu/) | `experts4bit-qlora`\n | [How do I run paged decode attention over an FP8 KV cache for a quantized MoE serving path, with sliding windows, attention sinks and a custom scale?](https://cerinamroth.com/ml/solutions/fp8-paged-attention-for-moe-serving/) | `grouped-nf4-gemm`\n | [How do I run a single-token INT4 decode GEMV over routed MoE experts, and pack calibrated (GPTQ) weights for it?](https://cerinamroth.com/ml/solutions/int4-decode-gemv/) | `grouped-nf4-gemm`\n | [How do I run MoE expert inference natively on MXFP4 (e2m1 + e8m0) weights, straight from the released checkpoint bytes?](https://cerinamroth.com/ml/solutions/native-mxfp4-moe-inference/) | `grouped-nf4-gemm`\n | [How do I run a grouped GEMM directly on NF4 packed MoE expert weights, without dequantizing to bf16 first?](https://cerinamroth.com/ml/solutions/nf4-grouped-gemm-without-bf16-materialization/) | `grouped-nf4-gemm`\n | [How do I run a MoE whose experts do not fit in VRAM, streaming them from pinned host RAM or an NVMe arena?](https://cerinamroth.com/ml/solutions/stream-moe-experts-from-host-or-nvme/) | `grouped-nf4-gemm`\n | [How do I verify that the quantized expert bytes I serve or train on are the released checkpoint bytes?](https://cerinamroth.com/ml/solutions/verify-quantized-checkpoint-provenance/) | `grouped-nf4-gemm`\n\n \n\n## Data and documentation\n\n \n\n[Site index](https://cerinamroth.com/site.json) · [Package data](https://cerinamroth.com/ml/packages.json) · [Compatibility manifest](https://cerinamroth.com/ml/system-manifest.json) · [Source record](https://cerinamroth.com/ml/system-manifest.provenance.json) · [Latest research data](https://cerinamroth.com/research/latest.json) · [AI reading guide](https://cerinamroth.com/ml/llms.txt)\n\n \n\n[OpenVINO upstream fixes](https://cerinamroth.com/ml/openvino/) · [Research notes](https://cerinamroth.com/research/) · [Hugging Face](https://huggingface.co/pjordanandrsn)\n\n \n\nrendered from experts4bit-qlora 0.58.1 · source: [pjordanandrsn/experts4bit-qlora@605d21e67958](https://github.com/pjordanandrsn/experts4bit-qlora/commit/605d21e67958fa94d5e73fc3a95ffcb5c8eae02d) · checked 2026-10-25\n\nrendered from grouped-nf4-gemm 0.51.0 · source: [pjordanandrsn/grouped-nf4-gemm@30adb1bfa76f](https://github.com/pjordanandrsn/grouped-nf4-gemm/commit/30adb1bfa76f95f9ce4a4c29734b8f36de907eca) · checked 2026-10-25\n\nSources and publication details\n\nThe generated ML documentation surfaces on this site (`/ml/` and its sub-pages, `/ml/packages.json`, both `llms.txt` files) are derived from the repositories below at pinned commits; each repository's `docs/claims.json` at that commit is the only source for numbers. Pages with their own source are listed separately.\n\n- [pjordanandrsn/experts4bit-qlora@605d21e67958](https://github.com/pjordanandrsn/experts4bit-qlora/commit/605d21e67958fa94d5e73fc3a95ffcb5c8eae02d) (2026-10-24) · package `experts4bit-qlora` 0.58.1 · PyPI 0.58.1 (published) · documents: [`docs/capabilities.json`](https://github.com/pjordanandrsn/experts4bit-qlora/blob/605d21e67958fa94d5e73fc3a95ffcb5c8eae02d/docs/capabilities.json), [`docs/claims.json`](https://github.com/pjordanandrsn/experts4bit-qlora/blob/605d21e67958fa94d5e73fc3a95ffcb5c8eae02d/docs/claims.json), [`docs/solutions/bitsandbytes-moe-load-in-4bit-still-ooms.md`](https://github.com/pjordanandrsn/experts4bit-qlora/blob/605d21e67958fa94d5e73fc3a95ffcb5c8eae02d/docs/solutions/bitsandbytes-moe-load-in-4bit-still-ooms.md), [`docs/solutions/mxfp4-moe-training-and-residency.md`](https://github.com/pjordanandrsn/experts4bit-qlora/blob/605d21e67958fa94d5e73fc3a95ffcb5c8eae02d/docs/solutions/mxfp4-moe-training-and-residency.md), [`docs/solutions/offload-moe-experts-to-cpu-or-nvme.md`](https://github.com/pjordanandrsn/experts4bit-qlora/blob/605d21e67958fa94d5e73fc3a95ffcb5c8eae02d/docs/solutions/offload-moe-experts-to-cpu-or-nvme.md), [`docs/solutions/qlora-fused-moe-experts.md`](https://github.com/pjordanandrsn/experts4bit-qlora/blob/605d21e67958fa94d5e73fc3a95ffcb5c8eae02d/docs/solutions/qlora-fused-moe-experts.md), [`docs/solutions/run-moe-larger-than-vram.md`](https://github.com/pjordanandrsn/experts4bit-qlora/blob/605d21e67958fa94d5e73fc3a95ffcb5c8eae02d/docs/solutions/run-moe-larger-than-vram.md), [`docs/solutions/serve-large-moe-on-a-consumer-gpu.md`](https://github.com/pjordanandrsn/experts4bit-qlora/blob/605d21e67958fa94d5e73fc3a95ffcb5c8eae02d/docs/solutions/serve-large-moe-on-a-consumer-gpu.md), [`docs/system-manifest.json`](https://github.com/pjordanandrsn/experts4bit-qlora/blob/605d21e67958fa94d5e73fc3a95ffcb5c8eae02d/docs/system-manifest.json), [`pyproject.toml`](https://github.com/pjordanandrsn/experts4bit-qlora/blob/605d21e67958fa94d5e73fc3a95ffcb5c8eae02d/pyproject.toml) · registers: capabilities, claims · synced 2026-10-25T00:58:32Z · [files at this commit](https://github.com/pjordanandrsn/experts4bit-qlora/tree/605d21e67958fa94d5e73fc3a95ffcb5c8eae02d) · [latest on main](https://github.com/pjordanandrsn/experts4bit-qlora) (unpinned; not the source of any fact rendered here)\n- [pjordanandrsn/grouped-nf4-gemm@30adb1bfa76f](https://github.com/pjordanandrsn/grouped-nf4-gemm/commit/30adb1bfa76f95f9ce4a4c29734b8f36de907eca) (2026-10-24) · package `grouped-nf4-gemm` 0.51.0 · PyPI 0.51.0 (published) · documents: [`docs/capabilities.json`](https://github.com/pjordanandrsn/grouped-nf4-gemm/blob/30adb1bfa76f95f9ce4a4c29734b8f36de907eca/docs/capabilities.json), [`docs/claims.json`](https://github.com/pjordanandrsn/grouped-nf4-gemm/blob/30adb1bfa76f95f9ce4a4c29734b8f36de907eca/docs/claims.json), [`docs/solutions/fp8-paged-attention-for-moe-serving.md`](https://github.com/pjordanandrsn/grouped-nf4-gemm/blob/30adb1bfa76f95f9ce4a4c29734b8f36de907eca/docs/solutions/fp8-paged-attention-for-moe-serving.md), [`docs/solutions/int4-decode-gemv.md`](https://github.com/pjordanandrsn/grouped-nf4-gemm/blob/30adb1bfa76f95f9ce4a4c29734b8f36de907eca/docs/solutions/int4-decode-gemv.md), [`docs/solutions/native-mxfp4-moe-inference.md`](https://github.com/pjordanandrsn/grouped-nf4-gemm/blob/30adb1bfa76f95f9ce4a4c29734b8f36de907eca/docs/solutions/native-mxfp4-moe-inference.md), [`docs/solutions/nf4-grouped-gemm-without-bf16-materialization.md`](https://github.com/pjordanandrsn/grouped-nf4-gemm/blob/30adb1bfa76f95f9ce4a4c29734b8f36de907eca/docs/solutions/nf4-grouped-gemm-without-bf16-materialization.md), [`docs/solutions/stream-moe-experts-from-host-or-nvme.md`](https://github.com/pjordanandrsn/grouped-nf4-gemm/blob/30adb1bfa76f95f9ce4a4c29734b8f36de907eca/docs/solutions/stream-moe-experts-from-host-or-nvme.md), [`docs/solutions/verify-quantized-checkpoint-provenance.md`](https://github.com/pjordanandrsn/grouped-nf4-gemm/blob/30adb1bfa76f95f9ce4a4c29734b8f36de907eca/docs/solutions/verify-quantized-checkpoint-provenance.md), [`docs/system-manifest.json`](https://github.com/pjordanandrsn/grouped-nf4-gemm/blob/30adb1bfa76f95f9ce4a4c29734b8f36de907eca/docs/system-manifest.json), [`pyproject.toml`](https://github.com/pjordanandrsn/grouped-nf4-gemm/blob/30adb1bfa76f95f9ce4a4c29734b8f36de907eca/pyproject.toml) · registers: capabilities, claims · synced 2026-10-24T14:51:49Z · [files at this commit](https://github.com/pjordanandrsn/grouped-nf4-gemm/tree/30adb1bfa76f95f9ce4a4c29734b8f36de907eca) · [latest on main](https://github.com/pjordanandrsn/grouped-nf4-gemm) (unpinned; not the source of any fact rendered here)\n- External versioned source: [Loggetta 0.7.0](https://pypi.org/project/loggetta/0.7.0/) · sections: `loggetta-front-door`, `models-supported-vs-tested` — the install and model matrix are quoted from the PyPI 0.7.0 release (tag v0.7.0 @ 3bbf7ab0d5ac); performance ratios below come from the pinned e4b claim register.\n- OpenVINO page ([/ml/openvino/](https://cerinamroth.com/ml/openvino/), separate source): [pjordanandrsn/ov-impact-bench@810ec5f49a9d](https://github.com/pjordanandrsn/ov-impact-bench/commit/810ec5f49a9d6e0f80010999dd29fa25dab93104) (2026-05-24) · documents: `results.jsonl`, `README.md` · fetched 2026-09-04T15:18:26Z · generated by `scripts/build_openvino.py`.\n- Site-authored (`scripts/build_ml_site.py` in Cerin-Amroth/cerinamroth-site, private): `start-with-the-need`, `released-and-research`, `upstream-work` — site-authored navigation and summary; the merged Unsloth Zoo contribution is linked directly to its upstream pull request.\n",
  "content_sha256": "4b369b57b5381ccc32a1c275dcd4f2186db8284e5607ee353bfb1c5a01d524bb"
}