# Research Notes & GPU Benchmarks — Cerin Amroth Source: https://cerinamroth.com/research/ [Home](https://cerinamroth.com/) › ML research notes # ML research notes Public benchmarks, new capabilities, and the experiments that did not work. ## Start with the software [Install Loggetta](https://cerinamroth.com/ml/) · [Runtime](https://cerinamroth.com/ml/experts4bit-qlora/) · [Kernels](https://cerinamroth.com/ml/grouped-nf4-gemm/) · [Model and hardware requirements](https://cerinamroth.com/ml/compatibility/) ## Research update · 8 October 2026 The Unsloth comparison is now quoted as GPU time. Unsloth spends more of each step on the CPU than e4b does, so its wall-clock time grows on a slower host. The GPU-time ratio is the figure that does not depend on the host; the wall-clock ratio is quoted beside it, on its host. The 6 October figures below were wall readings from before e4b's current defaults. | Workload | Result | Conditions | Qwen3-30B-A3B QLoRA | Unsloth spends 1.92× e4b's GPU time per step; wall-clock 2.80× | RTX 5090 on an AMD EPYC 7713 host, both frameworks on one stack, e4b at its defaults; comparable held-out loss; Unsloth peaked lower (24.27 vs 26.16 GB). [Evidence](https://github.com/pjordanandrsn/experts4bit-qlora/blob/72e6dfeb36dfefbfdb84d71b77b1399c64709581/bench/h2h-2026-10-02/tc1/RESULTS-tc1-pos69.md) | Packed 4K Qwen3 training | Unsloth spends 1.160× e4b's GPU time per step | RTX 5090, e4b at its defaults; wall-clock 1.472× on that host. [Evidence](https://github.com/pjordanandrsn/experts4bit-qlora/blob/72e6dfeb36dfefbfdb84d71b77b1399c64709581/bench/h2h-2026-10-02/tc1/RESULTS-tc1-pos68.md) | Wide decode steps (serving) | No measurable quality cost | Qwen3-30B-A3B int4 on one RTX 5090: one 64-row decode step against four 16-row pieces, within the 16-row arithmetic's own variation. [Evidence](https://github.com/pjordanandrsn/experts4bit-qlora/blob/72e6dfeb36dfefbfdb84d71b77b1399c64709581/bench/p117/RESULTS-p117.md) Released the same day: experts4bit-qlora 0.50.0, where `serve_paged` sizes its slots to the GPU's memory and decodes each step as one graph by default ([release notes](https://github.com/pjordanandrsn/experts4bit-qlora/releases/tag/v0.50.0)), and Loggetta 0.3.1, a documentation update. [Download the source revisions, result records, and release status as JSON](https://cerinamroth.com/research/latest.json). ## Research update · 6 October 2026 Bring your data, keep your adapters. New in [Loggetta 0.3.0](https://pypi.org/project/loggetta/0.3.0/): local or Hub data, validation before model loading, reusable adapters, and a run report. Save and reload pass CPU tests; the full CUDA test is still to run. | Workload | Result | Conditions | Qwen3-30B-A3B QLoRA | 2.352× vs Unsloth; 2.468× on a second host | RTX 5090, matched work and software, comparable held-out loss. Unsloth used 3.22 GB less peak VRAM. [Evidence](https://github.com/pjordanandrsn/experts4bit-qlora/blob/a41c857dd7d5c68f8735af16634ce08426e01271/bench/h2h-2026-10-02/tc1/RESULTS-tc1-samestack-host2.md) | Packed 4K Qwen3 training | 1.453× vs Unsloth, defaults as of 2026-10-06 | RTX 5090, torch 2.12.1, 28.23 vs 24.86 GB peak. Defaults have changed since. [Evidence](https://github.com/pjordanandrsn/experts4bit-qlora/blob/a41c857dd7d5c68f8735af16634ce08426e01271/bench/h2h-2026-10-02/tc1/RESULTS-tc1-packed4k-defaults.md) | Mixtral-8x7B QLoRA | 1.144× vs Unsloth | Same-stack RTX 5090 run. Comparable held-out loss; Unsloth used less VRAM and energy. [Evidence](https://github.com/pjordanandrsn/experts4bit-qlora/blob/a41c857dd7d5c68f8735af16634ce08426e01271/bench/h2h-2026-10-02/tc2/RESULTS-tc2-mixtral-samestack.md) The software stack matters. Packed training was slower on torch 2.8, and [bucketed padding is not the cause](https://github.com/pjordanandrsn/experts4bit-qlora/blob/4f56bdd5b0d2994211d241bb9768d01db80e6c64/bench/h2h-2026-10-02/tc1/RESULTS-tc1-padbk28.md). The decoded training route showed no measurable whole-step saving and stays opt-in. [Read the test](https://github.com/pjordanandrsn/experts4bit-qlora/blob/a41c857dd7d5c68f8735af16634ce08426e01271/bench/h2h-2026-10-02/tc1/RESULTS-tc1-decoded.md) [This update's records as JSON](https://github.com/Cerin-Amroth/cerinamroth-site/blob/main/docs/research-update-2026-10-06.json). ## Earlier research narratives These essays preserve the work as it stood when written. Use the [current software documentation](https://cerinamroth.com/ml/) for installation and support. ### [Receipts-driven engineering](https://cerinamroth.com/research/receipts-driven-engineering/)historical The methodology narrative behind grouped-nf4-gemm's blind-confirmed benchmarks: preregistration, OpenTimestamps stamping, and the published failures. Published 2026-07-14 · last revised 2026-07-31 (dates from this site's history before the narrative was moved here) · current documentation: [`grouped-nf4-gemm`](https://cerinamroth.com/ml/grouped-nf4-gemm/) ### [experts-4bit, the 2026 narrative](https://cerinamroth.com/research/experts-4bit/)historical The earlier experts-4bit story: fused 4-bit experts, QLoRA and offload as they stood in July 2026. Published 2026-07-09 · last revised 2026-07-31 (dates from this site's history before the narrative was moved here) · current documentation: [`experts4bit-qlora`](https://cerinamroth.com/ml/experts4bit-qlora/) Sources and publication details The generated documentation on this page is derived from the pinned source below; any site-authored section is listed as such, cites the same registers and copies no numbers. `docs/claims.json` at that commit is the only source for numbers. - Site-authored (`scripts/build_ml_site.py` in Cerin-Amroth/cerinamroth-site, private): `current-documentation`, `update-2026-10-08`, `update-2026-10-06`, `dated-narratives` — site-authored directory and dated research digests; each digest cites immutable repository evidence recorded in its docs/research-update-.json.