OpenVINO
Upstream contributions to OpenVINO, Intel's inference toolkit — keeping Intel-GPU LLM inference from silently falling back to the CPU on modern drivers.
The same practice as the NVIDIA work on this page's parent, aimed at the other accessible-hardware vendor: getting real inference out of the GPU a person already owns. Here that means fixing the toolkit itself, in the open, as external contributor pjordanandrsn — three merged pull requests and a fourth in review.
The bug class: a silent CPU fallback on modern Intel drivers
Intel's Compute Runtime (NEO) 23.x and newer stopped declaring the
__local-pointer overloads of intel_sub_group_block_read —
they moved to a separate extension that not every driver advertises. OpenVINO GPU
kernels that read shared local memory through those intrinsics then fail to compile
(CL_BUILD_PROGRAM_FAILURE) on current drivers, and under the
AUTO/HETERO device policies that failure forces the
whole model onto the CPU — a large, silent performance regression that
looks like "the GPU just isn't helping." The trigger is driver-determined,
not silicon-age: a Gen 9.5 UHD P630 on NEO 23.43 advertises no
extension and hits it, which is what disproved the tempting "old GPU = old code path"
assumption.
The fix
A two-tier dispatch for the affected block-read family, living in the
shared batch header so every kernel inherits it: the hardware intrinsic when the
driver advertises the extension, an inline per-lane gather emulation when it does
not — mirroring the pattern already used for the __global case. Landed
as a shared-header refactor migrating the affected kernels at once, plus a
driver-independent regression test that compiles the family through the
kernel cache so the symbols can never silently vanish again.
Contributions
| PR | What | Status |
|---|---|---|
| #35661 | Fix fully_connected_gpu_gemv compilation on modern NEO | merged |
| #35712 | Shared-header two-tier dispatch; migrate lora / MoE-SwiGLU / conv kernels | merged |
| #36017 | Driver-independent regression test for the __local block-read family | merged |
| #36543 | Core/transformations: internal-op input-validation sweep + matcher cleanup | in review |
Measured impact measured
Quantified with a purpose-built harness, ov-impact-bench (MIT), on the weakest affected GPU — a UHD P630. On the released build the GPU path throws the compile error and the model runs on the CPU; on the fixed build it runs on the GPU:
| UHD P630 | Decode | Time-to-first-token |
|---|---|---|
| CPU fallback (bug) | 13.9 tok/s | 355 ms |
| GPU (fixed) | 18.4 tok/s | 133 ms |
1.33× throughput and 2.7× faster first token on the least-capable affected part; the gap widens on Arc- and Lunar-Lake-class GPUs. The harness subprocess-isolates each device so a native driver crash is captured as a structured record rather than aborting the run, and auto-detects an energy source (wall-plug / RAPL / GPU hwmon) where one exists.