WebGPU for AI

Introduction and mental model

WebGPU is an explicit GPU API with compute capability; it is not an ML graph format and not an automatic replacement for WebAssembly. Application or runtime code requests an adapter and device, creates buffers and bind groups, builds compute pipelines from WGSL, encodes dispatches, submits work to a queue, and synchronizes only when results must cross back to the CPU. The GPU has its own resource and execution model, so upload, allocation, compilation, dispatch, and readback are part of the workload.

For AI, the central design question is tensor residency. A fast kernel can lose end-to-end if each operation uploads inputs and reads outputs back. Conversely, Wasm SIMD can be a strong choice for small, branch-heavy, or already-CPU-resident work. The relevant comparison names tensor shapes, precision, device limits, workgroup geometry, number of dispatches, transfer policy, and whether compilation is cold or warm.

WebGPU capabilities are negotiated, not universal. Optional features, limits, preferred formats, timestamp support, and implementation behavior vary. Code must request only needed capabilities, validate resource sizes and alignments, and maintain a measured fallback.

Why it matters

Browser inference runtimes increasingly use WebGPU execution providers, and custom compute can move preprocessing, tensor transforms, or postprocessing closer to resident model data. Understanding the underlying API makes provider diagnostics, I/O binding, copy costs, and profiling results intelligible.

It also prevents misleading performance work. Dispatch time alone omits JavaScript preparation, buffer writes, pipeline creation, queue delay, and readback. A disciplined WebGPU path measures both kernel behavior and the end-to-end boundary it is meant to improve.

Questions this branch answers

How do adapter discovery, device requests, features, limits, and device loss shape a compute path?
Which buffer usage flags, alignment rules, mapping states, and ownership transitions apply to tensor data?
How do bind group layouts, pipeline layouts, shader modules, and compute pipelines fit together?
How are tensor dimensions mapped to invocations, workgroups, dispatch counts, and bounds checks?
When should workgroup memory, barriers, or multiple passes be used?
What does queue submission guarantee, and which operations introduce CPU/GPU synchronization?
How can tensors remain GPU-resident across runtime inference and custom kernels?
Which timing mechanisms are available, and how should results be reported when timestamp queries are unavailable?
When does a Wasm SIMD implementation beat WebGPU after all transfers and setup are included?

Scope

Adapter and device acquisition, feature and limit negotiation, error scopes, uncaptured errors, and device loss.
GPU buffers, usage flags, mapping, queue writes, staging, alignment, reuse, and destruction.
WGSL compute shaders, storage and uniform bindings, shader validation, and numerical behavior.
Bind groups, pipeline layouts, compute pipelines, command encoders, passes, dispatch, and queue submission.
Invocation and workgroup geometry, bounds checks, workgroup memory, barriers, and reduction patterns.
Tensor upload, download, layout, packing, accelerator residency, runtime I/O binding, and custom-kernel handoff.
Cold compilation, warm execution, optional timestamp queries, wall-clock measurement, and correctness baselines.
Explicit comparison with Wasm SIMD for named tensor and postprocessing workloads.

Out of scope

Graphics rendering, scene pipelines, or a general WGSL language reference.
A claim that GPU execution is always faster or more energy efficient than CPU/Wasm.
Reimplementing a complete neural-network runtime or broad shader library in this index.
Assuming optional features, exact limits, shader precision, or browser availability across all devices.
Hiding WebGPU initialization or transfer costs inside an unlabeled inference number.

Expected outcomes

After completing this branch, a reader should be able to diagram a WebGPU compute submission from host tensor to validated result, choose resource usages and workgroup geometry from observed limits, and recognize synchronization and copy boundaries. They should be able to keep data resident across compatible stages, handle validation and device loss, and compare a WGSL kernel with a Wasm SIMD baseline using both correctness and end-to-end measurements.

Candidate note roadmap

WebGPU compute, capabilities, and explicit resource lifecycles — connect adapter, device, queue, buffers, shaders, pipelines, command encoding, submission, and completion while requesting minimal features, querying limits, documenting portability constraints, and retaining an explicit fallback.
GPU buffers, bind groups, and the tensor ABI — derive legal usage, alignment, mapping, transfer, allocation, reuse, and readback paths, then map shapes, strides, metadata, uniforms, storage buffers, and access modes into a stable kernel boundary.
WGSL indexing for multidimensional tensors — lower rows, channels, batches, and packed lanes into global invocation IDs with complete bounds checks.
Dispatch and workgroup geometry are measured choices — relate tensor size to workgroup size, occupancy constraints, tail handling, and portable limit checks.
Reductions, workgroup memory, and synchronization — build correct multi-stage sum, max, softmax support, or argmax patterns without assuming global barriers.
Tensor residency and runtime I/O binding — trace when an inference provider can accept or return GPU tensors and where ownership or compatibility forces a copy.
Profiling cold pipelines, warm kernels, and transfers — separate shader and pipeline creation, upload, dispatch, queue completion, readback, and total application time.
WebGPU versus Wasm SIMD for small tensor work — define crossover experiments by shape, reuse count, residence, transfer size, and device instead of declaring a winner.

Future runnable artifact

Build a paired kernel lab with two exactly specified operations: fused affine normalization over a contiguous float tensor and a row-wise score postprocessor that returns each row's maximum value and stable lowest-index argmax. Implement each operation once in WGSL and once in Wasm SIMD, with a scalar reference used only for correctness. Use deterministic tensors that include negative values, non-multiple tail lengths, equal-score ties, NaNs under an explicitly stated policy, and several row widths.

The WebGPU runner will expose cold pipeline creation, reusable GPU-buffer execution, upload-plus-dispatch, dispatch-plus-readback, and full upload-to-result modes. It will record adapter information exposed by the API, requested features, effective limits, workgroup size, dispatch geometry, buffer bytes, and validation failures. The Wasm runner will record scalar versus SIMD availability, memory growth, copies, and the same warmup and sample policy. Results will report exact argmax agreement, numerical error for normalized values, latency distributions, throughput, and the size or reuse point where the measured winner changes on that environment.

How to verify and measure

Query adapter/device features and limits and record only identifiers the platform exposes without assuming a specific physical GPU.
Validate shader modules, use error scopes for expected validation failures, listen for uncaptured errors, and handle device loss.
Assert buffer sizes, usages, alignment, mapping state, dispatch bounds, and output sizes before submission.
Compare WGSL and Wasm SIMD outputs against the same scalar fixtures, including tails, ties, special values, and zero-length policy.
Report cold pipeline creation separately from warmed reusable-pipeline runs.
Measure upload, command encoding, dispatch/queue completion, readback, and end-to-end latency separately where the APIs permit.
Use GPU timestamp queries only after feature detection; label CPU wall-clock timing as such and do not equate it with kernel-only time.
Run enough samples to report median and tail percentiles, and publish warmup, synchronization, and outlier policy.
Compare CPU-resident and GPU-resident scenarios so transfer costs are visible rather than amortized accidentally.
Repeat on multiple browsers/devices before making a scoped portability claim.

Primary sources

WebGPU specification — https://www.w3.org/TR/webgpu/ — W3C Candidate Recommendation Draft dated 2026-06-23, consulted 2026-07-16. The specification and implementations continue to evolve; optional features and limits must be queried.
WebGPU living specification — https://gpuweb.github.io/gpuweb/ — Editor-maintained source for current API details; changing and consulted 2026-07-16.
WebGPU Shading Language — https://www.w3.org/TR/WGSL/ — Latest published WGSL specification; status and language details are evolving and were consulted 2026-07-16.
WGSL editor's draft — https://gpuweb.github.io/gpuweb/wgsl/ — Editor's Draft dated 2026-07-14 when consulted 2026-07-16; use a pinned browser/toolchain when reproducing shader behavior.
ONNX Runtime Web WebGPU execution provider — https://onnxruntime.ai/docs/tutorials/web/ep-webgpu.html — Current provider, profiling, GPU tensor, and I/O-binding guidance; version-sensitive and consulted 2026-07-16.
ONNX Runtime Web performance diagnosis — https://onnxruntime.ai/docs/tutorials/web/performance-diagnosis.html — Current runtime-specific diagnostic guidance; version-sensitive and consulted 2026-07-16.

Connects to: Wasm Execution Model · SIMD, Threads, and Workers · Browser Inference Runtimes · WebNN and Adaptive Backends · Wasm AI Performance, Security, and Craftsmanship