Wasm AI Performance, Security & Craftsmanship

Introduction and mental model

Performance, security, and verification are not a final phase after the system works. They are the control plane used from the first WAT module through the final adaptive AI pipeline. Every optimization changes an assumption; every fallback changes an execution path; every model or runtime update can change numerical output.

The mental model is claim, context, measurement, evidence. “Faster,” “private,” “portable,” “sandboxed,” and “equivalent” are incomplete claims until the artifact, environment, method, tolerance, and observed fallback are recorded. A result that cannot be reproduced is a debugging clue, not a benchmark.

Why it matters

Browser and Wasm measurements are unusually easy to distort through cold compilation, warm-up, caching, background throttling, tensor copies, backend substitution, and hardware-specific scheduling. Numerical changes can be hidden by plausible model outputs, while deployment changes can silently weaken isolation or artifact integrity.

An always-active verification discipline prevents the Workbench from becoming a demo that reports only its best path. It makes unsupported configurations, regressions, and fallbacks first-class results.

Questions this branch answers

• What exactly belongs to cold start, first inference, warm inference, and steady state?
• How should p50, p95, throughput, sample count, warm-up, and variance be reported?
• Where do compilation, download, model load, tensor copies, and backend transfers appear in the cost model?
• Which golden outputs and tolerances establish numerical equivalence for each operation or model?
• How are browser, runtime, hardware, feature, and fallback matrices kept reproducible?
• Which CSP, cross-origin isolation, integrity, capability, and supply-chain controls protect deployment?
• How can binary size, model size, peak memory, and storage be tracked as release regressions?
• What evidence is required before an optimization or compatibility claim is published?

Scope

• Cold start, warm start, first inference, warm p50/p95, steady-state latency, throughput, and variance.
• Compilation, instantiation, model loading, preprocessing, inference, postprocessing, and copy timing.
• Peak memory, linear-memory growth, tensor residency, binary size, model size, and download time.
• Numerical regression tests, golden datasets, checksums, absolute/relative tolerance, and task-level agreement.
• Browser, operating-system, hardware, runtime, backend, feature, and fallback matrices.
• Browser profiling traces, CPU/GPU timelines, performance markers, and memory observations with named tooling and versions.
• Fuzzing, traps, malformed artifacts, capability tests, CSP, COOP/COEP, and model integrity.
• Reproducible builds, dependency provenance, artifact manifests, and release comparison.
• A common report emitted by every WasmAI Workbench milestone.

Out of scope

• Publishing performance numbers without raw context, method, and environment metadata.
• Declaring one backend universally fastest from a single machine or browser.
• Using visual plausibility as a numerical-equivalence test.
• Treating average latency as a substitute for distributions and tail behavior.
• Treating Wasm sandboxing as sufficient application security.
• Hiding failed capability checks or substituting a fallback without recording it.

Expected outcomes

After this branch, a reader should be able to design a benchmark before implementing an optimization, create golden fixtures with justified tolerances, and produce a report that another machine can interpret and reproduce. They should be able to separate startup phases, locate memory and copy costs, detect numerical regressions, verify artifact integrity, and explain exactly which security and compatibility boundary was tested.

Candidate note roadmap

• benchmark-protocols-for-steady-state-latency-throughput-and-tails — define environments, isolation, warm-up, repetition, concurrency, workload shape, sampling, raw records, and p50/p95 distributions for steady-state latency, throughput, and tail behavior.
• cold-start-compilation-instantiation-and-first-inference — separate startup phases that products experience differently.
• linear-memory-tensor-copies-and-peak-memory — attribute allocation, growth, host boundaries, and accelerator transfers.
• binary-model-download-and-storage-budgets — treat delivery size and local storage as performance constraints.
• golden-outputs-tolerances-and-numerical-regression — verify kernels, pipelines, quantization, and backend parity.
• browser-feature-matrices-and-fallback-testing — record support, selected paths, and graceful degradation.
• csp-cross-origin-isolation-and-artifact-integrity — secure deployment without obscuring runtime requirements.
• reproducible-builds-provenance-and-release-gates — bind source, toolchain, dependencies, model, and output evidence.

Future runnable artifact

Every WasmAI Workbench milestone will emit a machine-readable JSON report plus a human summary from the same schema. The report must include the exact fields below, with units, measurement method, sample metadata, and artifact checksums carried in adjacent structured metadata rather than hidden in prose:

Runtime
Backend
Browser
Operating system
Hardware
Model version
Model size
Wasm binary size
Cold start
Warm inference p50
Warm inference p95
Peak memory
Output checksum
Numerical difference
Fallbacks observed

The permanent artifact will also retain raw samples, the benchmark fixture identifier, golden-output version, tolerance policy, source revision, build commands, toolchain versions, feature-detection results, and pass/fail release gates. Reports from scalar, SIMD, threaded, WebGPU, WebNN, browser runtime, and WASI paths must remain comparable without pretending their environments are identical.

How to verify and measure

• Define the workload, fixture, environment controls, warm-up, sample count, and acceptance rule before collecting numbers.
• Preserve raw samples and calculate p50/p95 from the recorded distribution rather than UI-rounded values.
• Separate download, compilation, instantiation, model load, first inference, and warm inference timers.
• Measure peak memory and copy bytes at host/Wasm and CPU/accelerator boundaries where tooling permits.
• Compare output checksums and numerical differences against a versioned golden reference with documented tolerances.
• Run negative tests for corrupt models, unsupported features, denied capabilities, missing isolation headers, and forced fallbacks.
• Rebuild from the recorded revision and toolchain, then compare artifact checksums or explain the nondeterminism.
• Publish compatibility as a dated matrix of observed configurations, never as an unqualified universal claim.

Primary sources

• WebAssembly Core Specification — primary validation and execution semantics; living specification, consulted 2026-07-16.
• WebAssembly JavaScript Interface — primary browser host API for compilation, instantiation, memory, and feature behavior; living specification, consulted 2026-07-16.
• High Resolution Time — primary timing API and clock semantics for browser measurement; evolving specification, consulted 2026-07-16.
• Performance Timeline — primary model for timestamped performance entries; evolving specification, consulted 2026-07-16.
• Web Platform Tests — cross-browser conformance infrastructure and methodology reference; continuously evolving, consulted 2026-07-16.
• Content Security Policy Level 3 — primary browser policy specification for deployment controls; implementation details are browser-sensitive, consulted 2026-07-16.
• Subresource Integrity — primary integrity mechanism for fetched resources; consulted 2026-07-16.
• SLSA specification — supply-chain provenance and build-integrity framework; evolving specification, consulted 2026-07-16.

Connects to: Tensor Kernels & Inference from Scratch · SIMD, Threads & Workers · Browser Inference Runtimes · WebGPU for AI · WebNN & Adaptive Backends · Local-First, Offline & Multimodal AI · WASI, Components & WASI-NN · Sandboxed AI Tools & Plugins