SIMD, Threads, and Workers

SIMD, Wasm threads, and Web Workers solve different problems. SIMD performs several lane operations within one execution context. Wasm threads add shared linear memory and atomic instructions, while the host still creates and schedules workers. A plain worker can move computation off the browser main thread without making a kernel parallel or sharing its memory.

Introduction and Mental Model

Optimize along independent axes. First, improve the work done by one context through loop structure, locality, autovectorization, or explicit v128 operations. Second, partition independent work across a bounded worker pool. Third, choose how data reaches those workers: copied messages, transferred ownership, or shared memory with synchronization. Each axis has its own correctness conditions, startup cost, and compatibility gate.

Shared-memory browser execution is a deployment property as well as a compiler feature. The page, workers, response headers, subresources, browser, and module must support the chosen path. Cross-origin isolation commonly governs browser exposure of SharedArrayBuffer; atomics do not make a race-free algorithm; and blocking or proxying work through the browser main thread can destroy responsiveness or deadlock.

This branch uses concurrency and cache concepts from the Low-Level Atlas and scalar kernels from this atlas. It does not repeat general parallel-programming theory or the AI Atlas treatment of inference optimization.

Why It Matters

Browser AI needs throughput without freezing interaction, but more lanes or workers do not guarantee a faster end-to-end result. Small tensors may lose to dispatch overhead. Copies may erase kernel gains. False sharing may serialize workers through the cache-coherence system. Relaxed SIMD may change reproducibility and is not uniformly available. The workbench must expose the selected path and evidence rather than silently assuming the most ambitious backend.

Questions This Branch Answers

• Which loop and data-layout conditions allow a compiler to autovectorize a Wasm kernel?
• When are explicit SIMD128 intrinsics or WAT instructions justified over maintainable scalar source?
• What semantic latitude does Relaxed SIMD introduce, and how should availability and numerical drift be tested?
• How do shared Wasm memories and atomic instructions relate to JavaScript SharedArrayBuffer, Atomics, and Web Workers?
• Which COOP, COEP, subresource, and crossOriginIsolated conditions gate a browser deployment?
• When should work use copied messages, transferable buffers, shared memory, or isolated per-worker memories?
• How do pool creation, task granularity, backpressure, synchronization, and false sharing affect latency and throughput?
• How can optimized variants be selected without claiming universal browser or runtime support?

Scope

• Fixed-width 128-bit Wasm SIMD, lane operations, loads/stores, reductions, shuffles, and alignment considerations.
• Compiler autovectorization reports and explicit C/C++ or Rust intrinsics where they improve an evidenced bottleneck.
• Relaxed SIMD as a separately detected, separately validated capability with potentially varying results.
• Web Workers, bounded worker pools, task partitioning, startup, messaging, transfer, and backpressure.
• Shared Wasm memory, atomic read-modify-write operations, waits/notifications where allowed, and synchronization protocols.
• Emscripten pthread support and its browser/runtime constraints as one implementation strategy.
• COOP, COEP, cross-origin isolation, deployment checks, main-thread responsiveness, cache lines, and false sharing.
• Scalar, SIMD, and SIMD-plus-multithread benchmark variants with correctness gates and visible fallback.

Out of Scope

• Assuming SIMD width maps directly to a fixed speedup or worker count maps directly to CPU cores.
• Guaranteeing Relaxed SIMD, shared memory, pthread parity, or cross-origin isolation in every browser, embed, CDN, or server runtime.
• Blocking the browser main thread to imitate a native threading model.
• A general lock-free algorithms course or a production task scheduler.
• WebGPU, WebNN, GPU shaders, NPU execution, and runtime-specific execution providers.
• Optimizing before the scalar tensor kernels have independent correctness evidence.

Expected Outcomes

After this branch, the reader should be able to:

• Distinguish vectorization, parallel execution, and off-main-thread scheduling in architecture and measurements.
• Confirm whether a compiler vectorized the intended loop and inspect the resulting Wasm instructions.
• Implement or select a SIMD128 kernel without changing its documented mathematical contract.
• Build a bounded worker pool with explicit queue limits, cancellation policy, and result ordering.
• Deploy and feature-detect shared-memory variants without breaking the scalar fallback.
• Design synchronization and data partitioning that avoids races and reduces false sharing.
• Produce a reproducible benchmark report that includes correctness, latency, throughput, copies, module size, memory, and UI responsiveness.

Candidate Note Roadmap

• simd128-lanes-loads-and-reductions — map scalar tensor loops to v128 values, lane operations, tails, alignment behavior, and reduction strategies.
• autovectorization-intrinsics-and-the-scalar-oracle — use optimization remarks and disassembly to prove which loops vectorized, then introduce explicit C/C++ or Rust intrinsics only when evidence and differential tests against a maintainable scalar implementation justify them.
• relaxed-simd-and-reproducibility — identify relaxed operations, detect support, measure output variation, and define when the optimization is unacceptable.
• workers-shared-memory-atomics-and-happens-before — distinguish isolated worker scheduling from Wasm threads, trace message copies and transferable ownership, then build a shared-memory work queue with explicit module/instance lifecycle, atomics, state transitions, synchronization, and race tests.
• emscripten-pthreads-and-worker-pools — inspect compile/link flags, pool sizing, proxying, fallback builds, and behavioral differences from native assumptions.
• coop-coep-and-deployable-cross-origin-isolation — verify headers, subresource policies, embeds, service-worker interactions, and runtime feature detection in production-like hosting.
• false-sharing-partitioning-and-task-granularity — vary output ownership, padding, tiles, queue depth, and batch size while reading latency and throughput together.
• main-thread-budget-and-backpressure — cap in-flight work, preserve interaction, cancel stale requests, and report long tasks alongside kernel speed.

Future Runnable Artifact

WasmAI Workbench v2 will benchmark two deterministic FP32 workloads: a 256×256 matrix multiplication and a 3×3 NCHW convolution over a 224×224 single-channel input. Dimensions will remain configurable, but the report will always include the fixed comparison case and seeded inputs. Three separately built variants will share one logical ABI and output layout: scalar Wasm, SIMD128 Wasm, and SIMD128 Wasm using a prewarmed bounded worker pool over shared linear memory.

The parallel variant will partition disjoint output tiles, use atomics only for queue state and completion, pad contended control slots, and keep the main browser thread asynchronous. The host will check module validation, SIMD support, SharedArrayBuffer, crossOriginIsolated, and worker creation before selecting a variant. If any gate fails, it will choose and visibly label the next supported build; it will never pretend one binary can supply an untested threaded fallback.

The benchmark report will include output-error statistics against the scalar oracle, p50/p95 latency, operations per second at stated concurrency, worker startup and dispatch costs, copied/transferred/shared bytes, raw and compressed module bytes, peak linear-memory pages, queue depth, rejected or cancelled tasks, and main-thread long tasks. Raw samples and environment metadata will be exportable for reproduction.

How to Verify and Measure

• Require every optimized output to pass the scalar reference's documented absolute/relative tolerance and invariant checks before including its timing.
• Inspect generated Wasm or compiler vectorization remarks to prove SIMD instructions are present; do not infer vectorization from a flag or faster result.
• Test lengths and dimensions below, equal to, and above the vector width and tile size so scalar tails and boundary tiles execute.
• Run race-focused stress tests with randomized task order, small tiles, repeated cancellation, worker failure, and queue saturation; use state-machine assertions and canaries around output regions.
• Verify deployment headers and crossOriginIsolated from the served page, then test a deliberately non-isolated deployment and require a clear scalar/SIMD fallback rather than startup failure.
• Separate module compilation, worker creation, pool warm-up, dispatch, kernel, synchronization, and result-observation time.
• Report distributions after a fixed warm-up, retain raw samples, and record browser/runtime build, OS, CPU, logical processor count, power state, header configuration, module hashes, and compiler flags.
• Sweep worker count, tile size, queue depth, and input size. Treat the crossover point and regressions as results instead of choosing only the fastest configuration.
• Measure responsiveness with long-task or frame-budget observations while work is in flight; throughput gained by freezing the UI is not an acceptable browser result.

Primary Sources

• WebAssembly Core Specification — normative fixed-width and relaxed vector semantics plus ordinary memory semantics; threads and atomics are sourced separately below. Release 3.0; checked 2026-07-16.
• WebAssembly feature status — proposal stages and engine/tool implementation matrix. Changing compatibility data; checked 2026-07-16.
• WebAssembly Relaxed SIMD proposal — primary design history and tests for the feature incorporated into Core 3.0. Engine availability remains a runtime compatibility question; checked 2026-07-16.
• WebAssembly threads proposal — primary shared-memory, atomic-instruction, and memory-model proposal history/specification. Proposal repository status and integrations evolve; checked 2026-07-16.
• Emscripten SIMD documentation — primary compiler flags, intrinsics, autovectorization, and Relaxed SIMD guidance. Development documentation changes with releases; checked 2026-07-16.
• Emscripten pthreads documentation — primary worker-pool, proxying, shared-memory, fallback-build, and deployment constraints. Development documentation changes with releases; checked 2026-07-16.
• ECMAScript shared memory and Atomics — normative host-language semantics for SharedArrayBuffer and linked Atomics operations. Living ECMAScript specification; checked 2026-07-16.
• HTML crossOriginIsolated — normative browser isolation state used to gate powerful shared-memory capabilities. Living standard; checked 2026-07-16.

Connects to: Tensor Kernels and Inference from Scratch · Toolchains and Language Targets · WebGPU for AI · WebNN and Adaptive Backends · Wasm AI Performance, Security, and Craftsmanship