Traducción pendiente: esta página conserva la fuente canónica en inglés mientras la navegación sigue disponible en español.

Browser Inference Runtimes

Introduction and mental model

A browser inference runtime is the layer that turns a model contract into executable work. It parses a representation, resolves operators, allocates tensors, chooses or accepts an execution backend, schedules computation, and returns outputs whose ownership must be understood. The runtime is not the backend: ONNX Runtime Web or LiteRT.js can expose more than one execution path, while WebAssembly CPU, WebGPU, and WebNN have different capability and memory models.

This branch studies the complete lifecycle: capability discovery, model fetch, initialization, session creation, input tensor construction, execution, output interpretation, reuse, cancellation where available, and disposal. Backend selection and fallback must be observable. A successful call is not proof that the requested provider executed every operation; the runtime's documented partitioning and fallback behavior, diagnostics, and a forced-backend test all matter.

ONNX Runtime Web, LiteRT.js, Transformers.js, MediaPipe Tasks, and TensorFlow.js solve overlapping but non-identical problems. Some are general graph runtimes, some add model and preprocessing pipelines, and some target particular solution families. Comparisons must therefore name the model, runtime version, backend, operator set, precision, browser, and device.

Why it matters

The runtime boundary determines model compatibility, application startup, tensor-copy costs, memory lifetime, and the failure modes a user sees. It also determines whether a deployment can remain local and offline after assets are cached. Treating this layer as a single run() call hides shader compilation, Wasm initialization, CPU fallback, repeated allocations, and resources that survive longer than intended.

A small, explicit runtime contract makes experiments comparable without pretending that all runtimes have identical capabilities. It also makes backend decisions auditable and lets the application report why a preferred path was unavailable.

Questions this branch answers

Which responsibilities belong to a model format, a runtime, an execution provider, and application code?
How do ONNX Runtime Web and LiteRT.js expose Wasm CPU, WebGPU, and emerging WebNN paths?
When do Transformers.js, MediaPipe Tasks, or TensorFlow.js provide a more appropriate abstraction?
How can a model be checked against operator, data-type, shape, and backend coverage before deployment?
What work belongs in initialization, model loading, warmup, execution, and disposal?
When can tensors remain resident on an accelerator, and when do input or output copies occur?
How should unsupported operators, unavailable devices, context loss, and memory pressure be surfaced?
How can forced-backend tests prove that an apparently successful run did not silently use a fallback?
Which runtime metrics are comparable, and which measure different amounts of work?

Scope

ONNX Runtime Web, LiteRT.js, Transformers.js, MediaPipe Tasks for Web, and TensorFlow.js when its graph, layers, or ecosystem is relevant.
Wasm CPU execution, SIMD and worker/thread configuration, WebGPU execution providers, and WebNN integrations where exposed.
Model and operator compatibility, session options, capability discovery, and provider configuration.
Initialization, loading, warmup, tensor construction, repeated execution, output ownership, and cleanup.
Tensor layout, data type, memory placement, accelerator residency, I/O binding, and copy boundaries.
Explicit fallback policies, reason codes, telemetry, and user-visible diagnostics.
A conceptual common boundary for controlled comparisons, without erasing runtime-specific features.

Out of scope

Implementing a common runtime interface or adapters in this index.
Treating a high-level task pipeline as equivalent to a general graph runtime.
Training, fine-tuning, model conversion internals, or hand-written GPU kernels.
A universal backend ranking or performance claim across browsers and devices.
Silent fallback as an acceptable compatibility strategy.

Expected outcomes

After completing this branch, a reader should be able to map an application's inference lifecycle, choose a runtime for a named model and deployment constraint, configure a backend deliberately, and identify tensor-copy and cleanup boundaries. They should be able to build a compatibility test matrix, distinguish initialization from steady-state latency, and report requested, selected, and fallback providers without ambiguity.

Candidate note roadmap

Runtime, model format, execution provider, and task pipeline — define the layers and show why similar-looking JavaScript APIs may own different parts of inference.
ONNX Runtime Web from environment setup to session disposal — follow Wasm asset resolution, session options, execution providers, tensors, repeated runs, and resource release.
LiteRT.js as a new browser runtime surface — examine its LiteRT model contract, CPU and WebGPU paths, changing WebNN integration, package pinning, and fallback evidence.
Transformers.js pipelines and lower-level model execution — separate convenient tokenizer/model/postprocessor composition from backend and tensor details that still affect deployment.
MediaPipe Tasks and TensorFlow.js at different abstraction levels — compare packaged vision, text, or audio tasks with graph/layers ecosystem integration, including model constraints, backend selection, tensor lifetime, conversion assumptions, and explicit disposal.
Operator coverage, forced providers, and explicit fallback traces — turn a model inventory into provider-specific smoke tests, actionable unsupported-operation reports, and a decision trace containing request, probes, selected path, partitioning or fallback, reasons, and runtime evidence.
Tensor lifecycle, accelerator residency, and hidden copies — track allocation, reuse, I/O binding, readback, output ownership, and cleanup across asynchronous calls.
Benchmarking browser runtimes without timing different work — separate download, compilation, initialization, warmup, input preparation, inference, readback, and disposal.

Future runnable artifact

Create a two-adapter comparison harness for one deliberately small model available in both ONNX and LiteRT form with matching inputs and validated outputs. The target is a conceptual InferenceBackend boundary with four responsibilities: load a model, run named tensors, report structured capabilities and the selected provider, and dispose owned resources. This is a future design target only; this index neither declares an interface nor implements an adapter, and runtime-specific options must remain accessible rather than being reduced to a false common denominator.

One future adapter will use ONNX Runtime Web and one will use LiteRT.js. Each will support a cold lifecycle and a reused-session lifecycle, an explicitly requested backend, a no-silent-fallback mode where the runtime permits it, and a structured decision trace. The harness will compare raw outputs to a reference, run repeated calls to reveal allocation growth, force cleanup, and report model bytes, initialization, first run, warm runs, input upload, output readback, and total wall time. Unsupported providers and operators will be first-class results.

How to verify and measure

Pin the runtime package, model, browser, operating system, and device identifiers used by a result.
Inventory model operators, shapes, layouts, and data types, then test every requested provider independently.
Record requested provider, discovered capabilities, selected provider, fallback or graph partitioning, and reason codes.
Verify raw outputs against the same reference fixtures before comparing latency.
Separate asset fetch, module initialization, session creation, shader or graph compilation, warmup, inference, readback, and disposal.
Measure cold and warm paths and report sample count, warmup policy, distribution statistics, and input shape.
Track tensor and session reuse; run bounded repetition and inspect memory behavior after explicit disposal.
Simulate an unavailable provider, unsupported operation, corrupted model, aborted load, and device/context loss where the API exposes it.
Test offline reload from a fresh, deliberately populated cache rather than from an incidental developer cache.

Primary sources

ONNX Runtime Web documentation — https://onnxruntime.ai/docs/tutorials/web/ — Current runtime and execution-provider guidance; version-sensitive and consulted 2026-07-16.
ONNX Runtime Web: WebGPU execution provider — https://onnxruntime.ai/docs/tutorials/web/ep-webgpu.html — Current WebGPU, profiling, and GPU-tensor guidance; version-sensitive and consulted 2026-07-16.
ONNX Runtime Web environment flags and session options — https://onnxruntime.ai/docs/tutorials/web/env-flags-and-session-options.html — Current configuration surface; version-sensitive and consulted 2026-07-16.
ONNX Runtime Web build documentation — https://onnxruntime.ai/docs/build/web.html — As consulted 2026-07-16, the build documentation labels WebGPU and WebNN support as experimental and warns that instructions may change.
LiteRT.js for Web — https://developers.google.com/edge/litert/web — Google documentation for the new browser runtime; rapidly changing and consulted 2026-07-16. CPU/Wasm and WebGPU behavior must be checked against the pinned release, while its WebNN path is described as emerging and should be treated as experimental.
LiteRT.js announcement — https://developers.googleblog.com/en/litertjs-googles-high-performance-web-ai-inference/ — Initial public announcement dated 2026-07-09; consulted 2026-07-16. Its recency is a reason to pin versions and revalidate APIs, not a maturity guarantee.
Transformers.js documentation — https://huggingface.co/docs/transformers.js/ — Current task, model, and backend documentation; version-sensitive and consulted 2026-07-16.
MediaPipe Tasks setup for Web — https://ai.google.dev/edge/mediapipe/solutions/setup_web — Current task-oriented Web setup guidance; version-sensitive and consulted 2026-07-16.
TensorFlow.js guide — https://www.tensorflow.org/js/guide — Current official guide for models, backends, tensors, and browser execution; version-sensitive and consulted 2026-07-16.
WebNN specification — https://www.w3.org/TR/webnn/ — W3C Candidate Recommendation Draft dated 2026-06-26, consulted 2026-07-16. It remains work in progress; implementation and device coverage must be detected at runtime.

Connects to: Model Formats, Conversion, and Quantization · Preprocessing, Postprocessing, and Media Pipelines · WebGPU for AI · WebNN and Adaptive Backends · Local-First, Offline, Multimodal AI · Wasm AI Performance, Security, and Craftsmanship