We want to run inference directly in the browser. What is the best JavaScript library for running ML models client-side?

If your goal is general-purpose ML inference directly in the browser, I’d choose Hugging Face Transformers.js for most new projects.

My recommendation

Library	Best for	Browser GPU	Model ecosystem	Ease of use
Transformers.js	Modern NLP, LLMs, vision, audio	WebGPU	Excellent	⭐⭐⭐⭐⭐
ONNX Runtime Web	Maximum control/performance with ONNX	WebGPU	ONNX	⭐⭐⭐⭐
TensorFlow.js	TensorFlow models / custom JS ML	WebGPU	TensorFlow.js	⭐⭐⭐⭐

Why Transformers.js: it gives you a high-level pipeline() API while using ONNX Runtime underneath, supports text, vision, audio and multimodal tasks, and can run models through WebAssembly on CPU or WebGPU on compatible devices.Hugging Face+1

For example:

import { pipeline } from "@huggingface/transformers";

const classifier = await pipeline(
  "sentiment-analysis",
  "Xenova/distilbert-base-uncased-finetuned-sst-2-english",
  { device: "webgpu" }
);

const result = await classifier("This is surprisingly good!");
console.log(result);

For maximum low-level control, I'd use ONNX Runtime Web directly. It lets you load an ONNX model and explicitly select WebGPU, WASM, etc.; it's particularly attractive if you already have a model exported to ONNX or need to optimize the inference pipeline yourself.ONNX Runtime+1

import * as ort from "onnxruntime-web/webgpu";

const session = await ort.InferenceSession.create(
  "/model.onnx",
  { executionProviders: ["webgpu"] }
);

One important caveat: WebGPU isn't universal yet. Current browser support is substantial, but Transformers.js notes roughly 85% global support as of March 2026, with browser/platform differences. You should provide a WASM/CPU fallback if your application needs broad compatibility.Hugging Face+1

Rule of thumb

• Building an AI feature quickly → Transformers.js
• You control/own the ONNX model and care about performance → ONNX Runtime Web
• Already invested in TensorFlow → TensorFlow.js
• Running LLMs/embeddings/Whisper/etc. locally → Transformers.js + WebGPU
• Tiny conventional models where compatibility matters most → ONNX Runtime Web + WASM

Also, for browser inference, quantized models are usually worth prioritizing because model download size and client memory are major constraints. Transformers.js explicitly recommends quantized models for resource-constrained environments such as browsers.Hugging Face

If you tell me what model you want to run (LLM, embedding model, image model, Whisper, custom PyTorch model, etc.) and your target browsers, I can recommend the exact stack and model format.

If your goal is ML inference directly in a browser, my default recommendation today is huggingface.co—if your models are in the Hugging Face/Transformer ecosystem. It uses ONNX Runtime underneath and gives you a much higher-level API than working with tensors and execution providers directly.Hugging Face

How I'd choose

Library	Best for	Browser GPU	API level
Transformers.js	LLMs, embeddings, vision, speech, multimodal models	WebGPU	High-level
ONNX Runtime Web	Arbitrary/custom ONNX models, maximum runtime control	WebGPU	Low-level
TensorFlow.js	TensorFlow/Keras models and TF.js-specific workflows	WebGL/WebGPU	Medium

Transformers.js is particularly attractive for text, embeddings, image classification/object detection, speech recognition, and similar workloads. It can use WASM on CPU or WebGPU, and supports quantized models such as q8 and q4, which can substantially reduce browser download and memory requirements.Hugging Face

For example:

import { pipeline } from "@huggingface/transformers";

const classifier = await pipeline(
  "sentiment-analysis",
  "Xenova/distilbert-base-uncased-finetuned-sst-2-english",
  { device: "webgpu" }
);

const result = await classifier("This runs entirely in the browser!");
console.log(result);

When I'd use ONNX Runtime Web instead

Choose onnxruntime.ai if you have your own ONNX models or need fine-grained control over model execution. It supports WASM, WebGPU, WebGL, and WebNN depending on browser/platform.ONNX Runtime+1

import * as ort from "onnxruntime-web/webgpu";

const session = await ort.InferenceSession.create(
  "/model.onnx",
  { executionProviders: ["webgpu"] }
);

const output = await session.run({
  input: new ort.Tensor("float32", data, [1, 3, 224, 224])
});

WebGPU is generally the direction I'd target for compute-heavy browser inference; ONNX Runtime specifically recommends WebGPU over its older WebGL path for better performance.ONNX Runtime+1

Bottom line

• Building an AI feature around existing Hugging Face models → Transformers.js
• Deploying custom ONNX models / maximum control → ONNX Runtime Web
• Already invested heavily in TensorFlow/Keras → TensorFlow.js
• Need broad browser compatibility → use WASM fallback
• Need good performance on capable devices → WebGPU + quantized models

If you tell me what model you want to run (LLM, image model, Whisper, embeddings, etc.) and your target browsers, I can recommend the specific runtime/model format and architecture.

The "best" JavaScript library for client-side browser inference depends entirely on what kind of model you are trying to run . There is no single silver bullet, but the ecosystem has consolidated around a few standout champions based on your specific use case.

Here is the breakdown of the top contenders and when to use them:

• Transformers.js (by Hugging Face)
  • Best for: General-purpose AI (NLP, embeddings, computer vision, audio, and smaller generative tasks) directly using Hugging Face models.
  • Why it shines: It brings the Hugging Face ecosystem straight to the browser using ONNX Runtime Web under the hood. You can run thousands of pre-trained models (like BERT, Whisper, or small vision models) with just a few lines of JavaScript without setting up a backend server. It supports WebAssembly (WASM) and WebGPU for hardware acceleration.
  • Check it out: Transformers.js on Hugging Faceweb.dev+1
• WebLLM (by MLC-AI)
  • Best for: Running large language models (LLMs) like Llama 3, Phi-3, or Mistral completely client-side.
  • Why it shines: If your goal is a high-performance in-browser chatbot or local assistant, WebLLM leverages WebGPU and optimized compilation (MLC) to deliver native-like token generation speeds directly in the browser tab. It handles model caching and memory management exceptionally well for heavy generative models.reddit.com+1
• TensorFlow.js (TF.js)
  • Best for: Traditional ML, custom-trained Keras/TensorFlow models, and fine-grained tensor operations in the browser.
  • Why it shines: It is the veteran framework of the bunch. It supports hardware acceleration via WebGL, WebAssembly, and WebGPU. Choose TF.js if you trained a custom model in Python using TensorFlow/Keras, need to retrain or adapt a model on the client side, or need low-level linear algebra operations.
  • Check it out: TensorFlow.js Documentationweb.dev
• Google MediaPipe (Web)
  • Best for: Real-time, ultra-fast computer vision and audio tasks (face mesh, pose estimation, hand tracking, object detection, segmentation).
  • Why it shines: If you need real-time 60fps video processing on a user's webcam feed, MediaPipe Solutions is heavily optimized out-of-the-box for low latency and minimal resource consumption on mobile and desktop browsers alike.

Quick Decision Matrix

1. Want to run a standard open-source Hugging Face model (embeddings, text classification, image-to-text)? Use Transformers.js.
2. Want to run a heavy conversational LLM locally in the browser? Use WebLLM.
3. Migrating an existing Python TensorFlow/Keras model to the web? Use TensorFlow.js.
4. Doing real-time webcam/audio tracking or segmentation? Use MediaPipe.web.dev

To help narrow this down, could you tell me:

• What specific type of model or task are you trying to run (e.g., LLM chat, image classification, audio transcription)?
• Do you already have a pre-trained model file ready, or are you starting from scratch?

Running machine learning inference directly in the browser depends heavily on your specific use case, what kind of model you plan to run, and whether you are loading a pre-trained transformer or rolling out a custom architecture.pkgpulse.com+1

The client-side JavaScript ML ecosystem centers around three primary libraries, each serving a distinct purpose:

1. Transformers.js (by Hugging Face)
   • Best for: Running state-of-the-art Hugging Face models (like BERT, Whisper, or small LLMs and vision transformers) directly in the browser with minimal friction.
   • How it works: It uses ONNX Runtime Web under the hood, backed by WebAssembly (WASM) and WebGPU for acceleration. It provides high-level pipelines matching the Python transformers library.
   • Pros: Drop-in access to tens of thousands of pre-trained models; excellent task-level APIs (sentiment analysis, text generation, object detection); robust hardware fallback options.
   • Cons: Less optimized if you want to build and train custom, non-transformer layers from scratch.huggingface.co+2
2. ONNX Runtime Web (onnxruntime-web)
   • Best for: Production-grade performance and direct, low-level control over custom or converted ONNX models (exported from PyTorch, JAX, or TensorFlow).
   • How it works: Microsoft’s high-performance inference engine compiled for the web. It features execution providers targeting WebGPU, WebNN, WebGL , and WebAssembly.
   • Pros: Incredible execution speed, excellent hardware acceleration (leveraging modern WebGPU/WebNN standards), and fine-grained control over tensors and memory.
   • Cons: Lower-level API means you have to handle tokenization, pre-processing, and post-processing yourself compared to task-based libraries.huggingface.co+2
3. TensorFlow.js (TF.js)
   • Best for: Training models directly in the browser, interactive educational web apps, or deploying legacy models originally trained in the TensorFlow ecosystem.
   • How it works: Google’s end-to-end machine learning framework for JavaScript supporting WebGL and WebGPU acceleration.
   • Pros: The most mature and historically established framework; allows client-side fine-tuning or training from scratch; pairs nicely with high-level wrappers like ml5.js for creative coding and education.
   • Cons: Heavier bundle sizes and a larger dependency tree; for pure inference on modern LLMs or transformer pipelines, it has largely been outperformed in developer preference by ONNX-backed runtimes.reddit.com+3

Quick Decision Guide

• Choose Transformers.js if you want to quickly integrate a standard NLP, audio, or vision model from the Hugging Face Hub.
• Choose ONNX Runtime Web if you have a custom-trained model, need max throughput via WebGPU/WebNN, and want absolute control over tensor operations.
• Choose TensorFlow.js if you need client-side model training or are working within an existing TensorFlow-heavy stack.devcentrehouse.eu+2

To help narrow down the best fit, could you share:

• What type of model you plan to run (e.g., LLM, image classification, audio transcription)?
• Are you using a pre-trained model or a custom model you trained yourself?

For general-purpose ML inference directly in the browser, my default choice today would be Hugging Face Transformers.js.

huggingface.co

Why Transformers.js

• Very easy API: its pipeline() API makes loading and running models straightforward.
• Broad model support: NLP, LLMs, embeddings, computer vision, speech/audio, and multimodal models.GitHub+1
• WebGPU support: you can run inference on the user's GPU with device: "webgpu".Hugging Face
• WebAssembly fallback: browser inference can run through ONNX Runtime/WASM when GPU acceleration isn't available.GitHub
• Quantization: useful for keeping model downloads and memory usage manageable in browsers; Transformers.js supports formats such as q4 and q8.Hugging Face
• ONNX underneath: models can be converted from PyTorch, TensorFlow, or JAX to ONNX, giving you a fairly broad deployment path.GitHub

A minimal example:

import { pipeline } from "@huggingface/transformers";

const classifier = await pipeline(
  "sentiment-analysis",
  "Xenova/distilbert-base-uncased-finetuned-sst-2-english",
  {
    device: "webgpu",
  }
);

const result = await classifier("This runs entirely in my browser!");
console.log(result);

When I'd choose something else

Library	Best choice when...
Transformers.js	You want pretrained NLP/LLM/vision/audio models with the least development work
ONNX Runtime Web	You have your own ONNX model and want maximum control over inference/runtime behavior
TensorFlow.js	You're already invested in TensorFlow/Keras or want to train/retrain models in JavaScript
WebGPU directly	You're building a highly specialized inference engine and need low-level GPU control

ONNX Runtime Web is particularly compelling if your team owns the model pipeline. It exposes the WebGPU execution provider directly and gives you lower-level control over tensors, execution, and GPU memory.ONNX Runtime

onnxruntime.ai

TensorFlow.js remains a good option for TensorFlow-centric applications and supports both running existing models and training/retraining in the browser.TensorFlow+1

My recommendation

If you're starting a new project, I'd use:

Transformers.js → WebGPU when available → WASM fallback → quantized models → Web Worker for inference.

That gives you a relatively high-level developer experience while still taking advantage of the browser GPU. For a production application, I'd also put inference in a Web Worker so large model execution doesn't freeze the UI; current Transformers.js integrations provide worker-oriented patterns for this.GitHub

If you tell me what model you're trying to run (LLM, embeddings, image model, Whisper, custom PyTorch model, etc.) and your target browsers, I can recommend the specific library/runtime and model format.

There is no single "best" library because the ideal choice depends heavily on what kind of model you want to run (e.g., standard computer vision/embeddings vs. large language models) and your target ecosystem.

The modern client-side AI landscape centers on a few standout libraries, heavily accelerated by modern web compute abstractions like WebAssembly (WASM) and WebGPU:youtube.com+2

1. Best for LLMs & Generative AI: WebLLM

• Best for: Running heavy Large Language Models (like Llama 3, Phi-3, or Qwen) locally in the browser.
• How it works: It brings the MLC-LLM ecosystem to the web, leveraging WebGPU for high-throughput acceleration.
• Pros: Unmatched performance for generative text models in the browser; supports a wide range of open-weight chat models; manages model weights and caching smoothly.
• Cons: Strictly built for LLMs; requires WebGPU support (won't work gracefully on older hardware/browsers without fallback limitations).web.dev+3

2. Best for NLP, Audio, & Vision Pipelines: Transformers.js

• Best for: Running standard Hugging Face models (sentiment analysis, object detection, embeddings, Whisper audio transcription) completely client-side.altersquare.io+1
• How it works: Developed by Hugging Face, it is functionally equivalent to the Python transformers library, but runs in the browser using ONNX Runtime Web under the hood.altersquare.io+1
• Pros: Direct access to tens of thousands of pretrained models on the Hugging Face Hub; high-level pipeline API makes implementation trivial; robust WebGPU and WASM fallback support.altersquare.io+1
• Not designed for massive generative LLMs (though smaller ones are supported, heavy LLMs are better suited to ).

3. Best for Framework Agnosticism & Custom Models: ONNX Runtime Web

• Best for: Maximum flexibility when you have custom models trained in PyTorch, TensorFlow, or scikit-learn and want to export them to ONNX format for web deployment.intel.com+2
• How it works: Microsoft’s cross-platform inference engine compiled for the web. It automatically targets WebGPU, WebGL, or WebAssembly depending on client device capabilities.altersquare.io+1
• Pros: Extremely fast and optimized; framework-agnostic (export from PyTorch/TF); handles hardware acceleration gracefully.web.dev+2
• Cons: Verbose API. You have to handle your own tokenization, pre-processing, and post-processing logic compared to the plug-and-play nature of Transformers.js.

4. Best for Real-Time Computer Vision: MediaPipe (by Google)

• Best for: Face tracking, hand-pose estimation, selfie segmentation, and object detection.
• How it works: Highly optimized, production-ready pre-packaged solutions tailored specifically for real-time perception tasks.
• Pros: Incredible out-of-the-box performance on mid-range and mobile hardware; handles end-to-end media pipelines smoothly.
• Cons: Limited strictly to Google’s curated set of perception models; not a general-purpose ML runner.web.dev+1

(Note on TensorFlow.js : While historically the go-to standard, it has largely been superseded for high-performance inference by ONNX Runtime and WebGPU-native tooling, though it remains useful if you are deeply embedded in the TensorFlow ecosystem or need client-side training.)medium.com+1

Quick Decision Matrix

• Choose WebLLM if you want an in-browser chatbot or text-generation assistant.
• Choose Transformers.js if you want an easy, high-level API to pull standard models from Hugging Face for embeddings, audio, or vision.
• Choose ONNX Runtime Web if you trained a custom model and want the fastest, rawest ONNX execution layer.
• Choose MediaPipe if you are strictly building real-time camera/video tracking features.web.dev+3

To help narrow this down, tell me:

• What specific ML task are you trying to accomplish (e.g., text generation, image classification, audio transcription)?
• Are you bringing your own trained model , or looking for a pre-trained model?

The best JavaScript library for running machine learning models client-side depends entirely on your specific model architecture and use case, but Transformers.js by Hugging Face is the top recommendation for modern NLP, computer vision, and audio models, while ONNX Runtime Web (ORT-Web) or LiteRT.js are better suited for custom or lightweight architectures.youtube.com+3

Top Client-Side JavaScript ML Libraries

Key Library Breakdown

• Transformers.js (Hugging Face): The definitive choice if you want to run modern pre-trained models (like BERT, Llama-style small language models, or Whisper speech-to-text) without building pipelines from scratch. It mirrors the Python transformers API and handles tokenization and execution seamlessly.youtube.com+3
• ONNX Runtime Web: The underlying engine for many modern web frameworks. Choose ORT-Web if you train custom models in PyTorch or JAX, export them to the ONNX format, and need fine-grained control over the inference session and memory management in the browser.huggingface.co+1
• LiteRT.js (formerly TFLite Web): Google's high-performance native web runtime. It executes .tflite files via WebAssembly and taps straight into hardware accelerators like WebNN and XNNPACK, making it ideal for mobile-first web apps.developers.googleblog.com+1

Hardware Acceleration Considerations

To achieve near-native speeds in client-side inference, ensure your implementation targets modern browser acceleration APIs:

1. WebGPU: The gold standard for client-side GPU acceleration, offering massive performance boosts over older WebGL contexts.
2. WebNN: An emerging web API designed to hook directly into device NPUs (Neural Processing Units) for energy-efficient local execution.
3. WebAssembly (WASM): Used as a high-performance fallback when GPU/NPU acceleration is unavailable, heavily optimized via multithreading and SIMD.youtube.com+3

If you tell me what specific model or task (e.g., LLM text generation, image segmentation, speech recognition) you plan to run, I can recommend the ideal library configuration and quantization approach.

If your goal is ML inference entirely in the browser, my default recommendation today is huggingface.co.

It depends somewhat on what models you're running:

Library	Best for	Browser acceleration	Ease of use
Transformers.js	LLMs, embeddings, NLP, vision, audio	WebGPU + WASM	⭐⭐⭐⭐⭐
ONNX Runtime Web	Custom ONNX models / maximum control	WebGPU + WASM + WebNN	⭐⭐⭐⭐
TensorFlow.js	TensorFlow/Keras models, browser ML	WebGPU/WebGL/WASM	⭐⭐⭐⭐

My recommendation: Transformers.js

For modern AI applications, I'd start here. It lets you run pretrained Transformer models directly in the browser and provides a high-level pipeline() API, so you don't have to manually implement tokenization, preprocessing, inference, and postprocessing.Hugging Face+1

For example:

import { pipeline } from "@huggingface/transformers";

const classifier = await pipeline(
  "sentiment-analysis",
  "Xenova/distilbert-base-uncased-finetuned-sst-2-english",
  { device: "webgpu" }
);

const result = await classifier("This is fantastic!");
console.log(result);

Transformers.js uses ONNX-based execution under the hood, and its newer releases have significantly improved WebGPU support and performance.Hugging Face

When I'd choose ONNX Runtime Web instead

If you have your own model—for example, a PyTorch model that you've exported to ONNX—and you care about squeezing out performance or controlling the inference pipeline yourself, I'd use onnxruntime.ai directly.

import * as ort from "onnxruntime-web/webgpu";

const session = await ort.InferenceSession.create(
  "/models/model.onnx",
  { executionProviders: ["webgpu"] }
);

const output = await session.run({
  input: new ort.Tensor("float32", data, [1, 3, 224, 224])
});

ONNX Runtime Web supports WASM for broad compatibility and WebGPU for more compute-intensive workloads. Its documentation specifically recommends WebGPU when you want to take advantage of the client's GPU.ONNX Runtime+1

Practical decision

I'd use:

• LLM / embeddings / speech / computer vision model from Hugging Face → Transformers.js
• Your own ONNX model → ONNX Runtime Web
• Existing TensorFlow/Keras model → TensorFlow.js
• Maximum performance → ONNX Runtime Web + WebGPU, assuming your target browsers support it
• Need to work on weaker/older browsers → WASM fallback

One important architectural point: WebGPU should generally be your preferred backend for heavier models, with WASM as the fallback. ONNX Runtime's current browser documentation notes that WebGL is now in maintenance mode and recommends WebGPU for better performance.ONNX Runtime

If you tell me what model you want to run (e.g. Llama, Whisper, YOLO, BERT, a custom PyTorch model), I can recommend the exact browser stack and model format.

For example: