Chat with a local LLM running on your own GPU

A full chat interface backed by a small language model that lives in your browser. SmolLM2-135M loads in seconds on almost any machine; SmolLM2-1.7B and Qwen3-0.6B give noticeably better answers if you have a few gigabytes of VRAM to spare. Tokens stream as they are generated and the page reports real tokens-per-second, so you can see exactly what your hardware is doing. Because inference is local, the conversation never leaves the machine — useful for drafting anything you would not paste into a hosted chatbot.

browser llmlocal llm chatprivate ai chatchatgpt alternative offlinesmollm2 browserqwen3 webgpuwebllm chatllm without api keyrun llama in browseroffline chatbot

Frequently asked questions

How good is a 135M or 1.7B model compared to GPT-4?

Not close, and we will not pretend otherwise. Small local models are good at rewriting, summarising, extracting structure, brainstorming and simple Q&A. They hallucinate more on facts and are weaker at long multi-step reasoning. Pick the biggest model your GPU can hold.

How fast is it?

On a mid-range discrete GPU expect roughly 20–60 tokens/second for a 0.5B model and 8–20 for a 1.7B model at q4. Integrated graphics run maybe a third of that. The first message is slower because the weights must be uploaded to the GPU.

Is my data uploaded to a server?

No. The model weights are downloaded from Hugging Face to your browser once, and every inference after that runs on your own GPU through WebGPU. Your text, images and audio are never sent anywhere — hjLabs.in has no inference server and no way to see your input.

Does this work on a phone?

Small models do. Recent mobile Chrome and Safari expose WebGPU, but memory is the binding limit — stay under roughly 200 MB of weights on Android, and expect iOS to reclaim the tab sooner. The catalog shows every download size up front so you can pick something that fits.

Can I use the output commercially?

That depends on the individual model, not on this site. Each model page links to its Hugging Face repository, where the licence is stated — most of the catalog is Apache-2.0 or MIT, a few are research-only. We add nothing on top: hjLabs.in claims no rights over anything you produce here.

Models available for this task

SmolLM2 135M Instruct — HuggingFaceTB/SmolLM2-135M-Instruct, 186 MB, 135M params
SmolLM2 360M Instruct — HuggingFaceTB/SmolLM2-360M-Instruct, 392 MB, 360M params
SmolLM2 1.7B Instruct — HuggingFaceTB/SmolLM2-1.7B-Instruct, 1416 MB, 1.7B params
Qwen3 0.6B — onnx-community/Qwen3-0.6B-ONNX, 928 MB, 0.6B params
Qwen2.5 0.5B Instruct — onnx-community/Qwen2.5-0.5B-Instruct, 798 MB, 0.5B params
Gemma 3 1B Instruct — onnx-community/gemma-3-1b-it-ONNX, 880 MB, 1B params
Qwen3 1.7B — onnx-community/Qwen3-1.7B-ONNX, 2195 MB, 1.7B params
Llama 3.2 1B Instruct — onnx-community/Llama-3.2-1B-Instruct-ONNX, 1704 MB, 1.2B params
Llama 3.2 3B Instruct — onnx-community/Llama-3.2-3B-Instruct-ONNX, 3417 MB, 3.2B params
Gemma 3 270M Instruct — onnx-community/gemma-3-270m-it-ONNX, 348 MB, 270M params
DeepSeek-R1 Distill 1.5B — onnx-community/DeepSeek-R1-Distill-Qwen-1.5B-ONNX, 1976 MB, 1.5B params
LFM2 1.2B — onnx-community/LFM2-1.2B-ONNX, 854 MB, 1.2B params
Qwen3 4B — onnx-community/Qwen3-4B-ONNX, 2881 MB, 4B params
Qwen2.5 Coder 1.5B — onnx-community/Qwen2.5-Coder-1.5B-Instruct, 1932 MB, 1.5B params
Phi-3.5 Mini (3.8B) — onnx-community/Phi-3.5-mini-instruct-onnx-web, 2320 MB, 3.8B params
SmolLM3 3B — HuggingFaceTB/SmolLM3-3B-ONNX, 2854 MB, 3B params
LFM2 2.6B — onnx-community/LFM2-2.6B-ONNX, 2096 MB, 2.6B params

Explore the rest of the playground

Text & language models

Text embeddings — Generate text embeddings locally, in the browser
Translation — Translate text offline with a local neural MT model
Summarization — Summarise long text with a model running on your machine
Sentiment analysis — Sentiment analysis that runs in your browser
Zero-shot text classification — Classify text into labels you make up on the spot
Named entity recognition — Named entity recognition, running on your own machine
Question answering — Ask questions about a passage, get the exact answer span
Fill-mask — Watch a masked language model guess the missing word

Guides, comparisons & alternatives

Model catalog — The model catalog: what actually runs in a browser
WebGPU vs WASM — WebGPU vs WebAssembly for running models in a browser
Browser AI vs cloud API — Browser inference vs a hosted API: an honest comparison
Transformers.js vs WebLLM — Transformers.js vs WebLLM
Whisper model sizes — Choosing a Whisper size for browser transcription
fp32 vs q8 vs q4 — Picking a quantization level for browser inference
Biggest browser LLM — How big a model can a browser tab actually run?
Why no image generation — Why this playground has no text-to-image generation
MedGemma in the browser — MedGemma in the browser: why it does not work, and what does
Free alternatives to paid AI tools — Free, local alternatives to paid AI services
remove.bg alternative — A remove.bg alternative that never uploads your image
Ollama alternative — An Ollama alternative that installs nothing
Open WebUI alternative — An Open WebUI alternative with no server behind it
Whisper API alternative — A Whisper API alternative that runs on your machine
Embeddings API alternative — An embeddings API alternative that costs nothing per token
FAQ — Frequently asked questions about browser-based AI
About — About this project

This playground needs JavaScript and WebGPU. Models are downloaded to your browser and executed on your own GPU, so there is nothing to show without a script engine. Enable JavaScript, or read the FAQ for what the site does.