Inference in the browser
Browser-native LLM inference engine in WGSL compute shaders. Zero server cost.
- 45.5
- tok/s decode
- 334.9 MiB
- weights resident
- 5.63x
- compression
- 0
- server cost
LiveWGSL · WebGPU · TypeScript · Qwen2.5-0.5B
Enargeia runs Qwen2.5-0.5B-Instruct entirely in the browser. The visitor’s GPU does the inference, so serving it costs nothing beyond static files — no server, no API key, no per-token cost. Every kernel is WGSL running on WebGPU.
The model is quantized to int4, except the tied embedding and LM head which ship at int8 — 334.9 MiB resident against 1884.6 at fp32, for 13.5% worse perplexity. Decode runs at 45.5 tokens/second at 512 context and 43.1 at 2048, flat across context length. Time to first token is 219 ms on a short prompt. Cold load over the CDN is 14.6 seconds; warm is 1.4.
Measured on an Apple M2 with a 10-core GPU.