r2d4/react-llm
Easy-to-use headless React Hooks to run LLMs in the browser with WebGPU. Just useLLM().
Leverages Apache TVM and MLC Relax compiled to WebAssembly to execute Vicuna models entirely on-device, with model weights and tokenizer (SentencePiece) cached in browser storage for faster subsequent loads. Offloads inference to a WebWorker to avoid blocking the main thread, while conversation state persists to localStorage. The modular monorepo includes companion packages for pre-built retro UI components and a Chrome extension, allowing both headless integration and turnkey deployment.
702 stars. No commits in the last 6 months.
Stars
702
Forks
32
Language
TypeScript
License
MIT
Category
Last pushed
Jun 27, 2023
Commits (30d)
0
Get this data via API
curl "https://pt-edge.onrender.com/api/v1/quality/llm-tools/r2d4/react-llm"
Open to everyone — 100 requests/day, no key needed. Get a free key for 1,000/day.
Higher-rated alternatives
mlc-ai/web-llm
High-performance In-browser LLM Inference Engine
e2b-dev/desktop
E2B Desktop Sandbox for LLMs. E2B Sandbox with desktop graphical environment that you can...
geekjr/quickai
QuickAI is a Python library that makes it extremely easy to experiment with state-of-the-art...
chrisrobison/textweb
A text-grid web renderer for AI agents — see the web without screenshots
Azure-Samples/llama-index-javascript
This sample shows how to quickly get started with LlamaIndex.ai on Azure 🚀