rupeshs/ovllm_node_addon
OpenVINO LLM Node.js C++ addon
This helps developers integrate large language models (LLMs) into their Node.js applications, enabling local AI capabilities. It takes a pre-trained OpenVINO LLM model and allows a Node.js application to send text prompts and receive generated text responses. Developers building interactive applications or services that need on-device AI inference are the primary users.
No commits in the last 6 months.
Use this if you are a Node.js developer looking to add efficient, local large language model inference capabilities to your applications using OpenVINO.
Not ideal if you are not a developer or if you need to train or fine-tune LLMs, as this tool is focused on inference.
Stars
8
Forks
—
Language
C++
License
—
Category
Last pushed
Jul 13, 2024
Commits (30d)
0
Get this data via API
curl "https://pt-edge.onrender.com/api/v1/quality/llm-tools/rupeshs/ovllm_node_addon"
Open to everyone — 100 requests/day, no key needed. Get a free key for 1,000/day.
Higher-rated alternatives
lmstudio-ai/lmstudio-js
LM Studio TypeScript SDK
lmstudio-ai/lms
LM Studio CLI
samestrin/llm-interface
A simple NPM interface for seamlessly interacting with 36 Large Language Model (LLM) providers,...
nbonamy/multi-llm-ts
A Typescript library to use LLM providers APIs in a unified way.
token-js/token.js
Integrate 200+ LLMs with one TypeScript SDK using OpenAI's format.