yas-sim/openvino-llm-minimal-code
Most simple and minimal code to run an LLM chatbot from HuggingFace hub with OpenVINO
This project helps developers integrate large language models (LLMs) into their applications with minimal code. It takes an LLM from HuggingFace Hub and converts it into an optimized OpenVINO format. The output is a running chatbot that can process text inputs and generate responses, either all at once or word-by-word. This is for developers building AI-powered features who need efficient, local LLM deployment.
No commits in the last 6 months.
Use this if you are a developer looking for the simplest way to run an LLM from HuggingFace on your machine using OpenVINO for optimized performance.
Not ideal if you are an end-user without programming experience or if you need a solution that doesn't require direct code interaction.
Stars
5
Forks
—
Language
Python
License
—
Category
Last pushed
Feb 19, 2024
Commits (30d)
0
Get this data via API
curl "https://pt-edge.onrender.com/api/v1/quality/transformers/yas-sim/openvino-llm-minimal-code"
Open to everyone — 100 requests/day, no key needed. Get a free key for 1,000/day.
Higher-rated alternatives
jakobdylanc/llmcord
Make Discord your LLM frontend - Supports any OpenAI compatible API (Ollama, xAI, Gemini,...
xNul/chat-llama-discord-bot
A Discord Bot for chatting with LLaMA, Vicuna, Alpaca, MPT, or any other Large Language Model...
amanvirparhar/weebo
A real-time speech-to-speech chatbot powered by Whisper Small, Llama 3.2, and Kokoro-82M.
innightwolfsleep/old_llm_telegram_bot
Connect llama-cpp, transformers or text-generation-webui to telegram bot api.
ma2za/telegram-llm-bot
Telegram LLM bot backed by OpenAI, Whisper, Beam, LLaMA, Weaviate, MinIO and MongoDB