leoneversberg/llm-chatbot-rag
A local LLM chatbot with RAG for PDF input files
Leverages Hugging Face models with optional bitsandbytes GPU quantization for inference optimization, and provides a Streamlit-based UI for interactive chat sessions. Supports multiple LLM architectures (including Gemma) via configurable model loading, with RAG implementation that processes PDF documents into searchable context for prompt augmentation. Designed for local deployment with flexible hardware requirements—quantization targets NVIDIA GPUs, but falls back to CPU inference with adjusted configuration.
No commits in the last 6 months.
Stars
87
Forks
30
Language
Jupyter Notebook
License
MIT
Category
Last pushed
Apr 17, 2024
Commits (30d)
0
Get this data via API
curl "https://pt-edge.onrender.com/api/v1/quality/rag/leoneversberg/llm-chatbot-rag"
Open to everyone — 100 requests/day, no key needed. Get a free key for 1,000/day.
Higher-rated alternatives
Cinnamon/kotaemon
An open-source RAG-based tool for chatting with your documents.
BastinFlorian/RAG-Chatbot-with-Confluence
RAG Chatbot with Confluence
olioDuan/Domain-Specific-RAG-Chat-Course-Helper
This project is a Local RAG system built for the NYU Machine Learning course.
antoinelrnld/discord-rag
Easily create a RAG based on your Discord messages
Mohannadcse/DepsRAG
Interactive LLM Chatbot that constructs direct and transitive software dependencies as a...