Local LLM Orchestration Vector Databases

Complete platforms and toolkits for running, managing, and integrating multiple AI components (LLMs, vector databases, agents) entirely on-premises without external APIs. Does NOT include standalone vector databases, individual LLM models, or cloud-dependent AI services.

There are 56 local llm orchestration tools tracked. 2 score above 70 (verified tier). The highest-rated is neuml/txtai at 91/100 with 12,281 stars and 54,726 monthly downloads. 2 of the top 10 are actively maintained.

Get all 56 projects as JSON

curl "https://pt-edge.onrender.com/api/v1/datasets/quality?domain=vector-db&subcategory=local-llm-orchestration&limit=20"

Open to everyone — 100 requests/day, no key needed. Get a free key for 1,000/day.

#	Tool	Score	Tier	Stars	Language
1	neuml/txtai 💡 All-in-one AI framework for semantic search, LLM orchestration and...	91	Verified	12,281	Python
2	Mintplex-Labs/anything-llm The all-in-one AI productivity accelerator. On device and privacy first with...	71	Verified	56,148	JavaScript
3	pathwaycom/llm-app Ready-to-run cloud templates for RAG, AI pipelines, and enterprise search...	65	Established	56,145	Jupyter Notebook
4	DerwenAI/strwythura Strwythura: construct an entity-resolved knowledge graph from structured...	60	Established	214	Python
5	thebearwithabite/ai-file-organizer Intelligent file organization with computer vision, audio analysis,...	47	Emerging	30	Python
6	Corpus-OS/corpusos Open-source protocol suite standardizing LLM, Vector, Graph, and Embedding...	46	Emerging	144	Python
7	ArshVermaGit/nexus-ai-life-os NEXUS 🧠 — AI-powered second brain that watches, remembers, and assists in...	36	Emerging	5	Python
8	Afnanksalal/co-op Expert guidance across legal, finance, investor relations, and competitive...	35	Emerging	4	TypeScript
9	MicroClub-USTHB/mAIcro mAIcro is a reusable, open-source AI infrastructure designed for communities...	34	Emerging	8	Python
10	somritdasgupta/aide aiDe is a browser extension that combines a sideKick bar and web UI for...	34	Emerging	5	TypeScript
11	localcloud-sh/localcloud Stop paying for AI APIs during development. LocalCloud runs everything...	30	Emerging	46	Go
12	RobinMillford/Cortex-AI-Multi-Model-Insights-Hub Cortex AI: Multi-Model Insights Hub is an advanced platform that leverages...	30	Emerging	3	Python
13	codingforentrepreneurs/reflex-gpt Build a Python Reflex App with OpenAI, a Neon Postgres + Vector Database,...	30	Emerging	41	Python
14	kalisam/Amazon_Rose_Forest_01 Amazon Rose Forest is the foundational codebase of FLOSSI0ULLK: a...	27	Experimental	3	Rust
15	gonicolas12/My_AI Personal, private & local AI assistant — 9 specialized agents (code, web,...	26	Experimental	5	Python
16	Bitwarelabscom/Luna_chat_v7 [In Active Development] AI OS with Council Architecture, autonomous agents,...	26	Experimental	5	TypeScript
17	squaredev-io/squaredev The platform for developers building applications powered by LLMs	26	Experimental	32	TypeScript
18	araray/llmcore A unified, async Python framework for LLM applications—chat, autonomous...	25	Experimental	3	Python
19	agentic-dev-io/mao Multi-agent orchestration with MCP — LangChain, DuckDB vector storage,...	24	Experimental	2	Python
20	kdyck/paimons-codex Open source AI manhwa platform: Generate stories with LLMs + create artwork...	24	Experimental	1	Python
21	gekonge/No-one 🌐 Manage Java WebShells effortlessly with No-one, a modern tool for secure...	23	Experimental	1	TypeScript
22	damn124/ai-calling-agent 🎙️ Enhance customer interactions with a real-time AI voice agent, utilizing...	23	Experimental	1	Python
23	dsk-dev-ai/devos-ai 🚀 DevOS AI — Open-source AI operating system for developers. Chat with...	23	Experimental	1	Python
24	angrysky56/quantum-framework Theoretical basis and prototype concept for an AI native OS as an infinite...	23	Experimental	2	JavaScript
25	GrandpaSatan/Yggdrasil Distributed AI service mesh for local LLM orchestration, memory, code...	22	Experimental	—	Rust
26	pramayharsh/omni-mesh A decentralized Multi-Agent AI swarm featuring autonomous memory, security...	22	Experimental	—	Python
27	jgoy-labs/server-nexe Local AI server with persistent memory, RAG, and multi-backend inference...	22	Experimental	—	Python
28	kunashee/AnyToolCall 🛠️ Simplify tool calls for any LLM with AnyToolCall, an OpenAI-compatible...	22	Experimental	—	JavaScript
29	TensorHub-Technologies/THub-App-Dev-0.1 THub Apps - A Low Code No Code Generative AI Development Platform	22	Experimental	—	TypeScript
30	piyush-eon/ai-portfolio-nextjs Build an AI Driven Portfolio App with NextJS and Tailwind CSS. We will learn...	22	Experimental	10	JavaScript
31	PerminovEugene/messy-folder-reorganizer-ai 🤖 AI-powered CLI for file reorganization. Runs fully locally — no data...	22	Experimental	20	Rust
32	irsyadddddds/nexus-ai-toolkit Composable AI agents with vector memory and structured prompt pipelines.	22	Experimental	—	Python
33	Nabarup1/Silvanus Acoustic Nervous System for Forest Protection. An AI-powered multi-agent...	21	Experimental	2	Python
34	korvexai/axon Local-First Autonomous Development Infrastructure	20	Experimental	1	Rust
35	KingLeoJr/AITools A comprehensive toolkit for building AI-powered applications, covering model...	20	Experimental	12	—
36	Rinkatecam/AI.Stack Private AI Infrastructure - Deploy OpenWebUI + Ollama + Qdrant in one...	20	Experimental	1	Shell
37	mubseoul/local-ai-workspace Privacy-first, offline RAG desktop app — chat with local LLMs and your...	19	Experimental	—	TypeScript
38	24kr/Local_LLM_Assistant LoLA: Privacy-first desktop AI with RAG. Chat offline with your docs using...	19	Experimental	—	JavaScript
39	BrooksErica/voice-biometric-authentication Cloud-native voice biometric authentication system using deep learning...	19	Experimental	—	—
40	aicubetechnology/qilbee-mycelial-network Enterprise SaaS platform for AI agent communication inspired by fungal...	19	Experimental	—	Python
41	celersneha/DevZoku DevZoku is a hackathon collaboration platform designed to help developers...	18	Experimental	4	TypeScript
42	trustlelab/siteware-backend-v2 Siteware Backend - German Voice AI Agent provider - Deepgram + Twilio +...	17	Experimental	5	TypeScript
43	ActionpackdHQ/Ignite-AI-SDK Ignite AI SDK - A powerful toolkit for building AI-powered applications with...	16	Experimental	1	TypeScript
44	Maruch11/LLM-local Documentación técnica y roadmaps para implementación de un LLM local con...	16	Experimental	2	—
45	anubissbe/knowledgehub 🧠 Enterprise AI-Enhanced Development Platform with GraphRAG, LlamaIndex,...	16	Experimental	9	Python
46	mrkingsleyobi/trustswarm Real-time blockchain fraud detection using autonomous multi-agent AI swarms....	15	Experimental	—	TypeScript
47	buitoan112233/EleutherIA 🧠 Explore a FAIR-compliant knowledge graph that analyzes ancient debates on...	14	Experimental	—	Shell
48	LoganthP/NexusAI NexusAI is an enterprise-grade AI platform for autonomous agents, RAG...	14	Experimental	—	TypeScript
49	MassivDash/ai_tools AI Tools is a tester and companion web application designed to help you...	14	Experimental	—	Rust
50	Harsh-SOE/devmatch 🚀 A platform for developers to search for other similar developers by their...	12	Experimental	1	JavaScript
51	harshmriduhash/CognitiveHub-AI A unified, AI-powered SaaS platform designed to support engineering teams...	12	Experimental	1	Python
52	PinsaraPerera/intellihack_backend This is the backend of viewMo AI cloud platform. AI base graph generation...	12	Experimental	4	Python
53	malihanawshin/CloudVoice-agent MCP-based Voice-Ops Agent for Sustainable Cloud Infrastructure Operations.	11	Experimental	—	Python
54	littleAvel/voice-agent End-to-end voice AI system demonstrating ASR, LLM-based planning, vector...	11	Experimental	—	Python
55	gitsudhir/aicli aicli is a Rust-based terminal application that implements a...	11	Experimental	—	Rust
56	hunny345686/FitMetrics FitMetrics AI — Cinematic Resume Intelligence Platform	11	Experimental	—	TypeScript

Comparisons in this category

anything-llm and llm-app (71 vs 65) anything-llm and localcloud (71 vs 30) anything-llm and My_AI (71 vs 26) anything-llm and llmcore (71 vs 25)