LLM Orchestration Platforms Embedding Tools

Tools for unifying access to multiple LLM providers, managing model selection, routing requests, and coordinating AI agent execution across different backends. Does NOT include domain-specific applications, single-provider SDKs, or deployment infrastructure (unless orchestration is the primary purpose).

There are 101 llm orchestration platforms tools tracked. 2 score above 50 (established tier). The highest-rated is lfnovo/esperanto at 69/100 with 157 stars and 22,190 monthly downloads.

Get all 101 projects as JSON

curl "https://pt-edge.onrender.com/api/v1/datasets/quality?domain=embeddings&subcategory=llm-orchestration-platforms&limit=20"

Open to everyone — 100 requests/day, no key needed. Get a free key for 1,000/day.

#	Tool	Score	Tier	Stars	Language
1	lfnovo/esperanto A unified interface for various AI model providers	69	Established	157	Python
2	apocas/restai RESTai is an AIaaS (AI as a Service) open-source platform. Built on top of...	53	Established	480	Python
3	sbhjt-gr/InferrLM On-device AI for iOS & Android	48	Emerging	66	TypeScript
4	baidubce/bce-qianfan-sdk Provide best practices for LMOps, as well as elegant and convenient access...	45	Emerging	383	Jupyter Notebook
5	ProviderProtocol/ai 0-DEP AI DX SDK	43	Emerging	3	TypeScript
6	Muvon/octolib The lib to power AI tools.	43	Emerging	2	Rust
7	IBM/watsonx-ai-java-sdk The watsonx.ai Java SDK is an open-source library that simplifies the...	40	Emerging	5	Java
8	solygambas/python-openai-projects 13 projects using ChatGPT API, Whisper, Embeddings, and DALL-E with Python.	40	Emerging	51	Jupyter Notebook
9	ssoudan/gcp-vertex-ai-generative-ai An async client library for GCP Vertex AI Generative models	39	Emerging	5	Starlark
10	maragudk/gai Go Artificial Intelligence (GAI) helps you work with foundational models,...	39	Emerging	41	Go
11	ramanujammv1988/edge-veda On-device AI SDK for Flutter — LLM inference, vision, STT, TTS, image...	39	Emerging	74	Dart
12	spring-petclinic/spring-petclinic-langchain4j Spring Petclinic application with a chatbot powered by OpenAI's Generative...	38	Emerging	16	Java
13	lpalbou/AbstractCore A unified Python library for interaction with multiple Large Language Model...	38	Emerging	5	Python
14	IngressTechnology/jimbomesh-holler-server Open source AI inference server with Model Marketplace, Document RAG, and...	38	Emerging	4	JavaScript
15	itayzit/openai-async A light-weight, asynchronous client for OpenAI API - text completion, image...	37	Emerging	56	Python
16	jchristn/SharpAI SharpAI is an embeddable embeddings, completions, and model management...	36	Emerging	6	C#
17	netresearch/t3x-nr-llm The shared AI foundation for TYPO3 — one LLM setup for every extension on your site	35	Emerging	1	PHP
18	rrg92/powershai Powershell + AI	35	Emerging	61	PowerShell
19	SANSA-Stack/Archived-SANSA-ML SANSA Machine Learning Layer	34	Emerging	39	Scala
20	UBOS-tech/node-red-contrib-openai-ubos A Node-RED node that interacts with OpenAI machine learning models to...	34	Emerging	12	JavaScript
21	eliben/gemini-cli Access Gemini LLMs from the command-line	29	Experimental	148	Go
22	connerohnesorge/semanticrouter-go Fast & less costly AI decision making and intelligent processing of multi-modal data.	28	Experimental	11	Go
23	pinecone-io/gemini-cli-extension The official Pinecone Gemini CLI extension repo.	28	Experimental	18	Python
24	Prismadic/magnet the small distributed language model toolkit; fine-tune state-of-the-art...	27	Experimental	32	Python
25	cheahjs/gemini-to-openai-proxy Call Gemini (https://ai.google.dev) embedding models with OpenAI-compatible endpoints	27	Experimental	3	Go
26	skorotkiewicz/llmnet The Offline Internet.	27	Experimental	65	TypeScript
27	kingabzpro/easy-local-ai Run Ollama (Llama3.2), Langflow, Postgres, and qdrant server using Docker Compose.	26	Experimental	6	—
28	pavanad/beegen BeeGen is an intelligent command-line tool designed to assist developers...	25	Experimental	2	Python
29	proj-airi/webai-examples 🧠 Web AI / LLM in browser / Whisper in browser / WebGPU inference Examples	25	Experimental	28	Vue
30	priyanshua44/no-llm no-llm is a lightweight library designed to simplify the integration of...	24	Experimental	1	Python
31	unkn0wncode/openai A simple Go client for OpenAI’s API that supports most of the features you...	24	Experimental	2	Go
32	burgerkhan6227/tokenWise-Optimizer 🎯 Optimize LLM token usage by 70-90% with smart context ranking, reducing...	24	Experimental	2	Python
33	wizenheimer/tinylm Browser based ML Inference \| OpenAI compliant \| Run models like DeepSeek,...	24	Experimental	52	TypeScript
34	XNN-LLC/howtouseai Beginner-friendly guides for AI coding, self-hosting, RAG, embeddings,...	23	Experimental	1	—
35	H0NEYP0T-466/dataset-generator 🧠 DataForge — An intelligent LLM fine-tuning dataset generator that...	23	Experimental	1	Python
36	hekmon/go-honcho Go bindings for Honcho - Persistent memory layer for AI applications....	23	Experimental	1	Go
37	Netsleek/the-selection-layer Defines the Selection Layer — the decision system through which AI models...	23	Experimental	1	—
38	WizzyPluto2/ai-content-api Provide a unified REST API to generate AI content using multiple large...	23	Experimental	1	Python
39	XrecentX/vllm-skills 🚀 Deploy and manage vLLM with ready-made skills for modular automation,...	22	Experimental	—	Shell
40	Tberke0/go-llm 🤖 Simulate AI with go-llm, your unified Go SDK for various Large Language...	22	Experimental	—	Go
41	codingisloading/monkeys-with-typewriters Deliver AI capabilities with low latency and minimal power using a $3...	22	Experimental	—	C++
42	sahil-makandar/beyond-chatgpt-ai-systems Guest lecture presentation explaining modern AI architecture: LLMs, RAG...	22	Experimental	—	—
43	symfony/ai-open-ai-platform OpenAI platform bridge for Symfony AI	22	Experimental	3	PHP
44	nshkrdotcom/ollixir Ollixir provides a first-class Elixir client with feature parity to the...	22	Experimental	3	Elixir
45	austinmlang206/model-price 🛠️ Compare AI model pricing across major cloud providers effortlessly,...	22	Experimental	—	Python
46	bernard777/tool-selector-cascade Cascading tool selector for LLM agents — narrows 1000+ tools to the best...	22	Experimental	—	Python
47	m1thrandir225/galore-services The main services for the Galore mobile app.	22	Experimental	—	Go
48	GSejas/gacetachat Daily AI summaries of Costa Rica's official gazette. Simple. Fast. Open source.	22	Experimental	—	HTML
49	codenameakshay/ai_sdk_dart Dart/Flutter port of Vercel AI SDK v6 — provider-agnostic text generation,...	22	Experimental	—	Dart
50	fabriciosantos273738/llm-chat Manage multi-turn conversations efficiently with a lightweight C++ library...	22	Experimental	—	C++
51	darshjme/herald Semantic task router for multi-agent systems. Dispatch to specialists, not...	22	Experimental	—	Python
52	Aman00000007/lexilux 🚀 Simplify API calls with Lexilux, a unified LLM client that lets you access...	22	Experimental	—	Python
53	languageseed/valet-gateway AI Inference Gateway - orchestrates Ollama, vLLM, cloud providers, and...	21	Experimental	2	Python
54	actuallyrizzn/Venice-AI-SDK Comprehensive Python SDK for Venice AI - Advanced LLM, Image Generation,...	21	Experimental	2	Python
55	bgokden/llama-reranker High-performance, thread-safe reranking library using llama.cpp	19	Experimental	—	C++
56	leafspec/spec Framework-agnostic specification for AI-native applications. LEAF (Listen,...	19	Experimental	—	—
57	bgokden/llama-embedding High-performance, thread-safe embedding library using llama.cpp.	19	Experimental	—	C++
58	kamil5b/go-nl2query-lib A Go library for converting natural language queries into database queries...	19	Experimental	—	Go
59	AdarshTiwari3/ai-engineering-hub LLMs, LangChain, RAG, agents, embeddings, system-level AI experiments	19	Experimental	—	Python
60	bennyschmidt/next-token-prediction Next-token prediction in JavaScript — build fast language and diffusion models.	19	Experimental	143	JavaScript
61	douglasmitsue/ai-systems-engineering-portfolio Designing and building scalable AI systems - from deep neural networks to...	19	Experimental	—	—
62	Naseem77/tokenWise-Optimizer Smart Context Optimization for LLMs - Reduce tokens by 66%, save 40% on API...	18	Experimental	4	Python
63	liliang-cn/agent-go AI Agent SDK designed for Go developers	18	Experimental	6	Go
64	Cloudhabil/CLI Local-first AI orchestrator and CLI that spins up multi-agent LLMs, FastAPI...	17	Experimental	2	Python
65	Dragon-Born/go-llm Developer-friendly Go SDK for LLM apps: OpenAI, Anthropic, Gemini, Ollama,...	17	Experimental	2	Go
66	code-vygr/local-llm-ocr-ollama 🖼️ Extract text from images locally using Ollama's LLMs—100% free, offline,...	16	Experimental	2	Python
67	Ammar-Alnagar/Axion Axion is a high-performance LLM serving platform built with Rust that...	16	Experimental	1	Rust
68	inkybubble/agents-01-foundation An LLM enhanced with tools and retrieval (no memory for now)	15	Experimental	—	Python
69	Sabithulla-16/Prompt-Analyzer A local-first AI Prompt Analyzer that rewrites vague prompts, extracts...	15	Experimental	—	Python
70	superdupers1/ai-thought-visual 🎨 Transform language, voice, and images into structured AI concepts,...	15	Experimental	1	Lex
71	PabloSanchi/IBM-WatsonxAI-Spring-AI-Example Example of IBM watsonx.ai with Spring AI	15	Experimental	1	Java
72	Ethorbit/SoundPrompt Your sounds, triggered by AI prompts	15	Experimental	—	Python
73	linzeyang/minimax-python-client An (unofficial) python native client for easy interaction with MiniMax Open Platform	15	Experimental	—	Python
74	mshobari/local_ollama_project 🤖 Build a local AI assistant that categorizes user input with structured...	14	Experimental	—	—
75	dogunkim/llmnet 🔍 Transform your local LLMs into a private, high-speed search engine,...	14	Experimental	—	TypeScript
76	urmzd/agent-sdk Strongly-typed Go SDK for building streaming LLM agent loops — conversation...	14	Experimental	—	Go
77	Payton2cool4/LLMs-local 🚀 Explore tools and resources to run large language models locally,...	14	Experimental	—	—
78	orchestra-mcp/pack-ai AI and LLM integration skills with RAG, embeddings, and vector search	14	Experimental	—	—
79	jtgsystems/Ollama-Menu Interactive Windows batch menu for 160+ Ollama AI models. Browse, update,...	14	Experimental	—	Batchfile
80	sanketrana2598/ai-terminology 📘 Discover key AI terms and their alternatives in this concise guide to...	14	Experimental	—	Jupyter Notebook
81	urmzd/adk Strongly-typed Go SDK for building streaming LLM agent loops — conversation...	14	Experimental	—	Go
82	chukiagosoftware/alpaca Alpaca - Hotel Reviews Search Supercharged with AI	14	Experimental	—	Go
83	GhostGuyyt/llm-cpp Provide lightweight, zero-dependency C++17 single-header libraries to...	14	Experimental	—	JavaScript
84	polikyma/local-llm-project Local-first AI assistant. Open source models, RAG, zero cloud dependency.	14	Experimental	—	—
85	Al-Aswadd/Augment-BYOK-Proxy 🚀 Transform your Augment experience with the BYOK proxy for seamless LLM...	14	Experimental	—	Rust
86	onfiiva/ai-assistant-api LLM API with RAG, Agent, Embeddings and open source models	14	Experimental	—	Python
87	defrecord/llm-lab LLM Tools Lab: Testing environment for exploring LLM CLI tools (llm (0.21),...	12	Experimental	1	Shell
88	stiebo/spring-ai-samples Spring AI, chat client, vector store, RAG, multimodality samples	12	Experimental	1	Java
89	LLM-Gateway-ORG/llm-gateway-core A Python package to access different LLMs, embeddings, vector stores etc.	11	Experimental	2	Python
90	arunima10a/go-food-delivery A Full-Stack (Backend focused), Event-Driven Microservice System in Go....	11	Experimental	—	Go
91	redoh/openai-toolkit 🔧 OpenAI API toolkit — GPT-4, DALL-E, Whisper, embeddings with streaming and...	11	Experimental	—	—
92	redoh/ollama-recipes 🦙 Local LLM recipes with Ollama — fine-tuning, RAG, embeddings, multi-model pipelines	11	Experimental	—	—
93	xp-forge/openai OpenAI APIs for XP Framework	11	Experimental	—	PHP
94	vs4vijay/LLM-Ecosystem Code for Embeddings, VectorStore, SemanticSearch, and RAG using Azure OpenAI	11	Experimental	—	Python
95	dotcommander/syn Fast CLI for the Synthetic.new AI API — chat, search, vision, embeddings....	11	Experimental	2	Go
96	pipewrk/llm-core Lightweight, composable TypeScript library for semantic chunking, workflow...	11	Experimental	—	TypeScript
97	ArchitJ6/What-Beats-AI What Beats AI is an interactive word-challenge game that leverages...	11	Experimental	—	Python
98	zzarif/AI-Detector Detect AI generated coding answers	11	Experimental	—	JavaScript
99	johnamit/semantic-context-tokens A hybrid tokenization framework that combines coarse semantic context tokens...	11	Experimental	—	—
100	FANMixco/openai-outsystems-wrapper A simple wrapper for the OpenAI APIs for OutSystems	10	Experimental	1	—
101	iChetanRaval/Steganography-Project A professional-grade steganography platform that enables secure data hiding...	10	Experimental	3	Python