Kubernetes Llm Serving MLOps Tools

There are 9 kubernetes llm serving tools tracked. 4 score above 50 (established tier). The highest-rated is kubeflow/katib at 60/100 with 1,666 stars. 3 of the top 10 are actively maintained.

Get all 9 projects as JSON

curl "https://pt-edge.onrender.com/api/v1/datasets/quality?domain=mlops&subcategory=kubernetes-llm-serving&limit=20"

Open to everyone — 100 requests/day, no key needed. Get a free key for 1,000/day.

#	Tool	Score	Tier	Stars	Language
1	kubeflow/katib Automated Machine Learning on Kubernetes	60	Established	1,666	Python
2	beam-cloud/beta9 Ultrafast serverless GPU inference, sandboxes, and background jobs	58	Established	1,602	Go
3	sgl-project/rbg A workload for deploying LLM inference services on Kubernetes	54	Established	187	Go
4	kubeai-project/kubeai AI Inference Operator for Kubernetes. The easiest way to serve ML models in...	52	Established	1,161	Go
5	defilantech/LLMKube Kubernetes operator for GPU-accelerated LLM inference - air-gapped,...	41	Emerging	29	Go
6	scitix/arks Arks is a cloud-native inference framework running on Kubernetes	40	Emerging	46	Go
7	star-whale/starwhale an MLOps/LLMOps platform	38	Emerging	237	Java
8	tensorchord/openmodelz Autoscale LLM (vLLM, SGLang, LMDeploy) inferences on Kubernetes (and others)	33	Emerging	281	Go
9	Climatik-Project/Climatik-Project Carbon Limiting Auto Tuning for Kubernetes	33	Emerging	37	Go

Comparisons in this category

kubeai and LLMKube (52 vs 41)