tensorchord/openmodelz

Autoscale LLM (vLLM, SGLang, LMDeploy) inferences on Kubernetes (and others)

/ 100

Emerging

281 stars. No commits in the last 6 months.

Stale 6m No Package No Dependents

Maintenance 0 / 25

Adoption 10 / 25

Maturity 9 / 25

Community 14 / 25

Stars

281

Forks

Language

License

Apache-2.0

Category

Last pushed

Nov 03, 2023

Commits (30d)

Get this data via API

curl "https://pt-edge.onrender.com/api/v1/quality/mlops/tensorchord/openmodelz"

Open to everyone — 100 requests/day, no key needed. Get a free key for 1,000/day.

Higher-rated alternatives

kubeflow/katib

Automated Machine Learning on Kubernetes

beam-cloud/beta9

Ultrafast serverless GPU inference, sandboxes, and background jobs

sgl-project/rbg

A workload for deploying LLM inference services on Kubernetes

kubeai-project/kubeai

AI Inference Operator for Kubernetes. The easiest way to serve ML models in production. Supports...

defilantech/LLMKube

Kubernetes operator for GPU-accelerated LLM inference - air-gapped, edge-native, production-ready