michaelfeil/infinity
Infinity is a high-throughput, low-latency serving engine for text-embeddings, reranking models, clip, clap and colpali
57
/ 100
Established
Leverages pluggable inference backends (PyTorch, ONNX/TensorRT, CTranslate2) with dynamic batching and worker-threaded tokenization across NVIDIA CUDA, AMD ROCM, CPU, AWS INF2, and Apple MPS. Exposes an OpenAI-compatible REST API built on FastAPI, enabling drop-in replacement for embedding workflows while supporting multi-model orchestration and quantization (int8/fp8).
2,717 stars. Actively maintained with 1 commit in the last 30 days.
No Package
No Dependents
Maintenance
13 / 25
Adoption
10 / 25
Maturity
16 / 25
Community
18 / 25
Stars
2,717
Forks
181
Language
Python
License
MIT
Category
Last pushed
Feb 05, 2026
Commits (30d)
1
Get this data via API
curl "https://pt-edge.onrender.com/api/v1/quality/embeddings/michaelfeil/infinity"
Open to everyone — 100 requests/day, no key needed. Get a free key for 1,000/day.