laelhalawani/gguf_modeldb
A quick and optimized solution to manage llama based gguf quantized models, download gguf files, retreive messege formatting, add more models from hf repos and more. It's super easy to use and comes prepacked with best preconfigured open source models: dolphin phi-2 2.7b, mistral 7b v0.2, mixtral 8x7b v0.1, solar 10.7b and zephyr 3b
Used by 1 other package. No commits in the last 6 months. Available on PyPI.
Stars
12
Forks
3
Language
Python
License
—
Category
Last pushed
Jan 13, 2024
Monthly downloads
142
Commits (30d)
0
Dependencies
3
Reverse dependents
1
Get this data via API
curl "https://pt-edge.onrender.com/api/v1/quality/transformers/laelhalawani/gguf_modeldb"
Open to everyone — 100 requests/day, no key needed. Get a free key for 1,000/day.
Higher-rated alternatives
intel/auto-round
🎯An accuracy-first, highly efficient quantization toolkit for LLMs, designed to minimize quality...
ModelCloud/GPTQModel
LLM model quantization (compression) toolkit with hw acceleration support for Nvidia CUDA, AMD...
pytorch/ao
PyTorch native quantization and sparsity for training and inference
Picovoice/picollm
On-device LLM Inference Powered by X-Bit Quantization
NVIDIA/kvpress
LLM KV cache compression made easy