llmware-ai/llmware

Brings together prepackaged quantized models (50+ specialized for RAG tasks like extraction, classification, and summarization) and a modular RAG pipeline with multi-format document parsing, vector embedding with multiple backends (Chromadb, Milvus), and hybrid query capabilities (text, semantic, metadata filters). The unified ModelCatalog interface abstracts over diverse inference engines—GGUF, OpenVINO, ONNX-Runtime, HuggingFace—enabling the same code to run on-device across CPUs, GPUs, and NPUs on Windows, Mac, and Linux. Prompt objects orchestrate end-to-end knowledge retrieval and generation, automatically batching sources to fit model context windows while tracking provenance for fact-checking against source materials.

14,864 stars and 1,177 monthly downloads. Actively maintained with 12 commits in the last 30 days. Available on PyPI.