OpenNMT/OpenNMT-py

Supports quantization (4/8-bit) and LoRA fine-tuning for efficient adaptation of 7B-13B models on consumer GPUs, with optional tensor parallelism for larger deployments. Features multi-query attention, rotary embeddings, and Flash Attention 2 integration for optimized inference. Includes converters for popular model families (Llama, Mistral, Falcon, MPT) and interoperability with CTranslate2 for accelerated production serving.

7,000 stars and 13,033 monthly downloads. Used by 1 other package. Available on PyPI.