alpa-projects/alpa
Training and serving large-scale neural networks with auto parallelization.
ArchivedCombines JAX, XLA, and Ray to automatically apply data, operator, and pipeline parallelism across distributed clusters without manual sharding logic. Compiles single-device training code via a `@parallelize` decorator that generates optimal parallelization strategies, achieving linear scaling on billion-parameter models. Note: Core algorithms have been integrated into XLA's auto-sharding; this project is maintained as a research artifact.
3,188 stars. No commits in the last 6 months.
Stars
3,188
Forks
361
Language
Python
License
Apache-2.0
Category
Last pushed
Dec 09, 2023
Commits (30d)
0
Get this data via API
curl "https://pt-edge.onrender.com/api/v1/quality/transformers/alpa-projects/alpa"
Open to everyone — 100 requests/day, no key needed. Get a free key for 1,000/day.
Higher-rated alternatives
quic/efficient-transformers
This library empowers users to seamlessly port pretrained models and checkpoints on the...
ManuelSLemos/RabbitLLM
Run 70B+ LLMs on a single 4GB GPU — no quantization required.
deepreinforce-ai/CUDA-L2
CUDA-L2: Surpassing cuBLAS Performance for Matrix Multiplication through Reinforcement Learning
arm-education/Advanced-AI-Hardware-Software-Co-Design
Hands-on course materials for ML engineers to master extreme model quantization and on-device...
IST-DASLab/marlin
FP16xINT4 LLM inference kernel that can achieve near-ideal ~4x speedups up to medium batchsizes...