bytedance/lightseq

LightSeq: A High Performance Library for Sequence Processing and Generation

Archived
46
/ 100
Emerging

Built on CUDA with custom fused kernels optimized for Transformer architectures, it supports fp16 and int8 mixed-precision training and inference across BERT, GPT, ViT, and other sequence models. Integrates seamlessly with Fairseq, Hugging Face, and DeepSpeed, with TensorRT Inference Server backend for production deployment and includes decoding algorithms like beam search and CRF.

3,304 stars. No commits in the last 6 months.

Archived Stale 6m No Package No Dependents
Maintenance 0 / 25
Adoption 10 / 25
Maturity 16 / 25
Community 20 / 25

How are scores calculated?

Stars

3,304

Forks

335

Language

C++

License

—

Category

llm-cuda-optimization

Last pushed

May 16, 2023

Commits (30d)

0

GitHub
Llm Cuda Optimization · 25 models

Get this data via API

curl "https://pt-edge.onrender.com/api/v1/quality/transformers/bytedance/lightseq"

Open to everyone — 100 requests/day, no key needed. Get a free key for 1,000/day.

Higher-rated alternatives

ggml-org/ggml

Tensor library for machine learning

68

quic/efficient-transformers

This library empowers users to seamlessly port pretrained models and checkpoints on the...

58

ManuelSLemos/RabbitLLM

Run 70B+ LLMs on a single 4GB GPU — no quantization required.

52

SandAI-org/MagiCompiler

A plug-and-play compiler that delivers free-lunch optimizations for both inference and training.

49

alpa-projects/alpa

Training and serving large-scale neural networks with auto parallelization.

47

Explore Transformer Models

All categories Trending Transformer directory Insights