Attention Mechanism Implementations Transformer Models

There are 13 attention mechanism implementations models tracked. 2 score above 50 (established tier). The highest-rated is microsoft/LoRA at 67/100 with 13,320 stars and 207,985 monthly downloads.

Get all 13 projects as JSON

curl "https://pt-edge.onrender.com/api/v1/datasets/quality?domain=transformers&subcategory=attention-mechanism-implementations&limit=20"

Open to everyone — 100 requests/day, no key needed. Get a free key for 1,000/day.

#	Model	Score	Tier	Stars	Language
1	microsoft/LoRA Code for loralib, an implementation of "LoRA: Low-Rank Adaptation of Large...	67	Established	13,320	Python
2	jadore801120/attention-is-all-you-need-pytorch A PyTorch implementation of the Transformer model in "Attention is All You Need".	51	Established	9,651	Python
3	kyegomez/SparseAttention Pytorch Implementation of the sparse attention from the paper: "Generating...	43	Emerging	94	Python
4	bhavnicksm/vanilla-transformer-jax JAX/Flax implimentation of 'Attention Is All You Need' by Vaswani et al....	42	Emerging	15	Python
5	AbdelStark/attnres Rust implementation of Attention Residuals from MoonshotAI/Kimi	42	Emerging	47	Rust
6	takara-ai/go-attention A full attention mechanism and transformer in pure go.	38	Emerging	451	Go
7	tranquoctrinh/transformer This is a PyTorch implementation of the Transformer model in the paper...	37	Emerging	37	Python
8	SingleZombie/LLSA Official implementation of Log-linear Sparse Attention (LLSA).	36	Emerging	62	Python
9	sunnynguyen-ai/llm-attention-visualizer Interactive tool for analyzing attention patterns in transformer models with...	34	Emerging	14	Python
10	takara-ai/SwarmFormer A pytorch implementation of SwarmFormer for text classification.	32	Emerging	16	Python
11	HKUNLP/efficient-attention [EVA ICLR'23; LARA ICML'22] Efficient attention mechanisms via control...	32	Emerging	87	Python
12	mrcabbage972/simple-toolformer A Python implementation of Toolformer using Huggingface Transformers	24	Experimental	14	Python
13	tobifinn/ensemble_transformer Official PyTorch implementation of "Self-Attentive Ensemble Transformer:...	20	Experimental	14	Jupyter Notebook

Comparisons in this category

attention-is-all-you-need-pytorch and transformer (51 vs 37)