AnswerDotAI/ModernBERT
Bringing BERT into modernity via both architecture changes and scaling
Introduces FlexBERT, a modular encoder architecture configured via YAML, leveraging Flash Attention 2/3 for efficient long-context processing. Built on the Composer training framework with support for both streaming and decompressed MDS datasets, enabling reproducible pre-training at scale. Includes evaluation pipelines for GLUE tasks and retrieval applications via Sentence Transformers and PyLate integration.
1,642 stars.
Stars
1,642
Forks
144
Language
Python
License
Apache-2.0
Category
Last pushed
Mar 01, 2026
Commits (30d)
0
Get this data via API
curl "https://pt-edge.onrender.com/api/v1/quality/embeddings/AnswerDotAI/ModernBERT"
Open to everyone — 100 requests/day, no key needed. Get a free key for 1,000/day.
Higher-rated alternatives
MinishLab/model2vec
Fast State-of-the-Art Static Embeddings
Embedding/Chinese-Word-Vectors
100+ Chinese Word Vectors 上百种预训练中文词向量
tensorflow/hub
A library for transfer learning by reusing parts of TensorFlow models.
Santosh-Gupta/SpeedTorch
Library for faster pinned CPU <-> GPU transfer in Pytorch
twang2218/vocab-coverage
语言模型中文认知能力分析