Text Tokenization Libraries Embedding Tools
There are 2 text tokenization libraries tools tracked. The highest-rated is weaviate/contextionary at 24/100 with 17 stars.
Get all 2 projects as JSON
curl "https://pt-edge.onrender.com/api/v1/datasets/quality?domain=embeddings&subcategory=text-tokenization-libraries&limit=20"
Open to everyone — 100 requests/day, no key needed. Get a free key for 1,000/day.
| # | Tool | Score | Tier |
|---|---|---|---|
| 1 |
weaviate/contextionary
Weaviate's own language vectorizer, which allows for semantic context-based... |
|
Experimental |
| 2 |
Natooz/BPE-Symbolic-Music
Code of the paper "Byte Pair Encoding for Symbolic Music" (EMNLP 2023).... |
|
Experimental |