tokenizer and go-tokenizer
These are competitors offering overlapping functionality—both provide standalone tokenization libraries for Go without clear differentiation in scope or features, so a user would select one based on maturity (sugarme/tokenizer's higher star count suggests broader adoption) rather than using them together.
Maintenance
6/25
Adoption
10/25
Maturity
16/25
Community
22/25
Maintenance
6/25
Adoption
5/25
Maturity
9/25
Community
12/25
Stars: 316
Forks: 61
Downloads: —
Commits (30d): 0
Language: Go
License: Apache-2.0
Stars: 11
Forks: 2
Downloads: —
Commits (30d): 0
Language: Go
License: MIT
No Package
No Dependents
No Package
No Dependents
About tokenizer
sugarme/tokenizer
NLP tokenizers written in Go language
About go-tokenizer
euskadi31/go-tokenizer
A Text Tokenizer library for Golang
Scores updated daily from GitHub, PyPI, and npm data. How scores work