joshualoehr/ngram-language-model
Python implementation of an N-gram language model with Laplace smoothing and sentence generation.
Builds N-gram models with configurable smoothing (Laplace with adjustable lambda) and computes perplexity scores on test sets. The implementation prioritizes transparency by hand-coding core algorithms beyond NLTK's `ngrams` and `FreqDist`, while generating sentences with cumulative probability scores. Expects pre-tokenized, sentence-segmented input and provides a CLI for specifying model order, smoothing parameters, and generation count.
No commits in the last 6 months.
Stars
88
Forks
27
Language
Python
License
—
Category
Last pushed
Feb 09, 2018
Commits (30d)
0
Get this data via API
curl "https://pt-edge.onrender.com/api/v1/quality/nlp/joshualoehr/ngram-language-model"
Open to everyone — 100 requests/day, no key needed. Get a free key for 1,000/day.
Related tools
nlx-group/overlapy
Python package developed to evaluate textual overlap (N-Grams) between two volumes of text.
MannarAmuthan/kural-gen
KuralGen generates Thirukkural for a given English sentence
phughesmcr/SimpleNGrams
The easiest way to get n-grams from strings!
SpydazWebAI-NLP/BasicLanguageModelling2023
Basic Language Models , Bag of Words, Ngram Models Etc NLP modelling and associated tasks
simrann20/Hangman_Game_Project
Hangman Game implementation using n-gram language model in NLP, achieved an accuracy of more than 50%