madaan/minimal-text-diffusion

Implements discrete diffusion for text using a BERT encoder with tied embedding/output weights, where the model learns to predict clean embeddings from noisy versions via progressive denoising steps. Supports classifier-guided generation for conditional sampling and includes word-level tokenization with options for BPE. Built on PyTorch with configurable sequence lengths and diffusion schedules, designed for training on arbitrary text corpora with minimal dependencies.

410 stars. No commits in the last 6 months.