ildoonet/pytorch-gradual-warmup-lr

Implements a composable learning rate scheduler that linearly increases the learning rate over a warmup phase before delegating to a secondary scheduler (e.g., StepLR, ExponentialLR, CosineAnnealingLR) for the main training phase. Chains with PyTorch's native `torch.optim.lr_scheduler` interface through an `after_scheduler` parameter, enabling flexible scheduler composition without modifying optimizer code. Based on the large-batch SGD warmup strategy from Goyal et al., it prevents training instability during initial iterations when using aggressive learning rates or large batch sizes.

991 stars. No commits in the last 6 months.