The-AI-Summer/self-attention-cv

Provides modular PyTorch implementations of multi-head attention, axial attention, Vision Transformers, and positional embeddings using einsum/einops for efficient tensor operations. Includes pre-built architectures like ViT, ResNet50ViT, TransUnet, and BottleneckTransformer blocks that can be directly instantiated for image classification and segmentation tasks. Integrates with standard PyTorch workflows and includes relative/absolute positional embedding variants for 1D and 2D spatial attention.

1,215 stars. No commits in the last 6 months.