kyegomez/ScreenAI

Combines Vision Transformer patch encoding with multi-modal and LLM decoder layers to process paired image and text inputs through cross-attention and self-attention mechanisms. Configurable architecture supports variable patch sizes, embedding dimensions, and depth across distinct ViT, multi-modal encoder, and LLM decoder components. Installable via pip with straightforward PyTorch tensor-based inference.

380 stars and 45 monthly downloads. Available on PyPI.