SunzeY/AlphaCLIP

Incorporates alpha-channel (transparency/mask) conditioning into CLIP's vision encoder, enabling region-focused feature extraction by accepting binary foreground masks alongside images. Built on LoRA-based fine-tuning of standard CLIP backbones (ViT-B/16, ViT-L/14) trained on the MaskImageNet dataset. Integrates seamlessly with downstream applications like Stable Diffusion, LLaVA, and BLIP for improved performance in masked image understanding, zero-shot classification, and vision-language tasks.

869 stars. No commits in the last 6 months.