NVlabs/GroupViT

Employs hierarchical bottom-up spatial grouping to learn semantic segmentation purely from image-text pairs without mask labels, leveraging vision transformers with a contrastive learning framework. Integrates with MMSegmentation for evaluation on Pascal VOC, Context, and COCO datasets, and supports webdataset for scalable training on large caption-image corpora like GCC and YFCC. Available as a Hugging Face Space demo and compatible with timm backbone models and NVIDIA Apex for mixed-precision training.

783 stars. No commits in the last 6 months.