showlab/Image2Paragraph

A multi-stage vision-language pipeline that generates detailed paragraphs by composing outputs from BLIP2 (image captions), GRIT (dense object descriptions), and Semantic Segment Anything (region classification), then synthesizing results via ChatGPT/GPT4. The architecture optimizes for low GPU memory (runs on 8GB in ~20 seconds) through selective device offloading and includes a Gradio interface for interactive use. Generated paragraphs improve image-text retrieval performance over raw images, suggesting the compressed textual representation captures semantically relevant information more effectively.

824 stars. No commits in the last 6 months.