ltguo19/VSUA-Captioning

Constructs image representations as structured graphs with Visual Semantic Units (objects, attributes, relationships) extracted from scene graphs and bottom-up attention features, then aligns these units with caption words during generation. Implements dual training stages: cross-entropy pretraining followed by reinforcement learning optimization using CIDEr rewards. Built on PyTorch with integrated geometry and semantic relationship graphs, leveraging pre-extracted bottom-up features and scene graph annotations from COCO dataset.

258 stars. No commits in the last 6 months.