MILVLG/prophet
Implementation of CVPR 2023 paper "Prompting Large Language Models with Answer Heuristics for Knowledge-based Visual Question Answering".
Employs a two-stage framework that first trains a vanilla VQA model (MCAN) on knowledge-based datasets to extract answer candidates and answer-aware examples as heuristics, then uses these heuristics to prompt GPT-3 for improved reasoning. Supports multi-GPU training with PyTorch on OK-VQA and A-OKVQA datasets, leveraging pre-computed image features and extensive pre-training on VQA v2 before task-specific fine-tuning. Achieves SOTA performance by combining supervised VQA models with large language models through strategic prompt engineering.
279 stars. No commits in the last 6 months.
Stars
279
Forks
28
Language
Python
License
Apache-2.0
Category
Last pushed
Jun 14, 2025
Commits (30d)
0
Get this data via API
curl "https://pt-edge.onrender.com/api/v1/quality/prompt-engineering/MILVLG/prophet"
Open to everyone — 100 requests/day, no key needed. Get a free key for 1,000/day.
Higher-rated alternatives
OpenDriveLab/DriveLM
[ECCV 2024 Oral] DriveLM: Driving with Graph Visual Question Answering
ShiZhengyan/PowerfulPromptFT
[NeurIPS 2023 Main Track] This is the repository for the paper titled "Don’t Stop Pretraining?...
Lamorati92/LLMs-from-scratch
📚 Build and train your own GPT-like Large Language Model from scratch with clear guidance and...
mala-lab/NegPrompt
The official implementation of CVPR 24' Paper "Learning Transferable Negative Prompts for...
JindongGu/Awesome-Prompting-on-Vision-Language-Model
This repo lists relevant papers summarized in our survey paper: A Systematic Survey of Prompt...