GQA: A New Dataset for Real-World Visual Reasoning and Compositional Question Answering
GQA: новый набор данных для визуального рассуждения в реальных условиях и композиционного ответа на вопросы
2019-06-01
SCID: 54.1/hfssdu49
Discuss with AI
GQA datasetVisual Genome scene graphscompositional question answeringfunctional programsvisual reasoning
Figures from the paper
Abstract (AI)
We introduce GQA, a new dataset for real-world visual reasoning and compositional question answering, seeking to address key shortcomings of previous VQA datasets. We have developed a strong and robust question engine that leverages Visual Genome scene graph structures to create 22M diverse reasoning questions, which all come with functional programs that represent their semantics. We use the programs to gain tight control over the answer distribution and present a new tunable smoothing technique to mitigate question biases. Accompanying the dataset is a suite of new metrics that evaluate essential qualities such as consistency, grounding and plausibility. A careful analysis is performed for baselines as well as state-of-the-art models, providing fine-grained results for different question types and topologies. Whereas a blind LSTM obtains a mere 42.1%, and strong VQA models achieve 54.1%, human performance tops at 89.3%, offering ample opportunity for new research to explore. We hope GQA will provide an enabling resource for the next generation of models with enhanced robustness, improved consistency, and deeper semantic understanding of vision and language.
Key Findings
1
A tunable smoothing technique is introduced to reduce question-answer biases through tighter control of the dataset’s answer distribution.
2
Each GQA question includes a functional program representing its semantics, enabling controlled analysis of reasoning processes and answer distributions.
3
GQA introduces a real-world visual reasoning dataset containing 22 million diverse compositional questions generated from Visual Genome scene graphs.
4
Performance remains substantially below humans: blind LSTMs achieve 42.1%, strong VQA models 54.1%, and humans 89.3%.
5
The accompanying evaluation suite measures consistency, grounding, and plausibility, enabling more fine-grained assessment of visual question-answering systems.
Research Object
GQA, a dataset of real-world images paired with compositional visual question-answering questions and functional programs
Research Subject
The dataset’s support for robust, consistent, grounded, plausible, and semantically deep visual reasoning and compositional question answering, including answer-distribution control and evaluation across question types
Publication Details
Publication Date
2019-06-01
Journal
Publisher
ISSN
Cited by
1796
Open access PDF
Access Type
Author Information
Download PDF
Subscribe to digest
References available in scid.ai6
Adam: A Method for Stochastic Optimization2014
ImageNet: A large-scale hierarchical image database2009
WordNet: a lexical database for English1995
Multimodal Compact Bilinear Pooling for Visual Question Answering and Visual Grounding2016
YFCC100M2016
Don't Just Assume; Look and Answer: Overcoming Priors for Visual Question Answering2018