Published Paper Details:

The system is a research project on developing a Visual Inquiry (VI) system utilizing deep learning for visual comprehension and Natural Language Processing (NLP) for making context-based replies. VI systems are intended to understand and respond to visual content questions to deliver natural-style cognition and interaction. The system leverages Convolutional Neural Networks (CNNs) to obtain the visual features of the images to obtain salient facts like objects, scenes, and spatial relationships. They are then merged with NLP models that detect the input question in an attempt to display a compound presentation of text and image knowledge.Worthy of mention here are the use of state-of-the-art models such as attention mechanisms and Transformer models to project the image features onto the semantic features of the question. Attention layers enable the model to attend to the correct location in the image and enhance the accuracy of response generation. VI system is trained on vast amounts of data, for example, the VI v2 or Visual Genome dataset, with labeled images, questions, and answers.With the addition of vision and language processing, this VI system is able to answer appropriately to various types of questions, ranging from object recognition to more abstract reasoning questions.