A modular framework for vision & language multimodal research from Facebook AI Research (FAIR)

InternGPT (iGPT) is an open source demo platform where you can easily showcase your AI models. Now it supports DragGAN, ChatGPT, ImageBind, multimodal chat like GPT-4, SAM, interactive image editing, etc. Try it at igpt.opengvlab.com (支持しじDragGAN、ChatGPT、ImageBind、SAM的てき在ざい线Demo系けい统)

北京ぺきん航空こうくう航こう天てん大学だいがく大数たいすう据すえ高だか精しらげ尖とんが中心ちゅうしん自然しぜん语言处理研究けんきゅう团队开展了りょう智能ちのう问答的てき研究けんきゅう与あずか应用总结。包括ほうかつ基もと于知识图谱的问答（KBQA），基き于文本ほん的てき问答系けい统（TextQA），基き于表格かく的てき问答系けい统（TableQA）、基もと于视觉的问答系けい统（VisualQA）和かず机つくえ器き阅读理解りかい（MRC）等とう，每まい类任务分别对学がく术界和わ工こう业界进行了りょう相しょう关总结。

Bottom-up attention model for image captioning and VQA, based on Faster R-CNN and Visual Genome

The implementation of "Prismer: A Vision-Language Model with Multi-Task Experts".

Oscar and VinVL

Open-source evaluation toolkit of large vision-language models (LVLMs), support ~100 VLMs, 30+ benchmarks

[ICCV 2021- Oral] Official PyTorch implementation for Generic Attention-model Explainability for Interpreting Bi-Modal and Encoder-Decoder Transformers, a novel method to visualize any Transformer-based network. Including examples for DETR, VQA.

An efficient PyTorch implementation of the winning entry of the 2017 VQA Challenge.

Visual Question Answering in Pytorch

[CVPR 2021 Best Student Paper Honorable Mention, Oral] Official PyTorch code for ClipBERT, an efficient framework for end-to-end learning on image-text and video-text tasks.

A curated list of Visual Question Answering(VQA)(Image/Video Question Answering),Visual Question Generation ,Visual Dialog ,Visual Commonsense Reasoning and related area.

Implementation for the paper "Compositional Attention Networks for Machine Reasoning" (Hudson and Manning, ICLR 2018)

Visual Q&A reading list

Chatbot Arena meets multi-modality! Multi-Modality Arena allows you to benchmark vision-language models side-by-side while providing images as inputs. Supports MiniGPT-4, LLaMA-Adapter V2, LLaVA, BLIP-2, and many more!

PyTorch implementation for the Neuro-Symbolic Concept Learner (NS-CL).

PyTorch implementation of "Transparency by Design: Closing the Gap Between Performance and Interpretability in Visual Reasoning"

A lightweight, scalable, and general framework for visual question answering research

Strong baseline for visual question answering

[ICLR'24] Mitigating Hallucination in Large Multi-Modal Models via Robust Instruction Tuning