Multimodal Misinformation Detection using Large Vision-Language Models

This repository is the official implementation of Multimodal Misinformation Detection using Large Vision-Language Models published in CIKM 2024.

Please use the following citation:

@InProceedings{cikm_Tahmasebi24,
  author    = {Sahar Tahmasebi, Eric Müller-Budack and Ralph Ewerth},
  booktitle = {ACM International Conference on Information and Knowledge Management, CIKM 2024, Boise, Idaho, United States of America, October 21-25, 2024},
  title     = {Multimodal Misinformation Detection using Large Vision-Language Models},
  year      = {2024},
  doi       = {10.1145/3627673.3679826},
}

Requirements

To install requirements:

conda create -n LLM4FV python=3.8.10
conda install pytorch==1.11.0 torchvision==0.12.0 torchaudio==0.11.0 cudatoolkit=11.3.1 -c pytorch
pip install -r requirements.txt

Dataset

• MOCHEG: You can download dataset here.

  For more information on the dataset format and structure, please refer to their Please check their official repository.

• Factify: The dataset can be downloaded here.

  Additional details can be found on their official website.

Evaluation

To run the pipeline and evaluate on datasets, run:

bash eval.sh

Credit

This repository is built by Sahar Tahmasebi.

Contributing

Our work is licenced under the CC BY 4.0. This project includes code that is licensed under the Apache License 2.0, from the MOCHEG repository by Barry Menglong Yao. The original code has been modified to suit the needs of this project.