From Detection to Reflection: Using Explainable AI to Promote Critical Engagement with Multimodal Disinformation

  • Motivation

    The spread of disinformation is one of the most pressing societal challenges of our time, threatening informed public discourse and trust in institutions. Generative AI has dramatically lowered the barrier to creating and spreading misleading content at scale. The same technology, however, also holds the potential to help detect it. Since social media content is inherently multimodal, and manipulation increasingly happens across modalities, for example through decontextualized or AI-generated visuals, detection systems must reach beyond purely text-based analysis.

    At the same time, research frequently treats detection as a purely algorithmic classification problem, leaving the human reader as a passive recipient of a verdict ("fake" / "real"). In practice, such opaque verdicts are either distrusted or blindly followed, and neither outcome strengthens users' ability to navigate misleading content on their own. Effective solutions therefore require explanations (XAI) designed to promote critical engagement, meaning the reader's active reasoning about why content may be misleading, so that AI empowers rather than replaces human judgement.

    Research Objectives

    This master's thesis investigates explainable multimodal disinformation detection, addressing a key open question in the field: how explanations of AI verdicts can be designed to foster critical thinking about potentially misleading content. This thesis aims to:

    • Review relevant work at the intersection of multimodal disinformation detection, XAI, and human–AI interaction
    • Explore how explanations for multimodal content can be generated and presented to readers
    • Develop and evaluate a prototype, either technically, empirically with users, or both

    Methodology

    The concrete emphasis will be shaped together with the student, depending on their skill profile and interests. Building on a shared literature foundation, the thesis can take one of the following directions (or a combination):

    • Technical focus: Development of a novel approach for explainable multimodal disinformation detection, such as a text–image detection pipeline with integrated explanation generation, evaluated on suitable benchmarks regarding detection performance and explanation quality.
    • Human–AI collaboration focus: Design and prototyping of explanation interfaces on top of an existing detection approach, evaluated in a user study examining effects on users' reasoning, reliance, and trust.

    Requirements

    • Interest in interdisciplinary research on human interaction with AI systems
    • Understanding of ML/AI and programming skills (extent depending on the chosen focus)
    • Self-driven, curious, and open learning attitude
    • Good English skills

    Literature

    • Alam, F., Cresci, S., Chakraborty, T., et al. (2022). A Survey on Multimodal Disinformation Detection. Proceedings of COLING 2022.
    • Peng, L., et al. (2025). Multi-modal Fake News Detection: A Comprehensive Survey on Deep Learning Technology, Advances, and Challenges. Journal of King Saud University – Computer and Information Sciences.
    • Athira, A. B., et al. (2023). A Systematic Survey on Explainable AI Applied to Fake News Detection. Engineering Applications of Artificial Intelligence.
    • Saeidnia, H. R., Hosseini, E., Lund, B., Tehrani, M. A., Zaker, S., & Molaei, S. (2025). Artificial intelligence in the battle against disinformation and misinformation: a systematic review of challenges and approaches. Knowledge and Information Systems, 67(4), 3139-3158.
    • Wirtz, B. W., Weyerer, J. C., & Müller, T. F. (2026). AI-based digital disinformation: A theory-informed and integrated trilateral framework of digital disinformation for information systems research. International Journal of Information Management, 89, 103066.
    • Hetzner, T., Solopova, V., Schmitt, V., & Kolossa, D. (2025, June). Integrating video, text, and images for multimodal disinformation detection. In Proceedings of the 4th ACM International Workshop on Multimedia AI against Disinformation (pp. 1-16).
    • Guerrero-Sosa, J. D., Montoro-Montarroso, A., Romero, F. P., Serrano-Guerrero, J., & Olivas, J. A. (2026). A new hybrid intelligent approach for multimodal detection of suspected disinformation on TikTok. Multimedia Tools and Applications, 85(1), 37.
    • Kheddache, Y., & Lalonde, M. (2025). Effectiveness of Large Multimodal Models in Detecting Disinformation: Experimental Results. arXiv preprint arXiv:2509.22377.