Collaborate large and small language models for multi-modal emergency rumor detection

Area of Science:

• Artificial Intelligence
• Natural Language Processing
• Computer Vision

Background:

• Multi-modal emergency rumors proliferate online, causing societal disruption.
• Existing rumor detection models using individual SLMs or LLMs face limitations in knowledge capacity and analytical integration.
• Current multi-modal fusion techniques are superficial, hindering comprehensive rumor understanding.

Purpose of the Study:

• To propose an effective framework for multi-modal emergency rumor detection.
• To leverage the complementary strengths of LLMs and SLMs for improved rumor analysis.
• To enhance multi-modal feature fusion for deeper comprehension of emergency rumors.

Main Methods:

• Developed Collaborate Large and Small Language Models for Multi-Modal Emergency Rumor Detection (M2ERD).
• LLMs generate multi-dimensional rationales from multi-perspective prompts.
• SLMs selectively derive insights from LLM rationales.
• Implemented a multi-source cross-modal penetration fusion network for text and image data integration.

Main Results:

• M2ERD achieved a 2.6% improvement in accuracy and a 1.9% improvement in F1-score on benchmark datasets (Weibo, RumorEval, Pheme).
• Demonstrated superior performance compared to existing rumor detection baselines.
• Validated the effectiveness of collaborative LLM-SLM approach and advanced fusion network.

Conclusions:

• The proposed M2ERD framework significantly enhances multi-modal emergency rumor detection.
• Collaborative LLM-SLM synergy and advanced fusion techniques are crucial for tackling complex online rumors.
• The study provides a robust solution for combating the spread of misinformation in digital environments.