Multimodal-Disaster-Event-Identification-from-Social-Media-Posts
🥈SilverWe have proposed a multimodal approach. Where we first took the best unimodal for textual and visual data classification by testing and automation process. Then we fusion of the two models which can successfully classify the materials that have been damaged using the image and text data. EfficientNetB3+BERT multimodal better accuracy with 94.18%
610Updated 2w ago
Intermediate30min to implementautomation
Saves ~120 min per use
Quick InstallView Source
git clone https://github.com/SalehAhmedShafin/Multimodal-Disaster-Event-Identification-from-Social-Media-Posts.gitWorks with:
Claude
Overview
About This Skill
This skill uses multimodal machine learning to classify disaster events from paired social media images and tweets. It processes visual and textual data simultaneously through separate feature extraction paths—EfficientNetB3 for image analysis and BERT for tweet processing—then fuses the results for unified classification. The model achieves 94.18% accuracy across six disaster-related categories including building damage, infrastructure damage, human casualties, and non-damage cases. It handles imbalanced datasets through preprocessing techniques including image scaling, normalization, and augmentation. Organizations responding to disasters can leverage this skill to automatically filter and categorize social media reports for rapid situational awareness.
How to Use
["Set up real-time monitoring for [PLATFORM] using tools like Hootsuite, Brandwatch, or custom API integrations. Configure filters to capture posts with [TEXT_ANALYSIS_CRITERIA] and [IMAGE_ANALYSIS_CRITERIA].","Use a multimodal AI model (e.g., EfficientNetB3+BERT) to analyze text and images simultaneously. Set confidence thresholds (e.g., >0.9 for text, >0.85 for images) to filter high-probability events.","Integrate the output with emergency response systems like Sortd for Gmail to create kanban-style boards for task management. Assign tasks to teams (e.g., fire department, hazmat, road crews) based on urgency.","Validate flagged posts with human review for false positives. Use the confidence scores to prioritize which events require immediate attention.","Export the summary report in [OUTPUT_FORMAT] (e.g., JSON, CSV) and share with stakeholders. Include post IDs, confidence scores, and suggested actions for transparency and accountability."]
Use Cases
Automated classification of disaster damage reports from social media during emergency response
Real-time identification of casualty-related posts during natural disasters
Infrastructure damage assessment from crowdsourced image-text social media data
Training disaster management systems on historical disaster event datasets
Tags
Setup & Installation
Quick Install
No install command available. Check the GitHub repository for manual installation instructions.
Alternative Install (Git Clone)
git clone https://github.com/SalehAhmedShafin/Multimodal-Disaster-Event-Identification-from-Social-Media-PostsRequirements
- Claude Code or compatible AI agent
- Works with: Claude
Quick Start Guide
1
Install the Skill
Copy the install command above and run it in your terminal.
2
Open Your AI Agent
Launch Claude Code, Cursor, or your preferred AI coding agent.
3
Try It Out
Use the prompt template or examples below to test the skill.
4
Customize
Adapt the skill to your specific use case and workflow.
Usage Examples
Prompt Template
Analyze social media posts from [PLATFORM] (e.g., Twitter, Facebook, Instagram) to identify potential disaster events or material damages. Use the multimodal approach combining [TEXT_ANALYSIS_CRITERIA] (e.g., keywords like 'damaged', 'flood', 'fire', 'collapsed') and [IMAGE_ANALYSIS_CRITERIA] (e.g., structural damage, debris, hazardous materials). Flag posts that meet both textual and visual thresholds for immediate review. Prioritize posts with [URGENCY_LEVEL] (e.g., high-risk areas, recent timestamps). Provide a summary report with [OUTPUT_FORMAT] (e.g., JSON, CSV) including post IDs, confidence scores, and suggested actions for emergency responders.
Example Output
```json
{
"disaster_events": [
{
"post_id": "TW123456789",
"platform": "Twitter",
"timestamp": "2023-11-15T14:32:18Z",
"text": "Just saw a massive fire near the old warehouse district! The flames are spreading fast. #Emergency #Fire",
"image_url": "https://pbs.twimg.com/media/F_XYZ123.jpg",
"text_confidence": 0.98,
"image_confidence": 0.95,
"material_damages": ["structural_damage", "debris", "hazardous_materials"],
"location": "123 Main St, Warehouse District",
"urgency": "high",
"suggested_actions": [
"Dispatch fire trucks to 123 Main St immediately",
"Alert nearby residents within 0.5-mile radius",
"Coordinate with hazmat teams for potential chemical risks"
]
},
{
"post_id": "FB987654321",
"platform": "Facebook",
"timestamp": "2023-11-15T14:45:03Z",
"text": "Road completely blocked by fallen trees near the bridge. Can't get home! #Storm #Emergency",
"image_url": "https://scontent.xx.fbcdn.net/v/t1.0-9/123456789.jpg",
"text_confidence": 0.92,
"image_confidence": 0.91,
"material_damages": ["road_blockage", "fallen_trees"],
"location": "Elm St Bridge, Riverdale",
"urgency": "medium",
"suggested_actions": [
"Deploy road crews to clear debris",
"Alert local transit authorities for detours",
"Monitor for secondary hazards (e.g., downed power lines)"
]
}
],
"summary": {
"total_posts_analyzed": 1247,
"high_priority_events": 2,
"medium_priority_events": 8,
"low_priority_events": 15,
"false_positives": 3,
"recommendation": "Focus emergency response on the 2 high-priority events. Coordinate with local authorities to validate locations and deploy resources."
}
}
```Apply to these tools
Browse all toolsModal
Serverless cloud platform for AI and data workloads
Moda
Monitoring and error resolution for AI systems
Automa
Automate your browser workflows effortlessly
Microsoft Teams
Get more done every day with Microsoft Teams – powered by AI
Respell
Agentic AI Workflow platform
Notion
Connected workspace for docs, wikis, and projects
Compatible MCP servers
Browse all MCP servers
Find the right skills for your stack
Take a free 3-minute scan and get personalized AI skill recommendations.
Take free scan