Enhancing waste recognition with vision-language models: A prompt engineering approach for a scalable solution

Area of Science:

• Computer Vision
• Artificial Intelligence
• Waste Management

Background:

• Conventional computer vision models struggle with diverse waste image classification in material recovery facilities.
• Frequent fine-tuning and data augmentation are resource-intensive and unsustainable for these models.

Purpose of the Study:

• To implement and evaluate state-of-the-art Vision-Language Models (VLMs) for waste image classification.
• To assess VLM adaptability in data-scarce scenarios using zero-shot, few-shot, and fully supervised learning.
• To investigate the impact of prompt engineering on VLM performance and scalability.

Main Methods:

• Utilized state-of-the-art Vision-Language Models (VLMs) for multimodal image classification.
• Employed zero-shot, few-shot, and fully supervised learning paradigms.
• Implemented targeted prompt engineering to optimize VLM adaptability and scalability.

Main Results:

• Achieved a significant increase in zero-shot waste image classification accuracy from 82.71% to 90.48% through prompt engineering.
• Demonstrated a trade-off between accuracy and inference speed for optimal model selection.
• Attained an impressive fully supervised classification accuracy of 97.18% with the optimized VLM.

Conclusions:

• VLMs present a scalable and adaptable solution for waste image classification, outperforming traditional methods.
• Targeted prompt engineering is an effective strategy to enhance VLM performance without task-specific fine-tuning.
• This research advances waste management by offering an efficient and robust AI-driven approach to image classification.