Explainable Rice Disease Detection Via Swin Transformer and Large Language Model Powered Visual-Textual Interpretation

Open

Luis Andrei Espina Ouano, Paul France Mendoza Detablan, Christian Valencia Maderazo

2026 ASIG 2025 - Proceedings of the 3rd Asia Symposium on Image and Graphics Conference paper Cited by 0 Quartile

Abstract

Rice crop diseases pose a serious threat to global food security, particularly in regions like the Philippines, where they cause substantial yield losses. Effective disease management demands diagnostic tools that are both accurate and interpretable for non-expert users. While explainable artificial intelligence has improved transparency in crop disease detection, many existing methods remain too technical for non-expert stakeholders. This study presents a rice disease detection system that integrates image classification with natural language explanations. The system uses a Swin Transformer model to classify rice leaf images into four categories, including Rice Blast, Brown Spot, Bacterial Blight, and healthy leaves. The model’s outputs are processed through an interpretability layer that integrates Gradient-weighted Class Activation Mapping (GradCAM) to highlight the most influential image regions and a Large Language Model (LLM) to translate these visual cues into clear, symptom-based textual descriptions together with disease management strategies. Delivered through a mobile application designed using Expo and FastAPI, the system allows users to upload images and receive real-time predictions with corresponding heatmaps and explanations. The Swin Transformer model achieved an accuracy of 96.22%, with strong precision, recall, and F1-scores across all classes, demonstrating its reliability in differentiating between visually similar diseases. This work provides a reliable and interpretable decision-support tool to enhance early disease detection, enable informed crop management, and strengthen rice production resilience within precision agriculture frameworks. © 2025 Copyright held by the owner/author(s).

Affiliations

Department of Computer, Information Sciences, and Mathematics, University of San Carlos, Cebu, Philippines