In the heart of Bangladesh’s agricultural landscape, where the cultivation of rice and tea supports both livelihoods and food security, the constant danger of crop diseases is a substantial obstacle to economic stability. This research project aims to develop innovative disease detection models specifically designed for rice and tea leaves in Bangladesh. The study will utilize deep learning with sophisticated image analysis methods. Early detection of diseases is essential due to the extensive intake of these basic food items. Additionally, the initiative seeks to advance precision agricultural techniques, decrease the reliance on pesticides, and improve food security. The MobileNetV3 Large model stood out with its remarkable accuracy of 93.20% and required minimum training time, even though it was trained on a very small dataset gathered from real-world scenarios. This project seeks to modernize the agricultural sector in Bangladesh and empower smallholder farmers by transferring technology, promoting sustainable farming practices, and aligning with government objectives. The project aims to utilise the capabilities of MobileNetV3 Large and include a small amount of field data to reduce farming losses and significantly contribute to the resilience and profitability of Bangladesh’s important agricultural industry.