Bitter Gourd: Botanical Title is Memordica charantia L. Bitter gourd is known for its usage for medicinal purposes in Asian countries. Bitter gourd could be a wealthy source of vitamins and minerals. It contains press, magnesium, potassium, and vitamins like A and C. Different anti-oxidants and anti-inflammatory compounds are showed in Bitter gourd. Separating diseased vegetables from healthy ones of medium size vegetables is one tough job as well as costly for farmers. We propose a system which detects diseased or even defected bitter Gourd. We used Deep Learning (DL) to achieve the goal. Image processing is one of the most common yet interesting fields of DL. We used Convolutional Neural Network(CNN) to process images of Bitter Gourd. We created three different models by diversifying Convolutional Layer by number and its value. We are giving farmers a brief instrument utilizing Deep Learning that makes a difference select the non-absconded thing. The world is developing with AI within the future. So why not an agriculturist they are moreover utilizing this digitalization. We got some interesting results through the process and choose the model that performed best without test result and evaluation result over-fitting. It's M3 model with 99.70% accuracy.