The yellow mosaic disease is a common black gram leaf disease that causes severe economic losses to local farmers and a hindrance to healthy production which can be prevented by computer vision based fast and accurate recognition system. In this paper, Black Gram Convolutional Neural Network (BGCNN) has been proposed for the recognition of this disease, and the performance of BGCNN has compared with the state-of-the-art deep learning models such as AlexNet, VGG16, and Inception V3. All the models have trained with original dataset having 2830 images and expanded dataset generated with image augmentation having 16,980 images that increase test accuracy of all the models significantly. BGCNN realizes accuracy of 82.67% and 97.11% for the original and expanded dataset, respectively. While, AlexNet, VGG16, and Inception V3 have achieved 93.78%, 95.49%, and 96.67% accuracy for the expanded dataset, respectively. The obtained results validate that BGCNN can recognize yellow mosaic disease efficiently.