In this paper, an attempt is addressed towards accurate vegetable image classification. A dataset consisting of 21,000 images of 15 classes is used for this classification. Convolutional neural network, a deep learning algorithm is the most efficient tool in the machine learning field for classification problems. But CNN requires large datasets so that it performs well in natural image classification problems. Here, we conduct an experiment on the performance of CNN for vegetable image classification by developing a CNN model from the ground. Additionally, several pre-trained CNN architectures using transfer learning are employed to compare the accuracy with the typical CNN. This work proposes the study between such typical CNN and its architectures(VGG16, MobileNet, InceptionV3, ResNet etc.) to build up which technique would work best regarding accuracy and effectiveness with new image datasets. Experimental results are presented for all the proposed architectures of CNN. Besides, a comparative study is done between developed CNN models and pre-trained CNN architectures. And the study shows that by utilizing previous information gained from related large-scale work, the transfer learning technique can achieve better classification results over traditional CNN with a small dataset. And one more enrichment in this paper is that we build up a vegetable images dataset of 15 categories consisting of a total of 21,000 images.