Betel vine leaves diseases caused by regular endangerment to bacteria which causes a huge yield loss globally. Machine learning, the latest breakthrough in computer vision, is encouraging for fine-grained disease classification, as the method uses SVM classifier and Gaussian mixture model for image segmentation. Disease detection and classifications are considered as the two hardest works to the recognition of Betel vine disease. Two types of betel vine diseases are focused on the paper, Bacterial Leaf Spot and Stem Leaf. Pictures are taken using a phone camera or any kind of portable device and the dataset consists of almost 1275 images where each class contains 636 images. The proposed system reaches 83.69% accuracy in classification which appears to be good and promising in comparison to other relevant papers.