Herb leaves plays a significant role to cure various human disease throughout the world. Furthermore, it has had a great impact on clinical science. People have less information about herbs accordingly it turns into an issue to recognize them. Besides, utilizing harmful plants as a drug would genuinely expand the danger of life. As a result, it becomes a significant issue to recognize the useful herb leaves. Very few works have been done to localize and classify the herb leaves in real-time for helping people. Gradually the utilization of artificial intelligence makes our life simpler and user-friendly. Complex problems can be effectively solved with the help of artificial intelligence. In this paper, we will classify the five types of herbs respectively Mehndi, Betel, Mint, Basil, and Aloe Vera. We have proposed a neural network model using YOLO which will use for the classification of the herb leaves. In the future, we will carry out the prepared model into a versatile application that will assist the client with finding out about homegrown cures. The Classification accuracy of our model is about 95% which is promising enough compared to other relevant works.