Tea grading is a very prominent factor of the tea industry. The standard, fragrance and sweetness of tea mostly relies on this grading system. This research is a step to introduce machine learning with the tea industry, where image classification and recognition is deployed to digitize the grading system by eradicating human intervention in it. Three models are used in this system in which two were pre-trained. They are Faster RCNN (Inception-v2), and VGG16. The other one is manually trained, that is Sequential model or CNN. After a successful session of compulsory augmentation and scaling, we gathered 3000 raw images which were used to train and test the model spontaneously. Our productivity has rendered us tremendous satisfaction by supplying astonishing accuracy. So, it will be not wrong saying that this research has amalgamated machine learning technology with the grading system of tea very productively which can escort a great revolution to the tea industry.