Banana is amongst the most appealing and nutritious fruits worldwide, cultivated almost every part of the world round the year. Bangladesh ranks 14 th worldwide in producing this appealing fruit putting a substantial mark on the national economic growth. Classification and recognition of specific strains and identifying the quality of different agricultural products has been a challenge for mass production. With the continuous evolution of technological advancements now it has become a beneficial machine vision task to classify different strains and also determine the quality of the fruit to trash the affected ones that will minimize the loss to a great extent. In this paper, we have proposed a Convolutional Neural Network (CNN) based model that classifies five strains of different bananas namely cavendish, lady finger, shabri, green and the red banana and also identifies the rotten ones with great accuracy. We have successfully deployed the two deep learning models to find significant accuracy varying different parameters. We have also utilized the widely accepted precision, recall, F1-score and ROC evaluation metrics. The second model has outperformed the other in terms of accuracy with 93.4±0.8% and identifying the rotten bananas with an accuracy of 98.3±.8%.