Guava (Psidium guava) is one of the most popular fruits which plays a vital role in the world economy. To increase guava production and sustain economic development, early detection and diagnosis of guava disease is important. As traditional recognition systems are time-consuming, expensive, and sometimes their predictions are also inaccurate, farmers are facing a lot of losses because of not getting the proper diagnosis and appropriate cure in time. In this study, an automatic system based on Convolution Neural Networks (CNN) models for recognizing guava disease has been proposed. To make the dataset more efficient, image processing techniques have been employed to boost the dataset which is collected from the local Guava Garden. For training and testing the applied models named InceptionResNetV2, ResNet50, and Xception with transfer learning technique, a total of 2,580 images in five categories such as Phytophthora, Red Rust, Scab, Stylar end rot, and Fresh leaf are utilized. To estimate the performance of each applied classifier, the six-performance evaluation metrics have been calculated where the Xception model conducted the highest accuracy of 98.88% which is good enough compared to other recent relevant works.